Single-file scripts that download their dependencies
An ideal distributable script is fully contained in a single file. It runs on any compatible operating system with an appropriate language runtime. It is plain text, and you can copy and paste it. It does not require mucking about with a package manager, or several, to run. It does not conflict with other scripts’ packages or require managing a project environment to avoid such conflicts.
The classic way to work around all of these issues with scripts is to limit yourself to using the scripting language’s standard library. However, programmers writing scripts don’t want to; they want to use libraries that do not come with the language by default. Some scripting languages, runtimes, and script runners resolve this conflict by offering a means to download and cache a script’s dependencies with just declarations in the script itself. This page lists such languages, runtimes, and script runners.
Some of the following examples use “exec magic”. “Exec magic” means a shell command embedded in code in a different language that starts the interpreter for that language. It usually replaces the shell, hence exec
. Exec magic is a form of polyglot code. The term comes from the Tcl programming language community.
On modern systems
#! /usr/bin/env interp
has mostly replaced exec magic. However, there is still a use case for it: passing arguments to interp
. The POSIX shebang line can only pass a single argument to /usr/bin/env
. Everything starting with interp
is not split on spaces or in any other way.
Here is a real example that comes from pipx:
#! /usr/bin/env pipx run
in ./script.py
is equivalent to the shell command
/usr/bin/env 'pipx run' ./script.py
Recent GNU env(1) and FreeBSD env(1) add a new flag -S
to split argument.
#! /usr/bin/env -S pipx run
in ./script.foo
is equivalent to the shell command
/usr/bin/env pipx run ./script.foo
because env(1) splits the single argument -S pipx run
into -S
, pipx
, run
.
env -S
is not part of the POSIX standard. Where it is not availble, exec magic will be presented as a workaround. In the case of pipx, it can be
#! /bin/sh
"exec" "/usr/bin/env" "pipx" "run" "$0" "$@"
The Nix package manager can act as a #!
interpreter and start another program with a list of dependencies available to it.
#! /usr/bin/env nix-shell
#! nix-shell -i python3 -p python310 python310Packages.ansicolors
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
Scriptisto is a “language-agnostic ‘shebang interpreter’ that enables you to write scripts in compiled languages.”
#! /usr/bin/env scriptisto
# scriptisto-begin
# script_src: test.py
# build_once_cmd: python3 -m venv . && . ./bin/activate && pip install ansicolors
# build_cmd: . ./bin/activate && chmod +x ./run.sh
# target_bin: ./run.sh
# files:
# - path: run.sh
# content: |
# #! /bin/sh
# export DIR=$(dirname "$0")
# . "$DIR"/bin/activate
# python3 "$DIR"/test.py "$@"
# scriptisto-end
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
The unofficial .NET tool dotnet-script can download NuGet packages.
#! /usr/bin/env dotnet-script
#r "nuget: Spectre.Console, 0.46.0"
using Spectre.Console;
AnsiConsole.Markup("[bold]" + "Hello, world!".PadLeft(20, '-') + "[/]\n");
The Babashka interpreter runs scripts in (a large subset of) Clojure. It has dynamic dependency resolution.
#! /usr/bin/env bb
(require '[babashka.deps :as deps])
(deps/add-deps '{:deps {coldnew/left-pad {:mvn/version "1.0.0"}}})
(require '[coldnew.left-pad :refer [leftpad]])
(print (leftpad "Hello, world!" 20 "-"))
You can write scripts in Crystal with crun. It was inspired by gorun.
#! /usr/bin/env crun
# ---
# crinja:
# github: straight-shoota/crinja
# ...
require "crinja"
puts Crinja.render(
"{{ greeting }}!",
{"greeting" => "Hello, world".rjust(19, '-')}
)
D’s official package manager DUB supports single-file packages.
#! /usr/bin/env dub
/+ dub.sdl:
dependency "colorize" version="~>1.0.5"
+/
import std.conv : to;
import std.range : padLeft;
import colorize : color, cwriteln, mode;
void main() {
auto greeting = "Hello, world!".padLeft('-', 20).to!string;
cwriteln(greeting.color(mode.bold));
}
The configuration language Dhall has native support for remote imports. You can write shebang scripts in Dhall with env -S
.
#! /usr/bin/env -S dhall text --file
let Text/replicate = https://prelude.dhall-lang.org/Text/replicate
in Text/replicate 7 "-"
++ ''
Hello, world!
''
In Elixir 1.12 and later you can call Mix.install/2
from scripts.
#! /usr/bin/env elixir
Mix.install([
{:leftpad, "~> 1.0"}
])
Leftpad.pad("Hello, world!", 20, ?-)
|> IO.puts()
The .NET SDK has integrated support for F# scripts with dependencies.
#! /usr/bin/env -S dotnet fsi
#r "nuget: Spectre.Console, 0.45.0"
open Spectre.Console
AnsiConsole.Markup("[bold]" + "Hello, world!".PadLeft(20, '-') + "[/]\n")
gorun lets you embed go.mod
and go.sum
in the source file.
/// 2>/dev/null ; gorun "$0" "$@" ; exit $?
//
// go.mod >>>
// module foo
// go 1.18
// require github.com/keltia/leftpad v0.1.0
// <<< go.mod
//
// go.sum >>>
// github.com/keltia/leftpad v0.1.0 h1:b2jL8i1cRsuLITknjWZHRAwf1Jwh+twMUkhwDXD7fqc=
// github.com/keltia/leftpad v0.1.0/go.mod h1:f/6pIO5tikWLxSz68iAcl9OXDpV6k2+2mRm9jShMzoU=
// <<< go.sum
package main
import "fmt"
import "github.com/keltia/leftpad"
func main() {
padded, _ := leftpad.PadChar("Hello, world!", 20, '-')
fmt.Printf("%v\n", padded)
}
Groovy comes with an embedded JAR dependency manager.
#! /usr/bin/env groovy
@Grab(group='org.apache.commons', module='commons-lang3', version='3.12.0')
import org.apache.commons.lang3.StringUtils
println StringUtils.leftPad('Hello, world!', 20, '-')
The Haskell build tool Stack has integrated support for scripts.
#! /usr/bin/env stack
-- stack --resolver lts-6.35 script --package acme-left-pad
import Data.Text.LeftPad
main :: IO ()
main = do
putStrLn $ leftPad "Hello, world!" 20 "-"
JBang lets you write scripts in Java, Kotlin, and Groovy. It was inspired by kscript.
///usr/bin/env jbang "$0" "$@" ; exit $?
//DEPS org.apache.commons:commons-lang3:3.12.0
import static java.lang.System.*;
import org.apache.commons.lang3.StringUtils;
public class hello {
public static void main(String... args) {
out.println(StringUtils.leftPad("Hello, world!", 20, "-"));
}
}
Bun resolves NPM dependencies in scripts.
#! /usr/bin/env bun
import leftPad from "left-pad";
console.log(leftPad("Hello, world!", 20, "-"));
Deno downloads dependencies like a browser. Deno 1.28 and later can also import from NPM packages. Current versions of Deno require you to pass a run
argument to deno
. One way to accomplish this from a script is with a form of exec magic. Here the magic is modified from a comment by Rafał Pocztarski.
#! /bin/sh
":" //#; exec /usr/bin/env deno run "$0" "$@"
import leftPad from "npm:left-pad";
console.log(leftPad("Hello, world!", 20, "-"));
You can replace the magic with env -S
on systems that support it.
#! /usr/bin/env -S deno run
import leftPad from "npm:left-pad";
console.log(leftPad("Hello, world!", 20, "-"));
kscript is an unofficial scripting tool for Kotlin that understands several comment-based directives, including one for dependencies. Since version 1.4 Kotlin also has experimental integrated scripting.
New versions of kscript do not allow a space after #!
.
#!/usr/bin/env kscript
@file:DependsOn("org.apache.commons:commons-lang3:3.12.0")
import org.apache.commons.lang3.StringUtils
println(StringUtils.leftPad("Hello, world!", 20, "-"))
Python has standardized inline script metadata. The standard has made several existing and new script runners compatible. The standard specifies a TOML-based data format to use in a script comment. This format declares the Python version required for the script and a list of its dependencies.
Inline script metadata was originally standardized in PEP 723 in 2023. Earlier PEP 722 (which links to this document—woot!) proposed a line-oriented format that did not use TOML. It is implemented in pip-run and was briefly implemented in pipx. PEP 722 inspired PEP 723 and was rejected in its favor.
fades runs Python scripts with dependencies. It offers four ways to specify dependencies:
- In comments, normally on imports;
- With one or more command-line option
-d
or--dependency
; - With the command-line option
-r
or--requirements
and arequirements.txt
file; - In the script docstring.
Here is the comments method:
#! /usr/bin/env fades
from colors import bold # fades ansicolors>=1,<2
print(bold("Hello, world!".rjust(20, "-")))
The option -d
with env -S
:
#! /usr/bin/env -S fades -d ansicolors>=1,<2
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
A docstring:
#! /usr/bin/env fades
"""
fades:
ansicolors>=1,<2
"""
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
Hatch is a Python project-mamanagement tool. It can run tasks and scripts for projects with the commamd hatch run
. Version 1.10.0 extended this command to support inline script metadata derived from PEP 723.
The following example is identical to pipx and uv, which implements the same standard. You will need env -S
to make the script run as a command.
#! /usr/bin/env -S hatch run
# /// script
# dependencies = ["ansicolors>=1,<2"]
# requires-python = ">=3.8"
# ///
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
If your system does not support env -S
, you can instead use exec magic:
#! /bin/sh
"exec" "/usr/bin/env" "hatch" "run" "$0" "$@"
One feature of pip-run is running Python scripts with dependencies. The script can declare its dependencies in several ways:
- In the
env -S
shebang line (in the example) - Using the special variable
__requires__
- In a top commment, which can be either PEP 722-style or inline script metadata like Hatch and pipx.
An example of env -S
:
#! /usr/bin/env -S pip-run ansicolors>=1,<2
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
A PEP 722 top comment:
#! /usr/bin/env pip-run
# Requirements:
# ansicolors>=1,<2
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
pip-wtenv is my counterpart to pip.wtf. See the next item. pip-wtenv differs from pip.wtf in that it uses venvs, supports Windows, and has a free license. It requires newer Python.
#! /usr/bin/env python3
def pip_wtenv(*args: str, name: str = "", venv_parent_dir: str = "") -> None:
"""
Download and install dependencies in a virtual environment.
See https://github.com/dbohdan/pip-wtenv.
Warning: this function will restart Python
if Python is not running in a venv.
pip-wtenv requires Python >= 3.6 on POSIX systems
and Python >= 3.8 on Windows.
"""
from os import execl
from pathlib import Path
from subprocess import run
from sys import argv, base_prefix, platform, prefix
from venv import create as create_venv
me = Path(__file__)
venv_dir = (
Path(venv_parent_dir).expanduser() if venv_parent_dir else me.parent
) / f".venv.{name or me.name}"
if not venv_dir.exists():
create_venv(venv_dir, with_pip=True)
ready_marker = venv_dir / "ready"
venv_python = venv_dir / (
"Scripts/python.exe" if platform == "win32" else "bin/python"
)
if not ready_marker.exists():
run(
[venv_python, "-m", "pip", "install", "--quiet", "--upgrade", "pip"],
check=True,
)
run([venv_python, "-m", "pip", "install", *args], check=True)
ready_marker.touch()
# If we are not running in a venv, restart with `venv_python`.
if prefix == base_prefix:
execl(venv_python, venv_python, *argv)
pip_wtenv("ansicolors")
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
pip.wtf is a self-contained code snippet born out of its author’s frustration with Python packaging. It is a single function that you can copy into your script and call to install the script’s dependencies.
#! /usr/bin/env python3
# https://pip.wtf
def pip_wtf(command):
import os, os.path, sys
t = os.path.join(os.path.dirname(os.path.abspath(__file__)), ".pip_wtf." + os.path.basename(__file__))
sys.path = [p for p in sys.path if "-packages" not in p] + [t]
os.environ["PATH"] = t + os.path.sep + "bin" + os.pathsep + os.environ["PATH"]
os.environ["PYTHONPATH"] = os.pathsep.join(sys.path)
if os.path.exists(t): return
os.system(" ".join([sys.executable, "-m", "pip", "install", "-t", t, command]))
pip_wtf("ansicolors")
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
pipx version 1.3.0 and higher supports running scripts in addition to packages. Version 1.4.2 brought it in sync with the latest revision of PEP 723.
The following example is identical to Hatch and uv, which implements the same standard. You will need env -S
to make the script run as a command.
#! /usr/bin/env -S pipx run
# /// script
# dependencies = ["ansicolors>=1,<2"]
# requires-python = ">=3.8"
# ///
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
If your system does not support env -S
, you can instead use exec magic:
#! /bin/sh
"exec" "/usr/bin/env" "pipx" "run" "$0" "$@"
The package and project manager uv can run scripts with dependencies. This was implemented in version 0.3.0. The dependencies can be specified in the shebang line using env -S
or exec magic or as standard inline script metadata.
Use of the shebang line looks like this:
#! /usr/bin/env -S uv run --with ansicolors>=1,<2
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
The following example is identical to Hatch and pipx, which implements the same standard. You will need env -S
to make the script run as a command.
#! /usr/bin/env -S uv run
# /// script
# dependencies = ["ansicolors>=1,<2"]
# requires-python = ">=3.8"
# ///
from colors import bold
print(bold("Hello, world!".rjust(20, "-")))
If your system does not support env -S
, you can instead use exec magic:
#! /bin/sh
"exec" "/usr/bin/env" "uv" "run" "$0" "$@"
fades, Hatch, pip-wtenv, pipx, and uv create a virtual environment (venv) for the script on the first run and keep it for subsequent runs; pip.wtf does the same with a directory. By default, pip-run recreates the virtual environment on every run of a script. While this avoids the potential problem of venv rot and saves some disk space, it adds a noticeable delay to script startup. You can change this behavior by setting the environment variable PIP_RUN_RETENTION_STRATEGY
to persist
. An unset variable is equivalent to destroy
. You can set the variable in the env
shebang line of a script. pip-run will then cache the virtual environment.
On 2024-08-25, I compared the performance of:
- fades 9.0.2
- Hatch 1.12.0
- pip-run 13.0.0
- pip-wtenv 0.2.0
- pip.wtf
01e79be
- pipx 1.7.1
- uv 0.3.3
I used the import-comment example above for fades and the inline script metadata example for Hatch, pip-run, pipx, and uv. I measured the run time with the following script. The script runs each benchmark target in a separate Podman container using the Docker image python:3.12.4
.
Script source.
#! /usr/bin/env python3
from __future__ import annotations
import shlex
import subprocess
from pathlib import Path
CONTAINER_IMAGE = "python:3.12.4"
CONTAINER_RUNTIME = "podman"
DIR_IN_CONTAINER = Path("/mnt")
RUNS = 10
TARGETS_WITH_DEPS = [
("fades.py", ["fades==9.0.2"]),
("hatch.py", ["hatch==1.12.0"]),
("pip-run-destroy.py", ["pip-run==13.0.0"]),
("pip-run-persist.py", ["pip-run==13.0.0"]),
("pip-wtenv.py", []),
("pip.wtf.py", []),
("pipx.py", ["pipx==1.7.1"]),
("uv.py", ["uv==0.3.3"]),
]
def benchmark_script_text(target: str, deps: list[str]) -> str:
target_path = DIR_IN_CONTAINER / target
pip_install_line = (
f"python3 -m pip --quiet install {shlex.join(deps)}" if deps else ""
)
return f"""#! /bin/sh
set -eu
apt-get -qq update
apt-get -qq -y install time
{pip_install_line}
for _ in $(seq {RUNS}); do
time \\
--append \\
--format '%e' \\
--output {shlex.quote(str(target_path.with_suffix(".times")))} \\
{shlex.quote(str(target_path))}
done
"""
def main() -> None:
benchmark_script = Path("bench.sh")
benchmark_script.touch(0o755)
for target, deps in TARGETS_WITH_DEPS:
print(f"=== {target}")
benchmark_script.write_text(benchmark_script_text(target, deps))
subprocess.run(
[
CONTAINER_RUNTIME,
"run",
"--rm",
"--interactive",
"--tty",
"--mount",
f"type=bind,source={Path.cwd()},target={DIR_IN_CONTAINER}",
CONTAINER_IMAGE,
DIR_IN_CONTAINER / benchmark_script,
],
check=False,
)
if __name__ == "__main__":
main()
The run times I got were:
- fades
-
6.89, 0.14, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13
- Hatch
-
0.95, 0.16, 0.15, 0.14, 0.15, 0.15, 0.15, 0.15, 0.16, 0.15
- pip-run,
destroy
(default) -
1.15, 0.86, 0.86, 0.83, 0.83, 0.86, 0.87, 0.88, 0.87, 0.86
- pip-run,
persist
-
1.14, 0.11, 0.11, 0.12, 0.11, 0.12, 0.12, 0.11, 0.12, 0.12
- pip-wtenv
-
0.23, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06
- pip.wtf
-
0.08, 0.01, 0.01, 0.02, 0.01, 0.01, 0.01, 0.02, 0.02, 0.01
- pipx
-
6.27, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12
- uv
-
0.57, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02
You can see that when their respective scripts first ran, fades, Hatch, pip-run in persist
mode, pip-wtenv, pip.wtf, pipx, and uv took time to create the virtual environment and install the dependencies (the package ansicolors
). pip-run in destroy
mode also took slightly longer the first time to download and cache the package.
Results.
Platform: Debian 12 Docker container.
- fades 9.0.2
- Hatch 1.12.0
- pip-run 12.6.1
- pip-wtenv 0.2.0
- pip.wtf
01e79be
- pipx 1.6.0
- fades
-
6.19, 0.13, 0.12, 0.12, 0.13, 0.12, 0.12, 0.14, 0.12, 0.12
- Hatch
-
0.74, 0.14, 0.15, 0.15, 0.15, 0.14, 0.14, 0.14, 0.14, 0.15
- pip-run,
destroy
(default) -
1.07, 0.84, 0.85, 0.83, 0.84, 0.86, 0.83, 0.84, 0.85, 0.88
- pip-run,
persist
-
1.04, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.11, 0.12
- pip-wtenv
-
0.21, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06, 0.06
- pip.wtf
-
0.08, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01
- pipx
-
5.78, 0.13, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12
Results.
Platform: Debian 12 Docker container.
- fades 9.0.2
- pip-run 12.5.0
- pip-wtenv
6d6937a
- pip.wtf
01e79be
- pipx 1.4.3
- fades
-
5.56, 0.13, 0.12, 0.13, 0.12, 0.13, 0.13, 0.13, 0.12, 0.13
- pip-run,
destroy
(default) -
0.86, 0.69, 0.68, 0.68, 0.68, 0.68, 0.69, 0.68, 0.68, 0.68
- pip-run,
persist
-
0.87, 0.09, 0.09, 0.09, 0.09, 0.09, 0.08, 0.08, 0.09, 0.09
- pip-wtenv
-
5.00, 0.05, 0.04, 0.05, 0.05, 0.04, 0.04, 0.04, 0.04, 0.04
- pip.wtf
-
0.79, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01
- pipx
-
4.82, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13
Results.
Platform: Ubuntu 22.04.
- fades 9.0.2
- pip-run 12.4.0
- pipx 1.4.2
- fades
-
7.83, 0.18, 0.17, 0.17, 0.17, 0.18, 0.18, 0.17, 0.19, 0.18
- pip-run
-
1.53, 1.21, 1.20, 1.21, 1.20, 1.20, 1.22, 1.22, 1.22, 1.20
- pipx
-
1.17, 0.18, 0.17, 0.18, 0.17, 0.17, 0.18, 0.18, 0.18, 0.18
Results.
Platform: Ubuntu 22.04.
- pip-run v12.2.2
- pipx 1.2.0.1.dev0 (commit
271d0f1
)
- pip-run
-
1.36, 1.37, 1.36, 1.38, 1.36, 1.36, 1.37, 1.34, 1.36, 1.35
- pipx
-
0.98, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12, 0.12
When a script encountered an exception, older versions of pipx used to have <string>
instead of the script filename in the traceback. Now fades, pip-run, and pipx produce identical tracebacks except for their different file path normalization.
Scripty interactively prompts you to install the missing dependencies for a script in any Racket language.
#! /usr/bin/env racket
#lang scripty
#:dependencies '("base" "typed-racket-lib" "left-pad")
------------------------------------------
#lang typed/racket/base
(require left-pad/typed)
(displayln (left-pad "Hello, world!" 20 "-"))
The dependency manager Bundler has an API for single-file scripts.
#! /usr/bin/env ruby
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'left-pad', '~> 1.1'
end
puts 'Hello, world!'.leftpad(20, '-')
rust-script can evaluate expressions and run scripts in Rust.
#! /usr/bin/env rust-script
//! ```cargo
//! [dependencies]
//! leftpad-rs = "1.2.0"
//! ```
use leftpad_rs::pad_char;
fn main() {
println!("{}", pad_char("Hello, world!", 20, '-').unwrap());
}
The scripting environment in Ammonite lets you import Ivy dependencies.
#! /usr/bin/env amm
import $ivy.`org.apache.commons:commons-lang3:3.12.0`,
org.apache.commons.lang3.StringUtils
println(StringUtils.leftPad("Hello, world!", 20, "-"))
Scala CLI is a newer tool that understands a subset of Ammonite’s imports.
#! /usr/bin/env scala-cli
import $ivy.`org.apache.commons:commons-lang3:3.12.0`,
org.apache.commons.lang3.StringUtils
println(StringUtils.leftPad("Hello, world!", 20, "-"))
- Hacker News discussion. Thanks to everyone who made suggestions.