Single-file scripts that download their dependencies

An ideal distributable script is fully contained in a single file. It runs on any compatible operating system with an appropriate language runtime. It is plain text, and you can copy and paste it. It does not require mucking about with a package manager, or several, to run. It does not conflict with other scripts’ packages or require managing a project environment to avoid such conflicts.

The classic way to work around all of these issues with scripts is to limit yourself to using the scripting language’s standard library. However, programmers writing scripts don’t want to; they want to use libraries that do not come with the language by default. Some scripting languages, runtimes, and environments resolve this conflict by offering a means to download and cache a script’s dependencies with just declarations in the script itself. This page lists such languages, runtimes, and environments.

Introductory note: exec magic

Some of the following examples use “exec magic”. “Exec magic” means a shell command embedded in code in a different language that starts the interpreter for that language. It usually replaces the shell, hence exec. Exec magic is a form of polyglot code. The term comes from the Tcl programming language community.

On modern systems

#! /usr/bin/env interp

has mostly replaced exec magic. However, there is still a use case for it: passing arguments to interp. The POSIX shebang line can only pass a single argument to /usr/bin/env. Everything starting with interp is not split on spaces or in any other way.

Here is a real example that comes from pipx:

#! /usr/bin/env pipx run

in ./script.py is equivalent to the shell command

/usr/bin/env 'pipx run' ./script.py

Recent GNU env(1) and FreeBSD env(1) add a new flag -S to split argument.

#! /usr/bin/env -S pipx run

in ./script.foo is equivalent to the shell command

/usr/bin/env pipx run ./script.foo

because env(1) splits the single argument -S pipx run into -S, pipx, run.

env -S is not part of the POSIX standard. Where it is not availble, exec magic will be presented as a workaround. In the case of piox, it can be

#! /bin/sh
"exec" "/usr/bin/env" "pipx" "run" "$@"

Polyglot tools

Nix

The Nix package manager can act as a #! interpreter and start another program with a list of dependencies available to it.

#! /usr/bin/env nix-shell
#! nix-shell -i python3 -p python310 python310Packages.ansicolors

from colors import bold

print(bold("Hello, world!".rjust(20, "-")))

Scriptisto

Scriptisto is a “language-agnostic ‘shebang interpreter’ that enables you to write scripts in compiled languages.”

#! /usr/bin/env scriptisto

# scriptisto-begin
# script_src: test.py
# build_once_cmd: python3 -m venv . && . ./bin/activate && pip install ansicolors
# build_cmd: . ./bin/activate && chmod +x ./run.sh
# target_bin: ./run.sh
# files:
#   - path: run.sh
#     content: |
#       #! /bin/sh
#       export DIR=$(dirname "$0")
#       . "$DIR"/bin/activate
#       python3 "$DIR"/test.py "$@"
# scriptisto-end

from colors import bold

print(bold("Hello, world!".rjust(20, "-")))

C# (dotnet-script)

The unofficial .NET tool dotnet-script can download NuGet packages.

#! /usr/bin/env dotnet-script
#r "nuget: Spectre.Console, 0.46.0"

using Spectre.Console;

AnsiConsole.Markup("[bold]" + "Hello, world!".PadLeft(20, '-') + "[/]\n");

Clojure (Babashka)

The Babashka interpreter runs scripts in (a large subset of) Clojure. It has dynamic dependency resolution.

#! /usr/bin/env bb

(require '[babashka.deps :as deps])
(deps/add-deps '{:deps {coldnew/left-pad {:mvn/version "1.0.0"}}})

(require '[coldnew.left-pad :refer [leftpad]])
(print (leftpad "Hello, world!" 20 "-"))

Crystal (crun)

You can write scripts in Crystal with crun. It was inspired by gorun.

#! /usr/bin/env crun
# ---
# crinja:
#   github: straight-shoota/crinja
# ...

require "crinja"

puts Crinja.render(
  "{{ greeting }}!",
  {"greeting" => "Hello, world".rjust(19, '-')}
)

D

D’s official package manager DUB supports single-file packages.

#! /usr/bin/env dub
/+ dub.sdl:
dependency "colorize" version="~>1.0.5"
+/

import std.conv : to;
import std.range : padLeft;
import colorize : color, cwriteln, mode;

void main() {
    auto greeting = "Hello, world!".padLeft('-', 20).to!string;
    cwriteln(greeting.color(mode.bold));
}

Dhall

The configuration language Dhall has native support for remote imports. You can write shebang scripts in Dhall with env -S.

#! /usr/bin/env -S dhall text --file

let Text/replicate = https://prelude.dhall-lang.org/Text/replicate

in      Text/replicate 7 "-"
    ++  ''
        Hello, world!
        ''

Elixir

In Elixir 1.12 and later you can call Mix.install/2 from scripts.

#! /usr/bin/env elixir

Mix.install([
  {:leftpad, "~> 1.0"}
])

Leftpad.pad("Hello, world!", 20, ?-)
|> IO.puts()

F#

The .NET SDK has integrated support for F# scripts with dependencies.

#! /usr/bin/env -S dotnet fsi
#r "nuget: Spectre.Console, 0.45.0"

open Spectre.Console

AnsiConsole.Markup("[bold]" + "Hello, world!".PadLeft(20, '-') + "[/]\n")

Go (gorun)

gorun lets you embed go.mod and go.sum in the source file.

/// 2>/dev/null ; gorun "$0" "$@" ; exit $?
//
// go.mod >>>
// module foo
// go 1.18
// require github.com/keltia/leftpad v0.1.0
// <<< go.mod
//
// go.sum >>>
// github.com/keltia/leftpad v0.1.0 h1:b2jL8i1cRsuLITknjWZHRAwf1Jwh+twMUkhwDXD7fqc=
// github.com/keltia/leftpad v0.1.0/go.mod h1:f/6pIO5tikWLxSz68iAcl9OXDpV6k2+2mRm9jShMzoU=
// <<< go.sum

package main

import "fmt"
import "github.com/keltia/leftpad"

func main() {
	padded, _ := leftpad.PadChar("Hello, world!", 20, '-')
	fmt.Printf("%v\n", padded)
}

Groovy

Groovy comes with an embedded JAR dependency manager.

#! /usr/bin/env groovy
@Grab(group='org.apache.commons', module='commons-lang3', version='3.12.0')

import org.apache.commons.lang3.StringUtils

println StringUtils.leftPad('Hello, world!', 20, '-')

Haskell (Stack)

The Haskell build tool Stack has integrated support for scripts.

#! /usr/bin/env stack
-- stack --resolver lts-6.35 script --package acme-left-pad

import Data.Text.LeftPad

main :: IO ()
main = do
    putStrLn $ leftPad "Hello, world!" 20 "-"

Java (JBang)

JBang lets you write scripts in Java, Kotlin, and Groovy. It was inspired by kscript.

///usr/bin/env jbang "$0" "$@" ; exit $?
//DEPS org.apache.commons:commons-lang3:3.12.0

import static java.lang.System.*;
import org.apache.commons.lang3.StringUtils;

public class hello {
    public static void main(String... args) {
        out.println(StringUtils.leftPad("Hello, world!", 20, "-"));
    }
}

JavaScript

Bun

Bun resolves NPM dependencies in scripts.

#! /usr/bin/env bun

import leftPad from "left-pad";

console.log(leftPad("Hello, world!", 20, "-"));

Deno

Deno downloads dependencies like a browser. Deno 1.28 and later can also import from NPM packages. Current versions of Deno require you to pass a run argument to deno. One way to accomplish this from a script is with a form of exec magic. Here the magic is modified from a comment by Rafał Pocztarski.

#! /bin/sh
":" //#; exec /usr/bin/env deno run "$0" "$@"

import leftPad from "npm:left-pad";

console.log(leftPad("Hello, world!", 20, "-"));

You can replace the magic with env -S on systems that support it.

#! /usr/bin/env -S deno run

import leftPad from "npm:left-pad";

console.log(leftPad("Hello, world!", 20, "-"));

Kotlin (kscript)

kscript is an unofficial scripting tool for Kotlin that understands several comment-based directives, including one for dependencies. Since version 1.4 Kotlin also has experimental integrated scripting.

New versions of kscript do not allow a space after #!.

#!/usr/bin/env kscript
@file:DependsOn("org.apache.commons:commons-lang3:3.12.0")

import org.apache.commons.lang3.StringUtils

println(StringUtils.leftPad("Hello, world!", 20, "-"))

Python

fades

fades runs Python scripts with dependencies. It offers four ways to specify dependencies:

In comments, normally on imports;
With one or more command-line option -d or --dependency;
With the command-line option -r or --requirements and a requirements.txt file;
In the script docstring.

Here is the comments method:

#! /usr/bin/env fades

from colors import bold  # fades ansicolors>=1,<2

print(bold("Hello, world!".rjust(20, "-")))

The option -d with env -S:

#! /usr/bin/env -S fades -d ansicolors>=1,<2

from colors import bold

print(bold("Hello, world!".rjust(20, "-")))

A docstring:

#! /usr/bin/env fades
"""
fades:
    ansicolors>=1,<2
"""

from colors import bold

print(bold("Hello, world!".rjust(20, "-")))

pip-run

One feature of pip-run is running Python scripts with dependencies. The script can declare its dependencies:

In the env -S shebang line (in the example)
Using the special variable __requires__
In a top commment, which can be either PEP 722-style or PEP 723

An example of env -S:

#! /usr/bin/env -S pip-run ansicolors>=1,<2

from colors import bold

print(bold("Hello, world!".rjust(20, "-")))

A top comment:

#! /usr/bin/env pip-run
# Requirements:
#     ansicolors>=1,<2

from colors import bold

print(bold("Hello, world!".rjust(20, "-")))

pip-wtenv

pip-wtenv is my counterpart to pip.wtf. See the next item. pip-wtenv differs from pip.wtf in that it uses venvs, supports Windows, and has a free license. It requires newer Python.

#! /usr/bin/env python3


def pip_wtenv(*args: str, name: str = "", venv_parent: str = "") -> None:
    """
    Download and install dependencies in a virtual environment.
    See https://github.com/dbohdan/pip-wtenv.

    Warning: this function will restart Python
    if Python is not running in a venv.

    pip-wtenv requires Python >= 3.6 on POSIX systems
    and Python >= 3.8 on Windows.
    """

    from os import execl
    from pathlib import Path
    from subprocess import run
    from sys import argv, base_prefix, platform, prefix
    from venv import create as create_venv

    me = Path(__file__)
    venv_dir = (
        Path(venv_parent).expanduser() if venv_parent else me.parent
    ) / f".venv.{name or me.name}"

    if not venv_dir.exists():
        create_venv(venv_dir, with_pip=True)

    ready_marker = venv_dir / "ready"
    venv_python = venv_dir / (
        "Scripts/python.exe" if platform == "win32" else "bin/python"
    )

    if not ready_marker.exists():
        run([venv_python, "-m", "pip", "install", "--upgrade", "pip"], check=True)
        run([venv_python, "-m", "pip", "install", *args], check=True)
        ready_marker.touch()

    # If we are not running in a venv,
    # restart with `venv_python`.
    if prefix == base_prefix:
        execl(venv_python, venv_python, *argv)


pip_wtenv("ansicolors")

from colors import bold

print(bold("Hello, world!".rjust(20, "-")))

pip.wtf

pip.wtf is a self-contained code snippet born out of its author’s frustration with Python packaging. It is a single function that you can copy into your script and call to install the script’s dependencies.

#! /usr/bin/env python3

# https://pip.wtf
def pip_wtf(command):
    import os, os.path, sys
    t = os.path.join(os.path.dirname(os.path.abspath(__file__)), ".pip_wtf." + os.path.basename(__file__))
    sys.path = [p for p in sys.path if "-packages" not in p] + [t]
    os.environ["PATH"] = t + os.path.sep + "bin" + os.pathsep + os.environ["PATH"]
    os.environ["PYTHONPATH"] = os.pathsep.join(sys.path)
    if os.path.exists(t): return
    os.system(" ".join([sys.executable, "-m", "pip", "install", "-t", t, command]))


pip_wtf("ansicolors")

from colors import bold

print(bold("Hello, world!".rjust(20, "-")))

pipx

pipx version 1.3.0 and higher supports running scripts in addition to packages. Version 1.4.2 brought it in sync with the latest revision of PEP 723.

You will once again need env -S to make the script run as a command.

#! /usr/bin/env -S pipx run
# /// script
# dependencies = ["ansicolors>=1,<2"]
# requires-python = ">=3.8"
# ///

from colors import bold

print(bold("Hello, world!".rjust(20, "-")))

If your system does not support env -S, you can instead use exec magic:

#! /bin/sh
"exec" "/usr/bin/env" "pipx" "run" "$@"

Comparison

Performance

fades and pipx create a virtual environment (venv) for a script on the first run and cache it. By default, pip-run recreates the virtual environment on every run of a script. While this avoids the potential problem of venv rot and saves some disk space, it adds a noticeable delay to script startup. You can change this behavior by setting the environment variable PIP_RUN_RETENTION_STRATEGY to persist. An unset variable is equivalent to destroy. You can set the variable in the env shebang line of a script. pip-run will then cache the virtual environment.

In February 2024 I compared the performance of:

fades 9.0.2
pip-run 12.5.0
pip-wtenv 0.1.0
pip.wtf 01e79be
pipx 1.4.3

I used the import-comment example for fades and the PEP 723 example for pip-run and pipx. I measured the run time using the following script in a Debian 12 container on x86-64 Linux.

#! /bin/sh
set -eu

PATH=$HOME/.local/bin:$PATH

apt-get update
apt-get install -y python3 python3-pip python3-venv time

pip install \
    --break-system-packages \
    --user \
    fades==9.0.2 \
    pip-run==12.5.0 \
    pipx==1.4.3 \
    ;

for script in \
    fades \
    pip-run-destroy \
    pip-run-persist \
    pip-wtenv \
    pip.wtf \
    pipx \
    ;
do
    echo "=== $script"

    log_file="$script.log"
    rm "$log_file" || true
    pip cache purge

    for _ in $(seq 10); do
        command time -f '%e' ./"$script".py 2>> "$log_file"
    done
done

The run times I got were:

fades: 5.56, 0.13, 0.12, 0.13, 0.12, 0.13, 0.13, 0.13, 0.12, 0.13
pip-run, destroy (default): 0.86, 0.69, 0.68, 0.68, 0.68, 0.68, 0.69, 0.68, 0.68, 0.68
pip-run, persist: 0.87, 0.09, 0.09, 0.09, 0.09, 0.09, 0.08, 0.08, 0.09, 0.09
pip-wtenv: 5.00, 0.05, 0.04, 0.05, 0.05, 0.04, 0.04, 0.04, 0.04, 0.04
pip.wtf: 0.79, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01
pipx: 4.82, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13, 0.13

You can see that when their respective scripts first ran, fades, pip-run in persist mode, pip-wtenv, pip.wtf, and pipx took time to create the virtual environment and install the dependencies (the package ansicolors). pip-run in destroy mode also took slightly longer the first time to download and cache the package.

Tracebacks

When a script encountered an exception, older versions of pipx used to have <string> instead of the script filename in the traceback. Now fades, pip-run, and pipx produce identical tracebacks except for their different file path normalization.

Racket (Scripty)

Scripty interactively prompts you to install the missing dependencies for a script in any Racket language.

#! /usr/bin/env racket
#lang scripty
#:dependencies '("base" "typed-racket-lib" "left-pad")
------------------------------------------

#lang typed/racket/base

(require left-pad/typed)
(displayln (left-pad "Hello, world!" 20 "-"))

Ruby (Bundler)

The dependency manager Bundler has an API for single-file scripts.

#! /usr/bin/env ruby

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'left-pad', '~> 1.1'
end

puts 'Hello, world!'.leftpad(20, '-')

Rust (rust-script)

rust-script can evaluate expressions and run scripts in Rust.

#! /usr/bin/env rust-script
//! ```cargo
//! [dependencies]
//! leftpad-rs = "1.2.0"
//! ```

use leftpad_rs::pad_char;

fn main() {
    println!("{}", pad_char("Hello, world!", 20, '-').unwrap());
}

Scala

Ammonite

The scripting environment in Ammonite lets you import Ivy dependencies.

#! /usr/bin/env amm

import $ivy.`org.apache.commons:commons-lang3:3.12.0`,
  org.apache.commons.lang3.StringUtils

println(StringUtils.leftPad("Hello, world!", 20, "-"))

Scala CLI

Scala CLI is a newer tool that understands a subset of Ammonite’s imports.

#! /usr/bin/env scala-cli

import $ivy.`org.apache.commons:commons-lang3:3.12.0`,
  org.apache.commons.lang3.StringUtils

println(StringUtils.leftPad("Hello, world!", 20, "-"))