buckets

Throwaway runtime buckets for AI agents.

Resolve, fetch, and run any CLI tool in an isolated ephemeral environment — without installing it globally.

Borrowing concepts from pkgx (bottle format, dist server, env composition, companion deps) and exosphere's exo-hydra provisioning pipeline (resolve → install → compose → exec).

Why buckets?

AI agents need ad-hoc runtimes: "run this with Node 20", "test this with Python 3.11", "build with Go 1.22 + CMake". Installing these globally pollutes the host and creates version conflicts.

buckets gives each command its own throwaway runtime — fetched on demand from dist.pkgx.dev, cached locally, cleaned up when done.

Using it from the squad fleet

Any agent with shell access (bro-cli's bash tool, a dispatched Claude Code horse, any runner) can call buckets directly once it's on PATH — it's a plain CLI, no MCP/tool registration needed. For concurrent fleet agents, prefer bucket-bridge (jokersquad/bin/bucket-bridge, symlinked to ~/.local/bin/) over calling buckets directly: it's a transparent wrapper that points every agent at one shared BUCKETS_CACHE_DIR (~/.cache/squadron-buckets) instead of each agent maintaining its own cache, so N agents resolving the same spec (node@20) don't each redundantly download the same bottle. Safe under concurrency — see install.rs's atomic-rename install.

Usage

# Run one-off commands
buckets run node@20 -- script.js --flag arg
buckets run python@3.11 -- python3 -c "print('hello from a bucket')"
buckets run rust@latest -- rustc --version

# Multi-package environments
buckets run node@20 python@3.11 -- node -e "console.log('and python is at', process.env.PATH)"
buckets run go@1.22 cmake@latest -- go version

# Language registries (BUCKETS-15) — install into the shared cellar
buckets run pypi:six@1.16 -- python3 -c "import six; print(six.__version__)"
buckets run npm:is-number@7 -- node -e "console.log(require('is-number')(3))"
buckets env pypi:requests@2.31 npm:is-number@7   # PYTHONPATH + NODE_PATH

# Open an interactive shell with the runtime in PATH
buckets shell node@20
buckets shell python@3.11 --shell /bin/zsh

# Print the composed environment as shell exports (composable)
eval "$(buckets env node@20)"
buckets env node@20 python@3.11 --json

# See what a spec resolves to (no installation)
buckets info node@20
buckets info git@latest

# List cached installations
buckets list

# Reclaim disk space — the cache only grows by default, this is how you
# prune it back. A bare `buckets clean` with no target is refused (the
# cache dir is often shared across concurrent agents); pass a project,
# --all, or both --all and --older-than.
buckets clean node@18                    # remove one version
buckets clean node                       # remove every version of a project
buckets clean --all --dry-run            # preview a full wipe first
buckets clean --all --older-than 30d     # remove entries not touched in 30 days

# Persistent sessions — an OverlayFS mount shared across multiple `exec`
# calls, so files written by one command survive into the next (unlike
# `run`, which is a fresh sandbox every time).
buckets session start node@20
buckets session exec <session-id> -- node another-script.js
buckets session list
buckets session stop <session-id> --purge

# Isolated virtual networks — buckets in the same net share a private
# loopback (127.0.0.1) with no host network access, and a port can be
# forwarded in from the host.
buckets net create dev-net
buckets net run dev-net node@20 -- node server.js
buckets net expose dev-net 8080:3000
buckets net rm dev-net

# Herds — a fleet of bucket replicas with restart/scale/health-reconcile,
# each replica getting HERD_NAME/HERD_REPLICA_INDEX env vars to self-identify.
# `deploy` owns the processes and serves a Unix control socket so `scale` /
# `status` / `stop` from another CLI process can talk to it live.
buckets herd deploy worker --spec node@20 --replicas 5 --net herd-net -- node worker.js
buckets herd status worker
buckets herd scale worker --replicas 8
buckets herd stop worker

# Build/test/run a real project — clone (git URL) or use (local path),
# detect the build system, resolve the toolchain it needs, build inside
# a sandboxed bucket. Doesn't touch the host filesystem outside the
# project dir + the resolved toolchain's own cache.
buckets build /path/to/repo
buckets build https://github.com/owner/repo --test
buckets build . --test --run

# Ephemeral worktrees — a task gets its own working copy at a fresh
# branch (git worktree add, not a full clone — cheap, shares the repo's
# object store). Build/test it like any other local path. "Destroyed
# once you merge": removing an unmerged worktree's branch is refused by
# git itself (git branch -d's own safety check) unless --force.
buckets worktree create /path/to/repo my-task-branch
buckets build "$(buckets worktree create /path/to/repo my-task-branch)" --test
buckets worktree remove /path/to/repo /path/to/repo-my-task-branch my-task-branch
buckets worktree list /path/to/repo

# GUI buckets — run a GUI app against a fresh, isolated Xvfb X server
# (not the host's real display). Only the throwaway server's own socket
# + a session-scoped Xauthority cookie are visible inside the sandbox.
buckets gui --screenshot /tmp/out.png --timeout 5 -- glxgears
buckets gui node@20 --width 1280 --height 800 -- node gui-script.js

# Site buckets — open a URL with a real, OS-enforced per-origin storage
# sandbox (revives the intent behind exosphere-apps' site-capsulizer,
# which was found unenforced scaffolding — see the README's "Site
# buckets" section below). Defaults to a headless browser binary
# (surfer) on PATH; --gui runs a windowed one (super-surfer) in a
# fresh Xvfb.
buckets site https://example.com
buckets site https://example.com --gui --screenshot /tmp/site.png --timeout 5
buckets site https://example.com --incognito   # ephemeral storage, removed on exit

Admission-controlled builds (`--sched`)

buckets build <dir> --sched routes a cargo build through buildsched's admission-controlled pipeline (bob): per-crate jobs gated on memory watermarks, PSI pressure, and disk headroom instead of one monolithic cargo build. Opt-in at compile time — the buildsched cargo feature (OFF by default, cargo build --features buildsched) adds a path dep on the sibling buildsched repo; the default build stays fully self-contained. Cargo projects with a single [package] only: virtual workspaces and non-cargo projects print a one-line notice and fall back to the normal build path. Sched builds always run unsandboxed (the pipeline spawns its own cargo subprocess tree, which can't live inside the per-command bwrap wrap); the bucket toolchain env is still injected into every pipeline subprocess.

Real process isolation

run/shell/build (and gui, site, session, Bucketfile RUN) all execute under bubblewrap (bwrap). Only the resolved toolchain's own install dirs (read-only), the base OS dirs (/usr, /bin, /lib*, read-only) and the invocation/project directory (read-write) are visible inside, plus a private /tmp tmpfs. Network is off by default for run/shell and on for build/Bucketfile RUN/site (package registries and browsers need it).

Fail closed

If bwrap is not installed, buckets refuses to run and says how to fix it. Nothing silently downgrades. The two weaker modes are explicit:

flag	backend	what you lose
(default)	bwrap	—
`--allow-weaker-sandbox`	proot, only if bwrap is missing	network, PID and IPC namespaces are the host's; no seccomp. A warning is printed. cgroup limits and `--timeout` still apply
`--no-sandbox`	plain subprocess	all isolation. The environment is still clean (unless `--inherit-env`), cgroup limits and `--timeout` still apply, seccomp does not

A bwrap too old for a required flag (--clearenv, --die-with-parent, --new-session, --unshare-ipc, --unshare-uts, --seccomp) is also a refusal, not a downgrade. Optional flags (--cap-drop, --disable-userns, --unshare-cgroup-try, --hostname) are used when the installed bwrap has them (probed once from bwrap --help).

What is isolated (bwrap backend)

Namespaces: mount, PID, IPC, UTS (hostname bucket), cgroup, network (unless allowed / --net), and a user namespace with nested user namespaces disabled (--disable-userns).
Process hygiene: --die-with-parent (killing buckets kills the sandbox), --new-session (no TIOCSTI injection into your terminal; shell keeps your session for job control), --cap-drop ALL (the sandboxed process has empty capability sets).
Environment: --clearenv, then only the composed toolchain env (PATH, LD_LIBRARY_PATH, PYTHONPATH, NODE_PATH, ...), HOME=/tmp, and TERM, COLORTERM, NO_COLOR, LANG, LANGUAGE, LC_*, TZ from the host. Tokens and other host variables do not reach the sandbox (the bwrap process itself also starts with an empty environment). --inherit-env restores the old pass-everything behaviour.
seccomp (--seccomp off|default|strict, default default): a BPF program built with zandbox and handed to bwrap on an inherited memfd (--seccomp FD). bwrap installs it just before exec'ing your command.
- default: zandbox's Application profile (files, processes, threads, signals, timers, sockets) plus a compatibility list, minus unshare. Anything else fails with EPERM: ptrace, mount, setns, bpf, perf_event_open, keyctl, module loading, io_uring_* (libuv and cargo fall back), ... Measured with strace -f to cover python3 (threads, subprocess, multiprocessing, venv, pip), node (child_process, worker_threads), bash, uv, npm, git and cargo build.
- strict: zandbox's Default profile (no sockets); violations kill the process with SIGSYS. Anything that opens a socket dies, including NSS lookups (ls -l, id).
- gui and site run with seccomp off (GUI toolkits, and browsers that bring their own sandbox).

Resource limits

buckets run --memory 512M --pids 256 --cpu 50% --timeout 30s python@3.12 -- python job.py

--timeout kills the whole sandbox when it expires (exit status 124). Works without a cgroup.
--memory (memory.max, with memory.swap.max=0), --pids (pids.max; bwrap itself uses 2) and --cpu (cpu.max, percent of one CPU) are cgroup v2 limits on the whole sandbox, bwrap included. zandbox creates a per-run cgroup and the bwrap process joins it in pre_exec, before exec, so there is no attach race. They need a delegated cgroup v2 directory: --cgroup-parent DIR or $BUCKETS_CGROUP_PARENT. Without one, the run is refused. With systemd:
```
systemd-run --user --scope -p Delegate=yes bash -c '
  CG=/sys/fs/cgroup$(sed -n "s/^0:://p" /proc/self/cgroup)
  mkdir $CG/leaf $CG/sandboxes
  echo 0 > $CG/leaf/cgroup.procs           # move this shell out of the scope root
  echo "+pids +memory +cpu" > $CG/cgroup.subtree_control
  export BUCKETS_CGROUP_PARENT=$CG/sandboxes
  buckets run --memory 512M --pids 64 python@3.12 -- python job.py'
```

The sandbox flags (--allow-weaker-sandbox, --inherit-env, --seccomp, --memory, --pids, --cpu, --timeout, --cgroup-parent) exist on run, net run and shell.

Library API

sandbox::try_sandboxed_command(program, args, cwd, env, &profile) returns Result<Sandbox, SandboxError>. Sandbox::status() / spawn() + SandboxChild::wait() honour the timeout; into_command() gives the bare std::process::Command. SandboxProfile gained binds, fallback, env_policy, seccomp, limits, keep_terminal_session, share_ipc and allow_nested_userns, all defaulting to the strict choice, so SandboxProfile { .., ..Default::default() } literals keep compiling. The old infallible sandboxed_command still exists: when the sandbox can't be built it returns a command that runs nothing and exits 126 with the reason on stderr.

Real GUI isolation

buckets gui runs a GUI command against a brand-new Xvfb X server, not the host's real :0. Borrows the concept from x11docker — a session-scoped MIT-MAGIC-COOKIE Xauthority cookie, generated fresh per session and bound into the sandbox alongside exactly ONE file (the specific /tmp/.X11-unix/X<N> socket, never the whole directory) plus DISPLAY/XAUTHORITY env vars. Deliberately a nested server, not a --hostdisplay-style reuse of the real session — X11 has no native per-client window isolation, so a fresh Xvfb instance means nothing on the real display is ever exposed. Verified live: a client with the wrong/missing XAUTHORITY is refused by the X server ("Authorization required, but no authorization protocol specified"); the right cookie succeeds. Session cleanup (Xvfb process, socket, cookie file) happens on drop regardless of how the command exited.

vs. squadron/bin/display-up: a lighter-weight sibling tool exists in squadron for a different shape of problem — attaching automation (xdotool/import, or interactd's computer_use/vision_start/ atspi_start) to an app you're launching yourself rather than sandboxing a command for you. display-up allocates a per-agent Xvfb display number (flock-serialized, idempotent, keyed by agent identity) but does NOT provide buckets gui's Xauthority-gated/bwrap-sandboxed containment — any process that guesses the right :N can still connect. Use buckets gui/buckets site --gui when you want the command sandboxed too (real OS-level containment); use display-up when you just need a private display surface to avoid colliding with another agent's session on the shared host :0 (the incident that motivated it: blind xdotool automation during surfer-browser's SHELL-SPLIT-3 spammed input into another agent's window). The two don't conflict — buckets gui's XvfbSession could reuse display-up's allocation logic later if the two ever want to converge, but that's not needed today.

Site buckets

buckets site <url> runs a browser against a URL with a real, OS-enforced per-origin storage sandbox — reviving the intent behind exosphere-apps' exo-site-capsulizer (storage-capsule/net-capsule/worker-capsule), which was found to be ~95% unenforced scaffolding (check_request() never called, storage VFS dead code, workers never started) and replaced in surfer-browser with a shim explicitly documented as "not an enforcement layer" — real privacy there is a JS fetch/XHR interceptor, still in-process, still not real OS isolation.

buckets site gets the enforcement for free from the sandboxing this project already has: each host gets its own read-write bind ($BUCKETS_CACHE_DIR/sites/<host>/, persistent — real browsing-profile semantics — or --incognito for a tempdir removed on exit) and nothing else on the host is visible inside the sandbox. Headless by default (surfer on PATH); --gui runs a windowed browser (super-surfer) inside a fresh Xvfb, same mechanism as buckets gui. Neither browser binary is built by buckets itself — build it once in surfer-browser (cargo build --release -p surfer --features cli for headless, -p super-surfer --features bliss for GUI), then point buckets site at it (PATH lookup by default, or --browser-bin <path>).

Not built: per-domain/third-party network filtering (still just the existing coarse network on/off toggle) and building the browser binary itself (a separate, one-time cargo build, not part of this command).

Spec format

<tool>[@<version>]

Spec	Meaning
`node@20`	Node.js 20.x (latest in `^20` range)
`python@=3.11.0`	Exact Python 3.11.0
`rust@latest`	Latest stable Rust
`node`	Latest (same as `@latest`)
`go@^1.22`	Standard caret semver
`go@>=1.22`	Greater-or-equal
`rust@~1.70`	Tilde (patch-level) semver

Featured aliases (60+ tools)

Alias	Resolves to
`node`	`nodejs.org`
`python`	`python.org`
`rust`	`rust-lang.org`
`go`	`golang.org`
`git`	`git-scm.com`
`cmake`	`cmake.org`
`ripgrep` / `rg`	`BurntSushi/ripgrep`
`curl`	`curl.se`
`gh`	`github.com/cli`
`docker`	`docker.com`
`kubectl`	`kubernetes.io/kubectl`
`terraform`	`terraform.io`
`aws`	`amazon.com/aws-cli`
`neovim`	`neovim.io`
`tmux`	`tmux.github.io`
... and 45+ more	See `src/index.rs`

How it works

Parse spec → project + semver constraint
Resolve alias → full pkgx project name
Collect companions �� auto-include deps (e.g. openssl for curl, cmake)
Resolve versions → check cache first, then remote versions.txt
Install → download .tar.xz bottle, XZ-decompress, extract, atomic rename
Symlink → create v*, v<major>, v<major.minor> → latest version
Compose env → PATH, LD_LIBRARY_PATH, CPATH from installation dirs
Run or export the environment

Commands

Command	Description
`run <specs> -- <cmd>`	Resolve, install, and exec a command
`shell <specs>`	Open an interactive shell with the runtime
`env <specs>`	Print shell exports (`--json` for structured output)
`info <specs>`	Show resolution without installing
`list`	Show cached installations
`clean <project>[@version]`	Remove one cached project or version
`clean --all [--older-than <30d\|12h\|2w>] [--dry-run]`	Bulk-remove cached installations to reclaim disk space
`build <path-or-url> [--test] [--run]`	Detect + build (+ test/run) a real project, sandboxed
`worktree create <repo> <branch> [--from <base>]`	Create an ephemeral worktree (prints its path)
`worktree remove <repo> <path> <branch> [--force]`	Remove a worktree + its branch (git refuses if unmerged, unless --force)
`worktree list <repo>`	List existing worktrees
`gui [specs] -- <cmd> [--screenshot <path>] [--timeout <secs>] [--width <N>] [--height <N>]`	Run a GUI command in a sandboxed bucket against a fresh Xvfb X server
`site <url> [--browser-bin <path>] [--gui] [--incognito] [--screenshot <path>] [--timeout <secs>]`	Open a URL with a real per-origin storage sandbox
`session start <specs> [--tmpfs\|--zram] [--size <N>] [-- <cmd>]`	Start a persistent OverlayFS session (files survive across `exec` calls)
`session exec <session-id> -- <cmd>`	Run a command inside an existing session
`session stop <session-id> [--purge]`	Stop a session (`--purge` also removes its overlay state)
`session list [--json]`	List active sessions
`net create <name>` / `net rm <name>` / `net ls`	Create/remove/list named virtual networks
`net run <name> <specs> -- <cmd>`	Run a bucket joined to a named network
`net expose <name> <host_port>:<bucket_port>`	Forward a host port into a virtual network
`herd deploy <name> --spec <spec> [-n <replicas>] [--net <name>] [--restart <policy>] -- <cmd>`	Deploy a fleet of bucket replicas
`herd ls` / `herd status <name>`	List herds / show per-replica status
`herd scale <name> -n <replicas>`	Live hot-scale via the deploy process's Unix control socket
`herd stop <name>`	Stop via control socket (clean deploy shutdown); falls back to SIGTERM from state.json

Configuration

Env var	Default	Description
`BUCKETS_DIST_URL`	`https://dist.pkgx.dev`	Distribution server URL
`BUCKETS_CACHE_DIR`	`~/.cache/buckets/` or `~/.buckets/`	Local cache directory
`BUCKETS_WORKTREE_DIR`	sibling of the source repo	Parent directory for `worktree create` (see below)

BUCKETS_WORKTREE_DIR defaults to creating each worktree as a sibling of its source repo (/path/to/repo-my-branch next to /path/to/repo), not a fixed cache location — required for relative sibling path-dependencies (../other-repo) to keep resolving correctly from inside the worktree.

`--flare` — zram-backed cache instead of host disk

sudo flare-up --agent myagent --quiet   # once per session (needs root)
buckets --flare run node@20 -- node script.js

--flare points BUCKETS_CACHE_DIR at an already-provisioned zram-backed ext4 mount (squadron/bin/flare-up) instead of host disk — same resolve/install/sandbox pipeline, just backed by compressed RAM that evaporates on reboot instead of accumulating on an SSD. Deliberately does not provision a session itself: flare-up needs root and is meant to be session-scoped (run once per agent/session, reused across many buckets invocations), not re-provisioned per command. --flare errors clearly if no session is live rather than silently falling back to host disk — run sudo flare-up first. See /workspace/projects/FLARE_FIREFLY_DESIGN.md for the full design and why this doesn't repeat the earlier zram-for-inference performance problems.

Features borrowed from pkgx

Bottle format: .tar.xz from dist.pkgx.dev/<platform>/<arch>/<project>/v<version>.tar.xz
Cache layout: ~/.buckets/<project>/v<version>/bin/...
Symlink version scheme: v* → latest, v20 → latest v20.x.x
Companion packages: auto-included deps (openssl for curl, etc.)
Env composition: scan bin/, lib/, include/, share/, man/ per installation
Shell exports format: export PATH="/path/to/bucket/bin:$PATH"
JSON output: --json for structured consumption
Multi-package: buckets run node@20 python@3.11 -- node -e "..."

Installation

cargo install --path .
# or, from crates.io (published as `crush-buckets`, binary is still `buckets`)
cargo install crush-buckets

Claude Code plugin

This repo doubles as its own Claude Code plugin marketplace — install it directly from GitHub, no separate marketplace repo needed:

/plugin marketplace add nixpt/buckets
/plugin install buckets@buckets

The plugin ships a buckets-usage skill that teaches a model when to reach for a throwaway bucket instead of a global install, plus the core command reference. It does not vendor a compiled buckets binary (a public OSS repo shouldn't ship arch-specific blobs) — install buckets itself via one of the two commands above first.

Acknowledgments

buckets is a clean-room reimplementation, not a fork — no source from the projects below is copied here, only the formats/behaviors listed under "Features borrowed from pkgx" above, plus the general resolve→install→compose→exec pipeline shape.

pkgx (Max Howell, Jacob Heider; Copyright 2022–23 pkgx inc.; Apache-2.0) — the bottle format, dist.pkgx.dev distribution protocol, cache/symlink layout, companion-package resolution, and env-composition approach are all pkgx's design, reimplemented here.
exosphere's exo-hydra and exo-light's exo-hydra — the same pkgx-derived provisioning pipeline shape (resolve → install → compose → exec), adapted for this project's standalone/sync use case. See /workspace/projects/CLAUDE.md's provisioning-lineage note for how the three relate.

Real-world adoption note (2026-07-11): of the three pkgx-derived implementations above, buckets is currently the only one with real downstream consumers — interactd's BucketsTool (shells out to the buckets CLI) and flame's flame-core::kindle (a real compile-time Rust dependency, buckets = { path = "../../../buckets", optional = true }, one of flame's three "fuel" backends alongside local paths and language registries — see flame's own README: "Buckets remains the fleet tool-bottle CLI; Flame is the workspace OS"). Both exo-hydra copies remain fully built but honestly self-documented as unwired (their own READMEs say so) — resolve/install/manifest pipelines with no consumer calling into them yet.

License

Dual-licensed under MIT OR Apache-2.0, your choice — see LICENSE-MIT and LICENSE-APACHE (the standard convention across the Rust ecosystem). (Acknowledgments above are for design/format credit, not a licensing obligation — no pkgx source is redistributed.)

Name		Name	Last commit message	Last commit date
.claude-plugin		.claude-plugin
.dejavue		.dejavue
.jagent		.jagent
assets		assets
skills/buckets-usage		skills/buckets-usage
src		src
.gitattributes		.gitattributes
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE-APACHE		LICENSE-APACHE
LICENSE-MIT		LICENSE-MIT
README.md		README.md
validate_android.sh		validate_android.sh

Navigation Menu

Repository navigation

About

Repository files navigation

buckets

Why buckets?

Using it from the squad fleet

Usage

Admission-controlled builds (`--sched`)

Real process isolation

Fail closed

What is isolated (bwrap backend)

Resource limits

Library API

Real GUI isolation

Site buckets

Spec format

Featured aliases (60+ tools)

How it works

Commands

Configuration

`--flare` — zram-backed cache instead of host disk

Features borrowed from pkgx

Installation

Claude Code plugin

Acknowledgments

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

buckets

Why buckets?

Using it from the squad fleet

Usage

Admission-controlled builds (--sched)

Real process isolation

Fail closed

What is isolated (bwrap backend)

Resource limits

Library API

Real GUI isolation

Site buckets

Spec format

Featured aliases (60+ tools)

How it works

Commands

Configuration

--flare — zram-backed cache instead of host disk

Features borrowed from pkgx

Installation

Claude Code plugin

Acknowledgments

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Admission-controlled builds (`--sched`)

`--flare` — zram-backed cache instead of host disk