Contact: best bet to reach us is the #u-root-dev channel on the Open Source Firmware Slack (Sign Up Link).
Go Busybox is a set of Go tools that allow you to compile many Go commands into
one binary. The resulting binary uses its invocation arguments (os.Args) to
determine which command is being called.
| Feature | Support status |
|---|---|
| Go version | 1.27 and later (see src/go.mod) |
| Packaging | Go workspaces, Go modules, Go vendoring |
| Source | A checkout, or the module proxy (see gomodbb) |
GOOS |
any (linux is tested) |
GOARCH |
any (amd64, arm, arm64, riscv64 are tested) |
| CGO | Not supported |
An example:
go install github.com/u-root/gobusybox/src/cmd/makebb@latest
git clone https://github.com/u-root/u-root
cd u-root
makebb ./cmds/core/dmesg ./cmds/core/straceA binary named bb should appear. It can be invoked in one of two ways --
either with a symlink or using a second argument.
./bb dmesg
./bb strace echo "hi"It is meant to be used with symlinks for convenience:
# Make a symlink dmesg -> bb
ln -s bb dmesg
# Symlink means that argv[0] is the command name.
./dmesg
# Make a symlink strace -> bb
ln -s bb strace
./strace echo "hi"Go Busybox does this by copying all the source for these Go commands and rewriting it in a temporary directory.
Go Busybox can be used with any Go commands across multiple Go modules:
mkdir workspace
cd workspace
git clone https://github.com/hugelgupf/p9
git clone https://github.com/u-root/cpu
go work init ./p9
go work use ./cpu
makebb ./cpu/cmds/* ./p9/cmd/*Important
makebb works any time go build or go list also work.
For multi-module compilation, use Go workspaces or read below.
makebb builds commands that are already on disk, so every input has to be
cloned first, and commands from several projects have to be arranged into a Go
workspace before any of them can be named.
gomodbb skips both steps. The go command already knows how to fetch a module
and hand back its source, so gomodbb asks it to, and builds from the copy in
the module cache:
go install github.com/u-root/gobusybox/src/cmd/gomodbb@latest
gomodbb github.com/u-root/u-root/cmds/core/...@mainNo git clone, and no go work init even across projects:
gomodbb -o bb \
github.com/u-root/u-root/cmds/core/...@main \
github.com/u-root/cpu/cmds/...@main \
github.com/hugelgupf/p9/cmd/...@latestNothing there names a project gomodbb knows about. A pattern is handed to
go get, which decides which module provides it, so any module the go command
can fetch works.
Two wildcards, meaning what they mean to makebb:
| Pattern | Matches |
|---|---|
cmds/core/... |
every command below cmds/core, however deep |
cmds/core/* |
commands one level below cmds/core, and no deeper |
... is the go command's own wildcard and descends without limit, which picks
up more than a busybox usually wants. In u-root, cmds/... matches 193
packages, of which the four deepest are two go:generate helpers and two test
fixtures, not commands anybody means to build:
cmds/core/man/gen
cmds/exp/cbmem/gen
cmds/exp/cmd2pkg/findpkg/test/goglob/echo
cmds/exp/cmd2pkg/findpkg/test/goglob/foo
* stops at a path separator, so naming each level you want avoids them:
gomodbb -o bb \
github.com/u-root/u-root/cmds/core/*@main \
github.com/u-root/u-root/cmds/exp/*@mainThe go command has no *, so a globbed pattern is fetched as the ... that
contains it and matched back down afterwards. Quote the pattern if your shell
would otherwise expand it against the local directory.
A busybox reaches each command by the base name of its directory, so two
packages with the same base name cannot both be in one binary — the two gen
helpers above collide even though their import paths differ. Leave one out by
naming it with a leading -:
gomodbb github.com/u-root/u-root/cmds/...@main \
-github.com/u-root/u-root/cmds/exp/cbmem/gen-l lists what a build would contain and reports these collisions without
building, which is the cheap way to find them.
Packages and commands must be available in proxy.golang.org:
gomodbb sets GOPROXY to https://proxy.golang.org not https://proxy.golang.org,direct
as the standard go command sets it, so there is no fallback to git or some
other VCS. Pinning GOPROXY is not enough on its own, because GOPRIVATE and
GONOPROXY name paths that bypass the proxy and are fetched with git, so
gomodbb clears both and sets GOVCS=*:off, which turns any remaining attempt
to use a VCS into an error that says so. This i
8000
s helpful on systems like
Plan 9, which do not support the standard git command.
Everything after the last @ is a version query, and it means what it means to
go get:
| Query | Selects |
|---|---|
| (omitted) | The highest release tag, as go get would |
@main |
The tip of that branch |
@v0.16.0 |
That tag |
@98a84dad0e9c |
That revision |
Caution
@HEAD is not the tip of the default branch. It is a version query like any
other, and resolves through the proxy to whatever it last recorded under that
name, which can be months old. Use @main, or whatever the branch is called.
Omitting the query is the reproducible choice and the surprising one: for a
project that tags rarely, the newest release can be far behind the branch
git clone would have given you.
-l prints the commands a pattern matches and builds nothing. It applies
exclusions, so the list is exactly what a build with the same arguments would
contain, and it prints to stdout with everything else on stderr:
gomodbb -l github.com/u-root/u-root/cmds/core/...@main | wc -l-mod-dir keeps the module gomodbb writes to do the fetching, whose go.mod
records the version every pattern resolved to:
gomodbb -mod-dir ./pins github.com/u-root/u-root/cmds/core/...@main
grep 'u-root/u-root ' ./pins/go.mod
# github.com/u-root/u-root v0.16.1-0.20261010115301-98a84dad0e9c // indirectPass that directory again to build from the same versions. gomodbb reuses an
existing go.mod rather than overwriting it.
makebb and the APIs mentioned below all accept commands from file system
paths, Go package paths, and globs matching path.Match thereof.
In addition to the standard syntaxes supported by go list and go build,
makebb also accepts globs and exclusions.
git clone https://github.com/u-root/u-root
cd u-root
makebb ./cmds/core/ip ./cmds/core/init
# Escaping the * to show that the Go APIs know how to resolve this as well
makebb ./cmds/core/\*
makebb ./cmds/core/i\* ./cmds/boot/pxeboot
# All core commands except ip.
makebb ./cmds/core/\* -./cmds/core/ipTo compile commands from multiple modules, you may use workspaces.
mkdir workspace
cd workspace
git clone https://github.com/u-root/u-root
git clone https://github.com/u-root/cpu
go work init ./u-root
go work use ./cpu
makebb \
./u-root/cmds/core/init \
./u-root/cmds/core/gosh \
./cpu/cmds/cpud
# Also works for offline builds with `go work vendor` (Go 1.22 feature):
go work vendor
makebb \
./u-root/cmds/core/init \
./u-root/cmds/core/gosh \
./cpu/cmds/cpudFor a shortcut to specify many commands sharing common path elements (e.g. from
the same repository), the GBB_PATH environment variable exists. Paths are
concatenated with every colon-separated element of GBB_PATH from left to right
and checked for existence.
GBB_PATH=$(pwd)/u-root:$(pwd)/cpu makebb \
cmds/core/init \
cmds/core/gosh \
cmds/cpud
# matches:
# $(pwd)/u-root/cmds/core/init
# $(pwd)/u-root/cmds/core/gosh
# $(pwd)/cpu/cmds/cpudgo install github.com/u-root/gobusybox/src/cmd/goanywhere@latestgoanywhere creates a Go workspace temporarily on the fly from the packages'
modules in the given local file paths.
goanywhere then executes the command given after "--" in the workspace directory and amends the packages as args.
For example,
# -o $(pwd) is needed since goanywhere executes the command in the workspace
# directory, and by default the Go binary is created in the current wd.
goanywhere ./u-root/cmds/core/{init,gosh} -- go build -o $(pwd)
goanywhere ./u-root/cmds/core/{init,gosh} -- makebb -o $(pwd)
goanywhere ./u-root/cmds/core/{init,gosh} -- mkuimage [other args]
goanywhere ./u-root/cmds/core/{init,gosh} -- u-root [other args]
Like makebb, goanywhere supports GBB_PATH, globs and shell expansions.
GBB_PATH=$(pwd)/u-root:$(pwd)/cpu goanywhere \
cmds/core/{init,gosh} cmds/cpud -- makebb -o $(pwd)
makebb supports Go workspaces for locally checked out sources, as shown above.
For multiple Go module command dependencies that aren't checked out locally, we apply some Go module tricks.
To depend on commands outside of ones own repository, the easiest way to depend on Go commands is the following:
TMPDIR=$(mktemp -d)
cd $TMPDIR
go mod init foobarCreate a file with some unused build tag like this to create dependencies on commands:
//go:build tools
package something
import (
_ "github.com/u-root/u-root/cmds/core/ip"
_ "github.com/u-root/u-root/cmds/core/init"
_ "github.com/u-root/cpu/cmds/cpud"
)You can generate this file for your repo with the gencmddeps tool:
go install github.com/u-root/gobusybox/src/cmd/gencmddeps@latest
gencmddeps -o deps.go -t tools -p something \
github.com/u-root/u-root/cmds/core/{ip,init} \
github.com/u-root/cpu/cmds/cpud
Important
gencmddeps does not support file paths or exclusions, as these rely on a
go.mod already being present to resolve the Go package name.
The input to gencmddeps must be full Go package paths.
The unused build tag keeps it from being compiled, but its existence forces go mod to add these dependencies to go.mod:
go mod tidy
makebb \
github.com/u-root/u-root/cmds/core/ip \
github.com/u-root/u-root/cmds/core/init \
github.com/u-root/cpu/cmds/cpud
# Also works with vendored, offline builds:
go mod vendor
makebb \
github.com/u-root/u-root/cmds/core/ip \
github.com/u-root/u-root/cmds/core/init \
github.com/u-root/cpu/cmds/cpudBesides the makebb CLI command, there is a Go API at src/pkg/bb.
-
Any imported packages'
initfunctions are run for every command.For example, if some command imports and uses the
testingpackage, all commands in the busybox will have testing's flags registered as a side effect, becausetesting's init function runs with every command.While Go busybox handles every main commands' init functions, it does not handle dependencies' init functions. Done properly, it would rewrite all non-standard-library packages as well as commands. This has not been necessary to implement so far. It would likely be necessary if, for example, two different imported packages register the same flag unconditionally globally.
src/pkg/bb implements a Go source-to-source transformation on pure Go code (no cgo).
This AST transformation does the following:
- Takes a Go command's source files and rewrites them into Go package files (almost) without global side effects.
- Writes a
main.gofile with amain()that calls into the appropriate Go command package based onargv[0]orargv[1].
This allows you to take two Go commands, such as Go implementations of dmesg
and strace and compile them into one binary.
Which command is invoked is determined by argv[0] or argv[1] if argv[0] is
not recognized. Let's say bb is the compiled binary; the following are
equivalent invocations of dmesg and strace:
(cd ./src/cmd/makebb && go install)
(cd ./test/nested && makebb ./cmd/dmesg ./cmd/strace)
# Make a symlink dmesg -> bb
ln -s bb dmesg
./dmesg
# Make a symlink strace -> bb
ln -s bb strace
./strace echo "hi"./bb dmesg
./bb strace echo "hi"Principally, the AST transformation moves all global side-effects into callable
package functions. E.g. main becomes registeredMain, each init becomes a
numbered busyboxInitN, and each global variable assignment is moved into its
own busyboxInitN. busyboxInit0 calls the variable initializers in the order
go/types computed, and a registeredInit calls busyboxInit0 followed by
the bodies of the original init functions.
Then, these registeredMain and registeredInit functions can be registered
with a global map of commands by name and used when called upon.
Let's say a command github.com/org/repo/cmds/sl contains the following
main.go:
package main
import (
"flag"
"log"
)
var name = flag.String("name", "", "Gimme name")
func init() {
log.Printf("init %s", *name)
}
func main() {
log.Printf("train")
}This would be rewritten to be:
package bbsl // "bb" + the command name
import (
"flag"
"log"
bbmain "bb.u-root.com/bb/pkg/bbmain" // generated import path
)
// Type has to be inferred through type checking.
var name *string
func busyboxInit2() {
log.Printf("init %s", *name)
}
func registeredMain() {
log.Printf("train")
}
func busyboxInit1() {
name = flag.String("name", "", "Gimme name")
}
func busyboxInit0() {
// Order is determined by go/types.Info.InitOrder.
busyboxInit1()
}
func registeredInit() {
busyboxInit0()
busyboxInit2()
}
func init() {
bbmain.Register("sl", regist
9AE6
eredInit, registeredMain)
}By default only commands are rewritten, and the packages they import keep
their init and their initialised package-level variables, which the go tool
runs at program start for every command in the busybox -- whether or not the
command being dispatched imports them. In the 120-command u-root busybox, of
228 library packages, none is reached by every command and 125 are reached
by exactly one.
-lazy-init selects a library whose initialization moves to the commands that
import it. It is off by default and takes whole import paths, repeated:
makebb -lazy-init github.com/org/repo/pkg/table ./cmds/sl ./cmds/cowsay
A trailing /... reaches a whole subtree, as it does for the go command, and
covers the subtree's root as well as everything under it:
makebb -lazy-init github.com/org/repo/pkg/... ./cmds/sl ./cmds/cowsay
The two forms differ in what a package that cannot be deferred does to the build. An exact path is a claim about one package, so a build that cannot keep it fails, naming the package and the reason. A pattern is a request for whatever the subtree can give, so it defers what it can and reports the rest, failing only if it moved nothing at all. Either way the build never quietly defers less than it was asked to.
A selected package is rewritten like a command -- its package-level variables
and its init bodies become numbered busyboxInitN functions -- except that
it keeps its own package name and import path, and gains an exported hook in a
generated bbinit.go:
// Code generated by gobusybox for github.com/org/repo/pkg/table. DO NOT EDIT.
package table
var busyboxInitDone bool
func BusyboxInit() {
if busyboxInitDone {
return
}
busyboxInitDone = true
bbdep0.BusyboxInit() // this package's direct imports, if they too are deferred
busyboxInit0() // this package's hoisted variables
busyboxInit1() // the body of its original init
}Each rewritten command gets a bbinit.go of its own, calling the hooks of its
own direct imports; the recursion reaches the rest. The guard makes a package
reached by two paths initialize once, as Go initializes it once.
Three rules make this safe, and all three are enforced rather than documented:
- A package is deferred entirely or not at all. Go finishes an imported
package's initialization in full before the importing package's variables
run, so leaving
var x = table.Lookup()eager while deferringtablewould read a table that had not been built. - Naming a package defers everything that imports it. An eager package initializes before any deferred one, so it may not import one.
- Laziness is refused for a package carrying
//go:linkname, an assembly file or a.sysoobject, and for the target of a two-argument//go:linkname, because each can reach a package by a path the import graph does not have. It is refused too for a package whose hoisted variables would have a type that cannot be named outside the package defining it, which no generated declaration can spell. The standard library is never rewritten, and a command is never refused: its initialization already waits for its own dispatch. Naming a package that cannot be deferred fails the build rather than being ignored.
See ADR 0004 for what this recovers and what it does not.
The main.go file is generated from ./src/pkg/bb/bbmain/cmd/main.go.
package main
import (
"os"
"log"
"path/filepath"
// Side-effect import so init in sl calls bbmain.Register
_ "github.com/org/repo/cmds/sl"
"bb.u-root.com/bb/pkg/bbmain"
)
func main() {
bbmain.Run(filepath.Base(os.Argv[0]))
}All files are written into a temporary directory. All dependency Go packages are also written there.
The generated tree is a single, self-contained Go module named
bb.u-root.com/bb. The generated main.go sits at the module root, and every
rewritten command and every non-standard-library dependency is written into
vendor/ at its original import path, so that the rewritten sources compile
without any of their import statements being touched.
Whether the original commands come from a single Go module or several joined by
a Go workspace, we generate the same structure and compile it with GOWORK=off go build -mod=vendor. Because everything is vendored, -mod=vendor consults
neither the network nor the module cache, so in all cases traditionally offline
compilations remain offline.
Every command must live in a Go module. GOPATH mode (GO111MODULE=off) is not
supported: it is the only mode in which one import path can exist at two
versions at once, each reached through a different nested vendor/ directory,
and a module vendor directory is flat and can hold only one of them. See
docs/adr/0002-gopath-mode-is-not-supported.md.
If your commands span several modules, use goanywhere to put them in a
workspace.
go.mod and vendor/modules.txt are generated together from the same package
list; the go tool rejects a vendor directory whose modules.txt disagrees with
go.mod. A main module has no version of its own, so its packages are recorded
at a synthetic version that -mod=vendor never resolves.
/tmp/bb-$NUM/
├── go.mod << module bb.u-root.com/bb
├── main.go << ./src/pkg/bb/bbmain/cmd/main.go (with edits)
├── pkg
│ └── bbmain
│ └── register.go << ./src/pkg/bb/bbmain/register.go
└── vendor
├── modules.txt << generated manifest
├── github.com
│ └── u-root
│ ├── uio
│ │ ├── uio << dependency used by both
│ │ └── ulog << dependency used by both
│ ├── u-bmc
│ │ ├── cmd
│ │ │ ├── fan << generated command package
│ │ │ ├── login << generated command package
│ │ │ └── socreset << generated command package
│ │ └── pkg
│ │ ├── acme << dependency copied from u-bmc
│ │ ├── aspeed << dependency copied from u-bmc
│ │ ├── gpiowatcher << dependency copied from u-bmc
│ │ └── mtd << dependency copied from u-bmc
│ └── u-root
│ ├── cmds
│ │ └── core
│ │ ├── cat << generated command package
│ │ ├── ip << generated command package
│ │ └── ls << generated command package
│ └── pkg
│ ├── curl << dependency copied from u-root
│ ├── dhclient << dependency copied from u-root
│ ├── ip << dependency copied from u-root
│ ├── ls << dependency copied from u-root
│ └── uio << dependency copied from u-root
└── golang.org
└── x
└── sys
└── unix << dependency copied from golang.org/x/sys