8000 fix(minvmd): do not require libkrun when used as a rust dep by twitchyliquid64 · Pull Request #668 · gominimal/minimal · GitHub
[go: up one dir, main page]
Skip to content

fix(minvmd): do not require libkrun when used as a rust dep #668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
twitchyliquid64 merged 1 commit into from
Jul 8, 2026
Merged

fix(minvmd): do not require libkrun when used as a rust dep #668

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 34 additions & 17 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,19 +229,12 @@ jobs:
- name: Build release minvmd binary
run: cargo build --release -p minvmd --bin minvmd
- name: Point minvmd's libkrun load command at @rpath
# Linking against Homebrew's libkrun bakes its ABSOLUTE install name
# (/opt/homebrew/opt/libkrun/lib/libkrun.1.dylib) into minvmd as the
# load command. dyld resolves an absolute load command directly and
# NEVER consults the rpath, so the shipped bin/libkrun.1.dylib next to
# minvmd is ignored and the binary is unloadable on a host without that
# exact Homebrew keg. Rewrite the load command to `@rpath/…` so dyld
# walks minvmd's rpath instead: `@loader_path` (the adjacent shipped
# dylib) first, then `/opt/homebrew/lib` as a fallback (see the minvmd
# build.rs rpath ordering). Derive the current path from otool rather
# than hardcoding it, so a libkrun version/path bump can't silently
# skip the rewrite. install_name_tool invalidates the ad-hoc signature,
# so re-sign (the installer re-signs on download too, but keep the
# artifact valid in the meantime).
# Homebrew's libkrun bakes an ABSOLUTE install name into minvmd; dyld
# resolves it directly and never consults the rpath, ignoring the
# shipped bin/libkrun.1.dylib. Rewrite it to @rpath so dyld walks
# minvmd's rpath (@loader_path first, /opt/homebrew/lib fallback).
# Derive the path from otool so a libkrun bump can't skip the rewrite.
# install_name_tool breaks the signature, so re-sign.
run: |
current="$(otool -L target/release/minvmd | awk '/libkrun/ {print $1; exit}')"
if [ -z "$current" ]; then
Expand All @@ -250,17 +243,41 @@ jobs:
fi
install_name_tool -change "$current" @rpath/libkrun.1.dylib target/release/minvmd
codesign --sign - --force target/release/minvmd
- name: Verify minvmd loads libkrun via @rpath
# Gate the release on the rewrite above having stuck: an absolute or
# bare load command here would ship an unloadable binary (this bug).
- name: Verify minvmd linkage
# libkrun must resolve via @rpath (rewrite stuck), and every other
# dependency must be a system library (/usr/lib or /System). A stray
# /opt/homebrew dep would fail to load off the build host.
run: |
otool -L target/release/minvmd | grep -i krun
otool -L target/release/minvmd
if ! otool -L target/release/minvmd | grep -q '@rpath/libkrun\.1\.dylib'; then
echo "::error::minvmd does not reference @rpath/libkrun.1.dylib; the shipped dylib will not be found" >&2
exit 1
fi
bad="$(otool -L target/release/minvmd | tail -n +2 | awk '{print $1}' \
| grep -vE '^(/usr/lib/|/System/|@rpath/libkrun\.1\.dylib$)' || true)"
if [ -n "$bad" ]; then
echo "::error::minvmd links unexpected non-system libraries:" >&2
printf '%s\n' "$bad" >&2
exit 1
fi
- name: Build release minimal binary
# Separate cargo invocation from minvmd above, so its default
# `libkrun` feature can't unify into the CLI (which opts out via
# default-features = false). Gated next.
run: cargo build --release -p minimal --bin minimal
- name: Verify minimal links only system libraries
# The CLI calls no libkrun (it spawns minvmd as a subprocess), so
# every dependency must be under /usr/lib or /System. A krun line
# means the default-features = false opt-out regressed.
run: |
otool -L target/release/minimal
bad="$(otool -L target/release/minimal | tail -n +2 | awk '{print $1}' \
| grep -vE '^(/usr/lib/|/System/)' || true)"
if [ -n "$bad" ]; then
echo "::error::minimal links non-system libraries (allowed: /usr/lib, /System):" >&2
printf '%s\n' "$bad" >&2
exit 1
fi
- name: Rename binaries with platform suffix
# download-artifact merge-multiple flattens on basename, so the uploaded
# file must already carry the platform-suffixed name the release job expects.
Expand Down
6 changes: 5 additions & 1 deletion crates/minimal/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,9 @@ tokio.workspace = true
tracing.workspace = true
tracing-subscriber.workspace = true

minvmd = { path = "../minvmd" }
# default-features = false drops minvmd's `libkrun` feature: the CLI only uses
# the client helpers (sock/lifecycle) and calls no krun code, so it must not
# inherit libkrun's `#[link]` load command (would make it unloadable without
# libkrun).
minvmd = { path = "../minvmd", default-features = false }
ot.workspace = true
8 changes: 8 additions & 0 deletions crates/minvmd/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ version = "0.0.1"
edition.workspace = true
publish.workspace = true

[features]
default = ["libkrun"]
# Compile the real VM backend (the `krun` module's `#[link(name = "krun")]` +
# build.rs link/rpath wiring). On by default; the `minimal` CLI opts out with
# default-features = false so libkrun's load command isn't stamped into a binary
# that calls no krun code.
libkrun = []

[dependencies]
anyhow.workspace = true
clap.workspace = true
Expand Down
96 changes: 41 additions & 55 deletions crates/minvmd/build.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,20 @@
//! Without that cfg the crate compiles to a runtime-bailing stub and never
//! links libkrun.
//!
//! The rpath leads with a binary-relative entry (`$ORIGIN` / `@loader_path`) so
//! a relocated release binary prefers libkrun's shared objects shipped next to
//! it, and only then falls back to the absolute libkrun prefix. The prefix is
//! recorded except for a Linux `--release` build (its prefix is an ephemeral
//! build path that must not leak). A macOS release keeps the prefix because it
//! is the stable Homebrew lib dir (`/opt/homebrew/lib`), so a host that has
//! `brew install slp/krun/libkrun` but is missing the shipped dylib still
//! resolves libkrun. On Linux the standard system lib dirs ([`LINUX_LIB_DIRS`])
//! are added too, so a target with a system-installed libkrun resolves it
//! without bundling. See [`emit_rpaths`] for the full ordering.
//! The rpath leads with a binary-relative entry (`$ORIGIN` / `@loader_path`) so a
//! relocated binary prefers libkrun shipped next to it, then falls back to the
//! absolute prefix. That prefix is recorded except on a Linux `--release` build,
//! whose prefix is an ephemeral path that must not leak; macOS keeps it (stable
//! `/opt/homebrew/lib`) and Linux adds [`LINUX_LIB_DIRS`]. See [`emit_rpaths`].
//!
//! dyld only consults these rpaths for a load command that begins with
//! `@rpath/`. macOS links libkrun by its own (absolute Homebrew) install name,
//! so the release workflow rewrites minvmd's load command to
//! `@rpath/libkrun.1.dylib` with `install_name_tool` after the build — that is
//! what activates the `@loader_path`-first search above.
//! dyld only consults these rpaths for an `@rpath/` load command. macOS links
//! libkrun by its absolute Homebrew install name, so the release workflow
//! rewrites minvmd's load command to `@rpath/libkrun.1.dylib` post-build.
//!
//! Gated behind the `libkrun` cargo feature (ON by default). With it OFF every
//! target builds the stub and links nothing — how the `minimal` CLI depends on
//! this crate (`default-features = false`) without inheriting libkrun's `#[link]`
//! load command. Per-platform behaviour below applies only with the feature on:
//!
//! - **macOS**: libkrun is always linked (Hypervisor.framework backend). The
//! prefix defaults to `/opt/homebrew/lib` (the Homebrew tap install location);
Expand Down Expand Up @@ -56,17 +54,22 @@ fn main() {

let target_os = std::env::var("CARGO_CFG_TARGET_OS").unwrap_or_default();

let prefix = match target_os.as_str() {
// macOS always links libkrun: it is the only VM backend on the platform
// and the existing proof artifacts assume it is provisioned.
"macos" => Some(
std::env::var("LIBKRUN_PREFIX").unwrap_or_else(|_| "/opt/homebrew/lib".to_string()),
),
// Linux links libkrun only when it can be found, so a host without it
// (e.g. the stock Ubuntu CI runners) still builds the no-op stub.
"linux" => find_libkrun_prefix(),
_ => None,
};
// Real backend is opt-in via the `libkrun` feature (default on; the `minimal`
// CLI opts out). Off => stub, and no dependent binary inherits libkrun's
// `#[link]`. Cargo reruns build scripts on feature changes automatically.
let want_libkrun = std::env::var_os("CARGO_FEATURE_LIBKRUN").is_some();

let prefix = want_libkrun
.then(|| match target_os.as_str() {
// macOS: the only VM backend on the platform; always present.
"macos" => {
Some(std::env::var("LIBKRUN_PREFIX").unwrap_or_else(|_| "/opt/homebrew/lib".into()))
}
// Linux: link only when found, else stub (e.g. stock Ubuntu CI).
"linux" => find_libkrun_prefix(),
_ => None,
})
.flatten();

if let Some(prefix) = prefix {
println!("cargo:rustc-link-search=native={prefix}");
Expand All @@ -75,48 +78,31 @@ fn main() {
}
}

/// Record the runtime library search paths (rpath) baked into the binary. The
/// dynamic loader tries them in the order emitted here, so binary-relative comes
/// FIRST: a libkrun shipped next to the binary must win over any system or
/// Homebrew copy.
/// Runtime library search paths (rpath) baked into the binary, in the order dyld
/// tries them — binary-relative FIRST, so a libkrun shipped next to the binary
/// wins over any system or Homebrew copy.
fn emit_rpaths(target_os: &str, prefix: &str) {
let is_release = std::env::var("PROFILE").as_deref() == Ok("release");

// 1. Binary-relative (`@loader_path` on macOS, `$ORIGIN` on Linux) — FIRST,
// so a RELOCATED binary prefers libkrun's shared objects shipped alongside
// it (e.g. the release-staged `bin/libkrun.1.dylib` next to minvmd). dyld
// honours this only for an `@rpath/...` load command; on macOS the release
// workflow rewrites minvmd's absolute Homebrew load command to
// `@rpath/libkrun.1.dylib` (install_name_tool) precisely so this entry
// takes effect. Reaches the linker as a literal argv token (no shell
// expansion here), so it is recorded verbatim.
// 1. Binary-relative: finds libkrun shipped alongside a relocated binary
// (release-staged `bin/libkrun.1.dylib`). dyld honours it only for an
// `@rpath/` load command — the macOS release rewrites minvmd's to that.
// Recorded verbatim (literal argv token, no shell expansion).
rpath(if target_os == "macos" {
"@loader_path"
} else {
"$ORIGIN"
});

// 2. Absolute `{prefix}` — a FALLBACK consulted only when the binary-relative
// lookup above misses. Recorded EXCEPT for a Linux `--release` build:
// - Non-release (dev + every e2e CI run use `target/debug/minvmd`): resolve
// libkrun where it was materialized, no LD_LIBRARY_PATH needed.
// - macOS release: KEEP it. On Apple Silicon the prefix is the canonical
// Homebrew lib dir (`/opt/homebrew/lib`) — a stable system path identical
// on any target with `brew install slp/krun/libkrun`, so a host missing
// the shipped dylib still resolves libkrun. dyld consults this rpath when
// the load command is `@rpath/...` (see entry 1), and ignores it
// harmlessly when the install name is already absolute.
// - Linux release: DROP it — the prefix is an ephemeral build path (e.g.
// `$HOME/.krun`) that must not leak into the artifact; entries 1 and 3
// cover that binary instead.
// 2. Absolute {prefix}: fallback when entry 1 misses. Dropped on a Linux
// --release build (ephemeral path, must not leak); kept for dev (resolve
// where materialized) and macOS release (stable /opt/homebrew/lib).
if !is_release || target_os == "macos" {
rpath(prefix);
}

// 3. Standard system lib dirs on Linux — lets a target with a system-installed
// libkrun resolve it without bundling (the counterpart to macOS reusing the
// Homebrew prefix in entry 1). Stable absolute paths, so safe in a release
// artifact; nonexistent dirs are skipped by the loader.
// 3. Linux system lib dirs: resolve a system-installed libkrun without
// bundling. Stable paths; nonexistent ones are skipped by the loader.
if target_os == "linux" {
LINUX_LIB_DIRS.iter().for_each(|dir| rpath(dir));
}
Expand Down
12 changes: 5 additions & 7 deletions scripts/stage-release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,13 +113,11 @@ COMPONENTS=(
# macOS arm64 (darwin)
"minimal|darwin|arm64|file|bin/minimal|minimal-macos-arm64"
"minvmd|darwin|arm64|file|bin/minvmd|minvmd-macos-arm64"
# The trimmed libkrun minvmd links against (built from source by the release
# workflow's build-libkrun-macos-arm64 job). Stamped into bin/ next to minvmd
# for now — weird for a dylib, but it puts it on a known user-owned path the
# loader can be pointed at; the +x bit bin/ adds is inert. The basename MUST
# be libkrun.1.dylib: minvmd's load command is rewritten to
# `@rpath/libkrun.1.dylib` and `@loader_path` (bin/, next to minvmd) is its
# first rpath, so the loader looks for exactly this name beside the binary.
# The trimmed libkrun minvmd links against (built by the release workflow's
# build-libkrun-macos-arm64 job), staged into bin/ next to minvmd. Basename
# MUST be libkrun.1.dylib: minvmd's load command is @rpath/libkrun.1.dylib and
# @loader_path (bin/) is its first rpath, so the loader looks for that exact
# name beside the binary.
"libkrun|darwin|arm64|file|bin/libkrun.1.dylib|libkrun-macos-arm64.dylib"
"gvproxy|darwin|arm64|file|bin/gvproxy|gvproxy-darwin-arm64"
"initramfs|darwin|arm64|file|data/initramfs.cpio|initramfs-arm64.cpio"
Expand Down
Loading
0