8000 amd64: consume the bootstrap chain as sealed tarballs, like arm64 (interim; stops ladder rebuilds) by bryan-minimal · Pull Request #825 · gominimal/pkgs · GitHub
[go: up one dir, main page]

Skip to content

amd64: consume the bootstrap chain as sealed tarballs, like arm64 (interim; stops ladder rebuilds) - #825

Merged
bryan-minimal merged 5 commits into
mainfrom
amd64-ladder-off
Oct 14, 2026
Merged

bryan-minimal merged 5 commits into
mainfrom
amd64-ladder-off

Conversation

@bryan-minimal
@bryan-minimal bryan-minimal commented Oct 7, 2026 •
Copy link
Copy Markdown
Member

Interim fix for build-bot backing up. Every change under openssl/python/glibc currently rebuilds the whole amd64 hex0-rooted chain, because its build tools (bash, coreutils, sed, …) come from the live tree. arm64 already avoids this: the chain is empty stubs there, and go / rust take sha-pinned tarballs of the chain's last rung. This PR does the same on amd64, behind one switch.

Change

  • packages/hex0/amd64-ladder.ncl: { enabled = false }. Flip it to true to build the chain from hex0 again.
  • 38 chain packages (stage0 ladder, glibc-bedrock-2.42, gcc-15.2.0-glibc, binutils-2.46-glibc, go-1.4…1.24.9, mrustc*, rustc-1.91.1…1.97.1): their amd64 spec is an empty stub while the switch is off, exactly the arm64 shape.
  • go (amd64): go-1.24.9 comes from a prebuilt spec over build-bot's own output of that rung.
  • rust (amd64): same for rustc-1.97.1, with the rung's runtime_deps (libffi, llvm, openssl, sqlite, gcc libs).
  • Tarballs: both are build-bot's artifacts for these rungs at 9d20a830 (sha from its commit map), copied to gs://minimal-staging-archives/prebuilts/bedrock/sha256/<sha>/ and verified with an anonymous readback. They have the same layout as the rungs (usr/lib/go-1.24.9, usr/lib/rustc-1.97.1), so go and rust's build.sh are unchanged and see the same input tree.

Evidence

Spec hashes from mip dump --arch amd64 (no build):

main this PR
packages whose hash moves on an openssl change 548 511
…of which chain packages 38 / 38 0 / 38

Longer term

Seal the chain's build tools so it stops depending on the tree at all, then flip the switch back. That is hash-proven separately (0/23 rungs move with all tools sealed), but it needs sealed patch / python first.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Builds
    • AMD64 bootstrap builds are now controlled by a setting that is disabled by default. When disabled, affected toolchain stages are skipped, and Go and Rust use SHA-256-verified prebuilt artifacts where needed.
    • Enabling the setting selects the staged bootstrap build process on AMD64. ARM64 build behavior is unchanged.

…terim)

Every change under openssl/python/glibc rebuilt the whole amd64 hex0 chain
(38 packages: stage0 ladder, glibc-bedrock, gcc/binutils-*-glibc, go-1.4..1.24.9,
mrustc, rustc-1.91..1.97), because its build tools come from the live tree.
That backs up build-bot on every world-touching PR.

arm64 already avoids this: the chain is empty stubs there and `go` / `rust`
take sha-pinned tarballs of its last rung. This does the same on amd64, behind
one switch (packages/hex0/amd64-ladder.ncl, `enabled = false`):

- the 38 chain packages' amd64 spec is an empty stub while the switch is off
- go (amd64) takes go-1.24.9 as a prebuilt over build-bot's own output of that
  rung; rust (amd64) does the same for rustc-1.97.1, with the rung's runtime_deps.
  Both tarballs are build-bot's artifacts at 9d20a83 (commit map sha), copied to
  gs://minimal-staging-archives/prebuilts/bedrock/sha256/<sha>/, same layout as the
  rungs (usr/lib/...), so build.sh is unchanged.

Flip the switch to true to build the chain from hex0 again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bryan-minimal
bryan-minimal requested a review from a team as a code owner October 7, 2026 01:08
@coderabbitai
coderabbitai Bot commented Oct 7, 2026 •
Copy link
Copy Markdown
Contributor

Review in Change Stack →

Warning

Review limit reached

  • Run on-demand review

This review includes 4 billable files and costs up to $1.00.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 40 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 60 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: gominimal/pkgs/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 791dcc01-a039-4b0d-8736-4092784cbebb
📥 Commits

Reviewing files that changed from the base of the PR and between 199819e and 25187e1.

📒 Files selected for processing (4)
  • packages/binutils-2.46-glibc/build.ncl
  • packages/gcc-15.2.0-glibc/build.ncl
  • packages/glibc-bedrock-2.42/build.ncl
  • packages/hex0/amd64-ladder.ncl

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: gominimal/pkgs/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 7a60e898-61d0-41c5-997e-26b1a4f2ebfa
📥 Commits

Reviewing files that changed from the base of the PR and between 2bb548c and 199819e.

📒 Files selected for processing (42)
  • packages/binutils-2.46-glibc/build.ncl
  • packages/gcc-15.2.0-glibc/build.ncl
  • packages/glibc-bedrock-2.42/build.ncl
  • packages/go-1.17.13/build.ncl
  • packages/go-1.20.14/build.ncl
  • packages/go-1.22.6/build.ncl
  • packages/go-1.24.9/build.ncl
  • packages/go-1.4/build.ncl
  • packages/go/build.ncl
  • packages/hex0/amd64-ladder.ncl
  • packages/mrustc-rustc-1.90.0/build.ncl
  • packages/mrustc/build.ncl
  • packages/rust/build.ncl
  • packages/rustc-1.91.1/build.ncl
  • packages/rustc-1.92.0/build.ncl
  • packages/rustc-1.93.1/build.ncl
  • packages/rustc-1.94.1/build.ncl
  • packages/rustc-1.95.0/build.ncl
  • packages/rustc-1.96.0/build.ncl
  • packages/rustc-1.97.1/build.ncl
  • packages/rustc-1.98.1/build.ncl
  • packages/stage0-binutils-2.30/build.ncl
  • packages/stage0-binutils-2.41/build.ncl
  • packages/stage0-gcc-10.4.0/build.ncl
  • packages/stage0-gcc-15.2.0/build.ncl
  • packages/stage0-gcc-4.0.4/build.ncl
  • packages/stage0-gcc-4.7.4/build.ncl
  • packages/stage0-gmp-mpfr-mpc/build.ncl
  • packages/stage0-hex1/build.ncl
  • packages/stage0-kaem/build.ncl
  • packages/stage0-linux-headers-6.12.43/build.ncl
  • packages/stage0-mes/build.ncl
  • packages/stage0-mescc-full/build.ncl
  • packages/stage0-mescc/build.ncl
  • packages/stage0-musl-1.1.24-cc2/build.ncl
  • packages/stage0-musl-1.1.24/build.ncl
  • packages/stage0-musl-1.2.5/build.ncl
  • packages/stage0-tcc-0.9.26/build.ncl
  • packages/stage0-tcc-0.9.27-musl-s1/build.ncl
  • packages/stage0-tcc-0.9.27-musl-s3/build.ncl
  • packages/stage0-tcc-0.9.27-musl-s4/build.ncl
  • packages/stage0-tcc-0.9.27/build.ncl
🚧 Files skipped from review as they are similar to previous changes (2)
  • packages/rust/build.ncl
  • packages/go/build.ncl

Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 2 reviews per hour.

📝 Walkthrough

The amd64 build specifications now check a shared ladder switch. When disabled, most package specifications have no build dependencies. The Go and Rust bootstrap specifications select pinned prebuilt artifacts. Arm64 behavior remains unchanged.

Changes: AMD64 bootstrap configuration

Layer / File(s) Summary
Ladder switch and stage0 package gates
packages/hex0/amd64-ladder.ncl, packages/stage0-*/build.ncl
The switch defaults to disabled. Stage0 amd64 package specifications select their full specifications only when the switch is enabled; otherwise, they have no build dependencies.
Compiler and library package gates
packages/binutils-2.46-glibc/build.ncl, packages/gcc-15.2.0-glibc/build.ncl, packages/glibc-bedrock-2.42/build.ncl, packages/go-*/build.ncl, packages/mrustc*/build.ncl, packages/rustc-1.*/build.ncl
These amd64 package specifications select their full specifications only when the switch is enabled. When it is disabled, they return specifications with no build dependencies.
Go bootstrap artifact selection
packages/go/build.ncl
The amd64 bootstrap selects the Go 1.24.9 rung when the switch is enabled. Otherwise, it selects a SHA-256-pinned prebuilt artifact and exposes its usr/**/* contents as output data.
Rust bootstrap artifact selection
packages/rust/build.ncl, packages/rustc-1.98.1/build.ncl
The amd64 bootstrap selects the rustc 1.97.1 rung when the switch is enabled. Otherwise, it selects a SHA-256-pinned Bedrock archive with the rung’s runtime dependencies and usr/**/* output.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Suggested reviewers: norrietaylor

Merge Risk: ⚪ Minimal · up to 19981

No actionable issue was established; the change is mergeable after normal amd64 build validation.

Pre-merge checks · Passed 5
Passed Passed checks (5 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: amd64 will consume the bootstrap chain as sealed tarballs, matching arm64 behavior, to stop ladder rebuilds. The interim qualifier is relevan…
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@bryan-minimal
Copy link
Copy Markdown
Member Author

@coderabbitai full review

@coderabbitai
coderabbitai Bot commented Oct 7, 2026 •
Copy link
Copy Markdown
Contributor
✅ Action performed

Full review finished.

@norrietaylor norrietaylor left a comment
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified: both bedrock tarballs resolve on storage.googleapis.com and hash to the declared sha256; BUILDINFO in each matches the rung recipe (go-1.24.9 bootstrapped from go-1.22.6; rustc 1.97.1 built by stage0 1.96.0 against external llvm 21.1.8); layout matches what the unchanged go/build.sh and rust/build.sh expect at /usr/lib/...; prebuilt rustc runtime_deps equals the rung's; DT_NEEDED on rustc/cargo all covered. All 38 chain hunks are identical modulo name. Only go and rust consume the chain on main, so nothing else changes. CI green.

Not approving yet because of merge ordering, which git will not catch:

  1. #775 (jvm) depends on three packages this PR stubs out on amd64. Its root rungs import gcc-15.2.0-glibc, binutils-2.46-glibc and glibc-bedrock-2.42 as build_deps and hardcode SR=/usr/lib/glibc-bedrock-2.42 in build.sh. With enabled = false those are empty stubs, so the jvm ladder gets nothing hydrated and fails at its asserts. No file overlap, so no conflict warning. The body mentions #810 but not #775. Either #775 needs a bedrock tarball for those three too, or this lands after #775 (and #810).
  2. #810 conflicts textually on the amd64 arm of packages/rust/build.ncl, and its new rustc-1.98.1 rung takes rustc-1-97-1 as amd64 stage0, which this PR empties. Body acknowledges this one.
  3. "Interim" has no tracking issue. hex0/amd64-ladder.ncl:14-15 cites branch ladder-sealed-tools, which does not exist on origin, and no open issue mentions the ladder. Please file one and link it from the switch file so enabled = false is not forgotten.
  4. amd64 rustc tarball is more fragile than arm64's. host-llvm: external, link-shared; it ships only librustc_driver-*.so and relies on the live llvm package, whereas the arm64 comment says that tarball bundles its libLLVM.so.21.1. Previously an llvm soname bump rebuilt the rung; now it breaks rust on amd64 until someone flips the switch and regenerates. Worth a sentence in the comment for the next llvm bumper.

Lower priority:

  • Provenance is weaker than the arm64 precedent it mirrors: arm64 cites double-run byte reproduction and minimermetic#16; these cite only a build-bot commit map a reviewer cannot see, and there is no .intoto next to either tarball (stage0-* roots do have them). A build-bot run URL or attestation would close that.
  • Comments now stale while the switch is off: go/build.ncl:5 "amd64 bootstraps from the go-1.24.9 rung", rust/build.ncl:38-40, and the "This package exists on amd64 only" preambles in all 38 chain files.
  • packages/hex0/amd64-ladder.ncl is the first non-build.ncl .ncl under any package dir and governs the go/rustc/mrustc ladders too, not just hex0. Fine for an interim switch; noting the precedent.

bryan-minimal and others added 2 commits October 12, 2026 15:01
…tc-1.98.1 rung)

# Conflicts:
#	packages/rust/build.ncl
…m note, stale comments)

- #810 moved rust's stage0 to the rustc-1.98.1 rung, so the amd64 bedrock
  arm moves there (mirrors its arm64 arm); rust/build.ncl is back to main's.
  The 1.97.1 tarball is unchanged.
- llvm bumpers: the amd64 tarball links the live libLLVM, unlike arm64's.
- the switch file names its consumers and the new-consumer trap (#775).
- stale comments: go/build.ncl and all 38 chain preambles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bryan-minimal
Copy link
Copy Markdown
Member Author

@norrietaylor thanks, and sorry this sat. Point by point; the changes are in 199819e: a merge of main, then the review commit, no force-push.

  1. jvm: bootstrap OpenJDK from source (jikes/jamvm/classpath root, icedtea, openjdk-9..25) #775 (jvm). Agreed, and git really can't see it. We'll land this first, then jvm: bootstrap OpenJDK from source (jikes/jamvm/classpath root, icedtea, openjdk-9..25) #775 gets sealed tarballs for gcc-15.2.0-glibc / binutils-2.46-glibc / glibc-bedrock-2.42 the same way go/rustc-1.98.1 do. The switch file now says so explicitly for any new consumer: while it's off, importing a chain package gives you an empty stub on amd64.
  2. rust: 1.98.1 -> 1.99.0 (new rustc-1.98.1 rung; arm64 rung builds from the 1.97.1 bedrock) #810. Since merged. It moved rust's stage0 to the rustc-1.98.1 rung, so the amd64 bedrock arm now lives in that rung's amd64 stage0, mirroring its arm64 arm. rust/build.ncl is back to main's version. Same rustc-1.97.1 tarball, no new artifact. I re-checked consumers on current main: only go (go-1.24.9) and rustc-1.98.1 (rustc-1.97.1) import a chain package.
  3. "Interim" tracking. Filed amd64 bootstrap ladder is switched off: seal its build tools, then flip hex0/amd64-ladder.ncl back on #864 and linked it from hex0/amd64-ladder.ncl. ladder-sealed-tools is now pushed. It's an experiment branch with stand-ins, and amd64 bootstrap ladder is switched off: seal its build tools, then flip hex0/amd64-ladder.ncl back on #864 says so.
  4. amd64 rustc and llvm. Good catch. The tarball's comment now warns llvm bumpers that it loads the live libLLVM (host-llvm: external, link-shared). A soname bump breaks it until it's regenerated or the switch is flipped. The same note is in the switch file and amd64 bootstrap ladder is switched off: seal its build tools, then flip hex0/amd64-ladder.ncl back on #864.

Lower priority:

bryan-minimal and others added 2 commits October 13, 2026 08:57
…ts, not stubs

gcc-15.2.0-glibc, binutils-2.46-glibc and glibc-bedrock-2.42 now take the
same sha-pinned tarballs the production gcc/binutils/glibc packages already
anchor on (minimermetic#16), with their own outputs. The jvm ladder (#775)
imports them; it now builds either way (jikes-1.22 rebuilt and checked on
the sealed trio). The switch file's new-consumer note narrows to the rest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…spec.<field>

Reading a field back out of the contracted amd64_spec record broke the
whole-repo evaluation (mip dump: "not a function" in std contract/custom),
which per-package checks did not hit. Hoist both into lets, as the
production glibc does with all_outputs. With the switch on, all three spec
hashes are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bryan-minimal
bryan-minimal added this pull request to the merge queue Oct 14, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Oct 14, 2026
@bryan-minimal
bryan-minimal added this pull request to the merge queue Oct 14, 2026
Merged via the queue into main with commit 2d409d2 Oct 14, 2026
11 checks passed
@bryan-minimal
bryan-minimal deleted the amd64-ladder-off branch October 14, 2026 21:01
bryan-minimal added a commit that referenced this pull request Oct 21, 2026
Review follow-ups (mitodrummer):
- build.sh: the Widevine guard tests `find -print -quit` output instead
  of piping find into grep -q, which under pipefail could pass the guard
  on a SIGPIPE. Drop upstream's unused `chrome-wrapper` script with the
  Qt shims (layout_check asserts it is gone).
- build.ncl: one per-arch `release_url` let feeds both the Source and
  `binary_from` (it was written out four times).
- build.ncl: the provenance comment cites gominimal/pkgmgr-rs#1029
  (re-identify off microsoft/playwright, after a supply-chain cpe_remap)
  instead of #825, which closed and covered lifecycle bounds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

0