8000 perf: ship full-LTO dist builds and gate their size (#796) by adityamukho · Pull Request #801 · project-minigraf/minigraf · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .github/workflows/binary-size.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ permissions:

jobs:
binary-size:
name: Check release binary size
name: Check distribution binary size
runs-on: ubuntu-latest
env:
SIZE_LIMIT_BYTES: 1263616 # 1234 KiB — #277 error codes, #306 top-k, #723 ancestor join planning; see PHILOSOPHY.md §4
Expand All @@ -22,12 +22,13 @@ jobs:
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable

- name: Build release binary
run: cargo build --release
# The profile cargo-dist ships (#796)
- name: Build distribution binary
run: cargo build --profile dist

- name: Check binary size
run: |
BINARY=target/release/minigraf
BINARY=target/dist/minigraf
SIZE=$(stat -c %s "$BINARY")
SIZE_KB=$(( SIZE / 1024 ))
LIMIT_KB=$(( SIZE_LIMIT_BYTES / 1024 ))
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Performance

- **Released binaries use full LTO** (#796). The `dist` profile cargo-dist builds releases with no longer overrides `release`'s full LTO with thin LTO. The CLI is 7–31% smaller on the five release targets (x86_64 Linux 1,501,232 → 1,262,584 bytes, measured before #794) and 4–17% faster on query workloads; clean builds take up to 10 seconds longer. `binary-size.yml` now builds `--profile dist`, so it measures the shipped x86_64 Linux binary (still under the unchanged 1,263,616-byte cap). Windows packages keep cargo-dist's static C runtime. See `docs/benchmarks/796-dist-lto.md`.

- **Sort set element encodings without a stable sort** (#794). `set_elements` sorts its elements' byte encodings with `sort_unstable`: equal encodings are identical bytes and deduplicated straight after, so the encoded bytes are unchanged, and the stable sort's scratch allocation and code go. The native release binary is 3,024 bytes smaller (WASI 950, thin-LTO dist 4,112). On the bench machine, ingest, checkpoint, set-constant queries and scans of 100,000 32-element sets in seven input shapes stayed within run-to-run noise. No API, format or error change.

- **Unstable sorts for unique query metadata** (#795). Prepared-query slot names, an `or`'s branch variables and a subplan's imported variables sort with `sort_unstable`: their keys are unique, or, for a join variable named twice, the tied entries are identical, so order and plans are unchanged. Release builds shrink by 1,008 bytes (native CLI), 1,752 bytes (WASI) and 1,886 bytes (browser `--lib`); prepare and planning time is unchanged within noise. No API, dependency or file-format change.
Expand Down
4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -130,10 +130,10 @@ lto = true
codegen-units = 1
panic = "unwind"

# The profile that 'dist' will build with
# The profile that 'dist' will build with. It keeps release's full LTO, so the
# shipped binaries are the ones binary-size.yml measures (#796).
[profile.dist]
inherits = "release"
lto = "thin"

[lints]
workspace = true
Expand Down
7 changes: 6 additions & 1 deletion PHILOSOPHY.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,12 @@ SQLite's success comes from a clear philosophy: be a library, not a server. Be s
fit the 2 KiB left after the streaming engine (#473) and set values (#316).
The approved #723 ancestor-join optimization raises it to 1,234 KiB:
the measured increase is 2,720 bytes, avoiding a whole-attribute value
map for selective in-memory recursion without adding a storage index
map for selective in-memory recursion without adding a storage index.
The gate builds the x86_64 Linux CLI with cargo-dist's `dist` profile,
the one releases ship. From v3.0.0 `dist` keeps release's full LTO
rather than thin LTO, 7–31% smaller per target (#796; sizes per target
in `docs/benchmarks/796-dist-lto.md`). Other targets are measured, not
gated; a profile setting for one target needs its own measurement
- No runtime dependencies (no JVM, no Python, no Node.js)

**Anti-pattern**: Requiring external services, libraries, or runtimes to function.
Expand Down
100 changes: 100 additions & 0 deletions docs/benchmarks/796-dist-lto.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Full LTO for distribution builds (#796)

cargo-dist builds the released CLI with the `dist` profile. Before #796 it
inherited `release` but set `lto = "thin"`, while `binary-size.yml` built
`--release` (full LTO): the gate measured a binary nobody downloaded. `dist`
now keeps release's full LTO, and the gate builds `--profile dist`.

## Method

Revision: v3 `2ff91059e` (before #794) plus this change's profile edit, as
temporary commits `36598e361` (matrix and bench) and `ac7e661c2` (Windows
static-CRT check) that also added the measuring workflow; rustc 1.99.1
everywhere. Thin LTO is the same profile with
`CARGO_PROFILE_DIST_LTO=thin`.

- **CI matrix**: the five cargo-dist targets on the runners `dist plan` assigns
(macos-14, ubuntu-22.04-arm, macos-15-intel, windows-2022, ubuntu-22.04),
run once from a temporary branch (`measure/796`, workflow run 41012917889;
the branch is deleted). Per runner: [`builds.py`](796-dist-lto/builds.py)
times four clean builds (thin, full, full, thin; dependencies fetched, not
compiled), [`runtime.py`](796-dist-lto/runtime.py) runs 10 alternating
rounds, and `dist build --artifacts=local` packages the target, whose
archive is unpacked and run on `demos/demo_commands.txt`.
- **Bench machine**: the same two scripts on x86_64 Linux, bound to one NUMA
node, with 20 runtime rounds.
- **Runtime workloads** (each a CLI process fed a script on stdin; times include
start-up, ingestion and printing): `count` (#772's 2,000 facts, 400 counts),
`recursive` (closure of a 300-node chain, 10 queries), `aggregate` (5,000
people in 20 departments, a join and grouped sum, 50 queries), `file` (20
transactions of 500 facts into a `.graph` file, one count). Both builds must
print the same output, transaction timestamps aside.

## Size

| Target | Thin LTO | Full LTO | Saved |
| --- | ---: | ---: | ---: |
| x86_64-unknown-linux-gnu | 1,501,232 | **1,262,584** | 238,648 (15.9%) |
| aarch64-unknown-linux-gnu | 1,262,544 | 1,094,504 | 168,040 (13.3%) |
| x86_64-apple-darwin | 1,503,392 | 1,075,840 | 427,552 (28.4%) |
| aarch64-apple-darwin | 1,372,560 | 954,192 | 418,368 (30.5%) |
| x86_64-pc-windows-msvc, as packaged (static CRT) | 1,139,712 | 1,060,864 | 78,848 (6.9%) |

The x86_64 Linux full-LTO binary is what the gate measures: 1,262,584 bytes
against the unchanged 1,263,616-byte cap. It equals the `--release` binary, so
the change gives the gate no headroom. Sizes were identical across a runner's
two builds.

**Packaging.** On Linux and macOS the packaged binary has the full-LTO size
(aarch64-apple-darwin: 16 bytes more, the build is under a different target
directory); its bytes differ from the separate build. On Windows cargo-dist
links the C runtime statically (`+crt-static`), which the plain builds above
do not: a second run (41014187879) built thin and full LTO with
`RUSTFLAGS=-Ctarget-feature=+crt-static` and the `--target` cargo-dist uses.
The full build matches the packaged 1,060,864 bytes, differing only in the PE
header; thin is 1,139,712
([`package-x86_64-pc-windows-msvc-static-crt.json`](796-dist-lto/package-x86_64-pc-windows-msvc-static-crt.json)). Without the static CRT the Windows binaries are
1,041,920 (thin) and 963,072 (full). All five packaged binaries ran the demo.

The gate checks x86_64 Linux only. The other targets' sizes are recorded here,
not gated.

## Build time

Clean builds, seconds, in run order (thin, full, full, thin):

| Runner | Thin | Full | Full | Thin |
| --- | ---: | ---: | ---: | ---: |
| ubuntu-22.04 | 43 | 30 | 31 | 28 |
| ubuntu-22.04-arm | 30 | 28 | 28 | 26 |
| macos-15-intel | 124 | 97 | 77 | 67 |
| macos-14 | 42 | 41 | 42 | 37 |
| windows-2022 | 64 | 46 | 47 | 46 |
| bench, x86_64 Linux | 28 | 31 | 31 | 28 |

The first build on each runner is slower whatever its profile (cold caches).
Comparing the warmer pairs, full LTO costs 0–10 seconds (0–16%) per build,
3 seconds on the bench machine. A release builds each target once.

## Runtime

Median full-vs-thin change in elapsed time, and the rounds full LTO won:

| Runner | count | recursive | aggregate | file |
| --- | ---: | ---: | ---: | ---: |
| ubuntu-22.04 | −12.7% (10/10) | −9.4% (10/10) | −8.7% (10/10) | −9.0% (10/10) |
| ubuntu-22.04-arm | −9.7% (10/10) | −16.0% (10/10) | −17.4% (10/10) | −3.6% (10/10) |
| macos-15-intel | −7.4% (7/10) | −3.9% (9/10) | −7.8% (8/10) | −1.5% (6/10) |
| macos-14 | −6.3% (8/10) | −10.3% (10/10) | −5.7% (8/10) | +3.5% (5/10) |
| windows-2022 | −5.7% (10/10) | −9.4% (10/10) | −6.7% (10/10) | −2.1% (7/10) |
| bench, x86_64 Linux | −9.5% (20/20) | −8.4% (20/20) | −11.0% (20/20) | −4.3% (13/20) |

Full LTO is faster on every query-bound workload on every target. `file`
(about 0.1–0.3 s, mostly process start-up and file I/O) is within noise on
macOS. Hosted runners are shared machines; the bench rows are the controlled
measurement.

## Evidence

[`796-dist-lto/`](796-dist-lto/): `builds-*.json`, `runtime-*.json` (every
round) and `package-*.json` per target, plus the two scripts.
30 changes: 30 additions & 0 deletions docs/benchmarks/796-dist-lto/builds-aarch64-apple-darwin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"platform": "macOS-14.8.9-arm64-arm-64bit",
"machine": "arm64",
"builds": [
{
"round": 0,
"lto": "thin",
"build_seconds": 42.01392458400005,
"bytes": 1372560
},
{
"round": 0,
"lto": "full",
"build_seconds": 40.54684737499997,
"bytes": 954192
},
{
"round": 1,
"lto": "full",
"build_seconds": 42.17448854100002,
"bytes": 954192
},
{
"round": 1,
"lto": "thin",
"build_seconds": 36.50212337500005,
"bytes": 1372560
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"platform": "Linux-6.8.0-1070-azure-aarch64-with-glibc2.35",
"machine": "aarch64",
"builds": [
{
"round": 0,
"lto": "thin",
"build_seconds": 30.349435244999995,
"bytes": 1262544
},
{
"round": 0,
"lto": "full",
"build_seconds": 28.18664725100001,
"bytes": 1094504
},
{
"round": 1,
"lto": "full",
"build_seconds": 28.104224689000034,
"bytes": 1094504
},
{
"round": 1,
"lto": "thin",
"build_seconds": 25.75162138500002,
"bytes": 1262544
}
]
}
30 changes: 30 additions & 0 deletions docs/benchmarks/796-dist-lto/builds-bench-x86_64-linux.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"platform": "Linux-6.8.0-139-generic-x86_64-with-glibc2.39",
"machine": "x86_64",
"builds": [
{
"round": 0,
"lto": "thin",
"build_seconds": 27.877472235122696,
"bytes": 1501232
},
{
"round": 0,
"lto": "full",
"build_seconds": 30.649512137984857,
"bytes": 1262584
},
{
"round": 1,
"lto": "full",
"build_seconds": 30.725669295992702,
"bytes": 1262584
},
{
"round": 1,
"lto": "thin",
"build_seconds": 27.925453558098525,
"bytes": 1501232
}
]
}
30 changes: 30 additions & 0 deletions docs/benchmarks/796-dist-lto/builds-x86_64-apple-darwin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"platform": "macOS-15.7.9-x86_64-i386-64bit",
"machine": "x86_64",
"builds": [
{
"round": 0,
"lto": "thin",
"build_seconds": 123.61216050499996,
"bytes": 1503392
},
{
"round": 0,
"lto": "full",
"build_seconds": 96.794796571,
"bytes": 1075840
},
{
"round": 1,
"lto": "full",
"build_seconds": 76.51469797699997,
"bytes": 1075840
},
{
"round": 1,
"lto": "thin",
"build_seconds": 67.28196448799997,
"bytes": 1503392
}
]
}
10BC8
30 changes: 30 additions & 0 deletions docs/benchmarks/796-dist-lto/builds-x86_64-pc-windows-msvc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"platform": "Windows-2022Server-10.0.20348-SP0",
"machine": "AMD64",
"builds": [
{
"round": 0,
"lto": "thin",
"build_seconds": 63.82672539999999,
"bytes": 1041920
},
{
"round": 0,
"lto": "full",
"build_seconds": 46.1428391,
"bytes": 963072
},
{
"round": 1,
"lto": "full",
"build_seconds": 47.27537620000004,
"bytes": 963072
},
{
"round": 1,
"lto": "thin",
"build_seconds": 45.59298579999995,
"bytes": 1041920
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"platform": "Linux-6.8.0-1070-azure-x86_64-with-glibc2.35",
"machine": "x86_64",
"builds": [
{
"round": 0,
"lto": "thin",
"build_seconds": 43.43511368700001,
"bytes": 1501232
},
{
"round": 0,
"lto": "full",
"build_seconds": 30.330564745999993,
"bytes": 1262584
},
{
"round": 1,
"lto": "full",
"build_seconds": 30.577716478000013,
"bytes": 1262584
},
{
"round": 1,
"lto": "thin",
"build_seconds": 28.490207176000013,
"bytes": 1501232
}
]
}
26 changes: 26 additions & 0 deletions docs/benchmarks/796-dist-lto/builds.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/usr/bin/env python3
"""Time clean thin- and full-LTO `dist` builds, two rounds, alternating (#796).

Usage: builds.py OUTPUT_JSON (from the repository root; copies each binary
to bins/thin-minigraf and bins/full-minigraf for runtime.py)
"""
import json, os, platform, shutil, subprocess, sys, time
exe = "minigraf.exe" if os.name == "nt" else "minigraf"
rows = []
for rnd, order in enumerate([("thin", "full"), ("full", "thin")]):
for label in order:
tdir = f"tgt-{label}-{rnd}"
env = dict(os.environ)
if label == "thin":
env["CARGO_PROFILE_DIST_LTO"] = "thin"
start = time.perf_counter()
subprocess.run(["cargo", "build", "--locked", "--profile", "dist", "--target-dir", tdir], env=env, check=True)
secs = time.perf_counter() - start
size = os.path.getsize(os.path.join(tdir, "dist", exe))
rows.append({"round": rnd, "lto": label, "build_seconds": secs, "bytes": size})
print(rows[-1], flush=True)
os.makedirs("bins", exist_ok=True)
shutil.copy(os.path.join(tdir, "dist", exe), os.path.join("bins", f"{label}-{exe}"))
shutil.rmtree(tdir)
json.dump({"platform": platform.platform(), "machine": platform.machine(), "builds": rows},
open(sys.argv[1], "w"), indent=2)
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"target": "aarch64-apple-darwin",
"packaged_bytes": 954208,
"full_bytes": 954192,
"thin_bytes": 1372560,
"packaged_equals_full": false
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"target": "aarch64-unknown-linux-gnu",
"packaged_bytes": 1094504,
"full_bytes": 1094504,
"thin_bytes": 1262544,
"packaged_equals_full": false
}
Loading
Loading
0