Real-world benchmark and SLO gates for Githome, a self-hosted GitHub-compatible forge. It runs the same REST surface and git transport the real binary mounts, seeds real or synthetic corpora through the Githome realworld pipeline, and checks the measured p99 latency against the doc 10 budget.
Githome real-world doc 10 sets a latency budget: every interactive API (a point read or a paginated read) must answer in under 25 ms p99 on real-world repositories, and that budget must not grow with the repository. A commit read that takes 2 ms on a five-file fixture and 2 s on a 1.45M-commit kernel is a bug in the index strategy, not an acceptable scale trade-off.
The gates here enforce both conditions:
- Smoke gate (
gate.TestSmokeGate): runs on every change, no real repository needed. Seeds a synthetic corpus, fires a 200 RPS workload, and fails if any interactive class exceeds 25 ms p99. - Scale gate (
gate.TestScaleGate): runs against a real repository (point it at a bare linux clone or any large repo). Repeats the smoke run on the giant, checks the budget holds, and then runs the flatness assertion: the giant p99 must not exceed the smoke p99 by more than 3x. A regression that slips under the absolute ceiling but degrades steadily still fails here.
| Package | Role |
|---|---|
budget |
The doc 10 budget table as code: ceilings by class, bounded/unbounded distinction |
benchload |
Open-model HTTP load harness: fires arrivals independent of server response rate, records latency from intended arrival time (coordinated-omission correction) |
slo |
Budget checker and flatness asserter: turns a result into pass/fail verdicts |
workload |
Binds a realworld ReplayPlan to concrete HTTP endpoints for a seeded server |
setup |
Builds a seeded, served Githome for a test run: store + git content + in-process server |
server |
In-process Githome wired from the real domain and REST layers |
gate |
Ties the above together: plan, fire, check, report |
Ordinary go test ./... runs the smoke gate and all unit tests.
The scale gate needs a bare repository on disk:
GITHOME_BENCH_GITREPO=/path/to/linux.git \
go test ./gate -run TestScaleGate -v -timeout 20m
A full git clone --bare https://github.com/torvalds/linux works. A local mirror (--local) is what CI uses since it hardlinks the object database rather than copying it.
The benchmark prints per-class latency at the end of each gate run:
point-read count=680 errors=0 p50=1.5ms p95=2.0ms p99=2.8ms max=3.7ms
The p99 is the nearest-rank 99th percentile over the steady-state window (warmup excluded). The gate fails when p99 exceeds 25 ms or when errors are non-zero for a bounded class.
The flatness line shows the ratio:
flatness point-read small_p99=2.2ms giant_p99=2.8ms ratio=1.31x
1.31x on a 1.45M-commit, 6.4 GiB kernel versus a five-file fixture means the pooled cat-file point-read path (5.5 enhancement) is genuinely index-seek-cost, not scan-cost.
This repo imports githome as a module and runs its surface as a black box. It does not duplicate the githome contract tests (those live in the githome suite). What it adds is the real-corpus, open-model, latency-budget assertion the unit tests cannot make: a fast test on a fixture tells you the code compiles correctly; this tells you the read path holds its budget on real data at realistic concurrency.
The workload currently excludes the paginated commit-list endpoint (/repos/.../commits?sha=...) because the go-git log iterator still uses a fresh per-request handle that re-parses the 200 MB pack index on linux.git. That is the next enhancement (pooled log iterator). When it lands, the R-git PaginatedRead class will be wired back in and the gate will cover it.