8000 test: v8 golden files and v7 migration checks (#391) by adityamukho · Pull Request #456 · project-minigraf/minigraf · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/policy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ name: Policy
# - msrv: the crate builds and its unit tests pass on `rust-version`
# - semver-checks: no accidental public-API break against the latest release tag
# - cargo-deny: dependency licenses, duplicate versions and sources (deny.toml)
# - golden-corpus: golden files in tests/golden/ are only ever added (#391)

on:
push:
Expand Down Expand Up @@ -88,3 +89,25 @@ jobs:
with:
# Advisories are covered by cargo-audit.yml and osv-scanner.yml.
command: check bans licenses sources

golden-corpus:
name: Golden files are add-only
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Refuse modified, deleted or renamed golden files
env:
BASE_REF: ${{ github.base_ref }}
run: |
changed=$(git diff --name-only --diff-filter=MDRT "origin/${BASE_REF}...HEAD" -- \
'tests/golden/*.graph' 'tests/golden/*.wal')
if [ -n "$changed" ]; then
echo "::error::Golden files are never regenerated, only added (tests/golden/README.md):"
echo "$changed"
exit 1
fi
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,6 @@ minigraf-ffi/python/minigraf/minigraf_ffi/
minigraf-ffi/python/**/__pycache__/
minigraf-ffi/python/.venv/
minigraf-ffi/python/dist/

# golden-file generator build output (tests/golden/gen/*)
tests/golden/gen/*/target/
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- **B+tree reads no longer follow leaf sibling links** (v8 storage format, PR 1 of 5; #374). Every index scan uses a cursor that walks down from the root and keeps the path, with a forward `seek` that climbs only as far as needed. This prepares for copy-on-write index pages, which cannot keep sibling links. The file format and query results are unchanged. Concurrent 100k-fact scans take 25–37 % less time and checkpointing after one fact at 10k facts 9 % less; single point lookups and attribute scans are 2–6 % slower (about 0.4 µs per lookup). A corrupted child pointer now fails with STG-013 (wrong page type) or INT-049 (cycle) instead of following a bad link.

### Tests

- **Golden-file compatibility corpus (#391).** `tests/golden/` holds v7 files written by minigraf 2.0.3, each with a JSON manifest of its CRC32, `tx_count` and expected query results. They cover one checkpoint, several checkpoints, indexes rebuilt on open, a WAL left by a crash, a WAL whose entries were already checkpointed, and same-transaction multi-values (#371). `tests/golden_corpus_test.rs` opens a copy of each file and checks its manifest after open, after a checkpoint and reopen, and after one more write. Golden files are never regenerated, only added; a new `golden-corpus` job in `policy.yml` fails a pull request that changes one. The generator is a standalone crate pinned to `minigraf = "=2.0.3"` (`tests/golden/gen/v7/`). On `v3`, each v7 file must also carry a v8 meta page after its first open, and seven v8 files are frozen alongside them, written by the `v3` branch at `23aa57c` (`tests/golden/gen/v8/`): one checkpoint, seven copy-on-write checkpoints, a crash-left v2 WAL, an already-checkpointed v2 WAL, same-transaction multi-values, a v7 file migrated in place, and long values (a 4,068-byte string stored twice, the 64/65-byte inline boundary, a 1,000-byte attribute and keyword). All #371 queries now pass.

## v2.0.3 — 2026-10-06

Patch release on the v2.x line with two data-integrity fixes, under the [support policy](PHILOSOPHY.md#support-policy). File format is unchanged (v7). No API changes. Upgrading is recommended for every v2.x user.
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are

## Test Coverage

**1236 tests** (1227 passing, 9 ignored; unit + integration + doc).
**1238 tests** (1229 passing, 9 ignored; unit + integration + doc).
See `docs/TEST_COVERAGE.md` for the full per-file breakdown.

**Testing conventions** — see the Testing Conventions section below before writing any tests.
Expand Down
7 changes: 7 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,13 @@ Bugs that can lose, corrupt or hide data are handled so that users can always se

At release time, the maintainer updates the pinned issue and closes the issues whose fixes are in the release.

## Golden Files

`tests/golden/` holds database files written by released versions, each with a manifest of the results it must give (#391). See [`tests/golden/README.md`](tests/golden/README.md).

- Golden files are **never regenerated, only added**. CI fails a pull request that modifies, deletes or renames one.
- Every release that changes on-disk behavior (a new format version, a new page or WAL shape, a migration) adds golden files for it, with the recipe in a generator crate under `tests/golden/gen/`.

## Philosophy Check

Before submitting, ask yourself:
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[workspace]
members = ["."]
exclude = ["fuzz"]
exclude = ["fuzz", "tests/golden/gen/v7", "tests/golden/gen/v8"]
resolver = "2"

[package]
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ let r2 = pq.execute(&[("tx", BindValue::TxCount(2)), ("entity", BindValue::Entit

```bash
cargo run # interactive Datalog REPL
cargo test # run 1236 tests
cargo test # run 1238 tests
cargo run < demos/demo_recursive.txt # recursive rules demo
```

Expand Down
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Scope — format and storage integrity:

Scope — release-gate testing and documentation:

- Golden-file compatibility corpus for every format version, frozen before v8 work goes further (#391)
- Golden-file compatibility corpus for every format version (#391; done on `v3`: six v7 files from 2.0.3 checked after migration, seven v8 files frozen at `23aa57c`)
- SIGKILL crash test that checks committed data through EAVT, AEVT and full scan (#384)
- Model-based test of transact/retract/checkpoint/reopen (#385)
- Wider property tests: file-backed, batched, retracts, temporal, joins (#386)
Expand Down
Loading
Loading
0