Add protected official Comparator execution report by Mnehmos · Pull Request #64 · Mnehmos/MathOS

Mnehmos · 2026-07-22T06:38:09Z

Summary

add the closed canonical comparator_run_report/1 Rust contract and committed JSON Schema
add database-independent verification for the exact 20-file protected run bundle, including package reprojection, raw official markers, sandbox records, retained tools, and runner script
run official Comparator in a fresh protected GitHub-hosted job with strict Landlock V5 probing, live systemd controls, empty-stderr acceptance, attestation, and non-authority
document the trust transition in ADR-0015

Trust boundary

An accepted report may set comparator_verified: true, but authoritative remains structurally false. This PR does not ingest evidence, update research status, or complete Pilot C.

Refs #63

Local verification

cargo fmt --check
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace (179 unit/integration tests)
python -m compileall -q src tests
python -m unittest discover -s tests -v (39 tests)
Lean v4.32.0 axiom audit for MathOS.PilotA.every_prime_other_than_two_is_odd
ShellCheck 0.11.0
Actionlint 1.7.12
bash -n scripts/comparator-protected-run.sh

Summary by CodeRabbit

New Features
- Added protected, attested Comparator run execution on the main branch.
- Added CLI support for independently verifying Comparator run bundles and reports.
- Added strict validation for report contents, artifacts, tool identities, sandbox settings, and execution results.
- Added a structured Comparator run report schema with accepted, rejected, and failed classifications.
Documentation
- Documented the official Comparator execution and verification model, including its non-authoritative status.
Tests
- Added coverage for report validation, successful verification, tampered artifacts, and invalid run bundles.

coderabbitai · 2026-07-22T06:38:27Z

Warning

Review limit reached

@Mnehmos, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 40 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f4050ccb-d5af-4add-88e0-004bd24d982b

📥 Commits

Reviewing files that changed from the base of the PR and between a620d9e and 45fc586.

📒 Files selected for processing (8)

.github/workflows/publication.yml
docs/decisions/ADR-0015-protected-official-comparator-execution.md
schemas/release/comparator-run-report-1.schema.json
scripts/comparator-network-probe.py
scripts/comparator-protected-run.sh
src/comparator_run.rs
src/domain/comparator_run.rs
src/domain/mod.rs

📝 Walkthrough

Walkthrough

Adds an attested protected Comparator workflow, a strict run-report schema and domain model, offline bundle verification, a release CLI command, and acceptance/substitution tests.

Changes

Protected Comparator verification

Layer / File(s)	Summary
Comparator run contract `schemas/release/comparator-run-report-1.schema.json`, `src/domain/comparator_run.rs`, `src/domain/mod.rs`, `src/lib.rs`, `src/comparator_export.rs`	Defines strict report structure, fixed tool and workflow identities, sandbox predicates, artifact bindings, classification rules, and crate-level exports.
Protected execution workflow `.github/workflows/publication.yml`, `scripts/comparator-protected-run.sh`, `docs/decisions/ADR-0015-protected-official-comparator-execution.md`	Adds boundary hash outputs and a protected-main Comparator job that builds pinned tools, prepares a clean harness, runs under systemd and Landlock constraints, creates the canonical report, and attests it.
Offline verification and CLI integration `src/comparator_run.rs`, `src/cli/mod.rs`, `tests/phase1_cli.rs`	Verifies report hashes, inventories, bindings, tool pins, harness, sandbox evidence, systemd properties, output markers, and classification; exposes `release verify-comparator-run` and tests valid and invalid bundles.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related issues

Phase 6e: protected official Comparator execution report #63 — Covers the protected official Comparator execution report, isolated workflow, pinned tools, sandboxed run, verification, and attestation implemented here.
Phase 6d: deterministic Comparator-ready package boundary #60 — Provides the deterministic Comparator package boundary artifacts and hashes consumed by this protected execution path.

Possibly related PRs

Mnehmos/MathOS#61 — Introduces the deterministic Comparator package boundary artifacts and hashes used by the protected run verifier.

Sequence Diagram(s)

sequenceDiagram
  participant Boundary
  participant ComparatorJob
  participant ProtectedRunner
  participant Comparator
  participant Verifier
  participant Attestation
  Boundary->>ComparatorJob: provide artifact hashes and bundle
  ComparatorJob->>ProtectedRunner: invoke protected run
  ProtectedRunner->>Comparator: execute in gated systemd unit
  Comparator-->>ProtectedRunner: return stdout, stderr, and exit result
  ProtectedRunner-->>ComparatorJob: write report and run bundle
  ComparatorJob->>Verifier: verify report and package hashes
  ComparatorJob->>Attestation: attest and verify Comparator report

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 8.16% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	⚠️ Warning	It includes Summary, Trust boundary, and Local verification, but most required template sections are missing or unfilled.	Add the missing template sections: Change class, TDD evidence, Mathematical verification, Provenance/licensing, Risk/recovery, and Review readiness details.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise and accurately describes the main change: protected official Comparator execution report.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/63-protected-comparator-run

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a620d9ea07

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/publication.yml:
- Around line 313-319: Update the workflow step invoking
comparator-protected-run.sh to pass the boundary output values through an env
block, then reference corresponding shell environment variables in the run
command instead of inline ${{ needs.boundary.outputs.* }} expansions. Preserve
the existing output-to-argument ordering and all other command arguments.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 418a5a39-29f8-4af4-a35e-f9ad471da919

📥 Commits

Reviewing files that changed from the base of the PR and between 42e5a50 and a620d9e.

📒 Files selected for processing (11)

.github/workflows/publication.yml
docs/decisions/ADR-0015-protected-official-comparator-execution.md
schemas/release/comparator-run-report-1.schema.json
scripts/comparator-protected-run.sh
src/cli/mod.rs
src/comparator_export.rs
src/comparator_run.rs
src/domain/comparator_run.rs
src/domain/mod.rs
src/lib.rs
tests/phase1_cli.rs

Add protected official Comparator execution report

a620d9e

chatgpt-codex-connector Bot reviewed Jul 22, 2026

View reviewed changes

Comment thread scripts/comparator-protected-run.sh Resolved

Pin protected landrun build toolchain

3558413

coderabbitai Bot reviewed Jul 22, 2026

View reviewed changes

Comment thread .github/workflows/publication.yml Resolved

Mnehmos added 2 commits July 21, 2026 23:50

Actively verify Comparator network isolation

8f824c3

Harden Comparator workflow output handling

45fc586

Mnehmos merged commit 3a75ddc into main Jul 22, 2026
6 checks passed

This was referenced Jul 22, 2026

Phase 6e: protected official Comparator execution report #63

Closed

Phase 6f: controlled attested Comparator evidence authority #65

Closed

Workstation handoff: resume MathOS 1.0 loop from publication attestation verification #25

Open

coderabbitai Bot mentioned this pull request Jul 22, 2026

Add controlled attested Comparator evidence authority #66

Merged

Navigation Menu

Repository navigation

Add protected official Comparator execution report - #64

Add protected official Comparator execution report#64
Mnehmos merged 4 commits into
mainfrom
feat/63-protected-comparator-run

Uh oh!

Review limit reached

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (2 warnings)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Uh oh!

Summary

Trust boundary

Local verification

Summary by CodeRabbit

Uh oh!

Uh oh!

Review limit reached

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (2 warnings)

Uh oh!

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant