8000 feat(codebase-ai-readiness): binary signal verdicts, level gates, repo hygiene signals (v0.5.0) by krokoko · Pull Request #4 · krokoko/cairn · GitHub
[go: up one dir, main page]

Skip to content

feat(codebase-ai-readiness): binary signal verdicts, level gates, repo hygiene signals (v0.5.0) - #4

Draft
krokoko wants to merge 2 commits into
mainfrom
feat/readiness-binary-signals
Draft

krokoko wants to merge 2 commits into
mainfrom
feat/readiness-binary-signals

Conversation

@krokoko
@krokoko krokoko commented Sep 3, 2026 •
Copy link
Copy Markdown
Owner

Summary

Makes the readiness assessment reproducible and auditable, and fills gaps in what it checks.

Scoring model

  • Every row in the two category-definitions tables is a binary check: PASS / FAIL / N/A / NOT INSPECTABLE, each with a one-line evidence string. Those rows are the whole denominator; the category-specific references (feedforward, compound engineering, context engineering, spec-first, workflow artifacts) now define the PASS profile per row instead of carrying their own 0-100 bands.
  • Category score = PASS / (PASS + FAIL). N/A and NOT INSPECTABLE leave the denominator; a fully inapplicable category drops out and weights renormalize. Libraries and CLIs stop losing points for missing IaC or observability.
  • Autonomy level is set by gates: a level unlocks when at least 80% (round up) of its applicable transition requirements pass and every lower level is unlocked. Level-transition requirements were rewritten to be observable; practice items became informative notes. NOT INSPECTABLE requirements count as not passed and mark the level provisional when access would change it. The score-to-level table stays as a sanity check. Caps (spec-first, oracle ceiling, L5 hard prerequisites) only lower the level.
  • Assessments ground on a prior report: every check still runs, but identical evidence reuses the prior verdict and wording verbatim. Cuts run-to-run drift without permitting skipped checks.
  • Monorepos: app-scoped signals are checked per deployable app and recorded as n/m apps.

New signals

  • Documentation freshness as a smell (stale doc naming commands or paths that no longer exist), generated docs
  • Measured CI feedback time, required checks verified through gh api, secret and code scanning in CI
  • Pinned lockfile, dependency update automation, release automation and tag cadence, hygienic .gitignore
  • Operational legibility for deployed services: log redaction, health checks, error tracking and tracing, alert rules and runbooks (N/A for libraries)
  • Code-health scanners: complexity, dead code, duplication, unused dependencies, TODO-with-ticket lint
  • Instruction file validated by CI, agent co-authored commits in history
  • Issue templates and label taxonomy, backlog health
  • Circular-dependency and doubles-at-seams rows so every skill bullet has a table row

Report and roadmap

  • New sections: Level gates (per-level pass %), Signal evidence (one row per signal), Changes since last assessment. Category breakdown shows Pass/Applicable. Validator now requires Signal evidence; test fixture updated.
  • Levels table gains a "What agents can safely do" column; L3 is called out as the target most teams should reach first.
  • Roadmap gap analysis starts from failing gate requirements and marks pure file additions with [auto] so they can be batched first.

Housekeeping

Test plan

  • mise run build passes (markdownlint, manifests, cross-refs, reference integrity, 64 validator tests, gitleaks)
  • Run /assess-readiness on a library and on a deployed service; confirm N/A handling and the Level gates table
  • Run /assess-readiness twice on the same repo; confirm the second run reports no changes

bgagent added 2 commits September 3, 2026 15:36
… repo hygiene signals (v0.5.0)

Scoring: every signal is now PASS / FAIL / N/A / NOT INSPECTABLE with an
evidence string; category score = pass / applicable; N/A drops out of the
denominator and weights renormalize. Level is set by gates (80% of a
transition's requirements, cumulative) with the score table kept as a
sanity check. Assessments ground on a prior report to reduce drift.

New signals: doc freshness, measured CI time, branch protection via gh,
dependency update automation, secret/code scanning, pinned lockfiles,
release automation, gitignore hygiene, operational legibility (structured
logging, health checks, tracing, alerting, runbooks), code-health scanners,
instruction-file validation, agent co-authored commits, issue templates and
backlog health, monorepo app scoping.

Report gains Level gates, Signal evidence, and Changes since last
assessment sections; validator requires Signal evidence. Roadmap marks
[auto] file-only actions.
- Replace impression-based 0-100 bands in feedforward, compound engineering,
  context engineering, workflow artifacts, and spec-first references with
  PASS/FAIL profiles keyed to the category-definitions rows.
- Define the denominator: exactly the rows of the two category-definitions
  tables; add rows for circular dependencies and doubles at seams.
- Rewrite level transitions as observable requirements; move practice
  items to informative notes; L5 hard prerequisites stated as a cap.
- NOT INSPECTABLE requirements count as not passed in gates; level marked
  provisional when access would change it; 80% rounds up.
- Freshness is a smell (stale doc naming missing commands), not a gate;
  drop the 30-day threshold.
- Grounding re-runs every check and reuses verdicts on identical evidence.
- Branch protection scored once (CI reliability); dependency update
  automation moved beside pinned dependencies; dedupe structured logging.
- Roadmap gap analysis starts from failing gate requirements.
- CI duration from startedAt; monorepo wording; README category table.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0