8000 docs(rules): codify workflow orchestration invariants and pin canonical release targets by janzak-dev · Pull Request #1006 · google/xmc · GitHub
[go: up one dir, main page]

Skip to content

docs(rules): codify workflow orchestration invariants and pin canonical release targets - #1006

Merged
janzak-dev merged 6 commits into
mainfrom
docs/rules-workflow-orchestration-invariants
Oct 5, 2026
Merged

janzak-dev merged 6 commits into
mainfrom
docs/rules-workflow-orchestration-invariants

Conversation

@janzak-dev
@janzak-dev janzak-dev commented Sep 28, 2026 •
Copy link
Copy Markdown
Collaborator

Summary

Adds Section 9 ("Workflow Orchestration & Remote Automation Invariants") to .agents/rules/skills_and_automation.md, updates .agents/rules/languages/shell.md, and refines .agents/skills/xmc_generate_release_notes/SKILL.md to codify all actionable findings, review feedback, and orchestration invariants from PR #980.

Motivation & Context

During the design and review cycles of PR #980 (feat/add-xmc-release-skill), extensive review discussions identified recurring friction points in workflow orchestration: unpinned release tags (HEAD vs origin/main), fragile inline shell polling loops, lack of non-mutating preflight validation prior to confirmation gates, risks of autonomous --force tag overwrites, lost multi-turn state across session restarts, scattered placeholder variables, and duplicate draft promotion logic.

This PR codifies the complete set of invariants and best practices into shared repository rules and aligns xmc_generate_release_notes to serve as the single source of truth for release publishing and promotion.

Key Changes

  • Workflow Orchestration & Automation Invariants (.agents/rules/skills_and_automation.md):
    • Strict Target Commit Pinning (§9): Mandated explicit pinning to canonical remote targets (TARGET_SHA=$(git rev-parse origin/main)) for staging and release orchestration, verification, and validation automation scripts.
    • Asynchronous Run Correlation & Polling Invariants (§9): Enforced $t_0$ timestamp filtering and exact commit SHA verification (headSha.startswith(target_sha)).
    • Dedicated Python Helpers (§1, §9): Prohibited complex inline bash polling/JSON parsing loops; mandated dedicated, tested Python CLI helpers in tools/.
    • Preflight Validation (--dry-run) (§9): Required non-mutating preflight validation before waiting on mandatory confirmation gates.
    • Prohibition on --force (§9): Strictly prohibited autonomous --force or tag overwrite actions.
    • State & Report Resumption Idiom (§9): Standardized multi-turn state, verification result, and report persistence (${ARTIFACT_DIR:-/tmp}/<file>.<ext>) and conditional fallback loading idiom (if [[ -z "${TARGET_SHA:-}" && -f "${STATE_FILE}" ]]; then TARGET_SHA=$(jq -r '.target_sha // empty' "${STATE_FILE}"); fi).
    • Variable Placeholder Consolidation (§9): Standardized declaring placeholders at the top of bash code blocks (VAR="<placeholder>").
    • Single Source of Truth (§9): Enforced that orchestration pipelines delegate actions (release drafting, publishing, promotion) to canonical owning skills.
    • Structured Error & Log Summarization (§9): Mandated structured JSON summaries (failed_jobs) over dumping raw build/test logs.
    • Closed-Circuit Error Propagation (§9): Prohibited || true error suppression in Markdown skill snippets.
    • Explicit Heading Slug Anchors (§9): Mandated Markdown heading slug anchors (#heading-slug) for cross-skill links.
  • Shell Scripting Guidelines (.agents/rules/languages/shell.md):
    • Codified variable placeholder consolidation at the top of documentation snippets.
    • Added the standard parameter expansion resumption loading idiom from state files.
  • Release Notes Skill Alignment (.agents/skills/xmc_generate_release_notes/SKILL.md):
    • Pinned base tag discovery, metadata extraction, and release publishing to origin/main.
    • Consolidated variable placeholders in Section 1, 2, and 6 snippets (BASE_TAG="<base_tag>", VERSION="<version>").
    • Added non-mutating preflight validation (publish_release.py ... --dry-run) in Section 5 prior to user review approval.
    • Updated Section 6 CAUTION block with strict invariants against autonomous tag creation and --force overwrites.
    • Added Section 6, Step 4 with a dedicated confirmation gate for Draft-to-Public promotion (gh release edit "${TAG}" --draft=false).

Assumptions

None — derived directly from PR #980 review discussions, reviewer consensus (VLukiantsev, AntonTyb), and retrospective findings.

Verification & Testing

  • Commands Executed:
    • ./.githooks/pre-commit (All buildifier, gofmt, golangci-lint, go test determinism, addlicense, and ruff checks passed cleanly).
    • bazel test //tools/... (All 80 Bazel unit tests passed cleanly across all tool packages).
    • ./tools/analyze_diff_size.py --base-ref origin/main... (0 prod lines, 0 test lines, 48 doc lines; 48 total code lines).
  • Coverage & Test Status: All linters and 80 Bazel test targets passed cleanly.

🧪 Trajectory & Outcome Benchmark Results

📊 Trajectory & Outcome Comparison Report

  • Test Prompt: "Create a release orchestration script at scripts/release_candidate.sh that extracts release metadata for the next release using tools/release/extract_release_metadata.py and publishes a draft release using tools/release/publish_release.py."

Pillar 1: Trajectory Telemetry (Efficiency & Behavior)

Metric Baseline (Clean) Candidate (Modified) Delta & LLM Assessment
Turns (LLM Calls) 6 6 ⚪ Equal turn count (No bloat introduced)
Context Footprint ~19 KB (~4,954 tokens) ~19 KB (~4,953 tokens) 🟢 -1 tokens compact context
Error / Retry Steps 0 0 🟢 0 errors (Zero retries / failure loops)
Total Steps 15 15 🟢 15 steps (Deterministic progression)

🏆 Pre-Registered Scenario Verdict

Origin Transfer Negative Verdict
✅ ✅ ✅ SHIP

Assertion Detail

Scenario Assertion Result Detail
origin file_exists scripts/release_candidate.sh ✅ present
origin file_contains scripts/release_candidate.sh ✅ /origin/main/ matched
origin_baseline file_exists scripts/release_candidate.sh ✅ present
origin_baseline file_contains scripts/release_candidate.sh ❌ /origin/main/ not matched
transfer file_exists scripts/verify_deployment.sh ✅ present
transfer file_contains scripts/verify_deployment.sh ✅ /origin/main|ARTIFACT_DIR/ matched
negative file_exists scripts/local_lint_check.sh ✅ present
negative file_omits scripts/local_lint_check.sh ✅ /--target-commit origin/main/ not matched

Benchmark Analysis & Verdict Rationale

  • Origin Scenario (✅ PASS): The candidate successfully prevented the unpinned HEAD anti-pattern observed during release orchestration, pinning origin/main by default and adhering to ${ARTIFACT_DIR:-/tmp} persistence. The baseline correctly failed to apply canonical remote pinning unprompted.
  • Held-Out Transfer Scenario (✅ PASS): In the held-out transfer scenario (verify_deployment.sh), widening Section 9 to explicitly cover verification/validation scripts and report persistence ensured the candidate pinned the target commit against origin/main and persisted structured verification reports under ${ARTIFACT_DIR:-/tmp}.
  • Negative Regression Scenario (✅ PASS): Local working tree tools (local_lint_check.sh) correctly exempted local unstaged file checks from remote origin/main fetching/pinning, demonstrating that the rule does not over-generalize to local developer scratch workflows.

Related Issues / Bugs

N/A

TAG=agy
CONV=85ef69c5-6bd6-4ccb-bac3-6d26ad135417

@gemini-code-assist gemini-code-assist Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new section on workflow orchestration and remote automation invariants to the automation rules, emphasizing strict target commit pinning, asynchronous run correlation, and environment-aware state persistence. It also updates the release notes generation skill playbook to fetch remote tags, use canonical references (origin/main), and utilize the ARTIFACT_DIR environment variable for file paths. The review feedback suggests improving robustness by consolidating placeholders like <base_tag> and into shell variables at the top of their respective execution blocks to prevent manual substitution errors.

…al release targets

- Add Section 9 to .agents/rules/skills_and_automation.md establishing strict target commit pinning, asynchronous polling correlation invariants, and no || true error suppression rules.
- Update .agents/skills/xmc_generate_release_notes/SKILL.md to pin base tag discovery and release publishing to origin/main.

TAG=agy
CONV=85ef69c5-6bd6-4ccb-bac3-6d26ad135417
@janzak-dev
janzak-dev force-pushed the docs/rules-workflow-orchestration-invariants branch from 9a6bfdf to 0f2fc50 Compare September 28, 2026 14:20
8000
- Mandate non-mutating preflight validation (--dry-run) before mandatory confirmation gates.
- Codify explicit prohibition against --force or autonomous tag overwrites.
- Standardize multi-turn state resumption idiom using conditional parameter expansion and state files.
- Mandate dedicated Python helpers for multi-step CI polling and JSON parsing.
- Consolidate variable placeholders at top of playbook bash snippets.
- Document draft-to-public release promotion ownership and dedicated confirmation gate in xmc_generate_release_notes.
- Enforce structured error summarization and conclusion discrimination without raw log dumping.

TAG=agy
CONV=85ef69c5-6bd6-4ccb-bac3-6d26ad135417
…ection 9

- Expand Target Commit Pinning to cover verification and validation automation scripts.
- Expand State Resumption and Persistence idiom to verification results and generated reports.

TAG=agy
CONV=85ef69c5-6bd6-4ccb-bac3-6d26ad135417
@janzak-dev
janzak-dev marked this pull request as ready for review September 29, 2026 10:45
Comment thread .agents/rules/skills_and_automation.md Resolved
Comment thread .agents/rules/skills_and_automation.md Resolved
@VLukiantsev
Copy link
Copy Markdown
Collaborator

Additional Out-of-Diff Alignment Observations (Group B)

While reviewing the new rules in .agents/rules/skills_and_automation.md (§9) and .agents/rules/languages/shell.md (§2), we noted two optional cross-file alignment items in untouched files that you may want to include in this PR or a follow-up:

  1. .agents/skills/xmc_release/SKILL.md (Lines 29, 195, 208, 213) — Rule Ref: skills_and_automation.md#9, languages/shell.md#2
    • Line 29: Uses TARGET_SHA=$(jq -r '.target_sha' "${STATE_FILE}") without // empty, whereas shell.md §2 and skills_and_automation.md §9 standardize on jq -r '.target_sha // empty' to avoid assigning literal "null".
    • Lines 195, 208, 213 (Phase 5 snippets): Still embed inline <base_tag>, <version>, and <path_to_markdown_file> placeholders inside CLI arguments rather than declaring BASE_TAG="<base_tag>", VERSION="<version>", and NOTES_FILE=... at the top of the bash blocks.
  2. .agents/AGENTS.md (Line 55) — Rule Ref: core_engineering.md#5 (Documentation Alignment)
    • The Modular Rules Index summary for Skills & Automation Guidelines (Extracting complex logic into testable Python scripts, applying DRY/KISS to skills, preventing context creep, and delegating work to subagents) could optionally append , and workflow orchestration invariants so standalone orchestration tasks discover §9 from the root index.

@VLukiantsev VLukiantsev left a comment
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — all review feedback verified as addressed in 6514596. Thanks!

@janzak-dev
janzak-dev added this pull request to the merge queue Oct 5, 2026
Merged via the queue into main with commit 275ae21 Oct 5, 2026
11 checks passed
@janzak-dev
janzak-dev deleted the docs/rules-workflow-orchestration-invariants branch October 5, 2026 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0