8000 feat(skills): add xmc_release skill for end-to-end release orchestration by janzak-dev · Pull Request #980 · google/xmc · GitHub
[go: up one dir, main page]

Skip to content

feat(skills): add xmc_release skill for end-to-end release orchestration - #980

Merged
janzak-dev merged 18 commits into
mainfrom
feat/add-xmc-release-skill
Sep 29, 2026
Merged

janzak-dev merged 18 commits into
mainfrom
feat/add-xmc-release-skill

Conversation

@janzak-dev
@janzak-dev janzak-dev commented Sep 24, 2026 •
Copy link
Copy Markdown
Collaborator

Summary

Adds the new xmc_release agent skill (.agents/skills/xmc_release/SKILL.md) providing an end-to-end operational playbook for orchestrating XMC releases across 5 sequential phases, adds the tools/release/dispatch_and_watch.py workflow dispatch and monitoring CLI tool with accompanying Bazel tests, and registers the skill in .agents/AGENTS.md.

Motivation & Context

Preparing and publishing an XMC release involves coordinating multiple GitHub Actions workflows, failure-stop checks, visual verification, structured release notes synthesis, and release publishing. Standardizing this workflow into a documented agent skill with a dedicated helper CLI tool ensures deterministic execution, enforces strict confirmation gates before tag and release publishing, and prevents broken builds from reaching users.

Key Changes

  • New Skill (.agents/skills/xmc_release/SKILL.md): Added operational playbook defining:
    1. Triggering nightly-build.yml deployment on main.
    2. Monitoring deployment with mandatory abort-on-failure handling and concurrency cancellation recognition.
    3. Running nightly-examples-e2e.yml test suite with mandatory abort-on-failure handling and single approved flake retry.
    4. Generating and surfacing nightly-visual-snapshot.yml golden UI gallery links with GitHub Pages deployment step verification.
    5. Delegating structured release notes synthesis, interactive user review, preflight validation (--dry-run), and tag/release publishing to xmc_generate_release_notes.
  • Workflow Dispatch Tool (tools/release/dispatch_and_watch.py): Added CLI tool to dispatch GitHub Actions workflows, poll for registered runs with commit SHA verification, watch outcomes to completion, and persist release state.
  • Unit Tests (tools/release/dispatch_and_watch_test.py): Added unit test suite covering run dispatch, SHA mismatch detection, cancellation handling, GitHub Pages URL extraction, and state file updates.
  • Build & Package Metadata (tools/release/BUILD.bazel, tools/release/README.md): Configured Bazel targets for dispatch_and_watch and documented CLI usage, package architecture, and testing.
  • System Registration (.agents/AGENTS.md): Registered XMC Release Skill in the Task Intent Routing Matrix and Modular Rules Index.

Assumptions

None — requirements were fully specified.

Verification & Testing

  • Commands Executed:
    • bazel test //tools/release:all (All 8 release tool tests passed cleanly).
    • ./.githooks/pre-commit (All buildifier, gofmt, golangci-lint, go test determinism, addlicense, and ruff checks passed cleanly).
    • ./tools/analyze_diff_size.py --base-ref origin/main... (387 prod lines, 531 test lines, 264 doc lines; 918 total code lines).
  • Coverage & Test Status: All Bazel tests and pre-commit checks pass cleanly.

Related Issues / Bugs

N/A

TAG=agy
CONV=85ef69c5-6bd6-4ccb-bac3-6d26ad135417

Introduce the `xmc_release` skill guiding agents through the 7-phase XMC
release process:
1. Trigger nightly deployment workflow (`nightly-build.yml`).
2. Watch deployment and abort on failure.
3. Trigger examples E2E test suite (`nightly-examples-e2e.yml`) and abort on failure.
4. Trigger visual snapshot gallery (`nightly-visual-snapshot.yml`) and share URL.
5. Generate structured release notes using `xmc_generate_release_notes`.
6. Iteratively review release notes and version bump with user.
7. Publish draft GitHub release and tag upon explicit user confirmation.

TAG=agy
CONV=85ef69c5-6bd6-4ccb-bac3-6d26ad135417
@gemini-code-assist gemini-code-assist Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds a comprehensive playbook (SKILL.md) to orchestrate the end-to-end XMC release workflow across seven sequential phases. The review feedback suggests removing a hardcoded local repository path to ensure environment independence, and adding a short delay (sleep 5) after triggering asynchronous GitHub workflows to guarantee that the correct, newly created run IDs are retrieved.

Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
- Make repository working directory requirement generic instead of hardcoded `~/Documents/XMC`.
- Add a 5-second sleep buffer after `gh workflow run` across Phases 1, 3, and 4 to prevent fetching stale/previous workflow run IDs before GitHub registers the new runs.

Comment links:
- #980 (comment)
- #980 (comment)
- #980 (comment)
- #980 (comment)

TAG=agy
CONV=85ef69c5-6bd6-4ccb-bac3-6d26ad135417
…l pythonpath

- Eliminate prerequisite requirement to checkout or pull local `main`, allowing the release skill to run from any branch or git worktree without conflicts.
- Ensure helper tool invocations specify `PYTHONPATH=.`.

TAG=agy
CONV=85ef69c5-6bd6-4ccb-bac3-6d26ad135417
@janzak-dev
janzak-dev marked this pull request as ready for review September 25, 2026 08:19
AntonTyb
AntonTyb previously approved these changes Sep 25, 2026
@janzak-dev janzak-dev added the priority::3-medium needs review. label Sep 25, 2026
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
@VLukiantsev
Copy link
Copy Markdown
Collaborator

Out-of-diff observations (related to the inline comments above)

These are outside this PR's diff, so they can't be posted inline. They directly affect how safe xmc_release is and may be worth a follow-up PR:

# Location Priority Observation
B2 .agents/skills/xmc_generate_release_notes/SKILL.md §1 (L31) and §6 (L184) Blocker (release safety) Base-tag discovery is anchored on HEAD (git describe ... HEAD^), and the documented extract_release_metadata.py / publish_release.py commands pass no --target-ref / --target-commit, so both default to HEAD. This is the root cause of the inline comment on L188: any skill that delegates here inherits "tag whatever is checked out locally".
B3 same, §6 Actionable No publish_release.py --dry-run step before the confirmation gate, no explicit ban on --force when the tag already exists, and no owner for draft→public promotion (see the inline comment on L191).
B4 .github/workflows/nightly-visual-snapshot.yml L124 Clarity continue-on-error: true on deploy-pages hides Pages deploy failures from any caller, including gh run watch --exit-status (see the inline comment on L173). Consider surfacing the failure, e.g. with a final step that fails the job when page_url is empty.

- Register XMC Release Skill in .agents/AGENTS.md routing matrix and modular index
- Correlate workflow runs by DISPATCH_TIME, event, and verify against TARGET_SHA
- Persist TARGET_SHA and state paths in Prerequisites and direct entry
- Distinguish cancelled vs failure outcomes and offer single rerun for E2E flakes
- Tie E2E tests to Phase 1 deployment and inspect Pages deploy step conclusion
- Link heading anchors in Phase 5 to xmc_generate_release_notes/SKILL.md
- Enforce --target-commit and --target-ref pinned to TARGET_SHA with --dry-run preflight
- Remove duplicate draft promotion step from xmc_release

TAG=agy
CONV=85ef69c5-6bd6-4ccb-bac3-6d26ad135417
…d_watch helper

- Implement tools/release/dispatch_and_watch.py with Python 3.14+ syntax, strict type hints, and PEP 8
- Add Bazel test target and hermetic unit test suite in dispatch_and_watch_test.py
- Support --workflow, --target-sha, --actor, --find-existing, --state-file, and --watch
- Update .agents/skills/xmc_release/SKILL.md to invoke the Python tool across phases 2, 3, and 4
- Extend HandledError and run_tool to support custom exit codes and error details
Comment thread tools/release/dispatch_and_watch.py Resolved
Comment thread tools/release/dispatch_and_watch.py Resolved
Comment thread tools/release/dispatch_and_watch.py Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
Comment thread tools/release/dispatch_and_watch.py Resolved
@VLukiantsev
Copy link
Copy Markdown
Collaborator

Review Summary & Documentation Observations (Group B)

Great progress on addressing the prior review comments! The introduction of tools/release/dispatch_and_watch.py and its hermetic Bazel unit tests (//tools/release:dispatch_and_watch_test) substantially improves the determinism of the release workflow.

The 11 previously open review threads have all been verified and marked as resolved on GitHub. 5 new inline feedback items have been posted regarding function decomposition, polling condition safety, DRY subprocess helpers, and error propagation.

Additionally, here are two observations on files outside the immediate diff:

1. Synchronize tools/release/README.md with the new tool

  • tools/release/dispatch_and_watch.py is a newly introduced tool binary in tools/release/, but tools/release/README.md was not updated in this PR.
  • Per rules/core_engineering.md#5 and rules/languages/markdown.md, please add a section documenting dispatch_and_watch.py (CLI flags, usage examples) and register it in Section 6 ("Package Architecture") of tools/release/README.md.

2. E2E Closed-Circuit Image Tag Verification

  • Currently, Phase 3 runs nightly-examples-e2e.yml after Phase 1 finishes on main. As an optional future hardening improvement, Phase 3 could explicitly verify that the cluster's deployed image tag matches the tag generated in Phase 1 before running the tests.

Comment thread tools/release/BUILD.bazel Resolved
Comment thread tools/release/dispatch_and_watch.py Resolved
Comment thread tools/release/dispatch_and_watch.py Resolved
Comment thread .agents/skills/xmc_release/SKILL.md Resolved
@VLukiantsev VLukiantsev left a comment •
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! All review comments across the skill definition, tools/release/dispatch_and_watch.py CLI, Bazel targets, unit tests, and documentation have been cleanly addressed.

@AntonTyb AntonTyb left a comment
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will release titles/names be consistent with this?

@janzak-dev
Copy link
Copy Markdown
Collaborator Author

will release titles/names be consistent with this?

@AntonTyb They should be - when it comes to creation of the draft notes we defer this to the preexisting SKILL so there should be no naming drift.

@janzak-dev
janzak-dev requested a review from AntonTyb September 28, 2026 13:46
janzak-dev added a commit that referenced this pull request Sep 29, 2026
- Mandate non-mutating preflight validation (--dry-run) before mandatory confirmation gates.
- Codify explicit prohibition against --force or autonomous tag overwrites.
- Standardize multi-turn state resumption idiom using conditional parameter expansion and state files.
- Mandate dedicated Python helpers for multi-step CI polling and JSON parsing.
- Consolidate variable placeholders at top of playbook bash snippets.
- Document draft-to-public release promotion ownership and dedicated confirmation gate in xmc_generate_release_notes.
- Enforce structured error summarization and conclusion discrimination without raw log dumping.

TAG=agy
CONV=85ef69c5-6bd6-4ccb-bac3-6d26ad135417
@janzak-dev
janzak-dev added this pull request to the merge queue Sep 29, 2026
Merged via the queue into main with commit fd2942c Sep 29, 2026
11 checks passed
@janzak-dev
janzak-dev deleted the feat/add-xmc-release-skill branch September 29, 2026 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority::3-medium needs review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

0