8000 docs(rules): add harness encapsulation and sanitization rules, support pr title update by VLukiantsev · Pull Request #743 · google/xmc · GitHub
[go: up one dir, main page]

Skip to content

docs(rules): add harness encapsulation and sanitization rules, support pr title update - #743

Merged
VLukiantsev merged 2 commits into
mainfrom
docs/rules-update-harness-and-sanitization
Sep 2, 2026
Merged

VLukiantsev merged 2 commits into
mainfrom
docs/rules-update-harness-and-sanitization

Conversation

@VLukiantsev
@VLukiantsev VLukiantsev commented Sep 2, 2026 •
Copy link
Copy Markdown
Collaborator

Summary

Updates engineering guidelines and review tooling following iterative reflection:

  1. Adds Angular CDK Component Harness Standards to typescript_javascript.md enforcing 100% harness encapsulation (prohibiting raw DOM selectors/getEl in test specs) and internal DRY traversal helpers.
  2. Adds Environment & URL Sanitization Policy to github_authoring_and_prs.md prohibiting specific cluster names, internal domains, and test project IDs in PR descriptions.
  3. Enhances tools/pr_review/update_pr_description.py with an optional --title flag to update PR titles via the reliable GitHub REST API, accompanied by a comprehensive unit test suite in update_pr_description_test.py.

Key Changes

  • .agents/rules/languages/typescript_javascript.md: Added CDK Component Harness standards.
  • .agents/rules/github_authoring_and_prs.md: Added Environment & URL Sanitization Policy.
  • tools/pr_review/update_pr_description.py: Added --title support via GitHub REST API PATCH endpoint.
  • tools/pr_review/update_pr_description_test.py: Added 11 unit tests covering PR identifier parsing, validation errors, and title/body updates.
  • tools/pr_review/BUILD.bazel: Registered update_pr_description_test.

Assumptions

  • Requirements derived from iterative reflection and review comment trajectories.

Verification & Testing

  • Bazel Test Suite: bazel test //tools/pr_review/... (9/9 targets passed).
  • Unit Tests: python3 -m unittest tools/pr_review/update_pr_description_test.py (11/11 passed).
  • Pre-commit Hooks: Passed clean formatting, linting, and license headers.

🧪 Trajectory & Outcome Benchmark Results

📊 Trajectory & Outcome Comparison Report

  • Test Prompt: "Add an action button to the Angular Material work units table with unit tests asserting button attributes, and draft a PR description summarizing manual verification on a live GKE dev cluster."

Pillar 1: Trajectory Telemetry (Efficiency & Behavior)

Metric Baseline (Clean) Candidate (Modified) Delta & LLM Assessment
Turns (LLM Calls) 13 18 +5 turns (Thorough verification of candidate rules)
Context Footprint ~57 KB (~14,660 tokens) ~53 KB (~13,765 tokens) -4 KB (-895 tokens) (More concise tool payloads)
Error / Retry Steps 0 0 +0 errors
Total Steps 26 36 +10 steps

Pillar 2: Outcome & Quality Evaluation

🎯 LLM-as-a-Judge Outcome Scorecard

Evaluation Dimension Baseline Output Candidate Output Superior Output
Rule Adherence Leaked internal cluster environment details (... in project ...-env, zone us-central1-c) and raw log outputs into PR description. Strictly complied with the Environment & URL Sanitization Policy using generalized phrasing ("Live GKE Development Environment via deployment script") and zero dev leaks. 🟢 Candidate
Harness Architecture & DRY Identified harness methods but did not emphasize DRY cell traversal patterns. Explicitly highlighted 100% Harness Encapsulation and DRY row/cell traversal (getDataRows(), getUrlsCell()) avoiding raw DOM selectors. 🟢 Candidate
Completeness & Accuracy Complete analysis of table signals and templates. Complete analysis with explicit assumptions disclosure and compliant PR description. 🟢 Candidate

🏆 Final Benchmark Verdict: PASSED

Rationale: The candidate rules successfully eliminated environment leaks in drafted PR descriptions and enforced high-quality Angular CDK testing harness standards and DRY principles, while maintaining clean error-free execution and reducing overall context footprint.

TAG=agy
CONV=18ec6201-6e57-42c2-8346-1a62d59c3638

…tle update

Update engineering rules and review tooling following iterative reflection:
- Add Angular CDK Component Harness Standards to typescript_javascript.md enforcing 100% harness encapsulation and DRY traversal helpers in component tests.
- Add Environment & URL Sanitization Policy to github_authoring_and_prs.md prohibiting internal hostnames, cluster names, and test project IDs in PR descriptions and public commit messages.
- Enhance tools/pr_review/update_pr_description.py with optional --title argument, allowing title and/or body updates via GitHub REST API.
- Add unit test suite in update_pr_description_test.py and register py_test target in BUILD.bazel.

TAG=agy
CONV=18ec6201-6e57-42c2-8346-1a62d59c3638
@gemini-code-assist gemini-code-assist Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the update_pr_description.py tool to support updating a GitHub Pull Request's title in addition to its body, adds corresponding unit tests, and updates the repository rules and documentation. The review feedback suggests wrapping the file-reading logic in a try-except block to handle potential OSErrors gracefully, and resolving an inconsistency in how the title and body arguments are validated.

@VLukiantsev
VLukiantsev marked this pull request as ready for review September 2, 2026 12:30
…g title

- Wrap body_path.read_text in try-except OSError raising HandledError
- Check args.title with is not None to permit empty string titles
- Add unit tests for OSError handling and empty string title

Comment links:
- #743 (comment)
- #743 (comment)
@VLukiantsev
VLukiantsev added this pull request to the merge queue Sep 2, 2026
Merged via the queue into main with commit 7ffe8d8 Sep 2, 2026
11 checks passed
@VLukiantsev
VLukiantsev deleted the docs/rules-update-harness-and-sanitization branch September 2, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0