Repository navigation
feat(tools): make simple_job verification exhaustive with full protobuf resource and priority assertions - #908
Merged
Conversation
…uf resource and priority assertions TAG=agy CONV=606f1e41-6785-4137-b85b-cd0f56237339
This was referenced Sep 17, 2026
There was a problem hiding this comment.
Code Review
This pull request refactors and modularizes the XMC nightly test validation framework. It updates simple_job_verify.py to support customizable multi-unit job verification and introduces several new validation helpers in xmc_entities_validator.py for priority, resource requests, metadata, and jobsets, along with corresponding unit tests. Feedback is provided to add an explicit assertion in _assert_all_work_unit_specs to ensure the number of retrieved raw work units matches the expected count, preventing silent failures.
janzak-dev
temporarily deployed
to
xmc-nightly
September 17, 2026 11:57 — with 
GitHub Actions
Inactive
janzak-dev
had a problem deploying
to
xmc-nightly
September 17, 2026 11:58 — with
GitHub Actions
Failure
…ults take precedence
janzak-dev
temporarily deployed
to
xmc-nightly
September 17, 2026 12:02 — with
GitHub Actions
Inactive
janzak-dev
temporarily deployed
to
xmc-nightly
September 17, 2026 12:03 — with
GitHub Actions
Inactive
…ple_job_verify Comment link: #908 (comment)
janzak-dev
marked this pull request as ready for review
September 17, 2026 12:22
andrzejfudro
previously approved these changes
Sep 17, 2026
…ork_unit_priority
janzak-dev
force-pushed
the
feat/exhaustive-simple-job-verify
branch
from
September 17, 2026 13:08
ccc8b23 to
72c069b
Compare
janzak-dev
temporarily deployed
to
xmc-nightly
September 17, 2026 13:20 — with
GitHub Actions
Inactive
janzak-dev
temporarily deployed
to
xmc-nightly
September 17, 2026 13:22 — with
GitHub Actions
Inactive
andrzejfudro
approved these changes
Sep 17, 2026
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🥞 Stack Navigation
feat(tools): make simple_job verification exhaustive with full protobuf resource and priority assertions👈feat(ci): add unified nightly examples e2e pipeline workflowfeat(ci,tools): expand nightly examples suite to all 12 example workloads🏛️ Role in PR Stack Architecture
This PR serves as Layer 1 (Exhaustive simple_job Protobuf Verification & Foundation) within the nightly examples testing stack.
xmc-nightly.simple_job_verify.pyandxmc_entities_validator.pywith 100% full protobuf spec assertions (PriorityHIGH,cpu-queue, worker namingdemo-worker-1..3, replicas 1, CPU 500m, memory 512 MiB, tags, notes, completed state) and hermetic unit tests.Run
https://github.com/google/xmc/actions/runs/35218884252/job/105194293789
Summary
Implements exhaustive protobuf assertions for the
simple_jobverification CLI (tools/nightly_tests/simple_job_verify.py), decomposing validation helpers inxmc_entities_validator.pywith strict <= 25 line limits and adding unit tests.Motivation & Context
simple_jobverification script asserts not just that the WorkUnit completed, but that all specification parameters (PriorityHIGH,cpu-queue, worker namingdemo-worker-1..3, replicas 1, CPU 500m, memory 512 MiB, tags, and notes) strictly match expectations.xmc_entities_validator.py(validate_jobset_topology,validate_resource_requests,validate_work_unit_priority,validate_work_units,wait_for_experiment_state) to maintain high maintainability and testability.Key Changes
tools/nightly_tests/simple_job_verify.py):HIGH), scheduling queue (cpu-queue), worker naming (demo-worker-1..3), replicas (1), CPU requests (500m), memory requests (512 MiB), tags, and notes.tools/nightly_tests/xmc_entities_validator.py):tools/nightly_tests/):xmc_entities_validator_test.py: Adds comprehensive unit tests covering priority, resource requests, metadata, and JobSet validation.common.py: Adds common flag definitions for CLI runners (expected_queue,timeout_sec,expected_work_units,expected_title_pattern).BUILD.bazel: Registers test targets.Assumptions
None — requirements were fully specified.
Verification & Testing
bazel test //tools/nightly_tests:all(2/2 test targets passed)./.githooks/pre-commit check(100% clean)TAG=agy
CONV=606f1e41-6785-4137-b85b-cd0f56237339