Repository navigation
Conversation
Adds `examples/xmc_lite_job/`, an XMC Lite example that launches experiments directly on an existing GKE cluster with `xm_local.create_experiment` and the XManager SDK's built-in `xm_local.Kubernetes` client, without deploying the XMC control plane. - `launch.py` schedules one `batch/v1` Job per WorkUnit. - `setup.sh` fetches GKE credentials and creates the example venv. - `inspect_sqlite_state.py` decodes `~/.xmanager/experiments.sqlite3`. - `README.md` lists prerequisites, required IAM roles and Kubernetes RBAC permissions (verbs matched to the SDK's calls), local and Kubernetes representation, and teardown steps. - `requirements.txt` lists `python-dotenv`, which `launch.py` imports directly. First slice of a stack replacing #998. JobSet workloads and cluster provisioning follow in separate PRs. Bug: 555533775 TAG=agy CONV=328157cc-0621-4429-b4c2-19760fa03506
Adds JobSet submission to `examples/xmc_lite_job/`, matching the "experiment as a JobSet on GKE" scope of the story. Each WorkUnit becomes one non-restarting `jobset.x-k8s.io/v1alpha2` JobSet. - `k8s_workload_client.py` subclasses the SDK's Kubernetes client to build JobSet (or namespaced `batch/v1` Job) manifests, ensure the headless `experiments` Service, add an optional Kueue queue label, and support dry-run. - `k8s_workload_client_test.py` covers DNS-1035 naming, the JobSet manifest contract, Service idempotency (404/409), both submission paths and dry-run (17 tests). - `launch.py` adds `--workload_mode`, `--namespace`, `--local_queue` and `--dry_run`. - `README.md` documents JobSet mode, clarifies that Kueue is required only with `--local_queue`, ties the RBAC `delete` verbs to the new teardown steps, and adds 10000 the Full XMC comparison plus an explicit "Findings for XMC Lite Requirements" section. Bug: 555533775 TAG=agy CONV=328157cc-0621-4429-b4c2-19760fa03506
There was a problem hiding this comment.
Code Review
This pull request introduces support for Kubernetes JobSet workloads, custom namespaces, Kueue queue integration, and dry-run mode in the XMC Lite example. It adds a new k8s_workload_client module that overrides the default XManager Kubernetes client, along with corresponding unit tests and updates to the launcher script and documentation. The review feedback suggests implementing defensive checks in k8s_workload_client.py to prevent potential AttributeError exceptions when accessing v1_job.spec and v1_job.metadata.
- Close the SQLite connection with `contextlib.closing` and require a regular file (`os.path.isfile`) in `inspect_sqlite_state.py`; `sqlite3.connect` used as a context manager only commits or rolls back and never closes the connection. - Report a missing database as `app.UsageError` instead of a traceback, and update the README note. - Reject `--num_work_units < 1` and `--duration < 0` with absl `lower_bound` in `launch.py`. - Remove the `rm ~/.xmanager/experiments.sqlite3` teardown step; the file is shared by all `xm_local` experiments on the machine, so the README now explains that instead. Comment link: #1047 (comment) Comment link: #1047 (comment) Comment link: #1047 (comment) Comment link: #1047 (comment) Comment link: #1047 (comment) Bug: 555533775 TAG=agy CONV=328157cc-0621-4429-b4c2-19760fa03506
# Conflicts: # examples/xmc_lite_job/README.md # examples/xmc_lite_job/launch.py
… example - Raise `xmanager` to `>=0.8.2`. Up to 0.8.1 the SDK's Kubernetes client shell-quoted container args with `shlex.quote`, so `python3 -c '<script>'` evaluated a string literal and exited 0 without running the workload. 0.8.2 is also the version the example is tested with. - README: mark `KUBECONFIG` optional (default `~/.kube/config`) with a non-personal example path, write the `.env` snippet in the root `.env.example` `KEY="VALUE"` style, and load `.env` with `set -o allexport` (as in INSTALL.md) so `kubectl` sees `KUBECONFIG`. Comment link: #1047 (comment) Comment link: #1047 (comment) Bug: 555533775 TAG=agy CONV=328157cc-0621-4429-b4c2-19760fa03506
# Conflicts: # examples/xmc_lite_job/README.md
…mode # Conflicts: # examples/xmc_lite_job/README.md # examples/xmc_lite_job/launch.py
pachopepe
marked this pull request as draft
October 5, 2026 07:47
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applie
2EDB
d or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a JobSet workload mode to
examples/xmc_lite_job/. Each WorkUnit is submitted as one non-restartingjobset.x-k8s.io/v1alpha2JobSet (the default), with optional Kueue admission, a configurable namespace and dry-run. PR 2 of 3 in the stack replacing #998.Motivation & Context
Comment #2 on b/555533775 (now tracked as follow-up b/569821527) sets the scope as "an experiment as a JobSet on GKE". The SDK's built-in client only creates
batch/v1Jobs in thedefaultnamespace, so a thin subclass is needed. This PR also states that gap explicitly in a new "Findings for XMC Lite Requirements" README section, to feed the follow-up requirements discussion.Key Changes
k8s_workload_client.py: subclassesxmanager.cloud.kubernetes.Clientto build JobSet (or namespaced Job) manifests (maxRestarts: 0,backoffLimit: 0,restartPolicy: Never), ensure the headlessexperimentsService, add the Kueue queue label, and support dry-run. Same code as reviewed in feat(examples): add xmc lite kubernetes job and jobset example #998.k8s_workload_client_test.py(new, 17 tests): DNS-1035 naming, the JobSet manifest contract, Service idempotency (404/409/500), JobSet and Job submission paths, dry-run, client binding.launch.py: adds--workload_mode,--namespace,--local_queue,--dry_run.README.md: JobSet sections; Kueue documented as required only with--local_queue; RBACdeleteverbs tied to the teardown steps; Full XMC comparison; explicit SDK findings.Assumptions
unittestfrom the example venv rather than Bazel:examples/has no Bazel targets, andexamples/chart_artifact_job/training_metrics_test.pyfollows the same pattern. This is a documented deviation from the Bazel rule in.agents/rules/languages/python.md.Verification & Testing
python -m unittest discover -s examples/xmc_lite_job -p '*_test.py' -t .(17 tests, OK);python -m examples.xmc_lite_job.launch --dry_run;./.githooks/pre-commit(pass).k8s_workload_client.pyare exercised; no Bazel coverage, because the example is not a Bazel target.Related Issues / Bugs
Bug: 569821527
TAG=agy
CONV=328157cc-0621-4429-b4c2-19760fa03506