Repository navigation
RoboDojo-Sim: a program as the policy (first implementation) - #102
Draft
ruiling-smartbear wants to merge 33 commits into
Draft
ruiling-smartbear wants to merge 33 commits into
ruiling-smartbear wants to merge 33 commits into
Conversation
examples/aspire_libero/ adds LIBERO's Object, Spatial, Goal, Long (libero_10) and LIBERO-90 suites as tasks of ASPIRE's harness (cap-harness --benchmark libero-pro): 130 task rows, run.yaml (the four ten-task suites), run_90.yaml, smoke.yaml and a launcher that starts the Robosuite example's on this folder. The Robosuite example's engine now takes the benchmark from the task row: the venv that holds cap-harness, the init mode (LIBERO: seeded, so a seed is a scene of its own), the simulator's environment and the benchmark's docs page (LIBERO's without the section that names ASPIRE's worked programs per task). A row without the field is Robosuite's, as before. prepare.py's seeds are presets (--preset aspire|smoke) and its row and split writing is shared.
…out's pages The LIBERO configs granted $ASPIRE_ROOT/docs/libero-pro.md to the task sandbox, whose "Task examples" section names ASPIRE's worked programs per task; only the workspace copy was cut. launch.sh now stages the docs outside the checkout (stage_docs.py) and the Codex configs of both examples grant those (agent.params.aspire_docs). A benchmark page without its cut is refused rather than copied whole. LIBERO's demonstration datasets are hidden from the simulator's sandbox.
smoke_luna.yaml is smoke.yaml with GPT-5.6-Luna acting and coordinating on the Codex ChatGPT login. The launcher asks for FIREWORKS_API_KEY only when the config's models are Fireworks'.
examples/robodojo_sim/ adds RoboDojo-Sim's forty-two bimanual tasks. A Codex session writes policy.py (act(obs) -> actions), develops it on the task's development layout set through a simulator service on the host, and the judge scores it on the held-out layout sets with RoboDojo's own success check. RoboDojo evaluates the harness's XPolicyLab policy (utils/policy/hermit_program): its server never imports the program but runs it as a worker in a bubblewrap sandbox with no network that reads nothing of RoboDojo's checkout, which holds the layouts, the success conditions and the results; the episode loop that runs in the simulator's process is the harness's, and the engine refuses an installed policy that differs from it. The ASPIRE example's simulator service separates its seed rules from what it runs (_validate, _launch), which this example's service replaces.
…nor read the run's output An independent review found that a program could make RoboDojo abandon a layout unscored (a partial action raises in its simulator; a number that overflows float32 passed the finite check) and that the judge divided by the layouts that had a result. - Every action that reaches the simulator is whole and bounded: both arms of one mode and both grippers, what the program left out filled from the observation, numbers checked after the cast. - The judge scores the first eval_layouts of each held-out set against that fixed count: a layout with no result is a failure, and its replacement is not counted. - The program's worker reads a directory holding the program alone, not the run's output (where a held-out evaluation writes its results as it runs); the sandbox's own check finds the checkout and the run's directories missing inside. - RoboDojo's log and the sandbox record stay with the service, not in the workspace; a stopped run's resume manifest is removed from the checkout; the service stops its simulators; one simulator per GPU across the service and the judge (a file lock). - Host-side reads of a session's files and the program check take anything.
…t on an Object task
…ep's init mode is tested
…r on the host; run against RoboDojo
… service starts no simulator - worker.py: fd 0 is /dev/null and fd 1 stderr once the channel is taken, so a child process or a C library the program uses cannot write into the answer or read the observations. - dojo.Engine: a stop is checked where the simulator starts, with the start and the listing under one lock (the ASPIRE service's rule), so a run that waited for the GPU starts nothing after stop(); the GPU's lock is .cache/hermit-locks/gpu<N>.lock in the checkout, opened without truncating it; records.json says whether the run was stopped at its timeout, and a run's directories are made only after its paths are accepted. - dojo_service: a run stopped at its timeout answers 124 and says so, as the ASPIRE service does; otherwise RoboDojo's own exit code. - The stand-in takes an action as RoboDojo's take_action does: one mode, joint or end effector, with both arms of it and both grippers; the sandboxed test runs an end-effector program. Tests for stop, timeout and the service's answers. - The interface page says the program is loaded again after an episode it failed; CHANGELOG and examples/README state that one program has been evaluated on the real simulator.
LIBERO's grasp backend is on its fixed port 8115 (ASPIRE's SERVICE_PORTS for libero-pro); without it every generate_grasps call fails. The launcher checks it as the Robosuite launcher checks the other services.
… reported; the judge's own GPU; a cache per GPU
# Conflicts: # examples/robodojo_sim/utils/agent/dojo.py
…leanup awaited at shutdown; one port chosen at a time
# Conflicts: # tests/selection.py
…ver; any host failure reaches the session without paths; a stop waits only for started runs; a container left behind is logged
…s protocol; the method page names LIBERO's docs, aspire_docs and verifier seeds
…; an evaluation with no result is the host's failure; a test split for eval; the docs state the smoke and the program's memory across episodes
…'s tasks, the aspire method folder and a prompt_only one with LIBERO's own acting prompt; the Robosuite launcher runs another method folder's configs
# Conflicts: # CHANGELOG.md # examples/README.md
…it; its workspace leaves out ASPIRE's search page and assignment; the prompt is named method-neutral; the step limit is stated only when set
# Conflicts: # examples/README.md
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
Adds RoboDojo-Sim (arXiv:2607.04434; 42 bimanual ARX X5 tasks in Isaac Sim, five capability dimensions) as a task. A first implementation, written from RoboDojo's and XPolicyLab's source and tested against a stand-in; on the real simulator the engine, the service, the judge and the smoke run, and no training run has been made. Stacked on #100 (it uses that PR's simulator service); the change of this PR is
examples/robodojo_sim/and the service's_run.RoboDojo evaluates a policy served through XPolicyLab. Here the policy is a program: a Codex session writes
policy.py(act(obs) -> actions), develops it on the task's development layouts throughbin/sim, and the judge scores it on held-out layouts with RoboDojo's own success check.examples/robodojo_sim/utils/policy/: the harness's XPolicyLab policyhermit_program(model.py,deploy.py,deploy.yml, launch scripts), the program'sworker.py,install.py.utils/agent/dojo.py: the engine (onescripts/robodojo.sh eval, its result per layout; withROBODOJO_IMAGE, the simulator in RoboDojo's container and the policy server on the host,split_eval.sh, for a host on which Isaac Sim does not install);dojo_service.py,sim_cli.py: the simulator service andbin/sim;task.py, the prompt and the interface page.utils/prepare.py: the 42 tasks; development on layout set 2 (first 10 layouts), held-out on sets 0 and 1, each with as many layouts as RoboDojo evaluates a set (50, or 25 for the twelve tasks its_task.ymlsets to 25). This is not RoboDojo's leaderboard protocol, which scores all three sets; the README says so. The configs have a test split (the same tasks), so./launch.sh eval --adapter final(orroot) runs them again with an adapter given.prompt_only/:run.yaml,smoke.yaml,smoke_luna.yaml,launch.sh(prompt-only template, built-in Codex Evolver).examples/aspire_robosuite/utils/agent/sim_service.py:_runkeeps the shared rules (assigned seeds, the output moved without following a link) and calls_validate/_launchfor what is run; no behavior change for ASPIRE.What a program can reach (README, "What a program can reach"): RoboDojo's success check reads privileged state, and its checkout holds every set's layouts, the success conditions and the results beside the policy server. The server never imports the program: it runs it as a worker in a bubblewrap sandbox with no network that reads only the worker and a directory holding the program alone (not the run's output); every action that reaches the simulator is whole and bounded, and the judge's denominator is the fixed number of held-out layouts, so a program cannot drop a failing layout from its score; the episode loop in the simulator's process is the harness's, and the engine refuses an installed policy that differs from it; a session runs only its development set.
Validation
python -m pytest examples/robodojo_sim tests/methods/aspire -q: CI (tests.yml, Linux with bubblewrap) passes on the head.utils/tests/test_robodojo.py:FAKE_EVAL) runs the installed policy's realModel, worker anddeploy.pyunder the real sandbox. Under it: a program is scored by what it does; a program that tries to read a layout file of the checkout, write intoeval_result/, open a socket or find the run's output gets none of them and fails its layouts; a partial action reaches the stand-in whole; a number that overflows float32 never reaches it; a program that raises holds the joints and fails; an installeddeploy.pythat grants success is refused.split_eval.shruns with a stand-in for docker: the server's and the client's arguments, the mounts, the server stopped with the script, a container that does not start reported.f57e163with its pinned XPolicyLab5d0be14, the image of RoboDojo's Dockerfile and its Assets,ROBODOJO_IMAGEset, onstack_bowlswith programs written by hand:(480, 640, 3)image and the instruction; its exception fails both of two layouts while the robot holds. The worker ran in its sandbox.bin/sim(smoke preset): a session's program runs on its two development layouts (0/2, RoboDojo's result and videos delivered into the workspace, RoboDojo's log and the sandbox record kept out of it, no host path or held-out name in the workspace's files); a third layout is refused.0/2 held-out layouts, score 0.0.eval_result/or root-owned file.ROBODOJO_CONFIG=smoke_luna.yaml ./launch.sh, GPT-5.6-Luna on the machine's ChatGPT login, 83 minutes, 6.2M tokens): the run completes (two steps: session, judge, evolver,finaltag). Both tasks score 0/2 on the held-out layouts. The first session ended while its evaluation was still running; the Evolver's revision ofADAPTER.mdtells a session to wait for an evaluation and read its result, and the second session waited for all nine of its evaluations. The sessions ran only development layouts and no command that names the checkout, the held-out runs or the service's files; their sandbox records show enforcement on, the network isolated and none of those paths; the judged programs import only numpy andre; the worker's sandbox probe finds the checkout and the run's output absent and the network blocked.bin/simand the prompt now say that an evaluation takes minutes and returns when it has ended.dojo.NoResult; the session gets exit 3 without host paths), not scored as failed layouts; one stopped at its timeout is scored.scripts/robodojo.sh evalon a native install. The configs' timeouts and budgets are not measured. No program has solved a task: the default observation has no depth or calibration.Review
examples/robodojo_sim/utils/tests/test_robodojo.py; docs:examples/robodojo_sim/README.md,examples/README.md,CHANGELOG.md.hermit.sandbox.Sandbox(dojo.Engine._worker). RoboDojo's own processes (the policy server, Isaac Sim) run on the host unconfined, as the judge does: they are RoboDojo's and the harness's code, and need the GPU and the checkout. The worker's sandbox record of a real run is as described in Validation.play_tic_tac_toe, whose three sets are the same layouts (README). A second review, of the container path at2a2a72f, found no way for a session or a program to reach the policy server's port, the container's mounts or another run's result. Its findings, fixed ine13e654: a stopped run left root-owned files the engine could not remove; a missing image or a container that did not start was scored as failed layouts rather than reported; a killed script could leave its container; a relative cache path; the script itself was untested. In the same commit, found by running two evaluations at once: a cache per GPU (two simulators sharing one failed to build the scene). A third review, of2a2a72f..fc04e22, found the merge coherent and nothing that widens what a session or a program can read or influence. Its findings, fixed in9796dc9: a container or policy server that did not start reached the session as a refusal with a host path (now exit 3 and a message without one); a service that exits right after a stop could skip a run's cleanup (now awaited); the container's removal could raise over a finished evaluation; with the judge on its own GPU two evaluations could choose the same port (the choice is now made one at a time, and two evaluations started together on two GPUs each ran their own program). Not changed:bin/simreturns RoboDojo's own exit code; run by hand outside a process group of its own,split_eval.shstops only the policy server's shell. A fourth review, of9796dc9and the merge of LIBERO (and LIBERO-Pro) as a benchmark, with ASPIRE and prompt-only #100's latest, found the exit-3 message, the awaited cleanup and the port order as described, and the merge touching nothing RoboDojo uses. Its findings, fixed inf6fbc87: the policy server inherited the port lock, so a server that hung and outlived a failed start held it for every later evaluation (the server no longer gets its descriptor and the wait is bounded at 15 minutes); a host failure other than a start (an unreadable result, a policy installed from another commit) reached the session with host paths (now exit 3 with a message without them); a stop waited for runs still queued for their GPU; a container thatdocker rmcould not remove left no trace (now in the run's log); and tests for each. Not changed: if giving the container's files back takes longer than the minute a stop allows, root-owned files can stay in the checkout'seval_result/.