Repository navigation
boot-agents: codex resume-on-relaunch, with a verified fallback - #18
Merged
Merged
Conversation
Ported from the internal fleet deployment this package was extracted from.
A codex-runtime agent had no resume path at all. claude agents resume with
`claude --resume <name>`, but a codex agent launched a bare `codex`, so every
relaunch started genuinely fresh: a crash, a manual restart, or an incidental
relaunch as a side effect of another agent's boot silently discarded the whole
conversation. It bit live, where an unrelated boot run reset an agent mid-task
and read as a wedged pane rather than as a lost session.
WHAT LANDS
codex_latest_session_for_workspace scans the Codex sessions tree
mtime-descending, reads only the first line of each rollout file (a
session_meta record), and applies three filters that are all load-bearing: an
interactive originator, a user-owned thread, and an exact workspace cwd matched
through its closing quote so a short agent name cannot match a longer one that
starts with it. The id comes from the parsed payload rather than the filename,
and is cross-checked against the filename's own uuid so a format change cannot
quietly start yielding wrong ids. CODEX_SESSIONS_DIR is the test seam.
codex_verify_resume_or_fallback then confirms the launch actually worked. A
selected id is not a promise that `codex resume` accepts it: a corrupt session
file, a version incompatibility, or a session locked elsewhere drops the pane
to a bare shell, which is worse than never resuming at all, since the old
behaviour at least always produced a live agent. A pane that fails to come up
as codex/node gets a fresh codex re-sent into it carrying the same name and
flags as the primary send, and the offending session file is renamed, never
deleted, so the next boot stops re-selecting it.
The same verification window dismisses Codex's own update-available dialog.
Whenever a newer release is published, every launch shows a blocking dialog
waiting on a keystroke a headless boot never sends, and because Codex is itself
a node process the pane still reads as a live `node` while it sits frozen
behind it. The dismissal selects "skip until next version" rather than the
default-highlighted "update now", which would run an unattended global npm
install in the middle of a boot. It is deliberately not gated on a selected
session id: the dialog belongs to codex startup in general, so a first boot
meets it too.
CLASSIFYING THE OUTCOME IS THE HARD PART
Most of the care here is in telling three situations apart, because getting
them wrong is expensive in both directions.
- A dismissed dialog only proves the dialog is gone. It sits in front of
session loading, so a single check straight after it can read a live node
a moment before the load fails. That path therefore polls, bounded, and
requires three CONSECUTIVE healthy reads before trusting the pane, because
a crash in progress can still show one transient good read.
- A crash is a NON-EMPTY read that is not codex/node. An EMPTY read is not a
crash, on either path: it means the pane could not be observed at all, it
resets the healthy streak, and it classifies as inconclusive.
- Inconclusive never returns success. It logs wording distinct from a
confirmed crash and takes the same safe recovery, because declaring an
uncertain success is the one forbidden outcome.
The quarantine gate follows from that. Attempting a fallback send is always
worth it. But the rename requires BOTH a positively observed crash AND a
successful send. Send success alone answers only "did tmux accept keys
somewhere", and target existence only answers "is there somewhere to send";
neither converts an unobserved pane into a session fault. Quarantining a
healthy session silently drops the next boot onto an older conversation, which
is the exact continuity loss this mechanism exists to prevent. On an
inconclusive outcome the target's existence is checked directly with `tmux
list-panes`, a different question from the process-name read, and a
confirmed-gone window is escalated with no send attempted and no file touched.
Recreating a lost window is deliberately out of scope: creating windows belongs
to launch_agent, which already knows the name, directory and session.
ADAPTATIONS FOR THIS REPOSITORY
Paths are AGENTS_HOME-relative rather than assuming where agent homes live. The
fallback re-send carries this package's own launch prefix and the per-agent
model flag, which is what the internal version's environment prefix amounts to
here: the point of that argument is that a crashed process takes its launch
context down with it, so a bare re-send boots an agent that does not know what
it is or which model it should be running.
TESTS
New packages/base/test/boot-agents.test.sh, 31 assertions, registered in
shell-suites.test.ts so it actually runs under vitest rather than sitting where
it can never go red. No live tmux and no live codex: boot-agents.sh is sourced
behind a BOOT_AGENTS_LIB_ONLY guard that stops it before any live tmux call,
session fixtures live in a scratch directory, and the pane check and tmux are
stubbed with controllable exit codes so the send-success and target-existence
gates are exercisable.
Mutation-checked rather than assumed. Each of these goes red on exactly the
cases named, and nothing else:
drop the outcome == crash half of the quarantine gate -> D11b, T13
classify a no-dialog empty read as a crash -> T13
lower the consecutive-healthy threshold to one -> D10, D12
drop the launch flags from the fallback re-send -> 7 cases
match the dialog on its headline alone -> D2
stop resetting the streak on an empty read -> D12
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Owner
Author
Model-Diverse Peer Review: PR #18 (inkie-deck, commit 5fbed1a)Verdict: FULL APPROVAL 1. Audit of Codex Resume & Session Quarantine Architecture (
|
codex_pane_has_update_dialog required both of its anchors somewhere in the last 20 lines of the pane. A 20-line window is not a small place: a resumed Codex session's most recent exchange can quote or discuss this very dialog a few lines above a clean, ready composer, and both the quote and the composer then sit inside the same window. That pane read as "dialog pending" when nothing was pending. The consequence is worse than a missed dialog. The caller responds to a true return by sending the bare keystroke "3", Codex's "skip until next version", with no Enter. Against a live dialog the digit selects and confirms. Against an idle pane there is no dialog to swallow it, so it lands as a stray character typed into the composer text box. It is never submitted, so nothing errors and nothing is logged: it just sits there as staged text, and staged text is exactly what the live-push composer-clean gate refuses to type over. One false positive on one boot therefore takes that agent off the delivery path indefinitely, and the only evidence is a single "3" nobody typed. The footer anchor now has to be the last non-blank line of the tail window rather than merely present in it. A live dialog's footer IS the bottom of the pane, because Codex draws the dialog where the composer would be; a reference to the dialog in transcript text always has something under it, if only the composer prompt itself. Position alone rejects the whole class. The headline stays a loose match inside the window, since it sits several lines above the footer in a real render and is not the anchor doing the work. Containment on that line rather than equality with it, so indentation or a future boxed render cannot turn this into a false negative that leaves an agent frozen behind a real dialog. New negative case D3b covers the detector, and D6b covers what actually matters: no keystroke is sent for that pane. Both fail against the previous implementation, D6b with "send-keys -t <target> 3" in the log. The existing live-dialog positive case is unchanged and still passes. Found by independent review.
Owner
Author
Model-Diverse Re-Review Confirmation: PR #18 (inkie-deck, commit 73b03db)Verdict: FULL APPROVAL Audit of Bottom-Anchoring Fix (
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the internal fleet's largest fix of the three landed this week: Codex-runtime agents now get resume-on-relaunch (a relaunched agent whose session crashed or needs restarting resumes cleanly rather than losing state), plus auto-dismissal of Codex's own "update available" dialog on launch so a fresh boot doesn't get stuck behind a dialog nobody's there to click through.
Four new functions plus a dedicated
codexbranch inlaunch_agent. The core safety property: a session file is only ever quarantined (renamed aside) on a positive observation of a crash AND a successful fallback send, never on ambiguous evidence — an empty process read, mid-poll or single-check, is always classified inconclusive, never assumed to be a crash. A single transient hiccup can't quarantine a healthy session.Verification note
Every ported invariant was mutation-checked: broke each one in turn on a scratch copy (dropping the crash+send AND-gate, misclassifying an empty read as crash, weakening the healthy-streak threshold, dropping launch flags from the fallback, headline-only dialog matching, not resetting the streak) and confirmed the suite catches each one specifically, not just generally. I independently re-read the crash-classification function myself and confirmed the AND-gate and dual-path empty-read handling directly in the code, not just from the report.
Found and NOT silently fixed, flagging separately: porting this surfaced a genuine latent bug in the internal source file itself (not this repo) —
cmdis declared without an initialiser and read back in the inconclusive-path message; underset -u, if the poll's max-checks is ever configured to 0, that's an unbound-variable crash. This OSS port initialises it defensively. Flagging back to the internal side separately, out of scope for this PR.Test plan
node scripts/leak-gate.mjs,licence-gate.mjsboth cleanboot-agents.test.sh: new suite, 31/31 passing, registered inshell-suites.test.ts