8000 feat(runner): hold the Mac awake for each job and start runners without nohup by inkatze · Pull Request #265 · inkatze/dotfiles · GitHub
[go: up one dir, main page]

Skip to content

feat(runner): hold the Mac awake for each job and start runners without nohup - #265

Merged
inkatze merged 8 commits into
mainfrom
planwright/flight/mc-dotfiles-ad-hoc-b5c4b54c
Nov 14, 2026
Merged

inkatze merged 8 commits into
mainfrom
planwright/flight/mc-dotfiles-ad-hoc-b5c4b54c

Conversation

@inkatze
@inkatze inkatze commented Nov 14, 2026 •
Copy link
Copy Markdown
Owner

The Mac went into system sleep mid-job (a maintenance sleep) and froze every self-hosted runner. The runner now keeps the system awake for exactly the lifetime of each accepted job, and starts without nohup.

  • The job-started hook, once it accepts a job, starts /usr/bin/caffeinate -s -w <worker pid> in the background. -s holds only on AC power, the hold ends when the runner's worker process does, and nothing is left running between jobs. Run by hand with no worker above it, the hook holds nothing and says so.
  • runner-start.sh no longer uses nohup: jobs inherited SIGHUP ignored, which fails signal tests. The gate starts under job control (set -m), so SIGHUP and SIGINT stay at their defaults.
  • No pmset or system setting is touched; everything stays inside the fleet account. The gate's own caffeinate -i (idle sleep) is unchanged.
  • scripts/runner-hook-test.sh and scripts/runner-start-test.sh pin the -s hold via /usr/bin/caffeinate, its release at job end, no orphan, no pmset in the osx role, no nohup in the start path, and default signal dispositions for the gate. docs/offload-host.md describes it.

The comm= match on macOS (full path) is exercised by the lint job on the macOS runner, which runs the hook suite.

Step records

First run:

Steps at pre-implementation (run 000001)

# Step Kind Target Hosting Backend Session Head Start End Outcome Excerpt Output
none

Ended on a5fc5c13462d.

Steps at pre-ci (run 000001)

# Step Kind Target Hosting Backend Session Head Start End Outcome Excerpt Output
none

Ended on a5fc5c13462d.

Steps at convergence (run 000001)

# Step Kind Target Hosting Backend Session Head Start End Outcome Excerpt Output
1 polish skill polish in-session in-session a5fc5c1 to 1a09f31 2026-11-14T15:57:14Z 2026-11-14T16:41:23Z applied (degraded: ran in the worker session with a foreground lens fan-out, as the brief directs)
2 panel-review skill panel-review in-session gemini 1a09f31 2026-11-14T16:41:28Z 2026-11-14T16:50:59Z passed (degraded: ran in the worker session; gemini was the non-Anthropic backend)

Ended on 1a09f314049e.

Steps at pre-pr (run 000001)

# Step Kind Target Hosting Backend Session Head Start End Outcome Excerpt Output
none

Ended on 1a09f314049e.

Re-run after the macOS CI fix (delta passes only):

Steps at pre-implementation (run 000002)

# Step Kind Target Hosting Backend Session Head Start End Outcome Excerpt Output
none

Ended on 1a09f314049e.

Steps at pre-ci (run 000002)

# Step Kind Target Hosting Backend Session Head Start End Outcome Excerpt Output
none

Ended on 263589b43b1a.

Steps at convergence (run 000002)

# Step Kind Target Hosting Backend Session Head Start End Outcome Excerpt Output
1 polish skill polish in-session in-session 263589b 2026-11-14T17:28:23Z 2026-11-14T17:31:30Z passed (degraded: delta pass over the stub-stop commit, in the worker session, per the hub's usage steer)
2 panel-review skill panel-review in-session gemini 1ccd10a to 263589b 2026-11-14T17:21:30Z 2026-11-14T17:36:43Z applied (degraded: delta passes over the CI fix (applied, 1ccd10a to HEAD) and the stub-stop commit (clean), in the worker session)

Ended on 263589b43b1a.

Steps at pre-pr (run 000002)

# Step Kind Target Hosting Backend Session Head Start End Outcome Excerpt Output
none

Ended on 263589b43b1a.

Steps at post-pr (run 000002)

# Step Kind Target Hosting Backend Session Head Start End Outcome Excerpt Output
1 bot-review skill bot-review in-session pr-agent 263589b 2026-11-14T17:31:30Z 2026-11-14T17:46:02Z passed (degraded: PR-Agent after cubic's quota refusal; the one finding declined with a reproduction as the hub settled, reply posted; the start bound is the preceding step's end)

Ended on 263589b43b1a.

Steps at convergence (run 000003)

# Step Kind Target Hosting Backend Session Head Start End Outcome Excerpt Output
1 polish skill polish in-session in-session ab09da4 to e5f450b 2026-11-14T19:21:11Z 2026-11-14T19:27:21Z applied (degraded: lenses walked inline over a small delta)
2 polish skill polish in-session in-session e5f450b 2026-11-14T19:27:21Z 2026-11-14T19:29:47Z passed (degraded: lenses walked inline over a one-sentence delta)
3 panel-review skill panel-review in-session gemini e5f450b 2026-11-14T19:30:26Z 2026-11-14T19:42:26Z passed

Ended on e5f450be5db2.

Steps at post-pr (run 000003)

# Step Kind Target Hosting Backend Session Head Start End Outcome Excerpt Output
1 bot-review skill bot-review in-session pr-agent e5f450b 2026-11-14T19:42:26Z 2026-11-14T19:55:12Z passed (degraded: PR-Agent after cubic's quota refusal; its one finding is the SIGHUP-forwarding one already declined with a reproduction and replied to, now re-listed unchanged on the new head; no new reply)

Ended on e5f450be5db2.


Summary by cubic

Fixes the macOS runner host going to system sleep mid-job, which froze every self-hosted runner, and removes nohup from the start path to keep SIGHUP at its default for jobs.

  • An accepted job starts /usr/bin/caffeinate -s -w <worker pid> in the background, holding the system awake on AC power until the worker exits; run by hand, the hook holds nothing and says so.
  • runner-start.sh starts the gate under job control instead of nohup, so jobs no longer inherit an ignored SIGHUP; no pmset or system settings change.
  • Tests run the hook against a stub caffeinate so the suite never holds the Mac, and clear any hold a worker-parented case leaves behind; docs/offload-host.md documents the feature and attaches the redis and postgres gate note to the caffeinate -i sentence it extends.

Written for commit e5f450b. Summary will update on new commits.

View guided diff Turn on auto-fix

@inkatze
inkatze commented Nov 14, 2026
Copy link
Copy Markdown
Owner Author

@cubic-dev-ai review this PR

@cubic-dev-ai
cubic-dev-ai Bot commented Nov 14, 2026
Copy link
Copy Markdown

@cubic-dev-ai review this PR

@inkatze cubic can't start this review because your workspace has reached its free monthly review limit. cubic has reviewed 40,041 of the 40,000 allowed lines of code this month. Reviews resume on 2 December 2026 (in 19 days). Paid plans include much higher monthly review limits. Upgrade now to resume reviews.

To help optimise your usage, you can tune cubic to get the most out of your usage limits:

Learn more →

@github-actions
github-actions Bot commented Nov 14, 2026 •
Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit e5f450b)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

SIGHUP forwarding

With nohup removed, the gate now uses the default SIGHUP disposition, but it is still a background job tracked by the starting shell. If runner-start.sh itself receives SIGHUP — for example, because the SSH session or parent shell that launched it goes away — bash forwards the signal to all jobs, including the gate, so the runner will die. set -m only isolates the process group from terminal-generated signals; it does not prevent bash from propagating a SIGHUP it receives. Consider disowning the background job after & so the gate survives without inheriting an ignored SIGHUP.

set -m
${low_io[@]+"${low_io[@]}"} nice -n "$nice_level" \
    ${awake[@]+"${awake[@]}"} "$gate" "$dir" \
    >>"$cache/$prefix-$slug.out" 2>>"$cache/$prefix-$slug.err" </dev/null &
echo $! >"$pidfile"

@github-actions
github-actions Bot commented Nov 14, 2026 •
Copy link
Copy Markdown

PR-Agent run 43633565643.1 on e5f450b: success

@github-actions
Copy link
Copy Markdown

Persistent review updated to latest commit 263589b

@inkatze
inkatze commented Nov 14, 2026
Copy link
Copy Markdown
Owner Author

Declining the "SIGHUP forwarding" finding: a non-interactive bash does not forward SIGHUP to its background jobs, and runner-start.sh exits right after starting the gate. The gate also runs in its own process group under set -m. disown only removes the job from the shell's table, so it changes nothing here.

Reproduction: a set -m starter backgrounded sleep 30, then sent itself kill -HUP $$. The starter died with "Hangup", and kill -0 on the child still succeeded afterwards.

Context: cubic refused this PR's review on quota at 17:00:29Z (free monthly review limit, resets 2 December 2026), so this PR-Agent review is the bot review.

@github-actions
Copy link
Copy Markdown

Persistent review updated to latest commit e5f450b

@inkatze
inkatze marked this pull request as ready for review November 14, 2026 20:34
@inkatze
inkatze merged commit aad2c79 into main Nov 14, 2026
7 checks passed
@inkatze
inkatze deleted the planwright/flight/mc-dotfiles-ad-hoc-b5c4b54c branch November 14, 2026 20:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0