8000 Flaky: e2e-host-timeout.sh returns the command's 143 in place of 124 when its watcher is stopped before exit 3 · Issue #3450 · gominimal/minimal · GitHub
[go: up one dir, main page]

Skip to content

Flaky: e2e-host-timeout.sh returns the command's 143 in place of 124 when its watcher is stopped before exit 3 #3450

Closed

Description

@agent-137

scripts/e2e-host-timeout.sh can return the command's own exit code in place of 124 when it ends the command at the limit. The script is the host-side bash fallback used when PATH has no timeout or gtimeout. The race depends on load, so it can fail any merge group that runs the shell harnesses.

Seen in merge-group run 43647626472 (for #3442), in all_script_test_harnesses_pass (crates/common/tests/shell_harnesses.rs):

e2e-host-timeout_test.sh failed (exit status: 1):
FAIL - long sleep: got exit 143 after 1s, want 124 well before 30s

Main's native CI passed the same harness before and after that run (43643535654 on ce860bd, 43647017304 on d17573e). So it is a flake, not a regression from #3435, which changed only the in-box env_min_helper.sh.

The race

In e2e_timeout:

  1. The watcher subshell's sleep ends, and the watcher sends SIGTERM to CMD (if kill -TERM "$cmd_pid" ...; then exit 3; fi).
  2. CMD ends. The parent's wait "$cmd_pid" returns 143.
  3. The parent at once sends SIGTERM to the watcher (kill -TERM "$watch_pid").
  4. If that signal arrives before the watcher reaches exit 3, the watcher's TERM trap runs exit 0.
  5. watch_rc is 0, so e2e_timeout returns CMD's 143 and not 124.

Suggested fix

From the BCLI walk's analysis: the watcher writes a marker file before it sends the kill, and the parent checks for the marker in place of watch_rc. A signal to the watcher then cannot erase the fact that it ended CMD.

Found by the BCLI walk while it triaged a BOX merge-queue ejection.

Activity

  1. norrietaylor commented on Nov 18, 2026

    @norrietaylor
    Member

    Closing as fixed on main by #3454.

    What a read of main at 385add8 found:

    scripts/e2e-host-timeout.sh:47 now sets `trap '' TERM` after the watcher's sleep ends, so the parent's kill -TERM $watch_pid (line 60) can no longer run the exit-0 trap between the kill of CMD (line 56) and `exit 3`. Commit 969e26c5 (#3454) also adds a harness case to e2e-host-timeout_test.sh that holds the race open.
    

    Reopen this issue if any part of it is still missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0