Repository navigation
fix(base): force IPv4 on the classifier call, and log why it failed - #19
Merged
Merged
Conversation
Two independent changes, both inside model_raw(), the file's only curl call
site.
1. --ipv4 on the classifier request. Some hosts have no working IPv6 route to
the endpoint but DNS still returns an AAAA record for it, so a default
dual-stack curl can try v6 first and stall to the full --max-time before
falling back, rather than trying v4 straight away. This is a generic curl
dual-stack robustness fix, not specific to this tool: any self-hosted
deployment on a similarly-configured host can hit the identical stall.
Forcing v4 removes the failure mode outright rather than only diagnosing
it faster.
2. Log the HTTP status and the first 300 bytes of the response body on every
failure path, to a new append-only log (CLASSIFIER_ERROR_LOG, env-overridable
as STUCK_SWEEP_CLASSIFIER_ERROR_LOG, defaulting under STATE_DIR next to the
spend ledger -- same directory, same override convention). Deliberately NOT
the state file, which other things parse and must not grow a variable-shape
error blob. `curl -sS` does not fail on a non-2xx status, so the status is
now asked for explicitly via `-w` and split from the body; three failure
paths are distinguished and logged separately: transport failure, HTTP
error status, and an unparseable 2xx. Without this a "model-unavailable"
verdict carries no payload at all, so a quota outage, a server fault and a
genuine network failure are indistinguishable from routine noise and from
each other in the sweep's own records.
Only the response is ever logged, never the request or its headers -- the API
key travels in the x-goog-api-key header and must never reach this file. The
300-byte cap is a deliberate redaction on a response body of unknown shape,
not a formatting preference. Failing to write the log is non-fatal by design:
an observability addition must not become a new way for the classifier itself
to break.
Tests: 130 -> 146 assertions (16 new). None of the existing STUCK_MODEL_CMD-
seamed cases exercise any of this -- that seam stands in for the entire call
and has no argv, HTTP status or curl exit code of its own to assert on -- so
the new section adds a stub curl on PATH and drives the real (non-seamed)
branch instead, recording its own argv and emulating `-w '\n%{http_code}'`
(body, newline, status) so every failure shape can be dialled in with no
network call ever made.
Revert-to-red, three narrow reverts against the finished change:
- dropping --ipv4 alone reddens exactly the IPv4 assertion (1 fail)
- neutering the three classifier_error_log call sites alone reddens exactly
the nine log-content assertions (9 fails), leaving the IPv4 assertions and
the model-unavailable/success controls green -- i.e. those two properties
are genuinely independent of the logging calls
- widening the 300-byte cap to 100000 alone reddens exactly the truncation
assertion (1 fail, got 624)
inbox-push, pane-idle and activity-sweep suites are unaffected by this change
(89/49/84 passed, unchanged from baseline).
Ported from the internal deployment this package was extracted from, and
genericised on the way: that change's incident references, internal paths and
private document citations are not reproduced here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Follow-up to ecc3787 on this branch (not amended -- kept separate so the review history stays honest about what changed in response to feedback). A model-diverse peer reviewer (Codex-runtime) found a real, confirmed gap in that commit: it wired CLASSIFIER_ERROR_LOG into model_raw(), which only sees failures at the HTTP/envelope level. model_judge(), the caller, does further validation AFTER model_raw() succeeds -- extracts .verdict via jq, checks it is stuck/clear -- and on any failure there it just returned 1 with zero logging. A well-formed HTTP response whose own generated text is not JSON (the reviewer's confirmed repro, `{"candidates":[{"content":{"parts":[{"text":"not JSON"}]}}]}`) read as "unavailable" with no diagnostic trail: exactly the blind spot the whole feature was built to close, one layer up. Three new failure paths, each logged via the existing classifier_error_log() helper (no new mechanism): - jq -er '.verdict' failing (text isn't valid JSON, or lacks .verdict) -> kind=unparseable, REUSED from model_raw()'s own case rather than invented fresh. Conceptually the same failure -- a well-formed response whose CONTENT doesn't parse as expected -- one layer up. Matches the reviewer's own test, which used this kind for exactly this scenario. - a valid .verdict value outside {stuck, clear} -> kind=invalid-verdict, a NEW and DISTINCT kind. This is not a parse failure, it's an enum violation on content that parsed cleanly, and conflating it with "the model didn't even return JSON" would hide a real distinction from the operator this log is for. - empty extracted text (the existing `[ -n "$raw" ] || return 1` guard) -> kind=empty-text, also new: nothing to attempt to parse, different again from "got something and it was garbage". Since http-code/curl-exit genuinely don't apply at this layer, both columns are logged as "n/a" rather than a misleading 200/0. THE SEAM QUESTION, made explicit rather than left to infer: model_raw()'s own logging is deliberately unreachable through STUCK_MODEL_CMD (documented there -- an injected seam has no HTTP status or curl exit to report). That reasoning does NOT carry over to this new logging: model_judge() never asks whether $raw came from curl or the seam, and a STUCK_MODEL_CMD-swapped provider returning a malformed verdict is just as real a production failure as the built-in provider doing so (the seam's own doc comment names "a swap to any other provider" as a legitimate configuration, not only a test double). Said explicitly in a comment at both the classifier_error_log() and model_judge() ends, and proven by a test that drives the same malformed-text shape through STUCK_MODEL_CMD instead of curl. Checked whether any existing test asserted (implicitly or explicitly) that a malformed verdict through the seam produced no log line: none does -- the seam's existing coverage (the "model-down" stub, a transport-style exit 1) never inspects CLASSIFIER_ERROR_LOG at all, so nothing needed to change, only to be added. Also fixed, flagged by the same reviewer, non-blocking: the 300-byte truncation cap on logged bodies was described in a code comment (and in ecc3787's own commit message, which cannot be edited after the fact) as "a deliberate redaction". That overstates what it proves -- truncating arbitrary text bounds log size, it does not redact content, and the existing test only shows the API key doesn't reach the log via a fixture that never reflects it in the body, not that truncation redacts anything in general. Reworded the classifier_error_log() doc comment to say what the cap actually does and does not guarantee. Tests: 146 -> 156 assertions (10 new): the reviewer's repro verbatim, the invalid-verdict-enum case, the empty-text case, and the seam-parity case. Revert-to-red on exactly the three new classifier_error_log() call sites: 6 of the 10 new assertions go red (the ones reading log content), the other 4 (the tier=model-unavailable classification, unaffected by logging) stay green, and nothing pre-existing moves -- 150 passed, 6 failed on the reverted tree; restored back to 156/0. Full suite (vitest wrapper): 45/45 across all four ported suites. Adjacent suites run standalone, unaffected: pane-idle 49/49, inbox-push 89/89, activity-sweep 84/84, inbox 69/69, boot-agents 33/33. Gates: leak-gate, licence-gate and lib-version-gate all clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A model-diverse peer review confirmed an exploitable regression introduced by 9246343 on this branch. STUCK_MODEL_CMD's seam subprocess runs with the full inherited shell environment, which includes GOOGLE_GENERATIVE_AI_API_KEY whenever this tier is reachable at all (the curl path needs it too, so it is present on every run that gets this far). With STUCK_MODEL_CMD='printenv GOOGLE_GENERATIVE_AI_API_KEY', that prints the real key to stdout; model_judge() correctly classifies it as an unparseable verdict; and 9246343's own new logging then wrote the literal key into CLASSIFIER_ERROR_LOG. Before that commit the same leak failed with `return 1` and no persistent record -- 9246343 made a latent leak into a written one. Two independent fixes, deliberately not just one: 1. ROOT FIX, in model_raw(): the seam subprocess is now invoked via `env -u GOOGLE_GENERATIVE_AI_API_KEY --`, so it genuinely cannot read, echo, or crash-dump the key by any means. A provider swap -- the whole point of STUCK_MODEL_CMD -- has no legitimate need for this variable; a different provider authenticates through its own mechanism. 2. BACKSTOP, in classifier_error_log(): every body, on every call site, now has any literal occurrence of the configured key redacted to "[REDACTED]" before being written, ahead of the existing 300-byte cap (redacting first, not after, so truncation can't leave a partial fragment past the cut). Belt-and-braces -- it closes the "the key never reaches the log" claim even against a leak the env fix cannot see, and is proven to do exactly that by a test where the key reaches the log body through a path other than the environment. Each fix is tested alone as well as together, so neither test result is masking a failure in the other, plus the reviewer's own exact repro end-to-end. All three are revert-to-red checked: reverting either fix in isolation fails exactly its own tests and no others. Also closes the named functional gap in the same review: a nonzero STUCK_MODEL_CMD exit was never logged, unlike every other failure path in this tier (kind=seam-exit, in model_raw(), reusing the curl_exit column for the seam's own exit status since there is no HTTP code on this path). The existing model-down test exercised this path already; it now asserts on the log too. Fixed in passing: model_judge() checked for empty model text BEFORE stripping a ```json fence, so a response that is only an empty fence (e.g. "```json\n```") fence-stripped to empty AFTER the check and was misclassified as "unparseable" garbage instead of "empty-text". The strip now runs first, with a single emptiness check after. Left alone, as explicitly out of scope for this round: widening classifier_error_log()'s control-character handling beyond the existing newline/CR/tab normalisation. Doing it safely (without corrupting legitimate non-ASCII pane text under a naive tr/printable-class approach) is more than a "keep it simple" fix, so it is left for a dedicated pass rather than rushed alongside a security fix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports a fix from the internal deployment this package was extracted from into
packages/base/bin/stuck-pane-sweep.sh's model-tier classifier call (model_raw()).What changed
--ipv4on the classifier's curl call. Some hosts have no working IPv6 route to the endpoint but DNS still returns an AAAA record for it, so a default dual-stack curl can try v6 first and stall to the full timeout before falling back to v4. Generic robustness fix, not specific to this tool.CLASSIFIER_ERROR_LOG, env-overridable viaSTUCK_SWEEP_CLASSIFIER_ERROR_LOG, defaulting underSTATE_DIRnext to the spend ledger).curl -sSdoesn't fail on a non-2xx status, so without this a quota outage, a server fault, and a genuine transport failure are indistinguishable from each other and from routine noise in the sweep's own records. Only the response is ever logged, never the request or its headers (the API key travels in a header and must never reach this file).Verification
pane-idle(49),inbox-push(89),activity-sweep(84) unaffected.--ipv4flag alone, the three log call sites alone, the 300-byte cap alone), each reddening only its own assertions. Independently re-verified myself: dropped--ipv4alone, confirmed exactly 1 assertion reddens, restored.leak-gate.mjs,licence-gate.mjs,lib-version-gate.mjsall clean.Built by a sub-agent, independently reviewed by me: read the full diff, re-ran the suite and all three gates myself, and did my own revert-to-red spot check rather than trusting the report alone.
Note for whoever merges this alongside #(line-anchor PR): both touch
packages/base/test/stuck-pane-sweep.test.shby appending a new section at the same tail location, so whichever merges second will show a trivial append-only conflict there. Ping me and I'll rebase it.🤖 Generated with Claude Code