Repository navigation
fix(check): drop a shebang line rustc drops, and a byte order mark - #407
Merged
Merged
Conversation
inkless
force-pushed
the
gz/bz-404-rustcode-lexes-a-shebang-line-as-code
branch
2 times, most recently
from
November 22, 2026 22:35
89ab376 to
b81b479
Compare
rustCode lexed a file's first line as code. rustc drops it when it starts with `#!` and the next token isn't `[`. Whitespace and plain comments are passed over on the way to that token, but a doc comment is a token. So with `#!/** x */[] "` on line 1, rustc drops the line, while rustCode opened a string at the `"` that ran to a `// "` at the end of the file. That hid an unmarked `child.close(..)` in a `#[test]` from every check. The attribute rule read the line as `#! []` and let it through. rustCode now starts where rustc starts lexing: past a leading U+FEFF, which rustc removes first, and past a shebang line. That rule is matched exactly: Rust's Pattern_White_Space, not `\s`; `////`, `/***` and `/**/` as plain comments; the line's `\n`, and a `\r` before it, kept. The nested block-comment scan moves into one helper that both use. Output is unchanged for every tracked .rs file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
inkless
force-pushed
the
gz/bz-404-rustcode-lexes-a-shebang-line-as-code
branch
from
November 22, 2026 23:03
b81b479 to
cc72910
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
rustCode(scripts/rust-source.ts) now starts where rustc starts lexing. It skips a leading byte order mark, then a first line that rustc drops as a shebang.scripts/check-standards.tsreads all Rust throughrustCode, and so does the signal-path scan inscripts/signal-paths.ts, so both get the fix.Why
rustc drops a file's first line when it starts with
#!and the next token isn't[.rustCodelexed that line as code. Inrustc drops line 1, and the file compiles (
rustc --edition 2024 --test --emit=metadata, exit 0).rustCodeopened a string at line 1's", which ran to the"in the last comment. That hidchild.close(…)from every check, so the test passed the signalling-test rule unmarked. The attribute rule saw#! [](the doc comment became a space) and let it through as an inner attribute.There is a second way through. rustc removes a leading U+FEFF before it looks for a shebang, so
#!/** x */[] "hides the same call on main. This PR closes both.The rule
codeStartmatchesrustc_lexer::strip_shebang, afterSourceFileremoves the byte order mark:#!at offset 0, after an optional byte order mark. Leading whitespace means no shebang.\t \n \v \f \r, space, U+0085, U+200E, U+200F, U+2028, U+2029) and plain comments (//,////, nested/* … */,/**/,/***). JS\sis not that set: it includes U+00A0, U+3000 and U+FEFF and leaves out U+0085, U+200E and U+200F.///,//!,/** */or/*! */after#!makes the line a shebang. That is why the probe's line is dropped.[, the line is kept as an inner attribute. Otherwise the line is dropped up to its\n, which is kept so line numbers still match. A\rbefore the\nis kept too. The line is also dropped at end of input and after an unterminated plain block comment.rustCode's nested block-comment scan moves intoblockCommentEnd, so the shebang check and the/*branch share it.Tests
scripts/rust-source.test.ts› "drops a first line rustc reads as a shebang, and keeps one it reads as an inner attribute" puts 25 forms side by side, each followed by aclose(n), and asserts the whole output for each:[, U+00A0 and U+3000 before a[, an unterminated comment, and the byte-order-mark probe;#![a]across whitespace, U+0085, U+200E and plain comments of each kind,#!after a space or on line 2, and#![a].scripts/check-standards.test.ts› "tests that signal processes they didn't start" › "must be marked when a shebang line rustc drops opens a string before its close": the probe, with and without a byte order mark, both flagged.Both fail on main on their assertions. In the rust-source test, all 11 stripped forms fail, plus
#![a], whose mark main keeps; the other kept forms already pass. The check-standards case flags neither file on main.Evidence
Differential against rustc 1.100.0. Each first-line form F is compiled whole as
F + '\ncompile_error!("rest");'and again with line 1 cut out.rest, and the whole file gives the same result.#![cfg(any())]removes thecompile_error!.The forms: 46 hand-picked, plus 1500 fuzzed (
#!, an optional byte order mark, then 1–5 pieces of whitespace, comment openers,[,[cfg(any())],", …).rustCodeunder testNothing else changes.
rustCode's output on main and on this branch matches for all 221 tracked.rsfiles: 0 differ. 24 of them start with#!(18#![no_main], 6#![expect(), so the kept side runs on real files. Control: a version that strips every#!line differs on exactly those 24.check-standards.tsis unchanged. After this fix, a#!reaches the attribute rule only when rustc keeps it, and the rule's#\s*!?\s*\[matches it. Its\sdiffers from Rust's whitespace in only two ways:#[cfg(any())]compiles and is flagged as a#that doesn't begin an attribute. That is too strict, not a way through.#and[, fails to compile. rustc says "expected one of!or[, found doc comment".Cost:
codeStartreads only the first line, plus any comments after a#!.moon run :checkexits 0 on this branch, rebased onto main.Not in this PR
---blocks at the top of a file). rustc 1.100.0 rejects it (E0658, "frontmatters are experimental"), so it can't hide code in a build that passes. It's tracked separately, for when it stabilizes.🤖 Generated with Claude Code