Repository navigation
docs(grammar): document line comments and Unicode identifiers (#650) - #658
Merged
Merged
Conversation
The parser has always skipped `;` line comments and read Unicode letters and numerals in symbols, variables, keywords and bind slots. grammar.pest now says so: COMMENT is skipped like WHITESPACE, word_cont, plain_sym and bind_slot use ALPHABETIC and NUMBER, and `-` followed by a numeral is not a symbol (the parser reads it as a number). The verifier's Comment and UnicodeLetter drift entries are gone; GRAMMAR.md is regenerated, with the comment in the skip note and the new letter/numeral definitions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This was referenced Oct 20, 2026
adityamukho
added a commit
that referenced
this pull request
Oct 20, 2026
…660) Backport of #658 from v3. grammar.pest skips `;` line comments like whitespace, and symbols, variables, keywords and bind slots use ALPHABETIC and NUMBER, as the tokenizer does; `-` followed by a numeral is a number, not a symbol. The verifier's Comment and UnicodeLetter drift entries are removed. No parser change. Claude-Session: https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #650 by documenting, not restricting: the parser has accepted
;line comments since v1.1.0 and Unicode identifiers always, so restricting would break existing inputs.Changes
tests/grammar/grammar.pest:COMMENT = _{ ";" ~ (!"\n" ~ ANY)* };word_cont,plain_sym,bind_slotuseALPHABETIC/NUMBER(the classes behindchar::is_alphabetic/is_numeric);"-" ~ !NUMBER ~ word_cont+, since the parser reads-+ numeral as a number (without it the verifier finds 81-٣-style inputs pest accepts and the parser rejects with INT-008).verifier.rs:CommentandUnicodeLettercauses and theirKNOWN_DRIFTentries removed;is_plain_symbolmirrors the newplain_sym.tests/grammar_doc.rs: rendersCOMMENTinto the skip note (panics on any other comment shape), mapsALPHABETIC→letter,NUMBER→numeral; new testcomment_joins_the_note.docs/GRAMMAR.mdregenerated (the docs site reads it per tag, so no minigraf-docs PR).No parser behaviour changes.
Verification
cargo test --lib verifier: 4 passed (k = 3).MINIGRAF_GRAMMAR_BOUND=4structural level: 7,592,910 inputs, only Parser accepts inputs the grammar rejects (empty sections, unknown keywords, rule/call heads, lexical edge cases) #651 drift.cargo test --test grammar_doc,--test grammar_conformance, clippy-D warnings.🤖 Generated with Claude Code
https://claude.ai/code/session_01PtCdzbJMSS1V8qEPaJ4jXz