Repository navigation
Description
Summary
A block-shaped backtick literal cannot spell a file's exact content when the
literal itself is written indented. Two separate causes: the dedent amount is
taken from the closing delimiter rather than from the content, and ${...}
holes are spliced without re-indentation.
Current behaviour
Dedent is implemented in src/lexer/lexer.cxx (lexRawString, the
closing-backtick branch) and mirrored, for tag diagnostic offsets, in
src/validation/tag_validator.cxx (sourceLocationForOffset). The rule today:
- Dedent happens only when the closing
`sits alone on its line. - The amount stripped is the exact whitespace before that closing backtick.
A nonblank line less indented than it is a lex error; a margin mixing spaces
and tabs is a lex error. - Closer at column 0, or closer not on its own line → no dedent at all.
Repro (examples/backtick_margins.kex in the working tree), each line printed
with a leading | so indentation is visible:
--- A closer indented --- (closer at col 2, content at 4/6)
| def foo do
| :ok
| end
--- B closer at col 0 ---
| def foo do
| :ok
| end
--- C closer inline (`end` + backtick) ---
| def foo do
| :ok
| end
--- D interpolated block ---
| def foo do
| :ok
| :also_ok
|
| end
Why this is wrong
A — accidental indentation leaks into the value. The closer naturally lands
at the enclosing statement's column, the content one level in from it. The
difference (2 spaces here) is an artifact of where the literal sits in the
source, yet it ends up in the string. Getting exact file content requires
aligning the closing backtick with the content's leftmost line, which reads as
a typo and which nothing about the syntax suggests.
B/C — no dedent at all, so the same literal's meaning swings wildly on the
closer's placement.
D — holes are spliced after dedent, so a multi-line interpolated value gets
the slot's indentation on its first line only; lines 2..n land wherever the
nested string already sat, usually flush left. Its trailing newline also leaks
through as a blank line. This is the visible symptom that started this: nesting
generated fragments produces output drifting to the left margin.
Proposed fix
1. Content-relative dedent (<<~ semantics). A literal whose opening
backtick is immediately followed by a newline is block-shaped; for those:
- drop the opening newline;
- drop the closing delimiter's own line when it holds only whitespace (keeping
the newline before it, so block literals end in\n); - strip the longest leading-whitespace run shared by every nonblank line.
Blank lines do not participate and are emitted empty.
Compare the prefix by literal characters, so a body mixing tabs and spaces just
shares a shorter prefix — both current lex errors ("line is less indented than
its closing margin", "closing margin cannot mix spaces and tabs") disappear
rather than needing new wording. Under this rule A, B and C all yield the same
value, flush left, and the closer's placement stops being load-bearing.
2. Hole-aware re-indent for interpolating literals: when a ${...} hole is
the only thing on its line, prefix that line's indentation onto lines 2..n of
the substituted value and swallow one trailing newline. Optional and separable
from (1); without it, nested fragments still need an explicit
String#indentBy-style helper at each splice site.
Sites that must agree
src/lexer/lexer.cxx—lexRawStringsrc/validation/tag_validator.cxx—sourceLocationForOffsetkeeps a
cooked-byte → source-offset map and must dedent identically or tag
diagnostics point at the wrong columndocs/language-spec.md— "Raw Backtick Strings", the paragraph specifying
the closing-prefix ruletests/lexer_test.cxx— "dedents multiline raw strings by the closing
prefix", "preserves indentation when the closing backtick is flush left",
"rejects content left of the closing margin"tests/interpreter_test.cxx— "evaluates dedented multiline raw strings"- For (2), the BEAM lowering path as well, not just the interpreter
Breaking change to literal values, which is fine pre-stability, but it is a
deliberate semantic change rather than a bug fix at the edges.
Activity
Properly indent with backticks, resolves #158
Merge pull request #166 from kexhq/fix-dedent