8000 Backtick literals should dedent by their content, not by the closing delimiter · Issue #158 · kexhq/kex · GitHub
[go: up one dir, main page]

Skip to content

Backtick literals should dedent by their content, not by the closing delimiter #158

Closed

Description

@akoskovacs

Summary

A block-shaped backtick literal cannot spell a file's exact content when the
literal itself is written indented. Two separate causes: the dedent amount is
taken from the closing delimiter rather than from the content, and ${...}
holes are spliced without re-indentation.

Current behaviour

Dedent is implemented in src/lexer/lexer.cxx (lexRawString, the
closing-backtick branch) and mirrored, for tag diagnostic offsets, in
src/validation/tag_validator.cxx (sourceLocationForOffset). The rule today:

  • Dedent happens only when the closing ` sits alone on its line.
  • The amount stripped is the exact whitespace before that closing backtick.
    A nonblank line less indented than it is a lex error; a margin mixing spaces
    and tabs is a lex error.
  • Closer at column 0, or closer not on its own line → no dedent at all.

Repro (examples/backtick_margins.kex in the working tree), each line printed
with a leading | so indentation is visible:

--- A closer indented ---     (closer at col 2, content at 4/6)
|  def foo do
|    :ok
|  end
--- B closer at col 0 ---
|    def foo do
|      :ok
|    end
--- C closer inline (`end` + backtick) ---
|    def foo do
|      :ok
|    end
--- D interpolated block ---
|  def foo do
|      :ok
|  :also_ok
|
|  end

Why this is wrong

A — accidental indentation leaks into the value. The closer naturally lands
at the enclosing statement's column, the content one level in from it. The
difference (2 spaces here) is an artifact of where the literal sits in the
source, yet it ends up in the string. Getting exact file content requires
aligning the closing backtick with the content's leftmost line, which reads as
a typo and which nothing about the syntax suggests.

B/C — no dedent at all, so the same literal's meaning swings wildly on the
closer's placement.

D — holes are spliced after dedent, so a multi-line interpolated value gets
the slot's indentation on its first line only; lines 2..n land wherever the
nested string already sat, usually flush left. Its trailing newline also leaks
through as a blank line. This is the visible symptom that started this: nesting
generated fragments produces output drifting to the left margin.

Proposed fix

1. Content-relative dedent (<<~ semantics). A literal whose opening
backtick is immediately followed by a newline is block-shaped; for those:

  • drop the opening newline;
  • drop the closing delimiter's own line when it holds only whitespace (keeping
    the newline before it, so block literals end in \n);
  • strip the longest leading-whitespace run shared by every nonblank line.
    Blank lines do not participate and are emitted empty.

Compare the prefix by literal characters, so a body mixing tabs and spaces just
shares a shorter prefix — both current lex errors ("line is less indented than
its closing margin", "closing margin cannot mix spaces and tabs") disappear
rather than needing new wording. Under this rule A, B and C all yield the same
value, flush left, and the closer's placement stops being load-bearing.

2. Hole-aware re-indent for interpolating literals: when a ${...} hole is
the only thing on its line, prefix that line's indentation onto lines 2..n of
the substituted value and swallow one trailing newline. Optional and separable
from (1); without it, nested fragments still need an explicit
String#indentBy-style helper at each splice site.

Sites that must agree

  • src/lexer/lexer.cxx — lexRawString
  • src/validation/tag_validator.cxx — sourceLocationForOffset keeps a
    cooked-byte → source-offset map and must dedent identically or tag
    diagnostics point at the wrong column
  • docs/language-spec.md — "Raw Backtick Strings", the paragraph specifying
    the closing-prefix rule
  • tests/lexer_test.cxx — "dedents multiline raw strings by the closing
    prefix", "preserves indentation when the closing backtick is flush left",
    "rejects content left of the closing margin"
  • tests/interpreter_test.cxx — "evaluates dedented multiline raw strings"
  • For (2), the BEAM lowering path as well, not just the interpreter

Breaking change to literal values, which is fine pre-stability, but it is a
deliberate semantic change rather than a bug fix at the edges.

Activity

  1. added 2 commits that reference this issue on Aug 21, 2026
    This issue will close once commit 9b623bf is merged into the 'main' branch.
    9b623bf
    fc9a91d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0