8000 fix(blocks): an empty subject ends its header in a space, and trim() took it by simon-inkie · Pull Request #2 · simon-inkie/inkie-deck · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,8 @@ so far already had them:
all, so `LANG=C.UTF-8` belongs in any deployment that is not a desktop.

The test count doubles as a diagnostic here, with git and ripgrep the two that
move it: **1819 passing** means everything above is present, **1815** means
ripgrep is missing, and **1752** means neither ripgrep nor a git repository is.
move it: **1824 passing** means everything above is present, **1820** means
ripgrep is missing, and **1757** means neither ripgrep nor a git repository is.
All three are measured in a container on every rig run rather than remembered;
the numbers in this paragraph were wrong by twenty for two days before anything
asserted them.
Expand Down
17 changes: 16 additions & 1 deletion packages/base/conformance/corpus.json
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,21 @@
}
}
},
{
"id": "empty-subject-trailing-space",
"protocol": "5",
"why": "The spelling both writers actually emit for an empty subject: the label, one space, end of line. Its sibling above covers the bare-label spelling, and the application's deviation there is exactly why this one needs a case of its own - the answer recorded for the application on that case is already 'no blocks', so a reader that stopped recognising THIS line too would leave the sibling passing and say nothing. The application did precisely that for a day: its block-boundary test trimmed the line before matching, which took off the one space its own grammar requires, and a message with no subject was absorbed into the block above it.",
"input": "## 2026-08-13T10:00:00Z - from: bob - status: sent - subject: \n\nbody\n",
"expect": [
{
"ts": "2026-08-13T10:00:00Z",
"sender": "bob",
"status": "sent",
"subject": "",
"bodyRaw": "body"
}
]
},
{
"id": "status-read",
"protocol": "6",
Expand Down Expand Up @@ -339,7 +354,7 @@
],
"deviations": {
"deck": {
"reason": "The application tests `line.trim().startsWith('## ')`, so an INDENTED header-shaped line begins a block. Protocol section 4 anchors a block at column 0. This is the opposite direction from the CLI, which correctly ignores the indented line, so the two implementations disagree with each other here as well as with the specification.",
"reason": "The application strips LEADING whitespace off a line before matching the header grammar against it, so an INDENTED header-shaped line that is otherwise well formed begins a block. Protocol section 4 anchors a block at column 0. This is the opposite direction from the CLI, whose scan is anchored there and correctly reads the indented line as body, so the two implementations disagree with each other here as well as with the specification. (This text described the boundary test as `line.trim().startsWith('## ')` until 2026-08-17, two fixes after that stopped being what it did; the behaviour recorded below never moved.)",
"expect": [
{
"ts": "2026-08-13T10:00:00Z",
Expand Down
25 changes: 20 additions & 5 deletions packages/deck/app/lib/blocks/split.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,28 @@ import { HEADER_RE } from "./header.js";
* after it. That is a different defect - misattribution, not loss, and no
* bytes go missing - and closing it needs a write-time change, not a
* reader-side guess about what is quoted.
* - an INDENTED well-formed header. The trim() here keeps it a boundary, as
* it has always been; protocol section 4 anchors a block at column 0, and
* that departure is recorded against this implementation as corpus case
* indented-pseudo-header rather than quietly folded into this change.
* - an INDENTED well-formed header. Stripping the leading whitespace below
* keeps it a boundary, as it has always been; protocol section 4 anchors a
* block at column 0, and that departure is recorded against this
* implementation as corpus case indented-pseudo-header rather than quietly
* folded into this change.
*
* LEADING whitespace only, never a full trim(), and the two differ by a whole
* class of message. HEADER_RE requires `subject:\s+` ahead of the subject
* capture, so the space after that label is grammar, not padding, and a header
* whose subject is EMPTY matches only by way of it: serializeHeader
* interpolates the subject straight after "subject: ", so such a line ends in
* that space with nothing following. A full trim() takes it off along with the
* indentation, the line stops matching HEADER_RE, and the message stops opening
* a block - it merges into the one above it, attributed to that block's sender,
* with no error and nothing missing from disk. Empty subjects are ordinary
* traffic here, not a corner case; every message written without one
* serialises to exactly that shape. Stripping only what precedes the "##"
* normalises the indentation this has always tolerated and leaves the rest of
* the line as the writer emitted it.
*/
export function isBlockHeaderLine(line: string): boolean {
return HEADER_RE.test(line.trim());
return HEADER_RE.test(line.replace(/^\s+/, ""));
}

/**
Expand Down
123 changes: 121 additions & 2 deletions packages/deck/test/blocks/split.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/**
* Regression tests for the two ways splitBlocks has lost message content.
* Regression tests for the three ways splitBlocks has lost message content.
*
* 1. A standalone "---" line inside a message body silently truncated the
* rendered message in MC, dropping everything after it (hit Alex 1 + 15
Expand All @@ -16,14 +16,24 @@
* packages/base/test/inbox.test.sh). Corpus cases body-markdown-heading and
* pseudo-header-between-blocks state it against docs/protocol.md.
*
* 3. Bounding on the header GRAMMAR rather than the "## " prefix (the fix for
* 2) was first written as HEADER_RE.test(line.trim()), and the trim was the
* next defect: HEADER_RE requires a space after `subject:`, serializeHeader
* emits exactly one and nothing else when the subject is empty, and trim
* takes it back off. An empty-subject header stopped opening a block and
* was absorbed into the message above it, under that message's sender.
* Stripping only LEADING whitespace keeps the indentation tolerance that
* trim was there for without touching the end of the line.
*
* These tests drive the FULL pipeline (splitBlocks -> parseBlock) from a
* multi-block raw string, which is the only way to exercise BOTH bugs: a
* multi-block raw string, which is the only way to exercise ALL THREE bugs: a
* parseBlock-only test on a hand-built single block would never touch the split
* stage.
*/
import { describe, it, expect } from "vitest";
import { splitBlocks } from "~/lib/blocks/split.js";
import { parseBlock } from "~/lib/blocks/parse.js";
import { serializeHeader } from "~/lib/blocks/header.js";

describe("body-internal --- is preserved (not a block/body boundary)", () => {
it("mid-body --- followed by more content, a trailing separator, then a second block", () => {
Expand Down Expand Up @@ -224,3 +234,112 @@ describe("a body markdown heading is not a block boundary", () => {
expect(only.body).toContain("Everything from the fence down used to vanish.");
});
});

describe("an empty subject does not stop a header opening a block", () => {
// The line under test is produced by the writer rather than typed out here.
// An empty-subject header ends in the space after "subject:" and nothing
// else; that space is invisible in a source file and is the first thing an
// editor or formatter that strips trailing whitespace takes away - and
// taking it away leaves a test that still passes while no longer testing
// this. serializeHeader cannot drift from what the reader must accept, and
// the shape is asserted below rather than assumed.
const emptySubjectHeader = serializeHeader({
timestamp: "2026-08-17T09:05:00Z",
from: "atlas",
status: "read",
subject: "",
});

it("the writer emits a line ending in the space HEADER_RE requires", () => {
expect(emptySubjectHeader.endsWith("subject: ")).toBe(true);
});

it("keeps its own block between two ordinary ones", () => {
const raw = [
"## 2026-08-17T09:00:00Z - from: scribe - status: sent - subject: opener",
"",
"First block body.",
"",
"---",
emptySubjectHeader,
"",
"The message with no subject.",
"",
"---",
"## 2026-08-17T09:10:00Z - from: courier - status: sent - subject: closer",
"",
"Third block body.",
"",
"---",
"",
].join("\n");

const blocks = splitBlocks(raw);
// Two is the bug: a fully trimmed line no longer satisfies `subject:\s+`,
// so the middle header failed the boundary test and its message was served
// as the tail of the block above it, over scribe's name.
expect(blocks).toHaveLength(3);

const first = parseBlock(blocks[0]!);
expect(first.header.from).toBe("scribe");
expect(first.body).toBe("First block body.");

const middle = parseBlock(blocks[1]!);
expect(middle.header.from).toBe("atlas");
expect(middle.header.subject).toBe("");
expect(middle.body).toBe("The message with no subject.");

const last = parseBlock(blocks[2]!);
expect(last.header.from).toBe("courier");
expect(last.body).toBe("Third block body.");
});

it("opens the file when it is the first and only header", () => {
// The costlier half of the same defect. With nothing above it to be
// absorbed into, an unrecognised sole header leaves no headers at all, and
// a file with no blocks renders as an empty thread: the whole conversation
// gone from the view, not merely attributed to the wrong sender.
const raw = [
emptySubjectHeader,
"",
"The only message in the file.",
"",
"---",
"",
].join("\n");

const blocks = splitBlocks(raw);
expect(blocks).toHaveLength(1);

const >
expect(only.header.from).toBe("atlas");
expect(only.header.subject).toBe("");
expect(only.body).toBe("The only message in the file.");
});

it("still treats an indented well-formed header as a boundary", () => {
// The other side of the same expression, pinned here because the two are
// one decision: leading whitespace is stripped, trailing whitespace is
// not. Bounding on an indented header is a known departure from protocol
// section 4 (corpus case indented-pseudo-header), so it is deliberate
// behaviour to preserve, not incidental - a "fix" that stopped stripping
// whitespace altogether would change it silently.
const raw = [
"## 2026-08-17T10:00:00Z - from: scribe - status: sent - subject: outer",
"",
"body line",
" ## 2026-08-17T10:05:00Z - from: courier - status: sent - subject: indented",
"more body",
"",
].join("\n");

const blocks = splitBlocks(raw);
expect(blocks).toHaveLength(2);
expect(parseBlock(blocks[0]!).body).toBe("body line");
// Asserted on the split output, not through parseBlock: the block the
// indented line opens keeps its indentation, and parseHeader is anchored at
// column 0, so it goes on to parse as a plain bubble. Splitting there at
// all is the behaviour under test.
expect(blocks[1]!.startsWith(" ## 2026-08-17T10:05:00Z")).toBe(true);
});
});
9 changes: 8 additions & 1 deletion rig/expected.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -85,9 +85,16 @@ export const REQUIRED_TOOLS = ["node", "corepack", "pnpm", "git", "rg", "curl",
* deck moved 1813 -> 1819 on 2026-08-17: six tests for MC_BRIEFS_AGENT, which
* replaced a hardcoded agent name in the briefs path with a validated config
* value. base is unchanged.
*
* deck moved 1819 -> 1824 on 2026-08-17: four tests pinning that a header with
* an empty subject still opens a block, after the block-boundary test stopped
* trimming off the trailing space its own grammar requires, plus the corpus
* case for the same shape, which is one more case and so one more deck test.
* base holds at 38: its conformance runner is a single vitest case whatever
* the corpus holds, and its CLI never had the defect.
*/
export const EXPECTED_TESTS = {
deck: { passed: 1819, skipped: 11, files: 88 },
deck: { passed: 1824, skipped: 11, files: 88 },
base: { passed: 38, skipped: 0, files: 4 },
};

Expand Down
Loading
0