8000 MC inbox: scope receipt-flip + parse to loaded window (large threads wedge) + thread search · Issue #172 · simon-inkie/inkie-deck · GitHub
[go: up one dir, main page]

Skip to content

Navigation Menu

MC inbox: scope receipt-flip + parse to loaded window (large threads wedge) + thread search #172

Open

Description

@simon-inkie

Original-derived Linear description

Lane: platform maintainer (MC platform)

Symptom (2026-07-20): the maintainer's MC wedged/spun opening the <agent> thread. Root cause: that thread file (<inbox-root>/<agent>/inbox/<principal>.md) is 3.6MB / 4029 message blocks. The view renders only a 30-block window (DEFAULT_WINDOW_LIMIT), but every touch of the file pays for the whole thing:

  • the 3s revalidation loader re-parses all 4029 blocks on every poll
  • the read-receipt sent -> read flip rewrites the full 3.6MB file on thread open (INK-443)
  • all of it runs under the inbox file lock

Server responds ~0.7s but the sustained parse + full-file rewrite + lock is what spins the tab. Recurs on any large thread regardless of the render window.

Phase 1 (the real fix) - thread perf

Cap the receipt-flip and the parse to the loaded window rather than the whole file, so no thread wedges regardless of size. Files: app/routes/threads.$agent.tsx (loader), app/lib/inbox.server.ts (markRead, readBlocksWindow), app/routes/api.thread-window.tsx. Keep the receipt-flip correct (don't lose the whole-file semantics silently) - likely flip only within the loaded/visible range, or track a high-water read marker instead of rewriting the whole file. (platform maintainer: matches today's file-description cache fix pattern - whole-file cost on a windowed view.)

Phase 2 (the maintainer's ask, refined 2026-07-20) - inbox search

Index the inbox threads into the vector DB as their own source-tagged collection, and add MC thread search over it. the maintainer's steer: inbox messages are the better search corpus, not just friendlier.

Rationale:

  • Signal density. Inbox = the distilled communiques/decisions; the CLI transcript is the working (tool calls, reasoning, retries, noise). Inbox is far higher signal per token.
  • Clean chunking. The .md blocks already carry natural boundaries (the ## <ts> - from: headers) plus sender/thread/time metadata - much nicer to embed + filter than raw JSONL. Results can surface "coordinator to the maintainer, 20 Jul, re X".
  • Two-tier UX: search inbox by default; a "go deeper / verbatim" option drops to the CLI messages collection (already populated by the-brain index-messages) for full working. Keep indexing the CLI transcripts as the deep layer.

Guardrail (bake in from the start): index incrementally - only new blocks, never re-embed the whole thread on each append. Same whole-file-cost trap as Phase 1, and there was a May 2026 inbox-indexing cost spike from a runaway watcher (see core/indexer/cc-agent-mapping.ts note). Debounce + size-delta, not per-write.

Overlaps #149 (MC RHS file-list lazy-load + search) - check what that already covers before touching the search side.

Note: today the inbox .md thread files are NOT vector-indexed (file-indexer only watches memory observations/reflections + persona docs). Phase 2 changes that deliberately.

Stopgap available now: coordinator has a verified archive built (3.6MB -> 132KB, history to inbox-archive/<principal>-through-<date>.md, backup kept) but the swap onto the live the-brain inbox file is auto-mode-classifier gated. the maintainer opted for the proper fix over the manual archive; archiving still wanted at some point.

Low-pri check while in there: /threads/<principal-a> and /threads/<principal-b> return 500 in the maintainer's instance. the maintainer says that's the the maintainer<->principal-b human-to-human chat feature (not a bug per se) - just sanity-check it still renders. Related: #176 (human-to-human messaging).

Origin

Migrated from Linear: INK-700. Original detail is retained apart from the disclosed privacy and rendering edits.

Current scope and qualifications

The account above is historical, not a fresh claim that every described defect still exists. Current Linear state: Backlog. Prior source review: partial; remaining checks or work:

  • Bind incremental public indexing and inbox/deeper search end to end, plus the requested human-thread check.

Publication transfers tracking only. Original implementation/approval/freeze and acceptance gates remain; no live deployment, configuration, external messaging or provider spending is authorised.

Transfer edits

  • Historical people/fleet prose is aliased; permitted source links and code/path references remain.
  • Linear rich embeds and public punctuation are normalised where present.

Activity

  1. added
    bugSomething isn't working
    memoryMemory, recall, indexing and observation
    on Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingmemoryMemory, recall, indexing and observation

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0