8000 Brain: user-turn auto-recall hook with need detection and relevance verification · Issue #48 · simon-inkie/inkie-the-brain · GitHub
[go: up one dir, main page]

Skip to content

Brain: user-turn auto-recall hook with need detection and relevance verification #48

Open

Description

@simon-inkie

Original Linear description

Outcome

On each supported user turn, automatically recall relevant Brain evidence before the main LLM starts, avoiding repeated investigation. the maintainer requested this on 14 October 2026. This joins the earlier auto-recall design with the relevance-ranking work in INK-931.

Flow

  1. A user-turn hook supplies the latest directive and bounded current-task context to a typed classifier: would Brain recall help? Evaluate Jev and suitable small alternatives, including ScaleDown, before selecting a provider.
  2. When recall is warranted, run a bounded search through the existing Brain retrieval API, retaining candidate IDs, source paths, timestamps and access scope.
  3. A second typed classification selects useful candidate IDs or no_match and assesses relevance/coverage for the specific question. Reuse INK-931 rather than building a competing reranker.
  4. Inject only selected, budgeted evidence into the main LLM context: "Relevant Brain information found", excerpts, provenance and any partial/uncertain coverage. The main LLM still evaluates and answers the request. Relevance is not proof that the question is fully answered.
  5. On no_match, malformed output, timeout or provider failure, continue the turn normally without invented evidence. Preserve explicit user requests for recall and existing mandatory recall rules.

Bounded implementation

  • Identify actual supported user-turn lifecycle hooks for each runtime; prototype one adapter first. No recursive hook invocation, model-generated tool execution or unbounded search/retry loops.
  • Deterministic code builds the query from bounded input. Classifiers choose only supplied decisions/candidate IDs. Retrieved text is untrusted data, never hook instructions.
  • Respect principal/agent access controls before retrieval and injection. Minimise/alias content before an external provider; no raw full transcripts or credentials. Provider choice does not grant permission to transmit private memory.
  • Hard latency, token, candidate-count and call-count budgets; deduplicate/cache by message, task scope, index version and candidate hashes. Log decisions and timings without raw private payloads.
  • Start offline then shadow mode, disabled by default; live hook/config installation is a separate release decision.

Acceptance and measurement

  • Label historical/synthetic user turns for recall-needed versus not-needed, plus candidate relevance and partial/full coverage; use a held-out set for evaluation.
  • Measure missed useful recalls, unnecessary searches, unsupported/irrelevant injections, correct abstentions and provenance preservation.
  • Measure complete two-classifier-plus-search p50/p95 latency, error/timeout rate, actual usage/cost, main-model tokens and elapsed time saved against manual recall.
  • Compare with always-search and current manual recall baselines. Calibrate thresholds on the task; do not treat native confidence as factual correctness or copy thresholds across models.
  • Verify no-match, stale/conflicting sources, cross-principal access, embedded instructions, duplicate turn delivery and provider failure.
  • Produce a bounded SPEC with measured rollout gates and an independent review before enablement.

Earlier design notes: /home/simon/brain/ideas/2026-10-03-auto-recall-hook-routes-to-the-brain.md and /home/simon/brain/ideas/2026-09-22-jev-brain-retrieval-reranker.md.
Owner lane: platform maintainer / Brain and runtime adapters. Scope here is ticket and design, not live platform changes.

Origin

Migrated from Linear: INK-963. The original wording is retained except the disclosed privacy and rendering edits below.

Current scope and qualifications

The description above is the original Linear account, including its historical findings and scope. It is not a fresh claim that every described defect still exists.
Current Linear state is Backlog. Prior source review recorded uncertain, with this remaining scope:

  • Complete original bounded spec plus held-out evaluation and independent review; enablement remains separate.

Publication transfers the task; it does not authorise live recovery, deployment, spending or configuration changes.

Transfer edits

  • Historical principal names are aliased in prose; artificial shell-filename web links are rendered as code where present.
  • Historical fleet identities are aliased; code identifiers and permitted source paths remain references.
  • Linear rich issue embeds are rendered as clickable Markdown links.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestmajor-featureA substantial feature or new capability

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0