Repository navigation
feat(chat): recover when the model's context is full, and show what a summary covers - #558
Conversation
… summary covers
When the model server says a request is too long for the model, the answer
is tried again instead of failing: what tools returned is shortened, then
all but the last two messages are folded into the summary now. The window
the server names is kept as the model's. A 400 for a full context no longer
reads as a model that can't use tools.
GET /chats/{id} gives the summary with the messages it covers, and the chat
shows a line under the last of them, with Show summary.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZAK1YALs8mQRwgGnyju4F
|
Clicks and keystrokes per feature failed on the phone survey only: I've re-run the failed job once and am repeating the phone journey locally after a computer run, as CI does. If it fails again I'll treat it as this PR's and fix it here. Generated by Claude Code |
Requested by Adeel · project thread
Description
Before: when a request still didn't fit the model (a big tool result, a window smaller than Lens thought), the model server's "context length exceeded" error ended the answer. In the web app nothing showed which messages the assistant was reading only as a summary.
After: Lens reads that error (
llm.ContextFull, from 400/413 bodies, keeping the window the server names), shortens what tools returned (keeping each result's ends, so the untrusted-text markers stay whole), and if it still doesn't fit, folds all but the last two messages into the summary and asks once more (the search path also halves the excerpts).GET /api/v1/chats/{id}gainssummary({text, upto, at, covers}), and the chat shows a line under the last summarised message ("Summarised up to here", or "Cleared here" after/clear) with Show summary.How:
chat.fit,chat.room,chat.shrink(reuseschat.compact, force, keep 2),chat.stream_fitted/reply_fitted,tool_answer(shrink=); the web route and chat-room bridge passshrink.chat.coveredbuilds the summary view;components/chat/summary-mark.tsxdraws it.Motivation and Context
Adeel: "auto compress and manage the context of larger chats!" Follows #528 (summarise at the model's real window).
Types of changes
!in the title)Checklist:
make hookschecks it on commit).docs/accordingly.## UnreleasedinCHANGELOG.md. (n/a: the title says it)Done questions (AGENTS.md)
summaryis inGET /api/v1/chats/{id}.ai.compact(off: tool results are still trimmed, no fold).ai.compact_model, in the ledger under the chat) only when a request overflows.test_the_server_saying_the_context_is_full_is_understood,test_what_tools_returned_is_cut_to_fit_the_model,test_a_conversation_that_doesnt_fit_is_folded_in_and_asked_again,chat-summary-mark.test.tsx; docs/assistant.md, docs/requirements.md; OpenAPI and client regenerated with tooling.meta) doesn't yet say an answer was fitted or shrunk.chat.compactfor the fold; no second summariser.Generated by Claude Code