8000 feat(chat): deep research in the web app, a plan card to approve and a live progress card by adeelahmad · Pull Request #517 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content

feat(chat): deep research in the web app, a plan card to approve and a live progress card - #517

Merged
adeelahmad merged 29 commits into
mainfrom
claude/project-thread-r960xc-c8-ui
Oct 18, 2026
Merged

adeelahmad merged 29 commits into
mainfrom
claude/project-thread-r960xc-c8-ui

Conversation

@adeelahmad
Copy link
Copy Markdown
Owner

Description

Before: deep research (#516) worked through the API, MCP, chat rooms and the assistant's tool. In the web app, though, a plan was only text in the conversation, and the run showed nothing until its report arrived.

After:

  • Asking. The composer has a Deep research switch (a telescope). With it on, the next message is a research goal, and a chip says you approve a plan first. /research <goal> does the same, and neither offers Regenerate.

  • Plan card. The answer is a plan card with:

    • the follow-up questions, each with answer chips to tap and a field for your own answer;
    • the sub-questions as a list you can reword, add to (up to 8) or remove from, always keeping one;
    • where it looks, as ticks for only the sources you have (GET /chats/tools);
    • a tool-call stepper with the estimate beside it, which scales with the calls and is labelled as an estimate;
    • the report's outline.
  • Approving. Enter approves the plan with your edits. Once a question is answered, Enter revises the plan with your answers instead, and "Approve as it is" is still one click. Decline drops it. A research approval no longer also shows as a generic approval card.

  • Progress card. Once approved, it becomes a progress card read from GET /research/{id} every few seconds, so it carries on after a reload and on a phone. It shows:

    • the stage, and a bar of sub-questions done with each one ticked off;
    • sources, tool calls, cost (labelled when estimated) and minutes;
    • why a budget holds it, with "Go on anyway";
    • Stop.

    When the report is saved, the conversation is read again so the report shows.

  • Report. The report's actions are:

    • Save as note, asking which namespace when it could go in several;
    • Read aloud, using the voice settings;
    • Make a podcast, opening the episode;
    • Export, as a Markdown file with the sources it cites.
  • Notes. A chat: link opens the conversation.

  • Settings. Settings → AI assistant → Deep research holds research.max_tool_calls, max_minutes, model and agents_skip_review.

This is stacked on #516 (the backend). Merge that first, and this PR then retargets to main.

Motivation and Context

Module C8 of the chat-gaps plan, the web app half: plan card, progress card and report actions.

Screenshots (if appropriate):

None attached. The jest tests cover each card's behaviour.

Steps to reproduce (if appropriate):

  1. In chat, tap the telescope and type a goal. A gold plan card appears.
  2. Reword a sub-question and press Enter. The card turns into a progress card that ticks off sub-questions; reload the page and it carries on.
  3. When the report arrives, use Save as note, Read aloud, Make a podcast or Export under it.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change; ! in the title)

Checklist:

  • The title is a Conventional Commit.
  • My code follows the code style of this project (make hooks checks it on commit).
  • I added or updated tests.
  • My change requires documentation updates, and I have updated docs/ accordingly.
  • Users need more than the title to understand it, so I wrote it under ## Unreleased in CHANGELOG.md.

Done questions (AGENTS.md)

  • Goal: the web app half of C8. With feat(chat): deep research, a plan to approve and a cited report #516, deep research works end to end in the web app. Merge on green after feat(chat): deep research, a plan to approve and a cited report #516.
  • Agents: everything the cards do goes through the APIs from feat(chat): deep research, a plan to approve and a cited report #516. Those are /chats/{cid}/research, /research/{id} with /revise, /stop, /resume, /note and /podcast, plus /approvals/{id} and the MCP tools.
  • Zero setup: no new prerequisite.
  • Fewest clicks and keystrokes:
    • one switch, or /research;
    • Enter approves, or revises once a question is answered;
    • answers are one tap;
    • the stepper's defaults come from the plan;
    • Decline needs no confirmation.
  • Configurable: the research settings are now in Settings → AI assistant → Deep research, with the server's defaults (40 tool calls, 20 minutes, the conversation's model, agents need review).
  • Light: no new dependency. Polling runs only while a run is active, and every 20 s while a plan waits.
  • Private and secure: no new surface beyond feat(chat): deep research, a plan to approve and a cited report #516. A chat: link in a note opens a conversation only its owner can read.
  • Cost: the estimate is shown before approval and labelled. Cost so far comes from the activity ledger.
  • User decides: nothing runs until Approve. Existing defaults are unchanged, and the switch is off by default and turns off after one message.
  • Upgrades: n/a, there is no stored data change.
  • Honest: the docs and CHANGELOG move the cards from in progress to built. Estimates are labelled.
  • Tests, docs and generated files updated:
    • __tests__/chat-research.test.tsx has 14 tests: stream event, helpers, plan approve/revise/decline/edit, progress, stop, resume, report refresh, note namespace pick, read aloud, podcast, export, and the switch;
    • settings-web-search.test.tsx gains the server's research defaults;
    • docs/assistant.md › Deep research and the CHANGELOG are updated.
  • Open standards; dependencies keep Lens MIT. No new dependency.
  • Result reported with evidence. Refine later: use C3's export once merged (today's Export is a Markdown file with its sources), sub-questions in parallel, a report posted back into the chat room.
  • Reuse: approval and answer patterns, useVoice, the podcast and note APIs, settings fields.

Checks on 95e897d:

  • tsc --noEmit is clean;
  • eslint and prettier are clean on the changed files;
  • the full jest suite passes: 147 suites, 1016 tests;
  • mkdocs build --strict is clean.

Generated by Claude Code

claude added 27 commits October 16, 2026 23:28
Importing app.domain.ai_tools on its own failed: it imported automations,
which loads the actions, and the agent's action subclasses Toolbox before
ai_tools had defined it. Running one test file (tests/api/test_chats.py)
hit it. ai_tools now imports automations last.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Every answer keeps `meta`: the model and the host name of its server,
the tools offered and asked for, the scope it answered in (after
auto-scope), its limits and what it used, how it ended (answered,
direct, limit, fallback, stopped, error), tool results that read like
instructions to an AI, how long it took, Lens's version, and tokens and
cost from the ledger rows that name it. Questions keep how they were
sent (web, api, voice, room, automation) and their options.

The answer's id is taken once the question is saved and the answer
runs in activity.scope(chat:<cid>, chat_message:<mid>), in the web
chat, chat rooms and the automations' agent; the streamed answer holds
the scope across its steps (activity.scoped). App completions keep no
messages, so their record comes back in lens.record. The stream sends
`meta` before `done`; GET /chats/{cid}/messages/{mid} returns one
message with its record and ledger rows. Each tool step keeps its time
and whether it failed. The web app shows it all from an i in the
answer's header (a sheet on a phone); old answers say they weren't
recorded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
The last check asked for 2026-10-17's note after the call without a date had made today's note, so on that day it
found the note the deleted root had left behind. It now asks for a day no earlier call makes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
The answer's record uses C2's reserved answer_id for its ledger scope and is saved under the question (parent);
a message of any version can be read with its record and ledger rows.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Group the assistant's tools (GET /chats/tools lists the groups and tools a
person has, what each does and why a group is missing), offer an answer only
the tools picked (an admin's switch still wins), stop at a tool-call limit up
to the new ai.max_tool_calls setting, and narrow one answer to its own
namespaces. A conversation keeps its choice and new ones start with the last.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
…record' into claude/project-thread-r960xc-c4-tools
…ient

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
…t choice

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
…-r960xc-c4-tools

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Web search (#472) joins the tool groups as `web`: its tools send data out, and
the group says when no web search is set up (web.search). Main's SENDS_OUT is
kept; web citations stay readable in the shared citation filter.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
The find box empties while the list for what was typed loads, so on a busy
runner the first list could pass the wait and then vanish.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
The code sandbox (#473) joins the tool groups as `code`: there for those who
may use it, its runs labelled as sending data out while its network is on,
and the group says when the sandbox is off (sandbox.enabled) or for admins.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
The answer record (#469) is in main now; the client is regenerated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
classify (#494) joins the archive's tools.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Ported from #512 so this PR's upgrade check passes while that lands: the
entity-types step re-types weekday Terms as Dates, which the default entity
list leaves out, so the check read them as gone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
A goal (/research, research: true, POST /chats/{cid}/research, the assistant's and MCP's start_research) gets one planning call: follow-up questions and a plan with a labelled estimate, waiting as an approval of the assistant's (tool research). Approved in the web app, the inbox or with a yes in a chat room (an API key or an app only declines, unless research.agents_skip_review), it runs in the background: claimed by one process with a heartbeat, resumed from the last finished stage, one run per person, within its tool calls, research.max_minutes and budgets, stoppable. Gather uses only the plan's sources' read tools; write and check renumber citations and rewrite or mark unsupported claims. The report saves as a note (chat links) or makes a podcast.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
…a live progress card

The composer's Deep research switch (and /research) sends a goal; its answer is a plan card: the follow-up questions with answers to tap, the sub-questions to reword, add or remove, where it looks, a tool-call stepper with the estimate, and Enter approves it (or revises it once a question is answered), with Decline. Approved, it becomes a progress card read from GET /research/{id}, so it survives a reload: stage, sub-questions ticked off, sources, tool calls, cost and minutes, Go on anyway when a budget holds it, and Stop. The report saves as a note (picking the namespace when there are several), reads aloud, makes a podcast or exports as Markdown. Notes open chat: links, and Settings > AI assistant > Deep research holds the research limits.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
@adeelahmad adeelahmad self-assigned this Oct 17, 2026
Base automatically changed from claude/project-thread-r960xc-c8-research to main October 18, 2026 01:59
main now carries the research backend as one squashed commit (#516), so this keeps only the web app's plan,
progress and report cards on top of it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
…-r960xc-c8-ui

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
@adeelahmad
adeelahmad merged commit b0e193b into main Oct 18, 2026
10 checks passed
@adeelahmad
adeelahmad deleted the claude/project-thread-r960xc-c8-ui branch October 18, 2026 02:54
adeelahmad pushed a commit that referenced this pull request Oct 18, 2026
…ay and clipper sign-in as built

Since the last README refresh, #474, #484, #510, #511, #497, #495, #406, #516, #517
and #491 merged. The README's built list now says a conversation exports as Markdown,
PDF or JSON and its answers, questions and code copy; a Tools chip picks an answer's
tools, limit and namespaces; sandbox runs open under the answer with downloads, and
Settings starts the gateway; deep research plans, runs and returns a cited report;
ready-made MCP servers (the browser, GitHub) and the optional containers start from
the web app; and the web clipper signs in or takes an API token. Each has its checks
in docs/requirements.md. In progress drops the merged PRs and lists the open feature
PRs; the Planned proof item keeps only the Raspberry Pi timing (#481 and #482 are
closed); the docs index links requirements and checks, and published feeds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HDx7PPUauyjMn1LZs8JAsS
adeelahmad pushed a commit that referenced this pull request Oct 18, 2026
…ture

Main's deep research (#517) added a research section to Settings › AI assistant, so the fixture carries its defaults and Save sends only the voice change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dzm4YHdHn2qZy7aXer3KeH
adeelahmad pushed a commit that referenced this pull request Oct 18, 2026
…ay and clipper sign-in as built

Since the last README refresh, #474, #484, #510, #511, #497, #495, #406, #516, #517,
#491, #400 and #506 merged. The README's built list now says the assistant answers
questions about Lens itself; a conversation exports as Markdown, PDF or JSON and its
answers, questions and code copy; a Tools chip picks an answer's tools, limit and
namespaces; sandbox runs open under the answer with downloads, and Settings starts the
gateway; deep research plans, runs and returns a cited report; ready-made MCP servers,
the MCP servers page and the optional containers start from the web app; and the web
clipper signs in or takes an API token. Each has its checks in docs/requirements.md.
In progress lists only open PRs; the Nebula mesh (#546, merged, partly built) moves to
Planned with what's next; the Planned proof item keeps only the Raspberry Pi timing;
the docs index links requirements and checks, and published feeds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HDx7PPUauyjMn1LZs8JAsS
adeelahmad added a commit that referenced this pull request Oct 18, 2026
…ay and clipper sign-in as built (#536)

Since the last README refresh, #474, #484, #510, #511, #497, #495, #406, #516, #517,
#491, #400 and #506 merged. The README's built list now says the assistant answers
questions about Lens itself; a conversation exports as Markdown, PDF or JSON and its
answers, questions and code copy; a Tools chip picks an answer's tools, limit and
namespaces; sandbox runs open under the answer with downloads, and Settings starts the
gateway; deep research plans, runs and returns a cited report; ready-made MCP servers,
the MCP servers page and the optional containers start from the web app; and the web
clipper signs in or takes an API token. Each has its checks in docs/requirements.md.
In progress lists only open PRs; the Nebula mesh (#546, merged, partly built) moves to
Planned with what's next; the Planned proof item keeps only the Raspberry Pi timing;
the docs index links requirements and checks, and published feeds.


Claude-Session: https://claude.ai/code/session_01HDx7PPUauyjMn1LZs8JAsS

Co-authored-by: Adeel Ahmad <noreply@anthropic.com>
adeelahmad added a commit that referenced this pull request Oct 18, 2026
…ine (#538)

* feat(voice): read answers and podcasts aloud with Kokoro on this machine

Kokoro v1.0 (hexgrad's Kokoro-82M, Apache-2.0) as a text-to-speech provider, voice.tts_provider "kokoro": spoken
answers and podcast hosts are read on the server's own CPU through kokoro-onnx on ONNX Runtime, in any of its 54
voices across nine languages, with nothing sent anywhere and nothing billed. It's opt-in and fetched once it's picked:
the `kokoro` extra (espeak-ng and phonemizer are GPL-3.0, so they stay out of the default install), and the model
(voice.kokoro_model: fp32, fp16 or int8) and voices files from kokoro-onnx's release, checked against their SHA-256.
Settings → AI assistant → Voice offers it with the model size and the voice picker; a podcast's second host gets a
voice in the first host's language. kokoro.yml fetches and reads with each model on x64 and arm64.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dzm4YHdHn2qZy7aXer3KeH

* test(settings): carry ai.auto_title in the Kokoro settings fixture

main's conversation titles (#531) added ai.auto_title to the section the Kokoro settings test renders, so its fixture
names it too; without it Save sent an ai update beside the voice one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dzm4YHdHn2qZy7aXer3KeH

* docs(threat-model): count the Kokoro row in the JSON companion

The threat model workflow's validate job wants a change to threat-model.md to change a companion too (#507): the Kokoro row is documented, so the JSON's documented count goes up by one, as #518 did for Moondream.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dzm4YHdHn2qZy7aXer3KeH

* fix(voice): offer Kokoro in full precision and 8-bit only

On GitHub's arm64 runner, ONNX Runtime's CPU provider returned an empty waveform for kokoro-onnx's fp16 export, and kokoro-onnx's pause insertion then failed on a zero-size array. On x64 it was no faster than fp32, so voice.kokoro_model is now fp32 (the default) or int8, and the fp16 file is no longer pinned.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dzm4YHdHn2qZy7aXer3KeH

* test(settings): carry the research section in the Kokoro settings fixture

Main's deep research (#517) added a research section to Settings › AI assistant, so the fixture carries its defaults and Save sends only the voice change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dzm4YHdHn2qZy7aXer3KeH

* test(settings): carry ai.compact_at in the Kokoro settings fixture

Main added ai.compact_at (summarise long conversations early), so the fixture carries its default and Save sends only the voice change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dzm4YHdHn2qZy7aXer3KeH

* docs(readme): list answers about Lens itself as built, now that #400 is released

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dzm4YHdHn2qZy7aXer3KeH

* docs(changelog): name the Kokoro fragment after its pull request

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dzm4YHdHn2qZy7aXer3KeH

* docs(threat-model): count the Kokoro entries against main's newest model

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dzm4YHdHn2qZy7aXer3KeH

* docs(readme): take Kokoro out of In progress, now that it's built

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dzm4YHdHn2qZy7aXer3KeH

---------

Co-authored-by: Adeel Ahmad <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0