8000 feat(assistant): search the web and read pages, cited like excerpts by adeelahmad · Pull Request #472 · adeelahmad/lense · GitHub
[go: up one dir, main page]
Skip to content

feat(assistant): search the web and read pages, cited like excerpts - #472

Merged
adeelahmad merged 5 commits into
mainfrom
claude/project-thread-r960xc-c5-web
Oct 17, 2026
Merged

feat(assistant): search the web and read pages, cited like excerpts#472
adeelahmad merged 5 commits into
mainfrom
claude/project-thread-r960xc-c5-web

Conversation

@adeelahmad
@adeelahmad adeelahmad commented Oct 17, 2026
Copy link
Copy Markdown
Owner

Requested by Adeel · project thread

Description

Before: the assistant could only answer from what Lens holds. For anything else it had to say the archive doesn't cover it, or you had to capture a page into a namespace first (import_urls).

After: an admin turns on web search in Settings › AI assistant › Web search. It is off by default; one switch turns it on with DuckDuckGo, which needs no key. The assistant then gets two tools:

  • web_search returns titles, addresses, sites and snippets, with an optional time_range of day, week, month or year.
  • web_read reads a page or an online PDF, with find to keep only the paragraphs that matter.

Results and pages are numbered and cited [n] like archive excerpts. A web chip shows the site (example.org · Harbour news) and opens the page in a new tab, and the sources list shows a web card. A conversation keeps its web citations when it is opened again, and chat rooms list the page's address.

Providers:

  • DuckDuckGo, through ddgs (MIT).
  • SearXNG, your own instance.
  • Brave Search and Tavily, with an API key that is sealed like other secrets.

Motivation and Context

Module C5 of the chat-gaps plan (web search tools in Adeel's list). The plan's contract section 3 covers it, mainly 3.4 (tools) and 3.5 (citations).

How it works:

  • Network. Every request goes through netguard, the same guard as captured pages. It reaches public addresses only (plus documents.web_networks), on ports 80 and 443, and re-checks every redirect. A result's address or a redirect can't reach this machine's own services. Reads are capped at 10 MB and at web.read_chars for the model.
  • Pages that need scripts. With web.render on, a page that comes back nearly empty is loaded in Chromium and read again. Off by default, and only where Chromium is installed.
  • Prompt injection. Results reach the model marked as data. Once an answer has read outside text, a search or page read the model chose itself waits for approval. A query the person typed, an address they wrote, and an address a search or page returned all run at once. An approved call stores its result, and the same call again reuses it, so nothing is fetched twice.
  • Activity ledger. It gets web.search (provider, number of results) and web.read (host) rows, never the query or the page. Brave and Tavily rows are marked as estimates.
  • ddgs is an optional extra. It is the web extra in pyproject, about 11 MB of lxml and primp wheels, too heavy to be required. It is a component, web-search, fetched when an admin picks DuckDuckGo, like the smb extra. uv.lock was updated with uv.
  • Threat model. Added a §1.7 entry point. Updated P-SSRF-GUARD, P-INJECTION-EGRESS and N-EGRESS (SearXNG is an unguarded, admin-set address) and the admin-settings row. The YAML and JSON were regenerated, and ci/validate_threat_model.py passes.
  • Settings. The web settings are in the AI assistant section (backend ai, web, decisions, voice), so there is no new section.
  • Merges. Includes f6fd6c7 (test(notes): keep the daily notes test from failing on 2026-10-17 #470, the date-dependent notes test) and a merge of main, which brings in versions (feat(chat): versions: regenerate, edit and switch in a conversation tree #467). get_chat keeps both the conversation tree and web passages.

Screenshots (if appropriate):

n/a (covered by the jest tests below)

Steps to reproduce (if appropriate):

  1. Go to Settings › AI assistant › Web search and switch on "Let the assistant search the web and read pages". Save.
  2. Ask in chat: "What's the latest news on X?". The steps show "Searched the web for …" and "Read example.org/…".
  3. Click a web citation chip: the page opens in a new tab.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change; ! in the title)

Checklist:

  • The title is a Conventional Commit.
  • My code follows the code style of this project. ruff check, ruff format, mypy, eslint, prettier and tsc are clean.
  • I added or updated tests:
    • tests/api/test_web_search.py has 10 tests with a fake HTTP server and a fake ddgs, never the real internet;
    • __tests__/chat-web-citations.test.tsx;
    • __tests__/settings-web-search.test.tsx.
  • My change requires documentation updates, and I have updated docs/ accordingly:
    • docs/assistant.md: the new "Web search and reading pages" section, plus "Asks first";
    • docs/configuration.md: the web.* table;
    • docs/activity.md and docs/components.md.
  • Users need more than the title to understand it, so I wrote it under ## Unreleased in CHANGELOG.md.

Done questions (AGENTS.md)

  • Goal: delivers web search and page reading. Merged once CI is green; the coordinator merges.
  • Agents: these are assistant tools. An agent turns them on with PUT /api/v1/settings/web or through the setup chat's change_settings, which always asks first for web. MCP: n/a, because there is no MCP settings tool today.
  • Zero setup: no .env edit and no prerequisite. DuckDuckGo's client is fetched as a component when it is picked.
  • Fewest clicks and keystrokes: one switch turns it on with DuckDuckGo. The provider, key and address appear only when needed.
  • Configurable: all six web.* settings are in the web app with defaults, and none is in the setup wizard.
  • Light: off by default. ddgs is an optional extra, and Chromium rendering is off by default and only used where Chromium is installed.
  • Private and secure: off by default; guarded by netguard; the key is sealed; it asks first after reading outside text; the threat model is updated.
  • Cost: there are no extra model calls, and searches and reads are ledger rows. The paid providers are marked as estimates because Lens doesn't know the plan's price.
  • User decides: off by default, existing defaults unchanged, and calls the model chose after outside text wait for approval.
  • Upgrades: n/a, no change to stored data. The new Passage fields are optional and recording_id becomes optional.
  • Honest: answers cite web pages. The docs label the feature built and call the DuckDuckGo client unofficial.
  • Tests, docs and generated files updated. The OpenAPI schema and client were regenerated with tooling, and mkdocs build --strict is clean.
  • Open standards; dependencies keep Lens MIT. ddgs, lxml and primp are MIT/BSD and optional. SearXNG (AGPL) is only reached as the user's own server.
  • Result reported with evidence. The "Refine later" items are in docs/assistant.md:
    • an opt-in SearXNG Compose profile;
    • per-search prices for the paid providers;
    • per-namespace web access;
    • caching results.

Generated by Claude Code

claude added 4 commits October 16, 2026 23:28
@claude
fix(assistant): let the assistant's tools be imported first
db70c22 
Importing app.domain.ai_tools on its own failed: it imported automations,
which loads the actions, and the agent's action subclasses Toolbox before
ai_tools had defined it. Running one test file (tests/api/test_chats.py)
hit it. ai_tools now imports automations last.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
@claude
feat(assistant): search the web and read pages, cited like excerpts
b5d1bbc 
An admin turns web search on in Settings > AI assistant > Web search (off by
default). The assistant then gets web_search and web_read: it searches with
DuckDuckGo (no key; the ddgs client is the optional `web` extra, fetched as a
component when picked), the admin's SearXNG, Brave Search or Tavily, and reads
pages and online PDFs through netguard, public addresses only. Results and
pages are numbered and cited [n] like excerpts, with their site, and open in a
new tab; conversations keep them. Each search and read is an activity ledger
row (provider or host, never the query or page). Once an answer has read
outside text, a search or page the person or a tool didn't give it waits for
approval, and the approved call's result is reused.

Docs, changelog and threat model updated; generated client regenerated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
@claude
test(notes): keep the daily notes test from failing on 2026-10-17
b21b282 
The last check asked for 2026-10-17's note after the call without a date had made today's note, so on that day it
found the note the deleted root had left behind. It now asks for a day no earlier call makes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
@claude
Merge remote-tracking branch 'origin/main' into claude/project-thread…
3527dc5 
…-r960xc-c5-web

# Conflicts:
#	CHANGELOG.md
#	fastapi_backend/app/api/v1/routes/chats.py
#	nextjs-frontend/app/openapi-client/sdk.gen.ts
#	nextjs-frontend/openapi.json
@adeelahmad adeelahmad self-assigned this Oct 17, 2026
@claude
Merge remote-tracking branch 'origin/main' into port-c5
b15df06 
# Conflicts:
#	CHANGELOG.md
#	threat-model.md
#	threat-model.yaml
@adeelahmad
adeelahmad merged commit 6c4d952 into main Oct 17, 2026
18 checks passed
@adeelahmad
adeelahmad deleted the claude/project-thread-r960xc-c5-web branch October 17, 2026 01:12
adeelahmad pushed a commit that referenced this pull request Oct 17, 2026
@claude
Merge origin/main into claude/project-thread-r960xc-c3-export
abef0b0 
C1 (#469), web search (#472) and the sandbox (#473) are in main. The
routes keep C5's web pages in citations alongside the export, and the
export test now checks a cited web page stays through access changes.
The OpenAPI schema and client are regenerated with make openapi's two
commands, and the threat model's prose_version is recomputed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXPWXcSCQb9hWPr7xkZu7B
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@adeelahmad adeelahmad

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@adeelahmad @claude
0