8000 feat(chat): choose tools, a tool-call limit and a scope per answer by adeelahmad · Pull Request #510 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
ee327cf
fix(assistant): let the assistant's tools be imported first
claude Oct 16, 2026
defb487
feat(chat): each answer records what produced it
claude Oct 17, 2026
983d046
Merge remote-tracking branch 'origin/main' into claude/project-thread…
claude Oct 17, 2026
acf97d7
test(notes): keep the daily notes test from failing on 2026-10-17
claude Oct 17, 2026
db85b14
Merge origin/main into claude/project-thread-r960xc-c1-record
claude Oct 17, 2026
275692b
Merge remote-tracking branch 'origin/main' into port-c1
claude Oct 17, 2026
33a2c19
feat(chat): choose tools, a tool-call limit and a scope per answer
claude Oct 17, 2026
448565d
Merge remote-tracking branch 'origin/claude/project-thread-r960xc-c1-…
claude Oct 17, 2026
957c3f1
docs(chat): choosing tools, namespaces and a limit; regenerate the cl…
claude Oct 17, 2026
8759a60
feat(chat): a choice made as a conversation starts is the latest choice
claude Oct 17, 2026
73d10e7
fix(chat): setting the server up keeps every tool, whatever the lates…
claude Oct 17, 2026
554668b
Merge remote-tracking branch 'origin/main' into claude/project-thread…
claude Oct 17, 2026
a22d140
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
3f0b9de
test(entities): wait for the typed search before counting merge options
claude Oct 17, 2026
33e40f0
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
dde1384
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
e6b5cfa
test(settings): the AI section's server values include ai.max_tool_calls
claude Oct 17, 2026
cdf1f0c
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
a9e21e5
fix(chat): find_resources belongs to the archive's tools
claude Oct 17, 2026
daa8aaa
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
98b7a8b
docs(chat): a new built-in tool joins a tool group
claude Oct 17, 2026
093d4af
Merge origin/main into claude/project-thread-r960xc-c4-tools
claude Oct 17, 2026
1b55d2b
fix(ci): the upgrade check counts entities of every type
claude Oct 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,13 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version

## Unreleased

- **Choose the tools, a limit and the namespaces of an answer.** A conversation, or one question, can pick the tool
groups or tools its answers may use (`tools`), cap their tool calls (`max_tool_calls`, up to the new Settings → AI
assistant → Most tool calls a question may ask for, 50 by default) and narrow one answer to its own namespaces
(`scope`). `GET /api/v1/chats/tools` lists the groups a person has, what each tool does (reads, asks first, sends data
out) and why a group is missing. An admin's switches still win, and new conversations start with the person's last
choice. The web app's Tools chip is in progress
([Choosing tools, namespaces and a limit](docs/assistant.md#choosing-tools-namespaces-and-a-limit)).
- **Names get their real types, not Term.** The built-in extractor now tells people, places, organisations, events,
works and dates apart without a model: by a title or role before a name, a given name, the word a name ends or
starts with (Corp, University of, Summit, Act, River), lists of countries, regions and large cities and of
Expand Down
5 changes: 4 additions & 1 deletion ci/upgrade/lens_upgrade.py
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,10 @@ def facts(api, token):
c["text"] for c in api.get(f"/resources/{rid}/comments")
)
entities = api.get("/entities", namespaces=NS, limit=200)["items"]
out["entities"] = sorted(e["name"] for e in entities)
# every entity by name, whatever its type: an upgrade step may type a Term (Monday becomes a Date, which the list
# leaves out unless asked for), but none may go
quiet = api.get("/entities", namespaces=NS, limit=200, types="DATE,NUMBER")["items"]
out["entities"] = sorted(e["name"] for e in entities + quiet)
for e in entities:
if e["name"] == ENTITY["name"]:
out[f"entity {e['name']}"] = {
Expand Down
10 changes: 9 additions & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1493,6 +1493,7 @@ version.
```
GET /api/v1/chats
GET /api/v1/chats/capabilities
GET /api/v1/chats/tools
POST /api/v1/chats
GET /api/v1/chats/{cid}
PATCH /api/v1/chats/{cid}
Expand All @@ -1516,7 +1517,7 @@ that node runs without asking from now on). Approving, editing and `always` on a
session (403 for an API key or an app); each approval's `can` says what you may do with it.

`GET /chats/capabilities` tells anyone signed in whether a language model is set up (`configured`), which (`model`),
whether it uses tools (`tools`, `max_steps`), whether answers can be checked (`check`) and the `models` people may pick
whether it uses tools (`tools`, `max_steps`, `max_tool_calls`), whether answers can be checked (`check`) and the `models` people may pick
(the configured one first; `llm.chat_models`, else the server's list); never the model server's address or key. A
conversation's `model` (`POST /chats`, `PATCH /chats/{cid}`; null for the configured one) answers in it, and a
question can name another (`POST /chats/{cid}/messages {content, model}`, e.g. to retry); a model that isn't offered is
Expand All @@ -1529,6 +1530,13 @@ step it's on: the stream sends `stopped`, then `done` with the saved message, wh
everything after it are replaced by this question and its new answer; 404 when `edit` isn't a question in that
conversation. A conversation titled after the first question is retitled when that question is edited.

`GET /chats/tools` lists the tool groups and tools you have, each tool's `effect` (`read`, `asks` or `out`), why a
group isn't available and the setting that switches it on, the most tool calls a question may ask for
(`max_tool_calls`) and your latest choice (`remembered`); `chat_id` reads it for that conversation's scope. A
conversation's `tools` and `max_tool_calls` (`POST /chats`, `PATCH /chats/{cid}`; null for Auto) are what its answers
may use, and a question can set its own, with a `scope` for that answer alone (`POST /chats/{cid}/messages {content,
tools, max_tool_calls, scope}`) ([Choosing tools, namespaces and a limit](assistant.md#choosing-tools-namespaces-and-a-limit)).

A conversation's `scope` narrows what it draws on: `namespaces`, `recordings`, `collections`, `speakers`, `from` and
`to`; every key narrows it further, and an empty scope is everything you can read. `collections` are ids of
collections you can see (yours or shared); the conversation draws on their assets as they are each time it
Expand Down
56 changes: 55 additions & 1 deletion docs/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,8 @@ When the configured model supports function calling, chat becomes an agent.
(`save_automation` with `enabled`) and running one now wait for the person's approval; switching one off acts at once
([Automations](automations.md#agents)).
- **Limits:** everything is limited to what the person can read and to the conversation's scope, and capped by
`ai.max_steps` (Settings → AI assistant, Tool steps per answer; 6 by default) and `ai.max_transcript_reads`.
`ai.max_steps` (Settings → AI assistant, Tool steps per answer; 6 by default) and `ai.max_transcript_reads`, or by
the tool calls a question allows ([below](#choosing-tools-namespaces-and-a-limit)).
- **Always an answer:** the model is told its step budget. When the steps run out, or it replies with nothing, it is
asked once more to answer from what it found, with the tools off, and to say what it didn't get to check. A server
that calls tools anyway gets the conversation again as plain text, without tools. That last ask is one more model
Expand Down Expand Up @@ -239,6 +240,59 @@ gets two more tools, for everyone who may use the assistant:
Refine later: an opt-in SearXNG service in the Docker Compose file (SearXNG is AGPL, so it would stay an extra),
prices per search for the paid providers, web access per namespace, and keeping recent results to search less.

### Choosing tools, namespaces and a limit

By default (Auto) an answer may use every tool the person has. A conversation, or one question, can instead pick the
tools it may use, how many tool calls it may make, and which namespaces it reads.

- **Tool groups.** `GET /api/v1/chats/tools` lists the groups and the tools in each that the person has, and why a
group isn't there for them:
- `archive` (search and read assets);
- `graph` (entities, topics, the knowledge graph);
- `web` (searching the web and reading pages, once an admin has set up a search);
- `notes`;
- `memory` (a namespace's own assistant);
- `activity` (the timeline, approvals and automations);
- `changes` (proposals that become approvals);
- `files` (importing files and web pages);
- `code` (the [code sandbox](#code-sandbox): Python and bash, once an admin turns it on);
- `extensions`;
- `server` (admins);
- `browser`;
- `mcp:<name>` and `ext:<name>`, one for each of the person's extensions.

Each tool says its `effect`: `read`, `asks` (it waits for the person's approval) or `out` (it sends data out of
Lens, as a web search, the browser's open, click and type, and code runs while the sandbox's network is on do). A
group that isn't available says why (switched off by an admin, no web search is set up, the code sandbox is off,
for admins, or needs editor access to a namespace) and names the setting that switches it back on (`setting`, such
as `ai.browser`, `web.search` or `sandbox.enabled`).
- **Picking.** `tools` (on `POST /chats`, `PATCH /chats/<id>` or one question, `POST /chats/<id>/messages`) is a list
of group and tool names. The model is then offered only those tools, and a tool it calls that it wasn't offered isn't
run. Null is Auto. An empty list means no tools: the answer is written from a search, as when the model can't use
tools. A name that is no group or tool the person has is a 400. An admin's switches (`ai.disabled_tools`,
`ai.browser`, `ai.extensions`) still win over any pick.
- **A limit on tool calls.** `max_tool_calls` (in the same places) caps how many tool calls one answer may make in
all. The model is told the budget. Calls past it get a reply saying they weren't made, and then the answer is written
from what was found, with outcome `limit`. With a limit, the rounds of calls may go past `ai.max_steps`, up to the
limit. It is a whole number from 1 to `ai.max_tool_calls` (Settings → AI assistant, Most tool calls a question may
ask for; 50 by default, at most 500). `GET /chats/capabilities` gives it as `max_tool_calls`.
- **A scope for one answer.** `scope` on `POST /chats/<id>/messages` narrows that answer alone, with the same keys as
a conversation's scope. `{}` means everything the person can read, and a namespace they can't read is a 404. The
conversation keeps its own scope, and a question with a scope isn't narrowed by the
[namespace picker](#picking-the-namespace-for-a-conversation-over-everything).
- **Remembered.** Each answer records what it was offered and its limits (`tools`, `limits`, `scope`), and each
question records the options it was asked with ([What each answer records](#what-each-answer-records)). A
conversation keeps its `tools` and `max_tool_calls`. The person's latest choice (the last `POST /chats` or `PATCH
/chats/<id>` that set them) is kept on their account, so a new conversation starts with it (`remembered` in `GET
/chats/tools`) unless `POST /chats` says otherwise.
- **Web and MCP tools reach beyond your namespaces.** The namespaces picked narrow the archive's tools only. A web
search, an MCP server or the browser reads and sends wherever it reaches.

In the web app, a Tools chip beside the composer's scope picks these (in progress).

Refine later: tool choice in chat rooms and voice mode, a default choice per namespace, and the model suggesting the
tools a question needs.

## Code sandbox

Built: an admin turns it on in Settings → Code sandbox (`sandbox.enabled`, off by default) and points Lens at an
Expand Down
Loading
Loading
0