8000 feat(chat): versions: regenerate, edit and switch in a conversation tree by adeelahmad · Pull Request #467 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
`automation_actions`, `save_automation`, `test_automation` and `run_automation`, with problems in plain words. The
assistant saves them switched off and asks before switching one on or running it
([Agents](docs/automations.md#agents)).
- **Versions in conversations.** A conversation is now a tree: editing a question makes a new version beside it
instead of deleting what followed, regenerating an answer (`regenerate`, with the same model or another) writes one
beside it, and a question can go on from any message (`parent`). `GET /api/v1/chats/<id>` shows each message's
versions and every version's messages, and `PUT /api/v1/chats/<id>/head` switches between them; the model reads
only the version showing. The web app's switcher and Regenerate button are in progress. Existing conversations are
upgraded on the next start (`chat-tree`) ([Versions](docs/assistant.md)).
- **Describe an automation in words.** `POST /api/v1/automations/draft` asks the model once to turn a sentence into
a draft automation to try and save, and says what didn't fit and what it couldn't express
([Describe it in words](docs/automations.md#describe-it-in-words)).
Expand Down
17 changes: 13 additions & 4 deletions docs/assistant.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,22 @@ then.
is writing it); what came before is saved, marked stopped. A model call already under way finishes first.
- **Reopening:** a conversation's answers keep the tools the assistant used (and with what), any notice (the model
couldn't use tools), the error when there was no answer, and their latest source check.
- **Editing a question:** Edit on a question you asked asks it again as edited; the answer and everything after it
are replaced (`POST /api/v1/chats/<id>/messages` with `edit`).
- **Versions:** a conversation is a tree, and nothing in it is deleted. Regenerating an answer writes a new answer to
the same question beside it, with the same model or another (`POST /api/v1/chats/<id>/messages` with `regenerate`:
the answer's id, and `model`); editing a question makes a new version of it beside the old one, which keeps what
followed it (`edit`: the question's id, with the new `content`). The version showing is what the model reads and
what the next question goes on from; `parent` goes on from any other message instead. `GET /api/v1/chats/<id>`
gives the version showing as `messages`, each with its `versions` (`index` from 1, `count`, `ids`: it and the
others in its place, oldest first), every version's messages as `tree`, and `head`, the last message showing. `PUT
/api/v1/chats/<id>/head` with `{"message": <id>}` shows the version that message is in, to its newest message. A
new version streams `branch` first (`message`, `parent`, `versions`). Chat rooms ([Chat rooms](chat-rooms.md)) and
automations go on from the version showing. In the web app, Edit makes a version; switching between versions and
Regenerate are in progress.
- **Long conversations:** the model reads the latest six messages word for word. Once eight more have built up
behind them, they're folded into a running summary kept on the conversation (one model call, after an answer), and
every answer gets that summary plus the messages it doesn't cover yet. Room conversations (chat-rooms.md) go on for
good, so they rely on it most. Editing a question the summary covers drops the summary; it's made again from what's
left. A conversation that fills half of the model's context window (**Chat model's context**, `ai.context_tokens`,
good, so they rely on it most. A summary covers the version it was written on: on a version that leaves what it
covers, the model reads that version's messages, and the next summary is made from them. A conversation that fills half of the model's context window (**Chat model's context**, `ai.context_tokens`,
16000 when empty; about four characters a token) is folded in early, before the next answer, whatever the count.
**Model for summaries** (`ai.compact_model`) writes them with a smaller, cheaper model on the same provider; empty,
the chat's model does. Settings › AI assistant › Summarise long conversations (`ai.compact`, on) turns it off; off,
Expand Down
11 changes: 11 additions & 0 deletions docs/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,17 @@ tables, or changing or removing an index that already exists, is a named step in
* `lens migrations` lists every step (done, pending, failed, or unknown when a newer Lens ran it) without running
anything, so it works while an upgrade is failing; `lens migrations --run` runs the pending ones.

The steps so far, in the order they run:

* `access-levels`: IIIF-only access levels (public, transcript, signed-in, private) become the access setting.
* `collection-homes`: every namespace gets its default collection, and every asset a home in one.
* `tour-seen`: people who signed in before the welcome tour existed don't get it.
* `document-read-step`: documents and images queued to be transcribed are read instead, and failed ones run again.
* `chat-tree`: conversations become trees of versions ([Versions](assistant.md)): each message of a conversation
(`chat_message`) gets the one before it as its `parent`, and the conversation (`chat`) its last message as its
`head`. Only conversations without a `head` are touched, so it's safe to run twice; until it has run, a conversation
without a `head` is read as one line, and saved as one before anything is added to it.

To add a step:

1. Write a function `fn(db)` next to the code it serves. It must be safe to run twice, since a start that stops half
Expand Down
114 changes: 90 additions & 24 deletions fastapi_backend/app/api/v1/routes/chats.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""Conversations with the archive, the assistant's approvals, and checking an answer against its sources.

A question streams back server-sent events: ``scoped`` (the namespace a conversation over everything was narrowed
A question streams back server-sent events: ``branch`` (a new version: regenerated, edited, or beside others),
``scoped`` (the namespace a conversation over everything was narrowed
to) or ``suggested`` (namespaces to offer when it isn't clear, new ones included), ``step`` (a tool the assistant used), ``approval`` (work it proposed that waits for you), ``notice``, ``passages`` (the numbered excerpts), ``token`` (answer text), ``error`` and ``done``.
The assistant only ever sees, and cites, what the asker can read.
"""
Expand Down Expand Up @@ -33,6 +34,7 @@
ChatScope,
ChatSummary,
ChatUpdate,
HeadUpdate,
MessageCreate,
StopResult,
)
Expand Down Expand Up @@ -111,9 +113,28 @@ def create_chat(user: Writer, acl: Acl, db: Db, cfg: Cfg, body: ChatCreate | Non

@router.get("/chats/{cid}")
def get_chat(cid: int, user: CurrentUser, acl: Acl, db: Db) -> Chat:
"""A conversation with its messages. Citations follow your current access: ones you can no longer read are left out."""
"""A conversation with the version showing (`messages`, each with its `versions`) and every version's messages
(`tree`). Citations follow your current access: ones you can no longer read are left out."""
return _chat_view(db, acl, _own_chat(db, cid, user))


@router.put("/chats/{cid}/head")
def switch_version(cid: int, body: HeadUpdate, user: Writer, acl: Acl, db: Db) -> Chat:
"""Show another version: the one `message` is in, from the first message to the newest one below `message`. The
next question goes on from there. Returns the conversation as GET /chats/{cid} does."""
c = _own_chat(db, cid, user)
msgs, ok = chat.history(db, cid), set(acl.roles)
try:
c["head"] = chat.switch(db, cid, body.message)
except KeyError:
raise HTTPException(404, "no such message in this conversation") from None
return _chat_view(db, acl, c)


def _chat_view(db: DB, acl: Access, c: dict[str, Any]) -> Chat:
nodes = chat.tree(db, c["id"])
msgs, ok, versions = chat.path(db, c["id"], c.get("head"), nodes), set(acl.roles), chat.versions(nodes)
for m in msgs:
m["versions"] = versions.get(m["id"])
ids = {p["recording_id"] for m in msgs for p in m.get("passages") or []}
visible = (
{
Expand All @@ -126,7 +147,7 @@ def get_chat(cid: int, user: CurrentUser, acl: Acl, db: Db) -> Chat:
)
for m in msgs:
m["passages"] = [p for p in m.get("passages") or [] if p["recording_id"] in visible]
return Chat.model_validate(sign_urls({**c, "messages": msgs}, full=True))
return Chat.model_validate(sign_urls({**c, "messages": msgs, "tree": nodes}, full=True))


@router.patch("/chats/{cid}")
Expand Down Expand Up @@ -179,9 +200,14 @@ async def send_message(cid: int, body: MessageCreate, user: Writer, acl: Acl, db
reads with the question). Streams events: step, approval, notice, passages, token (answer text), error, stopped (POST
/chats/{cid}/stop: what came before is saved, marked stopped), done (the saved message id).

With `edit`, one of your earlier questions is edited: it and everything after it are replaced by this question and
its new answer (404 if it isn't a question in this conversation). Without `context`, it keeps the page it was asked
from (where, and the highlighted text; not the page's text, which isn't kept).
A conversation is a tree of versions (GET /chats/{cid}: `tree`, and each message's `versions`); the question goes
on from `parent` when given (any message of the conversation), else from the version showing (`head`). With
`regenerate` (an answer's id) its question is answered again, as a new version beside that answer (`content` isn't
needed; `model` may pick another). With `edit`, one of your earlier questions is edited: this question becomes a new
version beside it and is answered, and the old one stays with what followed it (404 if it isn't a question in this
conversation). Either keeps the page the question was asked from (where, and the highlighted text; not the page's
text, which isn't kept) unless `context` is given, and streams `branch` first ({message, parent, versions}: the
new version, the message it follows and its versions); so does a question beside others under `parent`.

`/clear` makes the model start afresh (the messages stay to read) and `/compact [what to keep]` folds the
conversation so far into its summary now; each streams cleared or compacted, then its reply as an answer.
Expand All @@ -190,17 +216,37 @@ async def send_message(cid: int, body: MessageCreate, user: Writer, acl: Acl, db
browser while the side panel is connected: open pages, read them, click, type and download files into the
conversation (docs/browser-assistant.md)."""
c = await run_in_threadpool(_own_chat, db, cid, user)
q = body.content.strip()[:4000]
files = await run_in_threadpool(_attachments, db, body.attachments, user)
if not q and files:
q = "I attached " + ("this file." if len(files) == 1 else "these files.")
if not q:
raise HTTPException(400, "ask something")
cmd = chat.command(q) if not files and body.edit is None else None
if body.regenerate is not None and body.edit is not None:
raise HTTPException(400, "regenerate an answer or edit a question, not both")
if body.parent is not None and (body.regenerate is not None or body.edit is not None):
raise HTTPException(400, "parent is for a new question: regenerate and edit go beside their message")
again: dict[str, Any] | None = None # regenerating: the question answered again
under: Any = chat.HEAD # the message the question follows
if body.regenerate is not None:
try:
again = found = await run_in_threadpool(chat.question_of, db, cid, body.regenerate)
except KeyError:
raise HTTPException(404, "no such answer in this conversation") from None
q, files = found["content"], found.get("attachments") or []
if chat.command(q):
raise HTTPException(400, f"{q.split()[0]} isn't answered by the model, so there's nothing to regenerate")
else:
q = body.content.strip()[:4000]
files = await run_in_threadpool(_attachments, db, body.attachments, user)
if not q and files:
q = "I attached " + ("this file." if len(files) == 1 else "these files.")
if not q:
raise HTTPException(400, "ask something")
if body.parent is not None:
try:
under = (await run_in_threadpool(chat.said, db, cid, body.parent))["id"]
except KeyError:
raise HTTPException(404, "no such message in this conversation") from None
cmd = chat.command(q) if not files and body.edit is None and again is None else None
if cmd: # /clear or /compact: done here, answered like a question, without asking the model anything else

def run() -> Iterator[str]:
reply, mid = chat.run_command(db, cfg, cid, *cmd)
reply, mid = chat.run_command(db, cfg, cid, cmd[0], cmd[1], parent=under)
yield _ev({"clear": "cleared", "compact": "compacted"}[cmd[0]], {"message": mid})
yield _ev("passages", [])
yield _ev("token", {"text": reply})
Expand All @@ -214,11 +260,16 @@ def run() -> Iterator[str]:
title = c["title"]
page = body.context.model_dump() if body.context else None
kept = chat.shared_context(page) # what the question keeps of the page
if body.edit is not None: # only after the question checks out, so a refused edit removes nothing
if again is not None:
under = again.get("parent")
if page is None and again.get("context"):
page = again["context"]
if body.edit is not None:
try:
old = await run_in_threadpool(chat.rewind, db, cid, body.edit)
old = await run_in_threadpool(chat.said, db, cid, body.edit, "user")
except KeyError:
raise HTTPException(404, "no such question in this conversation") from None
under = old.get("parent")
if title == old["content"][:80]: # titled after the question being edited: retitle it after the edit
title = "New conversation"
if page is None and old.get("context"):
Expand All @@ -227,17 +278,28 @@ def run() -> Iterator[str]:
asked = chat.with_context(q, page, cfg) + chat.attached_note(files)
readable = set(acl.roles)
summary: dict[str, Any] | None = None # what compaction keeps of the earlier messages (chat.compact)
question: int | None = again["id"] if again else None # the question being answered
answer_id = 0 # the answer's message id, taken once the question is saved (a reply is newer than what it follows)
branched: dict[str, Any] | None = None # the `branch` event, when this makes a new version

def prepare() -> tuple[list[dict[str, Any]], list[dict[str, Any]], Any]:
nonlocal summary
nonlocal summary, under, question, answer_id, branched
under = chat.head(db, cid) if under == chat.HEAD else under
past = chat.path(db, cid, under) # the version it goes on from: all the model reads of the conversation
try: # a conversation that fills the model's context is folded into its summary before it's answered
chat.compact(db, cfg, cid, full_only=True)
chat.compact(db, cfg, cid, full_only=True, line=past)
except Exception: # noqa: BLE001 - answered with what fits; tried again after the answer
log.warning("chat %s: couldn't update the summary", cid, exc_info=True)
past = chat.history(db, cid)
summary = chat.memory(db, cfg, cid)
chat.add(db, cid, "user", q, attachments=files, context=kept)
if not past and title == "New conversation":
summary = chat.memory(db, cfg, cid, past)
if again is None:
question = chat.add(db, cid, "user", q, attachments=files, context=kept, parent=under)
answer_id = db.next_id("chat_message")
if again is not None:
branched = chat.branch(db, cid, answer_id, question, "assistant")
elif body.edit is not None or body.parent is not None: # beside other questions, unless it went on from a leaf
made = chat.branch(db, cid, question, under, "user")
branched = made if made["versions"]["count"] > 1 else None
if not past and again is None and title == "New conversation":
db.q("UPDATE $r SET title = $t", r=R("chat", cid), t=q[:80])
# A conversation over everything gets the namespace its first question is about, when that's clear.
if past or c.get("kind") == "setup" or c.get("scope"):
Expand Down Expand Up @@ -267,7 +329,9 @@ def prepare() -> tuple[list[dict[str, Any]], list[dict[str, Any]], Any]:

def save(text: str, cited: list[dict[str, Any]], **kw: Any) -> int:
wrote = (model or cfg["llm"].get("model")) if llm.configured(cfg) else None
return int(chat.add(db, cid, "assistant", text, cited, steps=steps, notice=notice, model=wrote, **kw))
return int(
chat.add(db, cid, "assistant", text, cited, steps=steps, notice=notice, model=wrote, parent=question, mid=answer_id, **kw)
)

def stopped(text: str, cited: list[dict[str, Any]]) -> Iterator[str]:
yield _ev("stopped", {"message": "Stopped"})
Expand Down Expand Up @@ -350,6 +414,8 @@ def gen() -> Iterator[str]:
on = chat.Answering(db, cid)
saved = False
try:
if branched:
yield _ev("branch", branched)
if picked:
yield _ev("scoped", {"namespaces": c["scope"]["namespaces"], & B98C quot;confidence": picked["confidence"], "by": picked["by"]})
yield _ev("step", steps[0])
Expand Down
4 changes: 2 additions & 2 deletions fastapi_backend/app/domain/bridge.py
Original file line number Diff line number Diff line change
Expand Up @@ -327,8 +327,8 @@ def _answer(db, cfg, base, account, cid, q, files, act_on, outside):
chat.compact(db, cfg, cid, full_only=True)
except Exception: # noqa: BLE001 - answered with what fits; tried again after the answer
log.warning("bridge: couldn't update the summary of chat %s", cid, exc_info=True)
past = chat.history(db, cid)
summary = chat.memory(db, cfg, cid)
past = chat.path(db, cid) # a room goes on from the head (the version showing, should it be switched in the app)
summary = chat.memory(db, cfg, cid, past)
scope = (db.one("SELECT scope FROM $r", r=R("chat", cid)) or {}).get("scope") or None
chat.add(db, cid, "user", q, attachments=list(files) or None)
asked = q + chat.attached_note(list(files))
Expand Down
Loading
Loading
0