8000 feat(notes): page history, so any change (the assistant's too) can be undone by adeelahmad · Pull Request #139 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DEVLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ Todo:
- [x] Costs of refinement and filing to the activity ledger (`note_page:id` and its namespace; shown on the page)
- [x] Assistant tools: find, read, write and update notes (at once, no approval: notes are its notebook)
- [x] MCP tools to find, read and write notes (find_notes, read_note, write_note)
- [ ] Page history so the assistant's changes can be undone
- [x] Page history (note_version): a version before each change, one per sitting, restore that is itself undoable
- [x] Self-organising, first step: notes nobody filed are filed in PARA by a decision model (or the LLM), with a
suggestion when it isn't sure (ai.organise_notes)
- [ ] Self-organising, next: link notes to entities and topics, nest them under their project or area
Expand Down
14 changes: 11 additions & 3 deletions docs/notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ assistant write. The assistant is the main writer and organiser; people can do e
Status: **built**: pages (free notes in a tree, a page per recording, entity, topic, collection or speaker), @ and #
links with backlinks (# links topics), the tree in the left navigation, the page view with the BlockSuite editor, the
model keeping titles and summaries up to date, assistant tools to find, read, write and update notes, and MCP tools to
find, read and write them ([MCP](mcp.md)).
**Planned**: page history, linking notes to entities and topics and nesting them under their project or
find, read and write them ([MCP](mcp.md)), and page history with restore.
**Planned**: linking notes to entities and topics and nesting them under their project or
area on their own, attachments on encrypted object storage, pages in the graph.

## A page
Expand Down Expand Up @@ -68,6 +68,14 @@ Assistant lists them). It looks there first, and writes or updates a note when i
filed in PARA and linked with @ and #. Writing needs editor access to the namespace and takes effect at once, with no
approval: notes it writes say so. It never deletes a note.

## History

Before each change to a page's title, summary or text, Lens keeps what the page was as a version (`note_version`, up to
100 per page). A person's edits in one sitting (ten minutes) make one version; a change by the assistant, the model
refining it, or someone else always starts a new one. **History** on a page lists them, newest first, with who changed
it; pick one to read it, and editors can restore it. Restoring keeps the page as it was too, so it can be undone.
`GET /api/v1/notes/{id}/history`, `GET .../history/{version}` and `POST .../history/{version}/restore`.

## Access

Pages belong to a namespace: anyone with a role there reads them, editors write, move and delete them. The pages of
Expand All @@ -84,5 +92,5 @@ co-editing rather than inside Lens.

## Refine later

Pages for people who see only some collections of a namespace; page history and undo; attachments; live co-editing
Pages for people who see only some collections of a namespace; differences between versions; attachments; live co-editing
through OctoBase; pages and their links as nodes and relationships in the graph.
37 changes: 37 additions & 0 deletions fastapi_backend/app/api/v1/routes/notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
from app.domain.store import DB
from app.schemas.common import Ok
from app.schemas.notebook import (
NoteHistory,
NoteLinkTarget,
NotePage,
NotePageCreate,
Expand All @@ -23,6 +24,8 @@
NotePageMove,
NotePageUpdate,
NoteTree,
NoteVersion,
NoteVersionItem,
)

router = APIRouter(prefix="/notes", tags=["notes"])
Expand Down Expand Up @@ -151,6 +154,40 @@ def update_page(pid: int, body: NotePageUpdate, user: Writer, acl: Acl, db: Db)
return _out(acl, db, notebook.get(db, pid), user)


def _version_out(db: DB, v: dict[str, Any], cls: type[NoteVersionItem] = NoteVersionItem) -> NoteVersionItem:
who = db.one("SELECT email FROM $r", r=store.R("account", v["by"])) if v.get("by") else None
return cls(**{**{k: x for k, x in v.items() if k not in ("by", "page", "doc")}, "by": (who or {}).get("email")})


@router.get("/{pid}/history")
def page_history(pid: int, user: CurrentUser, acl: Acl, db: Db, limit: int = Query(50, ge=1, le=100)) -> NoteHistory:
"""What the page was before each change to its title, summary or text, newest first."""
_page(acl, db, pid)
return NoteHistory(versions=[_version_out(db, v) for v in notebook.history(db, pid, limit)])


@router.get("/{pid}/history/{vid}")
def page_version(pid: int, vid: int, user: CurrentUser, acl: Acl, db: Db) -> NoteVersion:
"""One earlier version, with its text."""
_page(acl, db, pid)
try:
v = notebook.version(db, pid, vid)
except KeyError:
raise HTTPException(404, "not found") from None
return _version_out(db, v, NoteVersion) # type: ignore[return-value]


@router.post("/{pid}/history/{vid}/restore")
def restore_version(pid: int, vid: int, user: Writer, acl: Acl, db: Db) -> NotePage:
"""Put an earlier version back. What the page was becomes a version too, so this can be undone. Needs editor access."""
_page(acl, db, pid, "editor")
try:
notebook.restore(db, pid, vid, user.id)
except KeyError:
raise HTTPException(404, "not found") from None
return _out(acl, db, notebook.get(db, pid), user)


@router.post("/{pid}/move")
def move_page(pid: int, body: NotePageMove, user: Writer, acl: Acl, db: Db) -> NotePage:
"""Move a free note in the tree: inside `parent` (null: the top), before `before` (null: at the end)."""
Expand Down
97 changes: 94 additions & 3 deletions fastapi_backend/app/domain/notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,11 @@
holds what Markdown can't, like drawings on the edgeless canvas. A change to the Markdown alone (the assistant's, say)
keeps that state but marks it stale, and the editor brings its text in line with the Markdown.

Refine later: pages for partial collection members, page history and undo, attachments, real-time co-editing.
Every change to a page's title, summary or text keeps what it was before as a version (`note_version`), so a page's
history can be read and any version restored, the assistant's changes included. Edits by one person in one sitting
make one version.

Refine later: pages for partial collection members, attachments, real-time co-editing, diffs between versions.
"""

from __future__ import annotations
Expand Down Expand Up @@ -255,11 +259,95 @@ def _doc(doc):


UNSET = object()
VERSION_SITTING = 600 # seconds: one person's edits this close together make one version
VERSIONS_MAX = 100 # versions kept per page; the oldest go first
VERSION_FIELDS = "record::id(id) AS id, page, at, by, author, title, summary, place, size, why"


def snapshot(db, p, account, author, why=None):
"""Keep page p (a row with its body) as it is, before a change by account (None: the model) as author. Changes by
the same person in one sitting share one version, the one from before they started; anyone else's change, or the
assistant's, starts a new one."""
last = db.one(
"SELECT at, by, author FROM note_version WHERE page = $p ORDER BY at DESC LIMIT 1",
p=int(p["id"]),
)
now = dt.datetime.now(dt.timezone.utc)
if last and why is None and author == "person" and last.get("author") == "person" and last.get("by") == account:
at = dt.datetime.fromisoformat(str(last["at"]).replace("Z", "+00:00"))
if (now - at).total_seconds() < VERSION_SITTING and str(p.get("updated_by") or "") == str(account or ""):
return None
body = p.get("body") or ""
vid = db.next_id("note_version")
db.q(
"CREATE $r CONTENT $d",
r=R("note_version", vid),
d=store.clean(
{
"page": int(p["id"]),
"space": p["space"],
"at": now.isoformat(timespec="milliseconds"),
"by": account,
"author": author,
"title": p.get("title"),
"summary": p.get("summary"),
"place": p.get("place"),
"body": body,
"doc": p.get("doc"),
"size": len(body),
"why": why,
}
),
)
old = db.rows(
"SELECT record::id(id) AS id, at FROM note_version WHERE page = $p ORDER BY at DESC START $n", p=int(p["id"]), n=VERSIONS_MAX
)
if old:
db.q("DELETE note_version WHERE id IN $ids", ids=[R("note_version", v["id"]) for v in old])
return vid


def history(db, pid, limit=50):
"""The page's earlier versions, newest first, without their bodies."""
return db.rows(f"SELECT {VERSION_FIELDS} FROM note_version WHERE page = $p ORDER BY at DESC LIMIT $n", p=int(pid), n=int(limit))


def version(db, pid, vid):
"""One earlier version with its body, or KeyError."""
v = db.one(f"SELECT {VERSION_FIELDS}, body, doc FROM $r", r=R("note_version", int(vid)))
if not v or v.get("page") != int(pid):
raise KeyError(vid)
return v


def restore(db, pid, vid, account):
"""Put an earlier version back: its title, summary and text (and the editor's state with them). What the page was
becomes a version too, so a restore can be undone."""
v = version(db, pid, vid)
update(
db,
pid,
account,
title=v.get("title"),
body=v.get("body") or "",
summary=v.get("summary"),
doc=v.get("doc"),
why=f"restored the version of {str(v['at'])[:16].replace('T', ' ')}",
)


def update(db, pid, account, title=None, body=None, summary=UNSET, date=UNSET, place=UNSET, author="person", doc=UNSET, view=UNSET):
"""Change what's given. A new body without `doc` marks the editor's state stale; the summary records who wrote it."""
def update(
db, pid, account, title=None, body=None, summary=UNSET, date=UNSET, place=UNSET, author="person", doc=UNSET, view=UNSET, why=None
):
"""Change what's given. A new body without `doc` marks the editor's state stale; the summary records who wrote it.
A change to the title, summary or text keeps what the page was as a version first (snapshot)."""
p = get(db, pid)
if (
(title is not None and _title(title) != p.get("title"))
or (body is not None and _body(body) != (p.get("body") or ""))
or (summary is not UNSET and _summary(summary) != p.get("summary"))
):
snapshot(db, p, account, author, why)
sets, args = ["updated_at = $t", "updated_by = $a"], {"t": store.now(), "a": account}
if title is not None:
sets.append("title = $title")
Expand Down Expand Up @@ -328,6 +416,7 @@ def delete(db, pid):
for child in db.values("SELECT VALUE record::id(id) FROM note_page WHERE parent = $p", p=int(pid)):
db.q("UPDATE $r SET parent = $up", r=R("note_page", child), up=p.get("parent"))
db.q("DELETE note_link WHERE page = $p", p=int(pid))
db.q("DELETE note_version WHERE page = $p", p=int(pid))
db.q("DELETE $r", r=R("note_page", int(pid)))


Expand Down Expand Up @@ -496,6 +585,8 @@ def refine(db, cfg, pid):
return {} # changed while the model was thinking: it's still pending, so a later pass looks again
changed = {k: v for k, v in (("title", title), ("summary", summary)) if v and v != p.get(k)}
sets = ["refine_pending = false", "refine_claim = NONE", "refined_at = $t"] + [f"{k} = ${k}" for k in changed]
if changed:
snapshot(db, get(db, p["id"]), None, "assistant", "the model refined the title and summary")
if "summary" in changed:
sets.append("summary_by = 'assistant'")
db.q(f"UPDATE $r SET {', '.join(sets)}", r=R("note_page", p["id"]), t=store.now(), **changed)
Expand Down
3 changes: 3 additions & 0 deletions fastapi_backend/app/domain/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -990,6 +990,9 @@ def close(self):
"DEFINE TABLE IF NOT EXISTS note_link SCHEMALESS",
"DEFINE INDEX IF NOT EXISTS note_link_page ON note_link FIELDS page",
"DEFINE INDEX IF NOT EXISTS note_link_target ON note_link FIELDS target",
# what a page was before each change (notebook.snapshot): its history, to look back at and restore
"DEFINE TABLE IF NOT EXISTS note_version SCHEMALESS",
"DEFINE INDEX IF NOT EXISTS note_version_page ON note_version FIELDS page",
# comments on resources, threaded, by everyone who can read them (app/domain/comments.py)
"DEFINE TABLE IF NOT EXISTS comment SCHEMALESS",
"DEFINE INDEX IF NOT EXISTS comment_rec ON comment FIELDS recording",
Expand Down
19 changes: 19 additions & 0 deletions fastapi_backend/app/schemas/notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,25 @@ class NoteBacklink(ResponseModel):
updated_at: str | None = None


class NoteVersionItem(ResponseModel):
id: int
at: str = Field(description="when the page changed from this")
by: str | None = Field(None, description="who changed it (an email); none for the model")
author: Literal["person", "assistant"] = Field("person", description="a person, or the assistant or model")
title: str | None = None
summary: str | None = None
size: int = Field(0, description="characters of text")
why: str | None = None


class NoteVersion(NoteVersionItem):
body: str = ""


class NoteHistory(ResponseModel):
versions: list[NoteVersionItem]


class NotePage(NotePageItem):
namespace: str
body: str = Field("", description="Markdown, with mentions written @[label](kind:id) and #[label](topic:id)")
Expand Down
41 changes: 41 additions & 0 deletions fastapi_backend/tests/api/test_notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -278,3 +278,44 @@ def test_filing_in_para(client, env, db, cfg, llm):
cfg["ai"]["organise_notes"] = False
assert notebook.file_due(db, cfg) == 0
llm.decision = None


def test_page_history(client, env, db, cfg, llm):
from app.domain import ai_tools

he, hv = env["he"], env["hv"]
p = _new(client, he, title="Plan", body="First draft.")
# a person's edits in one sitting make one version: what the page was before they started
client.patch(f"/api/v1/notes/{p['id']}", headers=he, json={"body": "Second draft."})
client.patch(f"/api/v1/notes/{p['id']}", headers=he, json={"body": "Third draft."})
client.patch(f"/api/v1/notes/{p['id']}", headers=he, json={"place": "project"}) # not the text: no version
h = client.get(f"/api/v1/notes/{p['id']}/history", headers=hv).json()["versions"]
assert [(v["author"], v["by"], v["size"]) for v in h] == [("person", "ed@x.io", len("First draft."))]
# the assistant's change is a version of its own, and can be undone
uid = db.one("SELECT record::id(id) AS id FROM account WHERE email = 'ed@x.io'")["id"]
pods = store.ns_id(db, "pods")
box = ai_tools.Toolbox(db, cfg, {"id": uid, "email": "ed@x.io"}, {pods}, {pods}, {}, None)
box.call("update_note", {"note_id": p["id"], "body": "Rewritten by the assistant."})
h = client.get(f"/api/v1/notes/{p['id']}/history", headers=he).json()["versions"]
assert [v["author"] for v in h] == ["assistant", "person"]
before = client.get(f"/api/v1/notes/{p['id']}/history/{h[0]['id']}", headers=hv).json()
assert before["body"] == "Third draft."
assert client.post(f"/api/v1/notes/{p['id']}/history/{h[0]['id']}/restore", headers=hv).status_code == 403
back = client.post(f"/api/v1/notes/{p['id']}/history/{h[0]['id']}/restore", headers=he).json()
assert back["body"] == "Third draft." and back["doc_stale"] is False
# restoring is itself undoable: the assistant's text is the newest version now
h = client.get(f"/api/v1/notes/{p['id']}/history", headers=he).json()["versions"]
assert (
h[0]["why"].startswith("restored")
and client.get(f"/api/v1/notes/{p['id']}/history/{h[0]['id']}", headers=he).json()["body"] == "Rewritten by the assistant."
)
# the model's refinement keeps a version too; another page's version isn't this page's
other = _new(client, he, title="Other", body="x")
assert client.get(f"/api/v1/notes/{other['id']}/history/{h[0]['id']}", headers=he).status_code == 404
_age(db, p["id"])
notebook.refine_due(db, cfg)
h2 = client.get(f"/api/v1/notes/{p['id']}/history", headers=he).json()["versions"]
assert h2[0]["author"] == "assistant" and h2[0]["by"] is None and len(h2) == len(h) + 1
# deleting the page drops its history
client.delete(f"/api/v1/notes/{p['id']}", headers=he)
assert not db.rows("SELECT id FROM note_version WHERE page = $p", p=p["id"])
18 changes: 18 additions & 0 deletions nextjs-frontend/app/openapi-client/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1624,6 +1624,7 @@ export type {
Note,
NoteBacklink,
NoteCreate,
NoteHistory,
NoteLink,
NoteLinkTarget,
NotePage,
Expand All @@ -1634,6 +1635,8 @@ export type {
NotePageUpdate,
NoteTree,
NoteUpdate,
NoteVersion,
NoteVersionItem,
NotifyDelivery,
NotifySecret,
NotifyTarget,
Expand Down Expand Up @@ -1668,6 +1671,16 @@ export type {
PageAboutResponse,
PageAboutResponses,
PageContext,
PageHistoryData,
PageHistoryError,
PageHistoryErrors,
PageHistoryResponse,
PageHistoryResponses,
PageVersionData,
PageVersionError,
PageVersionErrors,
PageVersionResponse,
PageVersionResponses,
Passage,
Passkey,
PasskeyAnswer,
Expand Down Expand Up @@ -1925,6 +1938,11 @@ export type {
ResourceTotalsErrors,
ResourceTotalsResponse,
ResourceTotalsResponses,
RestoreVersionData,
RestoreVersionError,
RestoreVersionErrors,
RestoreVersionResponse,
RestoreVersionResponses,
RetagRecordingsData,
RetagRecordingsError,
RetagRecordingsErrors,
Expand Down
Loading
Loading
0