8000 feat(notes): the assistant keeps notes, files them in PARA, and notes open on the edgeless canvas by adeelahmad · Pull Request #129 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions DEVLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,9 +163,11 @@ Todo:
text kept for search and the assistant
- [x] AI title and summary refinement (ai.refine_notes, on by default, can be switched off)
- [ ] Costs of refinement to the activity ledger (once "Activity history and budgets" lands)
- [ ] Assistant and MCP tools: find, read, write, link and file notes; changes by the assistant are undoable
- [ ] Self-organising: a routine files notes into PARA and links them to entities and topics, proposing what it isn't
sure of
- [x] Assistant tools: find, read, write and update notes (at once, no approval: notes are its notebook)
- [ ] MCP tools to read notes; page history so the assistant's changes can be undone
- [x] Self-organising, first step: notes nobody filed are filed in PARA by a decision model (or the LLM), with a
suggestion when it isn't sure (ai.organise_notes)
- [ ] Self-organising, next: link notes to entities and topics, nest them under their project or area
- [ ] # topics move to the SKOS topics once the graph thread splits them from entities
- [ ] Attachments on a page, encrypted through the keyring
- [ ] Object storage (S3, or an rclone remote served as S3) in the setup wizard, no local storage; a downloadable
Expand Down
31 changes: 24 additions & 7 deletions docs/notes.md
10BC8
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Every resource, entity and topic in Lens has a page of its own, and next to them
assistant write. The assistant is the main writer and organiser; people can do everything it can.

Status: **built**: pages (free notes in a tree, a page per recording, entity, collection or speaker), @ and # links
with backlinks, the tree in the left navigation, the page view with the BlockSuite editor, and the model keeping
titles and summaries up to date. **Planned**: assistant and MCP tools, a routine that files notes (PARA and the
ontology), attachments on encrypted object storage, pages in the graph, # links to SKOS topics.
with backlinks, the tree in the left navigation, the page view with the BlockSuite editor, the model keeping titles
and summaries up to date, and assistant tools to find, read, write and update notes. **Planned**: MCP tools, page
history, linking notes to entities and topics and nesting them under their project or area on their own, attachments on encrypted object storage, pages in the graph, # links to SKOS topics.

## A page

Expand Down Expand Up @@ -49,17 +49,34 @@ typed is never sent on each keystroke, empty notes are never sent, and nothing i
API and `lens worker` processes run this pass with the routines. Model calls are measured by telemetry; their costs go
to the activity ledger once it exists.

## Filing (PARA)

With **Settings → Assistant → File notes in projects, areas, resources and archives** (`ai.organise_notes`, on by
default), free notes nobody filed are filed once their summary is written. It's a routine decision, so a decision model
takes it when one is set up (Settings → Assistant), else the language model. When it's sure (`decisions.act_above`) the
note is filed (`place_by: assistant`); otherwise its best guess waits on the note as a suggestion to accept with one
click. What a person files is never refiled.

## The assistant

The chat assistant keeps notes as its notebook: `find_notes`, `read_note`, `write_note` and `update_note` (Settings →
Assistant lists them). It looks there first, and writes or updates a note when it learns something worth keeping,
filed in PARA and linked with @ and #. Writing needs editor access to the namespace and takes effect at once, with no
approval: notes it writes say so. It never deletes a note.

## Access

Pages belong to a namespace: anyone with a role there reads them, editors write, move and delete them. The pages of
recordings also need access to the recording.

## Editor

The page view uses AFFiNE's BlockSuite editor, whose documents are Yjs CRDTs, the format OctoBase stores and syncs.
The editor saves its document state in `doc` next to the Markdown; a change to the Markdown alone (the assistant's)
drops `doc`, so the editor rebuilds it from the Markdown. OctoBase is AGPL-3.0, so it would run as a separate service
for live co-editing rather than inside Lens.
The page view uses AFFiNE's BlockSuite editor, whose documents are Yjs CRDTs, the format OctoBase stores and syncs. A
page can be shown as a document or on the edgeless canvas (`view`), with BlockSuite's shapes, connectors, mind maps,
frames and free drawing. The editor saves its document state in `doc` next to the Markdown; the drawings live only
there. A change to the Markdown alone (the assistant's) keeps `doc` and sets `doc_stale`: the editor then replaces its
text with the Markdown and keeps the drawings. OctoBase is AGPL-3.0, so it would run as a separate service for live
co-editing rather than inside Lens.

## Refine later

Expand Down
18 changes: 16 additions & 2 deletions fastapi_backend/app/api/v1/routes/notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,18 @@ def create_page(body: NotePageCreate, user: Writer, acl: Acl, db: Db) -> NotePag
sid = acl.namespace(body.ns, "editor")
with domain_errors():
pid = notebook.create(
db, sid, user.id, body.title, body.body, body.summary, body.date, body.place, body.parent, body.about, doc=body.doc
db,
sid,
user.id,
body.title,
body.body,
body.summary,
body.date,
body.place,
body.parent,
body.about,
doc=body.doc,
view=body.view,
)
return _out(acl, db, notebook.get(db, pid), user)

Expand Down Expand Up @@ -126,10 +137,13 @@ def get_page(pid: int, user: CurrentUser, acl: Acl, db: Db) -> NotePage:

@router.patch("/{pid}")
def update_page(pid: int, body: NotePageUpdate, user: Writer, acl: Acl, db: Db) -> NotePage:
"""Change what's given. A new body without `doc` drops the editor's state, so it's rebuilt from the Markdown."""
"""Change what's given. A new body without `doc` keeps the editor's state (drawings on the canvas live there) but
marks it stale (`doc_stale`), so the editor brings its text in line with the body."""
_page(acl, db, pid, "editor")
given = body.model_fields_set
kw: dict[str, Any] = {k: getattr(body, k) for k in ("summary", "date", "doc") if k in given}
if body.view is not None:
kw["view"] = body.view
if "place" in given:
kw["place"] = body.place or None
with domain_errors():
Expand Down
155 changes: 155 additions & 0 deletions fastapi_backend/app/domain/ai_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
graph_ask,
graph_model,
entity_setup,
notebook,
ops_tools,
recsets,
render,
Expand Down Expand Up @@ -83,6 +84,58 @@
False,
),
("speaker_stats", "Speakers in a namespace with talk time and recordings.", {"namespace": _S}, ["namespace"], False),
(
"find_notes",
"Find notes (free notes and the pages of recordings, entities, collections and speakers) in scope by words in "
"their title, summary or text. Each has an id, title, one-line summary, where it's filed (PARA) and what it's "
"the page of.",
{"query": _S, "namespace": _S, "place": _S, "limit": _I},
[],
False,
),
(
"read_note",
"Read a note's Markdown, with its links and the notes linking to it: by note_id, or the page of a thing "
'(about, like "recording:12" or "entity:5").',
{"note_id": _I, "about": _S},
[],
False,
),
(
"write_note",
'Write a new note, or the page of a thing (about, like "entity:5"; one each). You keep notes as you learn: '
"a specific title, a one-line summary of what it holds, Markdown text, and where it's filed: project (an "
"outcome with an end), area (a responsibility kept up), resource (a topic of interest) or archive (done). Link "
"with @[label](recording:12), @[label](entity:5), @[label](page:3), and topics with #[label](entity:9) (ids "
"from find_entities, find_notes and list_recordings). Put it inside another note with parent_id.",
{
"namespace": _S,
"title": _S,
"body": _S,
"summary": _S,
"place": {"type": "string", "enum": list(notebook.PLACES)},
"parent_id": _I,
"about": _S,
},
["namespace", "title", "body"],
True,
),
(
"update_note",
"Change a note: its title, summary, place, or its text (body replaces it; append adds to the end). Move a free "
"note in the tree with parent_id (0: the top).",
{
"note_id": _I,
"title": _S,
"summary": _S,
"place": {"type": "string", "enum": list(notebook.PLACES)},
"body": _S,
"append": _S,
"parent_id": _I,
},
["note_id"],
True,
),
(
"graph_schema",
"What the graph holds (namespaces, collections, recordings, speakers, entities and how they link), with example "
Expand Down Expand Up @@ -642,6 +695,108 @@ def t_run_template(self, template_id, recording_ids=None, collection_id=None):
est = batches.estimate(self.db, self.cfg, ids, steps)
return self._approval("run_template", {"template_id": t["id"], "recordings": ids}, f"Run {label} on {len(ids)} recording(s)", est)

# ---- notes (notebook.py): the assistant reads and writes them at once; they're its notebook ----
def _note(self, note_id, edit=False):
try:
p = notebook.get(self.db, int(note_id))
except KeyError:
raise ValueError(f"no note {note_id}") from None
if p["space"] not in (self.editable if edit else self.readable):
raise ValueError(f"note {note_id} isn't {'yours to change' if edit else 'available'} in this conversation")
return p

def _space(self, namespace, edit=False):
names = {v: k for k, v in store.space_names(self.db).items()}
sid = names.get(namespace)
if sid not in (self.editable if edit else self.readable):
raise ValueError(f"no namespace called {namespace} {'you can write in' if edit else 'in scope'}")
return sid

def t_find_notes(self, query=None, namespace=None, place=None, limit=10):
spaces = [self._space(namespace)] if namespace else sorted(self.readable)
names = store.space_names(self.db)
q = " ".join(str(query or "").split()).casefold()
rows = self.db.rows(
"SELECT record::id(id) AS id, space, title, summary, place, about, updated_at, text FROM note_page WHERE space IN $s",
s=spaces,
)
hits = [
r
for r in rows
if (not place or r.get("place") == place)
and (not q or any(q in str(r.get(k) or "").casefold() for k in ("title", "summary", "text")))
]
hits.sort(key=lambda r: (q not in str(r["title"]).casefold(), r.get("updated_at") or ""), reverse=False)
out = [
store.clean(
{
"id": r["id"],
"title": r["title"],
"summary": r.get("summary"),
"place": r.get("place"),
"page_of": r.get("about"),
"namespace": names.get(r["space"]),
}
)
for r in hits[: min(int(limit or 10), 30)]
]
return {"total": len(hits), "notes": out}, f"Found {len(hits)} note(s)"

def t_read_note(self, note_id=None, about=None):
if note_id is None and not about:
raise ValueError("give note_id or about")
if note_id is None:
kind, _, key = str(about).partition(":")
sid = notebook.owner(self.db, kind, int(key)) if key.isdigit() else None
if sid not in self.readable:
raise ValueError(f"{about} isn't available in this conversation")
p = notebook.about(self.db, sid, about)
if not p:
return {
"page_of": about,
"note": None,
"hint": "nobody has written its page yet: write_note with about",
}, f"{about} has no page yet"
else:
p = self._note(note_id)
targets = [f"page:{p['id']}"] + ([p["about"]] if p.get("about") else [])
out = store.clean(
{
"id": p["id"],
"title": p["title"],
"summary": p.get("summary"),
"date": p.get("date"),
"place": p.get("place"),
"page_of": p.get("about"),
"parent_id": p.get("parent"),
"namespace": store.space_names(self.db).get(p["space"]),
"written_by": p.get("author"),
"body": (p.get("body") or "")[:20_000],
"links": notebook.links(self.db, p["id"]),
"linked_from": [b for b in notebook.backlinks(self.db, p["space"], targets) if b["page"] != p["id"]],
}
)
return out, f"Read the note {p['title']}"

def t_write_note(self, namespace, title, body, summary=None, place=None, parent_id=None, about=None):
sid = self._space(namespace, edit=True)
pid = notebook.create(self.db, sid, self.user["id"], title, body, summary, None, place, parent_id, about, author="assistant")
return {"note_id": pid, "url": f"/notes/{pid}"}, f"Wrote the note {title}"

def t_update_note(self, note_id, title=None, summary=None, place=None, body=None, append=None, parent_id=None):
p = self._note(note_id, edit=True)
if append:
body = ((body if body is not None else p.get("body") or "").rstrip() + "\n\n" + append).strip()
kw = {}
if summary is not None:
kw["summary"] = summary
if place is not None:
kw["place"] = place
notebook.update(self.db, p["id"], self.user["id"], title=title, body=body, author="assistant", **kw)
if parent_id is not None:
notebook.move(self.db, p["id"], int(parent_id) or None)
return {"note_id": p["id"], "url": f"/notes/{p['id']}"}, f"Changed the note {title or p['title']}"

def t_entity_setup(self, namespace):
names = {v: k for k, v in store.space_names(self.db).items()}
sid = names.get(namespace)
Expand Down
4 changes: 3 additions & 1 deletion fastapi_backend/app/domain/chat.py
Original file line number Diff line number Diff line change
Expand Up @@ -417,7 +417,9 @@ def request_stop(db, cid):

TOOL_SYSTEM = (
SYSTEM + " You can call tools to look things up in the archive. Cite moments with the [n] numbers the tools return, and "
"only those. Tools that run work or change data need the person's approval: say what you proposed and that it's waiting for them."
"only those. Tools that run work or change data need the person's approval: say what you proposed and that it's waiting for them. "
"Notes are your notebook and need no approval: look in them first (find_notes, read_note), and when you learn something "
"worth keeping, write or update a note (write_note, update_note), linking what it's about."
)


Expand Down
Loading
Loading
0