8000 feat(notes): keep a page's images and attachments where Settings → Storage says by adeelahmad · Pull Request #150 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions DEVLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,12 +212,12 @@ Todo:
one click; the assistant sees them as could_go_in)
- [ ] Self-organising, next: nest notes without a click when the fit is clear (decision model, like PARA filing)
- [x] # topics move to the SKOS topics once the graph thread splits them from entities (#135)
- [ ] Attachments on a page, encrypted through the keyring
- [x] Attachments on a page, encrypted through the keyring (in the file store)
- [x] The file store (blobs.py, docs/storage.md): this machine or any storage connection through rclone (S3, Drive,
Dropbox, OneDrive, SFTP, SMB, WebDAV, a folder), keyring-encrypted before it leaves, optional rclone crypt; the
`files` settings, a check, the assistant's file_storage tool. Adeel said go 2026-10-05, with rclone and crypt
- [x] Settings → Storage and the setup wizard step (pick or add a connection, folder, crypt, check); new installs choose
- [ ] Attachments on notes in the file store; BlockSuite images and files
- [x] Attachments on notes in the file store: BlockSuite's blob source keeps images and files there (note_file)
- [ ] Moving uploads and recordings' media to the file store (pipelines need local copies: a cache)
- [ ] A downloadable storage key (the server secret; opens every namespace that isn't a passkey vault)
- [ ] File storage over MCP (admins)
Expand Down
15 changes: 13 additions & 2 deletions docs/notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ assistant write. The assistant is the main writer and organiser; people can do e
Status: **built**: pages (free notes in a tree, a page per recording, entity, topic, collection or speaker), @ and #
links with backlinks (# links topics), the tree in the left navigation, the page view with the BlockSuite editor, the
model keeping titles and summaries up to date, assistant tools to find, read, write and update notes, and MCP tools to
find, read and write them ([MCP](mcp.md)), page history with restore, suggested links to what a note names, and the project or area a note fits under.
**Planned**: nesting notes under their project or area without a click, attachments on encrypted object storage, pages in the graph.
find, read and write them ([MCP](mcp.md)), page history with restore, suggested links to what a note names, the
project or area a note fits under, and images and attachments kept where Settings → Storage says.
**Planned**: nesting notes under their project or area without a click, pages in the graph.

## A page

Expand Down Expand Up @@ -58,6 +59,16 @@ API and `lens worker` processes run this pass with the routines. Each model call
the [activity ledger](activity.md) for the note (`note_page:5`) and its namespace; a note's page shows its costs and
activity at the bottom.

## Files and images

Images, attachments and other files dropped or pasted into a page are kept where **Settings → Storage** says
([Storage](storage.md)): on this machine, or on a storage connection (S3, Google Drive, Dropbox, OneDrive, SFTP, SMB,
WebDAV), encrypted with the namespace's key before they leave the machine. The editor names each by the SHA-256 of its
bytes, so the same file is kept once per page, and the server checks the name matches the bytes. Anyone who reads the
page reads its files; editors add them (`PUT /api/v1/notes/{id}/blobs?key=…`, up to `server.max_upload_mb`). Images
and PDFs show in place; anything a browser could run (HTML, SVG) only downloads. Deleting the page deletes its files.
The assistant sees a page's files (names, types, sizes) when it reads it.

## Filing (PARA)

With **Settings → Assistant → File notes in projects, areas, resources and archives** (`ai.organise_notes`, on by
Expand Down
4 changes: 2 additions & 2 deletions docs/storage.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Where Lens keeps its files

Status: **built**: the file store (this machine, or any storage connection through rclone, optionally wrapped in rclone
crypt), its settings, a check, the assistant's `file_storage` tool, Settings → Storage and the setup wizard's choice.
**In progress**: notes' attachments. **Planned**: moving uploads and recordings' media there too, and a downloadable
crypt), its settings, a check, the assistant's `file_storage` tool, Settings → Storage and the setup wizard's choice,
and notes' images and attachments ([Notes](notes.md#files-and-images)). **Planned**: moving uploads and recordings' media there too, and a downloadable
storage key.

The files Lens makes its own, starting with notes' attachments, go where **Settings → Storage** (the `files` section)
Expand Down
104 changes: 99 additions & 5 deletions fastapi_backend/app/api/v1/routes/notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,23 @@

from __future__ import annotations

import contextlib
from typing import Any
from urllib.parse import quote

from fastapi import APIRouter, HTTPException, Query
from fastapi import APIRouter, HTTPException, Query, Request
from fastapi.responses import StreamingResponse
from starlette.concurrency import run_in_threadpool

from app.api.deps import Access, Acl, CurrentUser, Db, Principal, Writer, domain_errors
from app.domain import auth, notebook, store
from app.api.deps import Access, Acl, Cfg, CurrentUser, Db, Principal, Writer, domain_errors
from app.api.v1.routes.files import _receive
from app.api.v1.routes.uploads import CHUNK
from app.domain import auth, keyring, notebook, store
from app.domain import files as resource_files
from app.domain.store import DB
from app.schemas.common import Ok
from app.schemas.notebook import (
NoteFile,
NoteHistory,
NoteHomeSuggestion,
NoteLinkSuggestion,
Expand Down Expand Up @@ -214,8 +222,94 @@ def move_page(pid: int, body: NotePageMove, user: Writer, acl: Acl, db: Db) -> N


@router.delete("/{pid}")
def delete_page(pid: int, user: Writer, acl: Acl, db: Db) -> Ok:
def delete_page(pid: int, user: Writer, acl: Acl, db: Db, cfg: Cfg) -> Ok:
"""Delete a page; the pages inside it move up a level. Needs editor access."""
_page(acl, db, pid, "editor")
notebook.delete(db, pid)
with _vault():
notebook.delete(db, pid, cfg)
return Ok()


# ---------- files on a page ----------
@contextlib.contextmanager
def _vault():
try:
yield
except keyring.Locked:
raise HTTPException(
423, "this namespace is a locked vault: its owners unlock it with a passkey, on its page in Admin, Namespaces"
) from None
except RuntimeError as e: # where files are kept didn't answer (Settings → Storage)
raise HTTPException(502, f"where files are kept didn't answer: {e}") from None


SAFE_INLINE = {"image/png", "image/jpeg", "image/gif", "image/webp", "image/avif", "application/pdf"}


@router.get("/{pid}/files")
def page_files(pid: int, user: CurrentUser, acl: Acl, db: Db) -> list[NoteFile]:
"""The page's files (images and attachments in its editor), oldest first."""
_page(acl, db, pid)
return [NoteFile(**f) for f in notebook.files(db, pid)]


@router.put("/{pid}/blobs", openapi_extra=CHUNK)
async def put_blob(
pid: int,
request: Request,
user: Writer,
acl: Acl,
db: Db,
cfg: Cfg,
key: str = Query(min_length=8, max_length=128, description="the editor's key for it"),
name: str | None = Query(None, max_length=255, description="its file name, when it has one"),
) -> NoteFile:
"""Keep a file the page's editor holds (an image, an attachment) as the raw request body, up to
server.max_upload_mb, where Settings → Storage says. Encrypted before it leaves the machine. Editors."""
p = await run_in_threadpool(_page, acl, db, pid, "editor")
path = await _receive(request, cfg)
try:
with domain_errors(), _vault():
f = await run_in_threadpool(notebook.add_file, db, cfg, p, key, path, name, request.headers.get("content-type"), user.id)
finally:
path.unlink(missing_ok=True)
await run_in_threadpool(auth.audit, db, user.as_audit(), "note.file", f"note_page:{pid}", {"key": key, "size": f["size"]})
return NoteFile(**f)


@router.get("/{pid}/blobs", response_class=StreamingResponse, responses={200: {"content": {"application/octet-stream": {}}}})
def get_blob(pid: int, user: CurrentUser, acl: Acl, db: Db, cfg: Cfg, key: str = Query(min_length=8, max_length=128)) -> StreamingResponse:
"""A file of the page, as it was kept. Images and PDFs show in place; anything a browser could run comes as bytes."""
_page(acl, db, pid)
with domain_errors(), _vault():
row, opened = notebook.open_file(db, cfg, pid, key)
f = opened.__enter__()
ctype = resource_files.served_type({"content_type": row.get("type")})
inline = ctype in SAFE_INLINE
name = row.get("name") or "file"
ascii_name = name.encode("ascii", "replace").decode().replace('"', "").replace("?", "_")

def body():
try:
while part := f.read(256 * 1024):
yield part
finally:
opened.__exit__(None, None, None)

disposition = "inline" if inline else "attachment"
headers = {
**resource_files.HEADERS,
"Content-Disposition": f"{disposition}; filename=\"{ascii_name}\"; filename*=UTF-8''{quote(name)}",
"Cache-Control": "private, max-age=86400",
}
return StreamingResponse(body(), media_type=ctype, headers=headers)


@router.delete("/{pid}/files")
def delete_page_file(pid: int, user: Writer, acl: Acl, db: Db, cfg: Cfg, key: str = Query(min_length=8, max_length=128)) -> Ok:
"""Remove a file from the page and from where it's kept. Editors."""
_page(acl, db, pid, "editor")
with domain_errors(), _vault():
notebook.delete_file(db, cfg, pid, key)
auth.audit(db, user.as_audit(), "note.file.delete", f"note_page:{pid}", {"key": key})
return Ok()
1 change: 1 addition & 0 deletions fastapi_backend/app/domain/ai_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -848,6 +848,7 @@ def t_read_note(self, note_id=None, about=None):
"links": notebook.links(self.db, p["id"]),
"linked_from": [b for b in notebook.backlinks(self.db, p["space"], targets) if b["page"] != p["id"]],
"could_link": [f"{x['sign']}[{x['label']}]({x['target']})" for x in notebook.suggest_links(self.db, p)] or None,
"files": [{"name": f.get("name"), "type": f["type"], "size": f["size"]} for f in notebook.files(self.db, p["id"])] or None,
"could_go_in": [
{"parent_id": x["page"], "title": x["title"], "place": x["place"], "why": x["why"]}
for x in notebook.suggest_homes(self.db, p)
Expand Down
99 changes: 97 additions & 2 deletions fastapi_backend/app/domain/notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,10 @@

from __future__ import annotations

import base64
import datetime as dt
import hashlib
import os
import re

from . import activity, store
Expand Down Expand Up @@ -410,9 +413,13 @@ def move(db, pid, parent=None, before=None):
db.q("UPDATE $r SET parent = $p, position = $pos, updated_at = $t", r=R("note_page", int(pid)), p=parent, pos=position, t=store.now())


def delete(db, pid):
"""Delete a page; the pages inside it move up to its parent. Links to it stay as text."""
def delete(db, pid, cfg=None):
"""Delete a page, with its files (given `cfg`, to reach where they are kept); the pages inside it move up to its
parent. Links to it stay as text."""
p = get(db, pid)
if cfg is not None:
for f in files(db, int(pid)):
delete_file(db, cfg, int(pid), f["key"])
for child in db.values("SELECT VALUE record::id(id) FROM note_page WHERE parent = $p", p=int(pid)):
db.q("UPDATE $r SET parent = $up", r=R("note_page", child), up=p.get("parent"))
db.q("DELETE note_link WHERE page = $p", p=int(pid))
Expand All @@ -426,6 +433,94 @@ def links(db, pid):
return [{"sign": s, "target": f"{k}:{i}", "label": label} for s, k, i, label in mentions(p.get("body"))]


# ---------- files on a page: images, attachments and other blobs the editor keeps (blobs.py stores them) ----------
FILE_KEY_RX = re.compile(r"^[A-Za-z0-9+/=_-]{8,128}$") # the editor's own key for a blob: a hash of its bytes
FILES_MAX = 500 # per page
FILE_FIELDS = "record::id(id) AS id, key, name, type, size, author, created_at"


def file_key(path):
"""What the editor names a file by: its bytes' SHA-256 in base64url, as BlockSuite's blob sync does."""
h = hashlib.sha256()
with open(path, "rb") as f:
while part := f.read(1 << 20):
h.update(part)
return base64.urlsafe_b64encode(h.digest()).decode()


def files(db, pid):
"""A page's files, oldest first."""
rows = db.rows(f"SELECT {FILE_FIELDS} FROM note_file WHERE page = $p", p=int(pid))
return sorted(rows, key=lambda f: f["id"])


def file_row(db, pid, key):
row = db.one(f"SELECT {FILE_FIELDS}, where FROM note_file WHERE page = $p AND key = $k LIMIT 1", p=int(pid), k=key)
if not row:
raise KeyError(key)
return row


def add_file(db, cfg, p, key, path, name=None, ctype=None, account=None, author="person"):
"""Keep a file on page p under the editor's `key`, where Settings → Storage says. The same key again is the same
bytes, so it's kept once. Returns its row."""
from . import blobs

if not FILE_KEY_RX.match(key or "") or key != file_key(path):
raise ValueError("the key isn't the file's: SHA-256 of its bytes, in base64url")
pid = int(p["id"])
try:
return file_row(db, pid, key)
except KeyError:
pass
if len(db.values("SELECT VALUE id FROM note_file WHERE page = $p", p=pid)) >= FILES_MAX:
raise ValueError(f"A page keeps up to {FILES_MAX} files.")
fid = db.next_id("note_file")
where = blobs.put(db, cfg, p["space"], f"notes/{p['space']}/{pid}/{fid}", path)
name = re.sub(r"[\x00-\x1f/\\]", "_", str(name or "").strip())[:200] or None
ctype = ctype if ctype and re.match(r"^[\w.+-]+/[\w.+-]+$", ctype) else "application/octet-stream"
db.q(
"CREATE $r CONTENT $d",
r=R("note_file", fid),
d=store.clean(
{
"page": pid,
"space": p["space"],
"key": key,
"name": name,
"type": ctype,
"size": os.path.getsize(path),
"where": where,
"author": author,
"created_by": account,
"created_at": store.now(),
}
),
)
return file_row(db, pid, key)


def open_file(db, cfg, pid, key):
"""(row, a context manager giving the file's plain bytes to read). KeyError when there's no such file."""
from . import blobs

row = file_row(db, pid, key)
return row, blobs.open_file(db, cfg, row.get("where"), f"notes/{_space_of(db, pid)}/{int(pid)}/{row['id']}")


def _space_of(db, pid):
return get(db, pid)["space"]


def delete_file(db, cfg, pid, key):
"""Remove a file from a page and from where it's kept."""
10BC8 from . import blobs

row = file_row(db, pid, key)
blobs.delete(db, cfg, row.get("where"), f"notes/{_space_of(db, pid)}/{int(pid)}/{row['id']}")
db.q("DELETE $r", r=R("note_file", row["id"]))


SUGGEST_CHARS = 20_000 # of a page's text looked through for things to link
SUGGEST_MAX = 8
_TOKEN = re.compile(r"[\w][\w'’.-]*")
Expand Down
3 changes: 3 additions & 0 deletions fastapi_backend/app/domain/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -1075,6 +1075,9 @@ def close(self):
# what a page was before each change (notebook.snapshot): its history, to look back at and restore
"DEFINE TABLE IF NOT EXISTS note_version SCHEMALESS",
"DEFINE INDEX IF NOT EXISTS note_version_page ON note_version FIELDS page",
# the files on a note page (images, attachments), kept where Settings → Storage says (notebook.py, blobs.py)
"DEFINE TABLE IF NOT EXISTS note_file SCHEMALESS",
"DEFINE INDEX IF NOT EXISTS note_file_page ON note_file FIELDS page, key",
# comments on resources, threaded, by everyone who can read them (app/domain/comments.py)
"DEFINE TABLE IF NOT EXISTS comment SCHEMALESS",
"DEFINE INDEX IF NOT EXISTS comment_rec ON comment FIELDS recording",
Expand Down
9 changes: 9 additions & 0 deletions fastapi_backend/app/schemas/notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,12 @@ class NoteHomeSuggestion(ResponseModel):
place: Literal["project", "area"]
score: int
why: list[str] = Field(description="what ties them: links between them, links they share, words of its title")


class NoteFile(ResponseModel):
key: str = Field(description="the editor's key for it (a hash of its bytes)")
name: str | None = None
type: str = "application/octet-stream"
size: int = 0
author: Literal["person", "assistant"] = "person"
created_at: str | None = None
46 changes: 46 additions & 0 deletions fastapi_backend/tests/api/test_notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@

from __future__ import annotations

import base64
import hashlib

import pytest

from app.domain import deletion, entities, moving, notebook, store, topics
Expand Down Expand Up @@ -387,3 +390,46 @@ def test_home_suggestions(client, env, db, cfg):

out = json.loads(box.call("read_note", {"note_id": q["id"]})[0])
assert out["could_go_in"] == [{"parent_id": health["id"], "title": "Health", "place": "area", "why": ["it links to this page"]}]


def test_page_files(client, env, db, cfg, folder):
he, hv, hx = env["he"], env["hv"], env["hx"]
p = _new(client, he, title="Diagrams")
png = b"\x89PNG\r\n\x1a\n" + b"pixels" * 100
svg = b"<svg >
k = lambda b: base64.urlsafe_b64encode(hashlib.sha256(b).digest()).decode() # noqa: E731
url = f"/api/v1/notes/{p['id']}/blobs"
put = lambda key, body, h=he, ctype="image/png", name="flow.png": client.put( # noqa: E731
url, params={"key": key, "name": name}, content=body, headers={**h, "Content-Type": ctype}
)
KP, KS = k(png), k(svg)
r = put(KP, png)
assert r.status_code == 200, r.text
assert (r.json()["key"], r.json()["name"], r.json()["type"], r.json()["size"]) == (KP, "flow.png", "image/png", len(png))
assert put(KP, png).status_code == 200 # the same key is the same bytes: kept once
assert [f["key"] for f in client.get(f"/api/v1/notes/{p['id']}/files", headers=hv).json()] == [KP]
# anyone who reads the page reads its files; images show in place
r = client.get(url, params={"key": KP}, headers=hv)
assert r.status_code == 200 and r.content == png
assert r.headers["content-type"] == "image/png" and r.headers["content-disposition"].startswith("inline")
assert r.headers["x-content-type-options"] == "nosniff"
# what a browser would run comes as bytes, to save
assert put(KS, svg, ctype="image/svg+xml", name="x.svg").status_code == 200
r = client.get(url, params={"key": KS}, headers=hv)
assert r.headers["content-type"] == "application/octet-stream" and r.headers["content-disposition"].startswith("attachment")
# only editors add; outsiders see nothing; keys are the editor's hashes
assert put(k(b"x"), b"x", h=hv).status_code == 403
assert client.get(url, params={"key": KP}, headers=hx).status_code in (403, 404)
assert put("bad key!", b"x").status_code == 400
assert put(k(b"other"), b"not the same bytes").status_code == 400 # the key is the bytes' hash
assert put(k(b""), b"").status_code == 400
assert client.get(url, params={"key": "hashOfNothing"}, headers=hv).status_code == 404
# kept where Settings → Storage says: here, data_dir/objects
kept = list((folder / "data" / "objects" / "notes").rglob("*"))
assert len([k for k in kept if k.is_file()]) == 2
# removing one, then the page, removes what was kept
assert client.delete(f"/api/v1/notes/{p['id']}/files", params={"key": KS}, headers=he).status_code == 200
assert client.get(url, params={"key": KS}, headers=hv).status_code == 404
assert client.delete(f"/api/v1/notes/{p['id']}", headers=he).status_code == 200
assert not [k for k in (folder / "data" / "objects" / "notes").rglob("*") if k.is_file()]
assert not db.rows("SELECT id FROM note_file")
Loading
Loading
0