8000 feat(entities): every entity has its own page at /wiki/namespace/Name by adeelahmad · Pull Request #393 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,16 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version

## Unreleased

- **Every entity has its own page.** An entity's page is named by its name, as a wiki names pages:
`/wiki/<namespace>/<Name>` (spaces as underscores), so Ada Lovelace in `work` is `/wiki/work/Ada_Lovelace`. It shows
what the entity is, the notes people and the assistant keep on it in the BlockSuite editor, every line that says it,
and the assets, people and entities it comes up with. Its name on the Entities page now opens it (the rest of
the row still opens the side panel), as do `@` links in notes, the graph panel and the entity's RDF URI. Any case, a
name it had before a rename, or the name of one merged into it opens it too ([Entity pages](docs/notes.md#entity-pages)).
- **The update notice no longer pushes pages off the screen.** While *Lens vX is out* shows above them, Chat, the
graph and a recording's page are that much shorter, so a phone's question box and a recording's player stay in
sight.

## 0.11.0 <small>October 15, 2026</small> {id="0.11.0"}

- **Installing an update from the web app works again in a microsandbox.** *Back up and install* stopped at the
Expand Down
5 changes: 3 additions & 2 deletions ci/clicks/journeys.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -492,17 +492,18 @@ export const JOURNEYS = [
ns: "",
module: "Entities",
feature: "Find an entity and see where it was mentioned",
note: "Its name opens its own page, /wiki/<namespace>/Raspberry_Pi, with its notes and every line that says it.",
from: "/",
async run(ctx) {
const { page, meter } = ctx;
const entity = page.getByRole("main").getByRole("button", { name: "Raspberry Pi", exact: true });
const entity = page.getByRole("main").getByRole("link", { name: "Raspberry Pi", exact: true });
await madeInBackground(ctx, "/entities", entity, "/", "the entity the imports' analysis makes");
await go(ctx, "Entities");
await meter.type(page.getByRole("searchbox", { name: "Find entities" }), "rasp", "part of its name");
// the list filters as you type: the other entity leaves it
await page.getByRole("main").getByRole("table").getByText("Lens", { exact: true }).waitFor({ state: "hidden" });
await meter.click(entity, "Raspberry Pi");
await page.waitForURL(/entity=\d+/);
await page.waitForURL(/\/wiki\/[^/]+\/Raspberry_Pi$/);
await page.getByText(/Raspberry Pi can run Lens/).first().waitFor();
},
},
Expand Down
6 changes: 6 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -991,6 +991,7 @@ POST /api/v1/entities/merge
POST /api/v1/entities/merges/{mid}/undo
POST /api/v1/entities/not-same
GET /api/v1/entities/{eid}
GET /api/v1/wiki/{ns}/{address}
GET /api/v1/entities/{eid}/mentions
GET /api/v1/entities/{eid}/connections
POST /api/v1/entities/{eid}/rename
Expand All @@ -1002,6 +1003,11 @@ GET /api/v1/graph/explore
GET /api/v1/graph/path
```

An entity's `page` is its page in the web app, by name: `/wiki/<namespace>/<Name_As_Written>`.
`GET /wiki/{ns}/{address}` says which entity an address names: by its name in any case and spacing (underscores for
spaces), or a name it had before a rename or a merge. It answers `{kind, id, namespace, title, path, canonical}`;
`canonical` is false when the address is another way of writing `path` ([Notes](notes.md#entity-pages)).

## metadata

```
Expand Down
2 changes: 1 addition & 1 deletion docs/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ front of the web app also needs `TRUST_PROXY_HEADERS=true` ([Authentication](aut
| `fetch` | an asset's whole text at once, as `[m:ss] Speaker: …` lines |
| `cite` | a citation for a moment (a line or a second, and a few lines from there): the words, who said or wrote them, the asset and date, the time, the link, and all of it as Markdown |
| `list_entities` | people, organisations, products, places, events, works and terms mentioned, by name, type, namespace or asset |
| `get_entity` | one entity: its names, how often and when it is mentioned, what it is mentioned with, who mentions it most, and the lines that mention it |
| `get_entity` | one entity: its names, its page's link, how often and when it is mentioned, what it is mentioned with, who mentions it most, and the lines that mention it |
| `explore_graph` | the knowledge graph around an entity (`e12`) or a person (`s3`), one or two steps out |
| `find_path` | the shortest chain of links between two of them, with lines that show each link |
| `sparql` | a read-only SPARQL query (SELECT, ASK, CONSTRUCT or DESCRIBE) over one namespace's linked data ([RDF](rdf.md)): JSON results, or Turtle |
Expand Down
31 changes: 29 additions & 2 deletions docs/notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@ Status: **built**: pages (free notes in a tree, a page per asset, entity, topic,
links with backlinks (# links topics), the tree in the left navigation, the page view with the BlockSuite editor, the
model keeping titles and summaries up to date, assistant tools to find, read, write and update notes, and MCP tools to
find, read and write them ([MCP](mcp.md)), page history with restore, suggested links to what a note names, the
project or area a note fits under, and images and attachments kept where Settings → Storage says.
**Planned**: nesting notes under their project or area without a click, pages in the graph.
project or area a note fits under, images and attachments kept where Settings → Storage says, and every entity's
own page at a wiki address, `/wiki/<namespace>/<Name>` ([Entity pages](#entity-pages)).
**Planned**: nesting notes under their project or area without a click, pages in the graph, wiki addresses for topics,
recordings and the other things that have pages.

## A page

Expand All @@ -29,6 +31,28 @@ draft with its name and the pages that already link to it. A page follows its as
thing is deleted, or an entity is merged into another, its page stays as a free note at the top of the tree, so
nothing written on it is lost.

## Entity pages

Every entity (a person, organisation, product, place, event, work or term) has a page of its own, named by its name
the way a wiki names pages: spaces become underscores and the rest stays as written, so Ada Lovelace in namespace
`work` is at `/wiki/work/Ada_Lovelace`. Nothing is stored for the address: it is the name. The page shows what the
entity is (its type, the other ways it's said and its description), the notes people and the assistant keep on it in
the BlockSuite editor below, every line that says it with a link to that moment, the assets and the people it comes
up with most, the entities it's mentioned together with (each a link to its own page), the same entity in other
namespaces, and similar content. **Edit** opens the panel that changes it, as on the Entities page; renaming it moves
the page to the new name.

The address is read the way analysis matches names: in any case and spacing (`/wiki/work/ada_lovelace`), and by a name
the entity had before a rename or the name of one merged into it. Those open the page and switch the address to its
own. A name with `/` or `?` in it is escaped in the address (`AC/DC` is `/wiki/work/AC%2FDC`). The entity's name on the
Entities page, in a note's `@` link, in the graph and in the RDF URI `/id/entity/<id>` all lead to its page, and
`/entities/<id>` (and an entity's page's old address, `/notes/about/entity/<id>`) does too.

`GET /api/v1/entities/{eid}` and the MCP tool `get_entity` give the page's address (`page`, `url`); the assistant's
`find_entities` gives it too. `GET /api/v1/wiki/{ns}/{address}` says which entity an address names (`id`, `path`, and
`canonical: false` when the address is another way of writing it). An address only answers for the namespace's members;
to anyone else it is not there.

## Links

The body links with mention tokens, which the editor and the assistant both write:
Expand Down Expand Up @@ -126,5 +150,8 @@ co-editing rather than inside Lens.

## Refine later

Wiki addresses for topics, assets, collections and speakers, and a namespace's wiki index page; two entities whose
names differ only by `_` and a space share an address (the one with spaces wins).

Pages for people who see only some collections of a namespace; differences between versions; live co-editing through
OctoBase; pages and their links as nodes and relationships in the graph.
12 changes: 8 additions & 4 deletions docs/processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,8 +110,10 @@ Nodes are people and named things. Edges are:
People, organisations, products, places, events, works and topics are extracted from every asset's text. Dates and
numbers are extracted too, but hidden unless asked for.

The **Entities** page in the web app lists a namespace's entities (search, type and collection filters, a Hidden tab)
and opens each in a side panel where editors rename, retype, describe, merge and hide it.
The **Entities** page in the web app lists a namespace's entities (search, type and collection filters, a Hidden tab).
An entity's name opens its own page, `/wiki/<namespace>/<Name>`, with its notes, every line that says it and what it
comes up with ([Entity pages](notes.md#entity-pages)); the rest of its row opens a side panel where editors rename,
retype, describe, merge and hide it.

- **Merge from the side panel:** **Merge** opens a find box; type a name and press Enter, and the entity picked is
merged into the open one at once. Its **Maybe the same** list offers likely doubles (same letters, an acronym, a near
Expand All @@ -128,8 +130,10 @@ and opens each in a side panel where editors rename, retype, describe, merge and
mentions, and hidden entities.
- Sorts: most mentioned, most assets, most recent, rising, and name.
- Options: grouping by name across namespaces, a twelve-month sparkline, and facets.
- **Entity page:** its details and aliases, paged mentions (each line with the words highlighted, linking to its moment),
a timeline by month or week and namespace, and connections (entities mentioned together, people, assets).
- **Entity page** (`/wiki/<namespace>/<Name>` in the web app; `GET /api/v1/wiki/{ns}/{address}` finds it by name): its
details and aliases, its notes, paged mentions (each line with the words highlighted, linking to its moment), and
connections (entities mentioned together, people, assets); the API also gives a timeline by month or week and
namespace.
- **Explorer:**
- `GET /api/v1/graph/explore?focus=e12` gives the neighbourhood of an entity or person. Depth is one or two hops, and it
can be filtered by type, edge kind and strength, within one namespace or all shared ones.
Expand Down
2 changes: 1 addition & 1 deletion docs/rdf.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ URIs use the address Lens publishes (`iiif.base_url`, or the address a request a

Each URI uses content negotiation. An `Accept` of `text/turtle`, `application/ld+json`, `application/n-triples` or
`application/rdf+xml`, or `?format=turtle|json-ld|nt|xml`, returns RDF. A browser is sent to the page instead
(303 See Other).
(303 See Other); an entity's page is named by its name, `/wiki/<namespace>/<Name>` ([Notes](notes.md#entity-pages)).

A public asset's URI is open to everyone, just like its IIIF manifest. Visitors get its metadata and published
custom fields. Members also get the entities it mentions (`dcterms:references`), its access and its internal fields.
Expand Down
9 changes: 6 additions & 3 deletions fastapi_backend/app/api/linked_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

from app.api.deps import Acl, Cfg, Db, OptionalUser
from app.api.iiif import _rec, base_url
from app.domain import rdf
from app.domain import rdf, store, wiki
from app.domain.store import R

router = APIRouter(include_in_schema=False, tags=["linked-data"])
Expand Down Expand Up @@ -75,9 +75,12 @@ def collection(cid: int, request: Request, acl: Acl, db: Db, cfg: Cfg, format: s

@router.get("/id/entity/{eid}")
def entity(eid: int, request: Request, acl: Acl, db: Db, cfg: Cfg, format: str | None = FORMAT) -> Response:
_member_of(acl, (db.one("SELECT space FROM $r", r=R("entity", eid)) or {}).get("space"))
e = db.one("SELECT space, name FROM $r", r=R("entity", eid)) or {}
_member_of(acl, e.get("space"))
base, fmt = base_url(request, cfg), wanted(request, format)
return rdf_response(rdf.entity_graph(db, base, eid), fmt) if fmt else _see(f"{base}/entities/{eid}")
if not fmt: # its page, by name
return _see(base + wiki.path(store.space_names(db).get(e["space"]) or "", e["name"]))
return rdf_response(rdf.entity_graph(db, base, eid), fmt)


@router.get("/id/topic/{tid}")
Expand Down
5 changes: 3 additions & 2 deletions fastapi_backend/app/api/mcp_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -914,8 +914,8 @@ def _entities(
@tool(
"get_entity",
"Get an entity",
"One entity: its names, how often and when it is mentioned, the entities mentioned together with it, who mentions "
"it most, and the lines that mention it (newest first), each with a url to that moment.",
"One entity: its names, its page's url, how often and when it is mentioned, the entities mentioned together with "
"it, who mentions it most, and the lines that mention it (newest first), each with a url to that moment.",
Arg("entity_id", "integer", "the entity (from list_entities, get_recording or the graph)", required=True),
Arg("mentions", "integer", "how many mentioning lines", default=10, minimum=0, maximum=100),
Arg("mentions_offset", "integer", "skip this many lines (for the next page)", default=0, minimum=0, maximum=10000),
Expand All @@ -930,6 +930,7 @@ def get_entity(ctx: Context, entity_id: int, mentions: int, mentions_offset: int
"type": d.type,
"type_label": d.type_label,
"namespace": d.namespace,
"url": f"{ctx.web}{d.page}" if d.page else None,
"aliases": d.aliases or None,
"mentions": d.mentions,
"recordings": d.recordings,
Expand Down
16 changes: 15 additions & 1 deletion fastapi_backend/app/api/v1/routes/entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
from fastapi import APIRouter, HTTPException, Request

from app.api.deps import Access, Acl, CurrentUser, Db, Writer, domain_errors
from app.domain import auth, entity_map, entity_setup, graph_history, hierarchy, jobs
from app.domain import auth, entity_map, entity_setup, graph_history, hierarchy, jobs, wiki
from app.domain import entities as ents
from app.domain.store import DB, R
from app.schemas.common import Ok
Expand All @@ -31,6 +31,7 @@
MentionList,
MentionMove,
MentionMoved,
WikiPage,
)

router = APIRouter(tags=["entities"])
Expand Down Expand Up @@ -203,6 +204,19 @@ def get_entity(eid: int, user: CurrentUser, acl: Acl, db: Db) -> EntityDetail:
return EntityDetail.model_validate(ents.detail(db, eid, set(acl.roles)))


@router.get("/wiki/{ns}/{address:path}")
def find_wiki_page(ns: str, address: str, user: CurrentUser, acl: Acl, db: Db) -> WikiPage:
"""The page /wiki/<ns>/<address> names: the entity called that (spaces as underscores, any case), or the one a name
it had before a rename or a merge now belongs to. When `canonical` is false, show `path` instead."""
sid = acl.namespace(ns)
hit = wiki.find(db, sid, address)
if not hit:
raise HTTPException(404, "not found")
eid, canonical = hit
name = db.one("SELECT VALUE name FROM $r", r=R("entity", eid))
return WikiPage(kind="entity", id=eid, namespace=ns, title=name, path=wiki.path(ns, name), canonical=canonical)


@router.get("/entities/{eid}/mentions")
def list_entity_mentions(
eid: int,
Expand Down
2 changes: 2 additions & 0 deletions fastapi_backend/app/domain/ai_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
topics,
untrusted,
uploads,
wiki,
)

R = store.R
Expand Down Expand Up @@ -917,6 +918,7 @@ def t_find_entities(self, query=None, type=None, namespace=None, limit=10): # n
"name": e["name"],
"type": e["type_label"],
"namespace": e["namespace"],
"page": wiki.path(e["namespace"], e["name"]),
"mentions": e["mentions"],
"recordings": e["recordings"],
**({"description": e["description"]} if e.get("description") else {}),
Expand Down
3 changes: 2 additions & 1 deletion fastapi_backend/app/domain/entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
import re
from collections import Counter, defaultdict, deque

from . import analyze, entity_setup, graph_history, notebook, render, store
from . import analyze, entity_setup, graph_history, notebook, render, store, wiki

R = store.R
TYPES = {
Expand Down Expand Up @@ -239,6 +239,7 @@ def detail(db, eid, spaces):
"defined": bool(e.get("defined")),
"builtin": e.get("builtin"),
"collection": e.get("collection"),
"page": wiki.path(names.get(e["space"]) or "", e["name"]),
}


Expand Down
53 changes: 53 additions & 0 deletions fastapi_backend/app/domain/wiki.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
"""Wiki addresses: every entity has a page at /wiki/<namespace>/<Name_As_Written>, as wikis name their pages.

The name is the address: spaces become underscores and the rest stays as written, so `Ada Lovelace` in namespace `work`
is /wiki/work/Ada_Lovelace. Nothing is stored for it. An address is read the way analysis matches names (any case,
any spacing, the same entity key), so /wiki/work/ada_lovelace finds the page too; a name the entity was known by
before a rename or a merge (its aliases) finds the entity it is part of now. Either way the page answers with the
address to show (`canonical` False means the browser should switch to it).
"""

from __future__ import annotations

import re
import urllib.parse

from . import analyze, store

R = store.R


def slug(name: str) -> str:
"""Ada Lovelace -> Ada_Lovelace."""
return re.sub(r"\s+", "_", (name or "").strip())


def _part(s: str) -> str:
"""Escaped as the web app's encodeURIComponent escapes it, so a name with / or ? in it stays one part."""
return urllib.parse.quote(s, safe="!*'()")


def path(namespace: str, name: str) -> str:
"""The page's address in the web app, like /wiki/work/Ada_Lovelace."""
return f"/wiki/{_part(namespace)}/{_part(slug(name))}"


def find(db, sid: int, address: str) -> tuple[int, bool] | None:
"""(entity id, whether `address` is already its own address) for an address in namespace `sid`, or None."""
address = (address or "").strip().strip("/")
keys = [k for k in dict.fromkeys((analyze.ent_key(address.replace("_", " ")), analyze.ent_key(address))) if k]
eid = None
for key in keys:
row = db.one("SELECT record::id(id) AS id FROM entity WHERE ekey = $k LIMIT 1", k=f"{sid}:{key}")
if row:
eid = row["id"]
break
if eid is None: # a name it had before a rename, or the name of one merged into it
for key in keys:
row = db.one("SELECT entity FROM entity_alias WHERE space = $s AND key = $k LIMIT 1", s=sid, k=key)
if row and db.one("SELECT VALUE id FROM $r", r=R("entity", row["entity"])):
eid = row["entity"]
break
if eid is None:
return None
return eid, slug(db.one("SELECT VALUE name FROM $r", r=R("entity", eid))) == address
14 changes: 14 additions & 0 deletions fastapi_backend/app/schemas/entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,20 @@ class EntityDetail(ResponseModel):
defined: bool = Field(False, description="on the fixed list people defined")
builtin: str | None = Field(None, description="unknown or unlabeled: one of the two entities that are always there")
collection: int | None = Field(None, description="a defined entity of one collection (and those inside it)")
page: str | None = Field(None, description="its page in the web app, by name: /wiki/<namespace>/<Name_As_Written>")


class WikiPage(ResponseModel):
"""What a wiki address names, and the address to show for it."""

kind: str = Field(description="entity")
id: int
namespace: str
title: str = Field(description="its name")
path: str = Field(description="its own address, like /wiki/work/Ada_Lovelace")
canonical: bool = Field(
description="false: the address asked for is another way to write it (any case, or a name it had before a rename or merge); show `path`"
)


class MentionList(ResponseModel):
Expand Down
Loading
Loading
0