8000 fix(entities): give names their real types instead of Term by adeelahmad · Pull Request #395 · adeelahmad/lense · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version

## Unreleased

- **Names get their real types, not Term.** The built-in extractor now tells people, places, organisations, events,
works and dates apart without a model: by a title or role before a name, a given name, the word a name ends or
starts with (Corp, University of, Summit, Act, River), lists of countries, regions and large cities and of
well-known companies and products, what a name is called ("a company called Lumen") and the namespace's named
speakers. Only names nothing tells stay Terms, and "and"
no longer joins two names into one. Existing entities analysis left as Terms are typed once on upgrade, keeping
every type people set; the change is in the graph's history to roll back
([How a name gets its type](docs/processing.md#entities-and-the-graph-explorer)).
- **Entities found by spaCy, fetched when picked.** Settings → Analysis → Entity extraction → spaCy now fetches spaCy
(MIT) and its small model the first time an asset needs it (about 60 MB): English, German, or one for many
languages. The defined entities still come first, a lone word like "Okay" isn't taken for a name, and until spaCy is
Expand Down
15 changes: 15 additions & 0 deletions DEVLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,21 @@ from a LanguageTool server, spaCy fetched for named entities, and dates found an

Refine later: revert a whole cleanup at once; cleanup rules for other languages' fillers; per-namespace switch.

## 2026-10-15 · Entity types beyond Term

Adeel: "every entity comes as term? why!!". The default rules extractor typed a capitalised name PERSON only after a
title, ORG only before Inc/Corp-like words and PRODUCT with a digit; everything else (people, places, most
organisations) was TERM. `name_types.py` now types names from built-in lists (given names, places, well-known
organisations and products, holidays), the words a name starts or ends with, roles and "called" phrases before it,
verbs of saying after it and the namespace's named speakers; a recording gives each name one type. Upgrade step
`entity-types` retypes old analysis-made Terms, skipping types people or agents set (graph history and audit log).

Refine later: the decision model (Jev or Laya) for the names the rules leave as Terms, in analysis and as an action
on existing Terms (next PR); place, name and brand lists for more languages and regions (Custom vocabulary in
Settings covers one install's own names); type existing entities again when a later recording gives better evidence
(needs a marker for types people set, kept out of the versioned rows); "Google Cloud"-like names (a known company,
then a word) as products.

## 2026-10-14 · Long recordings in pieces a speech provider takes

Adeel: large audio files are segmented automatically to each transcription provider's limits. `speech.Engine` cuts a
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
server (off by default, per namespace), entities and topics in a knowledge graph you can explore, query with Cypher or
SPARQL, and see as it was at any version. Every entity has its own page at a wiki address
(`/wiki/<namespace>/<Name>`), and entities, topics and people are edited and merged from their lists, the graph or an
asset, with Undo instead of a confirm. Find what is most like a passage, an asset, a topic or an entity, in a Similar
asset, with Undo instead of a confirm. Names get their real types (people, places, organisations, events, works) by
rules, without a model. Find what is most like a passage, an asset, a topic or an entity, in a Similar
tab on every asset or by selecting text anywhere, and, with anytopdf, photos by what they show.
([Graph](docs/graph.md), [entity pages](docs/notes.md#entity-pages),
[similar content](docs/processing.md#similar-content), [graph history](docs/graph-history.md),
Expand Down Expand Up @@ -206,7 +207,6 @@ These are open pull requests, not yet part of Lens:
- **Sources:** RSS and Atom news feeds ([#453](https://github.com/adeelahmad/lense/pull/453)).
- **Pictures:** described on this machine with Florence-2 ([#477](https://github.com/adeelahmad/lense/pull/477)), and
low-quality images enhanced before they're read ([#479](https://github.com/adeelahmad/lense/pull/479)).
- **Entities:** names given their real types instead of Term ([#395](https://github.com/adeelahmad/lense/pull/395)).
- **Web clipper:** signing in, or using your API token, not only an import webhook's
([#406](https://github.com/adeelahmad/lense/pull/406)).
- **Automations:** more operators for conditions ([#434](https://github.com/adeelahmad/lense/pull/434)).
Expand Down
2 changes: 1 addition & 1 deletion docs/graph-history.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ it shows in both entities' histories.
| --- | --- |
| `version` | The graph's version after this change. |
| `at` | When. |
| `op` | What happened: `entity.rename`, `entity.describe`, `entity.retype`, `entity.hide`, `entity.show`, `entity.define`, `entity.undefine`, `entity.aliases`, `entity.delete`, `entity.merge`, `entity.unmerge`, `link.add`, `link.remove`, `distinct.add`, `mention.move`, `mention.remove`, `merge.apply`, `link.apply`, `merge.accept`, `link.accept`, `merge.dismiss`, `merge.undo`, `link.undo`, `analysis`. |
| `op` | What happened: `entity.rename`, `entity.describe`, `entity.retype`, `entity.typed` (an upgrade gave old Terms a type), `entity.hide`, `entity.show`, `entity.define`, `entity.undefine`, `entity.aliases`, `entity.delete`, `entity.merge`, `entity.unmerge`, `link.add`, `link.remove`, `distinct.add`, `mention.move`, `mention.remove`, `merge.apply`, `link.apply`, `merge.accept`, `link.accept`, `merge.dismiss`, `merge.undo`, `link.undo`, `analysis`. |
| `actor` | Who: the person's email, `routine:<id>`, `analysis`, or `system`. |
| `via` | Through what: `web`, `token` (an API token), `oauth` (an app), `assistant`, `mcp`, `routine`, `workflow`, `analysis`, `cli`, `system`. |
| `why` | The reason given: a graph change's reason, the model's verdict, why an entity was hidden. |
Expand Down
28 changes: 28 additions & 0 deletions docs/processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,34 @@ places). spaCy's other languages' models carry GPL, LGPL or Creative Commons lic
installed by hand works. Until spaCy is there the rules find the entities and the job log says so. The workflows'
Extract entities (rules) node uses the same extractor.

**How a name gets its type.** The built-in extractor (`analysis.entities: rules`, the default) needs no model. It
reads each name and the words around it (`app/domain/name_types.py`):

- **Person:** a title (Dr Lee, President Obama) or a role before it (our CEO Maria Garcia, my colleague Tom); a
given name first (Sarah Johnson, Maria); a full name followed by a verb of saying (Sarah Connor said); or one of the
namespace's named speakers.
- **Organisation, event, work, place:** the word it ends or starts with (Acme Corp, University of Oxford; Web Summit,
World Cup; Clean Air Act; Hudson River, Mount Fuji), or what it's called (a company called Lumen, an app called
Notion, a town called Hope).
- **Built-in lists:** countries, regions, states and large cities (Place); well-known companies and institutions
(Organisation: Google, Anthropic, NASA) and products and tools (Product: ChatGPT, Kubernetes, Excel); holidays
(Event); days and months on their own (Date). A name with a number in it is a Product (iPhone 16).
- **Term:** anything none of these tells, such as a single unknown word (a brand or a project). A Term can be made a
topic, or retyped on the Entities page (one at a time or ticked together). Names listed in Settings › Analysis ›
Custom vocabulary (`Name | TYPE`) always get the type given there.

A name gets one type per recording, the one most of its lines give it, so "Lee" after "Dr Lee" is a person too. "And"
no longer joins two names into one ("Kenya and Nigeria" are two places). The given names are mostly the US Census
Bureau's frequent first names of 1990 (public domain); the other lists are written for Lens. With
`analysis.entities: spacy`, spaCy's types are used instead, and with
[Find places and dates](configuration.md#anytopdf) anytopdf's places join them.

Older installs get the same types once, as an upgrade step (`entity-types`, [Upgrades](database.md#upgrades)) when
they first start on this version: entities analysis left as Terms get the type these rules give them from their names
and the lines that mention them. Types people or agents set, entities on a fixed list, hidden ones, Unknown and
Unlabeled stay as they are. The changes are recorded in the graph's history as **Gave a type to** events, so they can
be rolled back there.

The **Entities** page in the web app lists a namespace's entities (search, type and collection filters, a Hidden tab).
An entity's name opens its own page, `/wiki/<namespace>/<Name>`, with its notes, every line that says it and what it
comes up with ([Entity pages](notes.md#entity-pages)); the rest of its row opens a side panel where editors rename,
Expand Down
1 change: 1 addition & 0 deletions docs/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ request, to another check that proves the same thing; loosening a requirement or
| The graph as it was at any version | [test_graph_history.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_graph_history.py) `test_the_explorer_and_cypher_as_of_a_version`, `test_the_canvas_as_of_a_version`; [test_graph_history.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_graph_history.py) `test_every_curation_is_an_event_and_every_version_replays` |
| Every entity's own page at `/wiki/<namespace>/<Name>` | [test_entities.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_entities.py) `test_wiki_addresses`; [wiki-page.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/wiki-page.test.tsx); [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `entity-find` |
| Entities, topics and people edited and merged from their lists, the graph or an asset, with Undo instead of a confirm | [test_entities.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_entities.py) `test_suggestions_and_merge_undo`; [entities-curate.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/entities-curate.test.tsx); [topics-curate.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/topics-curate.test.tsx); [recording-entities-edit.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/recording-entities-edit.test.tsx); [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `entity-merge`, `graph-merge`, `speaker-rename` |
| Names get their real types (people, places, organisations, events, works) by rules, without a model; older Terms are typed on upgrade | [test_name_types.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_name_types.py) `test_names_get_the_type_their_words_give`, `test_analysis_types_people_places_and_events`, `test_the_upgrade_types_the_old_terms_and_keeps_what_people_chose` |
| What is most like a passage, an asset, a topic or an entity | [test_similar.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_similar.py) `test_like_a_text_by_its_key_words`, `test_like_a_resource_by_words_topics_and_entities`, `test_like_a_topic_and_an_entity`; [similar.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/similar.test.tsx) |
| With anytopdf, photos found by what they show | [test_photos.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_photos.py) `test_photos_found_by_what_they_show`, `test_off_by_default` |

Expand Down
75 changes: 50 additions & 25 deletions fastapi_backend/app/domain/analyze.py
10BC8 C014
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import urllib.request
from collections import Counter, defaultdict

from . import enrich, entity_map, entity_setup, graph_history, store, telemetry
from . import enrich, entity_map, entity_setup, graph_history, name_types, store, telemetry

STOP = set(
"""a about above after again against all almost also am an and any are aren as at be because been before being
Expand Down Expand Up @@ -47,7 +47,7 @@ def words(text):
return out


# ---------- named things (rules; spaCy optional) ----------
# ---------- named things (rules, typed by name_types.py; spaCy optional) ----------
STOPCAP = set(
(
"I A The This That These Those It Its So And But Or If When While What Why How Who Where Which Yes No Okay Ok Right "
Expand All @@ -65,12 +65,7 @@ def words(text):
ACR_STOP = {"I", "A", "OK", "AI", "TV", "PM", "AM", "US", "UK", "EU", "OMG"}
TITLES = {"Dr", "Mr", "Mrs", "Ms", "Prof", "Professor", "Sir", "Dame", "Senator", "President", "Minister"}
CONTRACTION = re.compile(r"['’](?:m|re|ve|ll|d|t)$", re.I)
CONNECT = {"of", "for", "and", "de", "la", "van", "von", "the", "&"}
ORG_END = re.compile(
r"^(Inc|Ltd|Corp|Corporation|Company|Co|LLC|Group|Bank|Therapeutics|Labs?|Laboratories|University|Institute|"
r"Foundation|Agency|Department|Committee|Council|Association|Society|Commission|Ministry|Office|Board|"
r"Partners|Capital|Systems|Technologies|Holdings|Hospital|School|College)$"
)
CONNECT = {"of", "for", "de", "la", "van", "von", "the", "&"} # not "and": "Kenya and Nigeria" are two names
TOK = re.compile(r"[^\W_](?:[\w’'&.-]*[^\W_])?")
MONTHS = "January|February|March|April|May|June|July|August|September|October|November|December"
DATE_RX = re.compile(
Expand Down Expand Up @@ -98,6 +93,11 @@ def _acr(t):
return re.fullmatch(r"[A-Z0-9&]{2,8}", t) is not None and any(c.isupper() for c in t)


def _role(t, nxt):
"""A role said before a name, as in "CEO Maria" or "Director Maria", rather than the start of a name."""
return t.casefold() in name_types.ROLES and _base(t) not in TITLES and (t.isupper() or nxt.casefold() in name_types.given_names())


def ent_key(name):
return re.sub(r"\s+", " ", _base(name).lower()).strip()

Expand All @@ -112,7 +112,10 @@ def parse_gazetteer(items):
return sorted(out, key=lambda g: -len(g[0]))


def extract_entities(text, gaz=()):
def extract_entities(text, gaz=(), people=frozenset()):
"""[(name, type)] in the order said: the gazetteer's names, dates, runs of capitalised words (typed by
name_types.kind from the name and the words around it; TERM when nothing tells) and numbers. `people`: keys of names
known to be people."""
out, taken = [], []

def add(s, e, t, typ):
Expand Down Expand Up @@ -161,6 +164,8 @@ def initial(k):
or CONTRACTION.search(toks[a][0])
):
a += 1
while a < b and _role(toks[a][0], toks[a + 1][0]):
a += 1 # "CEO Maria Garcia" is Maria Garcia, said after her role
while b >= a and toks[b][0] in CONNECT:
b -= 1
span = toks[a : b + 1]
Expand All @@ -175,16 +180,7 @@ def initial(k):
continue
s0 = span[0][1]
name = _base(text[s0 : span[-1][2]])
last = _base(span[-1][0])
typ = (
"PERSON"
if first.rstrip(".") in TITLES and len(span) > 1
else "ORG"
if ORG_END.match(last)
else "PRODUCT"
if any(ch.isdigit() for x in span for ch in x[0]) and not _acr(name)
else "TERM"
)
typ = name_types.kind([x[0] for x in span], text[:s0], text[s0 + len(name) :], people) or "TERM"
add(s0, s0 + len(name), name, typ)
for m in NUM_RX.finditer(text):
s = m.group(0).strip()
Expand Down Expand Up @@ -256,13 +252,14 @@ def spacy_entities(text, nlp, gaz=()):
return [(t, typ) for _, t, typ in out]


def entity_finder(cfg, gaz=()):
def entity_finder(cfg, gaz=(), people=frozenset()):
"""The extractor Settings → Analysis picks, as a function of a line's text: the rules, or spaCy. While spaCy or its
model isn't installed here the rules find them, and `finder.fell_back` says so."""
model isn't installed here the rules find them, and `finder.fell_back` says so. `people` are the keys the rules
type as people (the namespace's named speakers)."""
nlp = spacy_nlp(cfg["analysis"].get("spacy_model") or "en_core_web_sm") if cfg["analysis"]["entities"] == "spacy" else None

def finder(text):
return spacy_entities(text, nlp, gaz) if nlp else extract_entities(text, gaz)
return spacy_entities(text, nlp, gaz) if nlp else extract_entities(text, gaz, people)

finder.fell_back = cfg["analysis"]["entities"] == "spacy" and nlp is None
return finder
Expand Down Expand Up @@ -426,6 +423,29 @@ def _place(db, cfg, nid, rec, setup, listed, first, segs, ch):
return known


def people_of(db, nid):
"""Keys of the names of the namespace's named speakers, whole and first names: they're people where they're said."""
out = set()
for n in db.values("SELECT VALUE name FROM speaker WHERE space = $s AND name != NONE AND name != ''", s=nid):
k = ent_key(str(n))
if k and not k.startswith("speaker"):
out.add(k)
if " " in k and len(k.split()[0]) >= 3:
out.add(k.split()[0])
return frozenset(out)


def one_type(seg_ents):
"""Each name gets one type in a recording: the one most of its lines give it, a term only where none gives another
(a Dr Lee in one line is a person in every line)."""
votes = defaultdict(Counter)
for es in seg_ents:
for n, t in es:
votes[ent_key(n)][t] += 1
best = {k: max(c, key=lambda t: (t != "TERM", c[t])) for k, c in votes.items()}
return [[(n, best.get(ent_key(n), t)) for n, t in es] for es in seg_ents]


def analyze_recording(db, cfg, rid, seg_ents=None, say=None):
"""Entities, keywords, sections and talk statistics. `seg_ents` (per line, [(name, type)]) replaces the extractor:
a workflow's save entities node passes what its nodes found."""
Expand All @@ -439,7 +459,7 @@ def analyze_recording(db, cfg, rid, seg_ents=None, say=None):
)
extra = entity_map.terms(db, nid, entity_map.chain_of(db, rec.get("collection"))) if listed else []
gaz = parse_gazetteer(list(cfg["analysis"].get("gazetteer") or []) + extra) # the defined entities, however written
find = entity_finder(cfg, gaz) if seg_ents is None else None
find = entity_finder(cfg, gaz, people_of(db, nid)) if seg_ents is None else None
if find and find.fell_back and say:
say(
f"spaCy or its model {cfg['analysis'].get('spacy_model')} isn't installed here (Settings → Components): the rules found the entities"
Expand All @@ -465,17 +485,22 @@ def analyze_recording(db, cfg, rid, seg_ents=None, say=None):
es[:] = [(n, typed.get(ent_key(n), t) if t == "TERM" else t) for n, t in es] # a bare term anytopdf knows
have = {ent_key(n) for n, _ in es}
es.extend(h for h in hints if ent_key(h[0]) not in have)
seg_ents = one_type(seg_ents)
if setup["types"] and not listed: # the namespace (or collection) keeps only some types
seg_ents = [[(n, t) for n, t in es if entity_setup.keeps(setup, t)] for es in seg_ents]
surface = {t: c.most_common(1)[0][0] for t, c in ts.items()}
starts = [0] + tiling(toks, wc) if segs else []
titles = section_titles(starts, toks, surface, seg_ents, len(segs)) if segs else []
keys = {ent_key(n) for es in seg_ents for n, _ in es} - {""}
alias = {k: max((L for L in keys if len(L) > len(k) and L.endswith(" " + k)), key=len, default=k) for k in keys}
first = {}
first, votes = {}, defaultdict(Counter)
for es in seg_ents:
for name, typ in es:
first.setdefault(alias.get(ent_key(name), ent_key(name)), (name, typ))
k = ent_key(name)
first.setdefault(alias.get(k, k), (name, typ))
votes[alias.get(k, k)][typ] += 1
# a short form takes its full name's type ("Johnson" after "Dr Sarah Johnson"), a term only when nothing else
first = {k: (n, max(votes[k], key=lambda t: (t != "TERM", votes[k][t]))) for k, (n, _) in first.items()}
with graph_history.acting(via="analysis", recording=rid), graph_history.change(db, "analysis", recording=rid) as ch:
known = _place(db, cfg, nid, rec, setup, listed, first, segs, ch)
over = {
Expand Down
Loading
Loading
0