8000 fix(entities): give names their real types instead of Term by adeelahmad · Pull Request #395 · adeelahmad/lense · GitHub
[go: up one dir, main page]
Skip to content

fix(entities): give names their real types instead of Term - #395

Merged
adeelahmad merged 7 commits into
mainfrom
claude/project-thread-qfebqj
Oct 17, 2026
Merged

fix(entities): give names their real types instead of Term#395
adeelahmad merged 7 commits into
mainfrom
claude/project-thread-qfebqj

Conversation

@adeelahmad
@adeelahmad adeelahmad commented Oct 15, 2026 •
edited
Loading
Copy link
Copy Markdown
Owner

Requested by Adeel · project thread

Description

Before: almost every entity analysis found was a Term. The default rules extractor typed a capitalised name as a person only after a title (Dr, Mr), as an organisation only before Inc/Corp-like words and as a product only with a digit; people, places and most organisations all came out as Term.

After: names get their real types without any model. app/domain/name_types.py reads each name and the words around it:

  • Person: a title or role before it (our CEO Maria Garcia), a given name first (Sarah Johnson, Maria), a full name followed by a verb of saying, or one of the namespace's named speakers (in analysis and in the workflows' Extract entities (rules) node).
  • Organisation / event / work / place from the word a name ends or starts with (Acme Corp, University of Oxford; Web Summit, World Cup; Clean Air Act; Hudson River, Mount Fuji), and from "a company called Lumen", "an app called Notion", "a town called Hope".
  • Built-in lists: countries, regions, states and large cities; well-known companies and products (Google, Anthropic, NASA; ChatGPT, Kubernetes); holidays; days and months.
  • Only names nothing tells stay Terms. A name gets one type per recording (the one most lines give it), "and" no longer glues two names into one ("Kenya and Nigeria"), and a role before a name is no longer part of it ("CEO Maria Garcia" is Maria Garcia).

Existing installs: upgrade step entity-types types the Terms analysis made, from their names and the lines that mention them. Types people or agents set (found in the graph history and the audit log), entities on a fixed list, hidden ones, Unknown and Unlabeled are left alone. The changes are recorded as entity.typed events ("Gave a type to") in the graph history, so they can be rolled back.

Motivation and Context

Adeel: "every entity comes as term? why!!"

Steps to reproduce (if appropriate):

Before this PR, analyze.extract_entities("Yesterday I met Sarah Johnson in London to talk about the Google deal.") gave [('Sarah Johnson', 'TERM'), ('London', 'TERM'), ('Google', 'TERM')]. It now gives PERSON, PLACE, ORG.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change; ! in the title)

Checklist:

  • The title is a Conventional Commit.
  • My code follows the code style of this project (make hooks checks it on commit).
  • I added or updated tests (tests/domain/test_name_types.py; tests that expected Berlin or Amazon Web Services to be Terms now expect their real types).
  • My change requires documentation updates, and I have updated docs/ accordingly (processing.md "How a name gets its type", graph-history.md op list).
  • Users need more than the title to understand it, so I wrote it under ## Unreleased in CHANGELOG.md.

Done questions (AGENTS.md)

  • Goal: names get real types in analysis, and old Terms are typed on upgrade. Merged on green CI. Single unknown words still need a model; that is the next PR (decision model).
  • Agents: agents see the types through the existing API, MCP and assistant entity tools; nothing new to call.
  • Zero setup: built-in lists, no new dependency or step.
  • Fewest clicks: fewer manual retypes needed.
  • Configurable: n/a, no new setting. Settings › Analysis › Custom vocabulary (Name | TYPE) still wins over the rules; with spaCy picked, spaCy's types are used.
  • Light: about 40 KB of text lists, loaded once; no model, nothing heavy.
  • Private and secure: n/a, no new surface; nothing leaves the machine.
  • Cost: no model calls at all.
  • User decides: types people or agents set are kept; the upgrade is recorded in the graph history and can be rolled back.
  • Upgrades: entity-types step, safe to run twice (tested).
  • Honest: docs say what the rules do and that unknown single words stay Terms.
  • Tests, docs and generated files updated (no API change, so no OpenAPI change). The README lists it as built (Understand it), with its checks in docs/requirements.md, and its In progress line is gone.
  • Open standards; given names are US Census 1990 data (public domain), other lists written for Lens (MIT). No new dependency.
  • Reported in the thread; refine-later items in DEVLOG.md.
  • Reuse: builds on the existing rules extractor, entity_finder and graph history events; no new library.

Local CI (ci/local-ci.sh) on the merged head: backend, Next.js, pre-commit and title pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_018MeaEgM8m7rTjbkKVQVsrT

@claude
fix(entities): give names their real types instead of Term
b3616d4 
The default rules extractor typed a capitalised name as a person only after a
title, an organisation only before Inc/Corp-like words and a product only with
a digit, so people, places and most organisations came out as Term.

name_types.py types each name from the name and the words around it: titles
and roles, given names, verbs of saying, the words a name starts or ends with,
"a company called X", built-in lists of places, well-known organisations and
products, holidays and dates, and the namespace's named speakers. A name gets
one type per recording, "and" no longer joins two names, and a role before a
name is not part of it.

Upgrade step entity-types types the Terms analysis made on existing installs,
keeping types people or agents set; it is recorded in the graph history as
entity.typed so it can be rolled back.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018MeaEgM8m7rTjbkKVQVsrT
@adeelahmad adeelahmad self-assigned this Oct 15, 2026
claude added 6 commits October 15, 2026 10:56
@claude
Merge remote-tracking branch 'origin/main' into claude/project-thread…
3c79512 
…-qfebqj

# Conflicts:
#	CHANGELOG.md
@claude
test(clicks): wait for the imports before merging in the graph
9196139 
The graph shows Raspberry Pi only once both imports are analysed (one
recording's mentions are below graph.min_edge_weight). Converting the
uploaded file took 42-46 s on some runners, so graph-merge opened the graph
before the second import was done and timed out. It now waits, uncounted,
until the top bar's activity says nothing is running or queued.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018MeaEgM8m7rTjbkKVQVsrT
@claude
Merge remote-tracking branch 'origin/main' into claude/project-thread…
40911aa 
…-qfebqj

# Conflicts:
#	CHANGELOG.md
#	DEVLOG.md
#	ci/clicks/journeys.mjs
#	docs/processing.md
#	fastapi_backend/app/domain/migrations.py
#	fastapi_backend/tests/domain/test_migrations.py
@claude
Merge remote-tracking branch 'origin/main' into claude/project-thread…
bab7d45 
…-qfebqj

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018MeaEgM8m7rTjbkKVQVsrT
@claude
Merge remote-tracking branch 'origin/main' into claude/project-thread…
88cb713 
…-qfebqj

Main's entity_finder (spaCy fetched when picked) now also takes the namespace's
named speakers for the rules, in analysis and in the workflows' rules node.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018MeaEgM8m7rTjbkKVQVsrT
@claude
docs(readme): list typed names as built, with their checks
10ff564 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018MeaEgM8m7rTjbkKVQVsrT
@adeelahmad
adeelahmad merged commit ccfca12 into main Oct 17, 2026
11 of 12 checks passed
@adeelahmad
adeelahmad deleted the claude/project-thread-qfebqj branch October 17, 2026 04:56
adeelahmad pushed a commit that referenced this pull request Oct 17, 2026
@claude
fix(ci): the upgrade check counts entities of every type
9c46640 
Since #395 the entity-types upgrade step types Terms such as Monday as
Dates, which the entities list leaves out unless asked for, so the check
read them as gone. It now lists the quiet types too: an entity may change
type in an upgrade, but none may go.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013sC1p4dJXjteytbxDXnk5g
This was referenced Oct 17, 2026
Merged
Merged
Merged
Merged
Merged
adeelahmad pushed a commit that referenced this pull request Oct 17, 2026
@claude
fix(ci): the upgrade check counts entities of every type
703d600 
Since #395 the entity-types upgrade step types Terms such as Monday as
Dates, which the entities list leaves out unless asked for, so the check
read them as gone. It now lists the quiet types too: an entity may change
type in an upgrade, but none may go.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013sC1p4dJXjteytbxDXnk5g
adeelahmad added a commit that referenced this pull request Oct 17, 2026
@adeelahmad @claude
fix(ci): the upgrade check counts entities of every type (#512)
8fa6f87 
Since #395 the entity-types upgrade step types Terms such as Monday as
Dates, which the entities list leaves out unless asked for, so the check
read them as gone. It now lists the quiet types too: an entity may change
type in an upgrade, but none may go.


Claude-Session: https://claude.ai/code/session_013sC1p4dJXjteytbxDXnk5g

Co-authored-by: Adeel Ahmad <noreply@anthropic.com>
@adeelahmad adeelahmad mentioned this pull request Oct 17, 2026
Merged
20 of 22 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@adeelahmad adeelahmad

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@adeelahmad @claude
0