adeelahmad · Oct 5, 2026 · Oct 5, 2026
diff --git a/docs/assistant.md b/docs/assistant.md
@@ -6,8 +6,11 @@ Conversations belong to one person and can be scoped to namespaces, recordings,
 collection in the scope is read each time the assistant answers, so it draws on the collection's recordings as they are
 then.
 
-- **Retrieval:** the question's keywords go through the full-text index (English stemming), limited to namespaces the
-  person can read. Hits are widened to their neighbouring lines and numbered.
+- **Retrieval:** the question's keywords go through the word index (English stemming) and, when search by meaning is
+  set up, passages about the question in other words are found too
+  ([Search by meaning](processing.md#search-by-meaning)); the two are ranked together by reciprocal rank, limited to
+  namespaces the person can read and to the conversation's scope. Hits are widened to their neighbouring lines and
+  numbered.
 - **Answers:** the model answers only from those excerpts and cites them as [n]. Each citation carries the recording
   and timestamp.
 - **Streaming:** answers arrive over server-sent events: `passages`, then `token`s, then `done`.
@@ -32,8 +35,6 @@ then.
 - **No model configured:** chat returns the best-matching passages instead. Anyone signed in can see whether a model
   is set up, and which (`GET /chats/capabilities`), so the app says so before the first question.
 
-Retrieval is keyword-based for now; vector search is not built yet.
-
 ### Picking the namespace for a conversation over everything
 
 When a conversation has no scope (the assistant home starts it over everything), its first question goes to the
@@ -48,12 +49,12 @@ never narrowed this way.
 
 ### Assistant mode and voice
 
-Home opens in **assistant mode** (one field and a big mic, like a search page) when the archive has any content, and
-on the overview when it's empty; the switch at the top right remembers the person's pick in the browser. The field
-opens a new conversation over everything the person can read (`/chat?global=1`, plus `q=` with what was typed); the
-mic opens one in voice mode (`/chat?global=1&voice=1`). Voice mode listens, sends what was heard, reads the answer
-aloud (without citation marks) and listens again, until the mic is tapped off or nothing is said twice in a row. The
-chat composer's mic turns it on in any conversation.
+Home opens in **assistant mode** (one field and a big mic, like a search page) when the archive has any content, and on
+the overview when it's empty; the switch at the top right remembers the person's pick in the browser. Enter in the field
+opens a new conversation over everything the person can read and sends what was typed as its first question
+(`/chat?global=1&q=…&send=1`); the mic opens one in voice mode (`/chat?global=1&voice=1`). Voice mode listens, sends
+what was heard, reads the answer aloud (without citation marks) and listens again, until the mic is tapped off or
+nothing is said twice in a row. The chat composer's mic turns it on in any conversation.
 
 Voice goes through one hook, `useVoice()` in `nextjs-frontend/lib/voice.ts` (`listen`, `speak`, `stop`).
 

diff --git a/docs/configuration.md b/docs/configuration.md
@@ -50,16 +50,18 @@ A fresh install (no accounts when the API first starts) walks its first admin th
    `LENS_NAMESPACE` names namespaces, or an install script made one.
 3. **Model provider**: an OpenAI-compatible server (OpenAI, Ollama, llama.cpp, LM Studio, vLLM), its model and key,
    with a test.
-4. **Storage**: where data lives (shown; set in archive.yaml and `SURREAL_URL`), the largest upload, and optionally a
-   folder inside `sources.local_roots` to watch.
+4. **Storage**: where data lives (shown; set in archive.yaml and `SURREAL_URL`), where Lens keeps its own files (this
+   machine or a storage connection, [Storage](storage.md)), the largest upload, and optionally a folder inside
+   `sources.local_roots` to watch.
 5. **Apps and AI**: whether apps and AI assistants (Claude, ChatGPT, Cursor and other MCP clients) may sign people in
    with their Lens account ([OAuth](authentication.md#oauth)), on unless turned off, how long their tokens last, and
    the MCP server's address to add to an assistant.
 6. **Telemetry**: off unless chosen ([Telemetry](telemetry.md)).
 
 Every step can be skipped, and the whole wizard too; all of it stays in Settings. Values from the environment win and
 show locked. Installs that already had accounts never see the wizard. The API side is `GET /api/v1/setup`,
-`POST /api/v1/setup/namespace`, `PUT /api/v1/setup/llm`, `PUT /api/v1/setup/storage`, `PUT /api/v1/setup/oauth`,
+`POST /api/v1/setup/namespace`, `PUT /api/v1/setup/llm`, `GET /api/v1/setup/llm/detect` (model servers running on
+this machine or the Docker host, with their models), `PUT /api/v1/setup/storage`, `PUT /api/v1/setup/oauth`,
 `PUT /api/v1/setup/telemetry` and `POST /api/v1/setup/finish`
 (admins); `GET /api/v1/auth/status` says whether it is still pending (`wizard_pending`).
 
@@ -86,11 +88,27 @@ Start from `fastapi_backend/archive.example.yaml`, which documents every key. Th
 
 ## Settings in the app
 
-Admins can change transcription, diarisation, voice-ID thresholds, analysis, LLM provider and key, graph, search
-(and search by meaning),
-reports, workers, IIIF, the assistant, video, uploads, documents and images, and server options (embed frame ancestors, transcript upload
-limit, allowed hosts, trusted proxies, session length). The API refuses an allowed-host list that leaves out the address
-you are using.
+Admins can change, in Settings: transcription, speaker separation, speech providers, voice IDs, analysis, the LLM
+provider and key, a local model, the AI assistant, search (word search and search by meaning), reports and graph,
+video, OCR, faces and objects, workers, sign-in, components, access and embedding (embed frame ancestors, transcript
+upload limit, allowed hosts, trusted proxies, session length), remote access, email, chat rooms, notifications,
+telemetry, the Fedora repository, sensors, uploads, storage, documents, API keys, and IIIF and metadata. The API
+refuses an allowed-host list that leaves out the address you are using.
+
+## Word search
+
+Searching by the words goes through the index of `search.engine` (Settings → Search → Word search; how each works is
+in [Database](database.md#schema)):
+
+| Setting | Default | |
+|---|---|---|
+| `search.engine` | `sqlite` | `sqlite`: a SQLite FTS5 file per database in `data_dir/search/`. `surrealdb`: SurrealDB's own full-text index, which needs several times the memory. `opensearch`: an OpenSearch (or Elasticsearch-compatible) cluster |
+| `search.opensearch_url` | empty | the cluster's address, such as `http://search.lan:9200`; with `opensearch` and no address, the built-in index is used |
+| `search.opensearch_user` / `opensearch_password` | empty / none | for a cluster that asks for them; the password is stored encrypted |
+| `search.opensearch_verify` | true | whether its certificate is checked; turn it off only for a self-signed one on your own network |
+| `search.stemming` | `english` | `none` for archives that aren't in English |
+
+After changing the engine or stemming, run `lens reindex` (or Settings → Search → Reindex now).
 
 ## Search by meaning
 
@@ -146,13 +164,14 @@ shot is a request, so a long document costs as many: `llm.describe_max` caps how
 
 ## Uploads
 
-Audio, video, documents (PDF) and images uploaded in the web app (Import → Upload) go up in pieces
+Audio, video, documents (PDF, Office and OpenDocument files, text, Markdown, saved web pages and emails) and images
+uploaded in the web app (Import → Upload) go up in pieces
 ([API](api.md#uploads)). Settings → Uploads:
 
 | Setting | Default | |
 |---|---|---|
 | `uploads.max_mb` | 4096 | the largest file, in MB |
-| `uploads.extensions` | the types folder scans import, PDFs and images | which types can be uploaded: any of `.m4a .mp3 .wav .flac .ogg .opus .aac .amr .aif .aiff .wma .mp4 .m4v .mov .mkv .webm .avi .mpg .mpeg .3gp .pdf .jpg .jpeg .png .tif .tiff .webp .gif .bmp`. A list saved before documents and images came leaves them out until they're added |
+| `uploads.extensions` | the types folder scans import, documents and images | which types can be uploaded: any of `.m4a .mp3 .wav .flac .ogg .opus .aac .amr .aif .aiff .wma .mp4 .m4v .mov .mkv .webm .avi .mpg .mpeg .3gp .pdf .doc .docx .odt .rtf .ppt .pptx .odp .xls .xlsx .ods .txt .text .md .markdown .mdx .html .htm .eml .msg .jpg .jpeg .png .tif .tiff .webp .gif .bmp`. A list saved before documents and images came leaves them out until they're added |
 | `uploads.chunk_mb` | 8 | how much the web app sends per request, 1–64. Keep it below the request-body limit of any proxy in front of the web app (nginx's `client_max_body_size` is 1 MB unless set) |
 | `uploads.expire_hours` | 24 | how long an unfinished upload waits for its next piece before what arrived is deleted, 1–720 |
 
@@ -238,7 +257,7 @@ video's sampled frames and on a document's or an image's pages ([API](api.md#obj
   offer them its source, so it's in no image and no extra.
 * `off`.
 
-Without one the step is skipped, and its job says why. Settings → Video:
+Without one the step is skipped, and its job says why. Settings → Video, OCR, faces and objects:
 
 | Setting | Default | |
 |---|---|---|

diff --git a/docs/database.md b/docs/database.md
@@ -1,7 +1,8 @@
 # Database: SurrealDB
 
 Everything Lens stores lives in SurrealDB: recordings, transcript segments, speakers and voiceprints, entities and
-the knowledge graph, full-text indexes, jobs, accounts, settings and audit.
+the knowledge graph, jobs, accounts, settings and audit. The word search index is kept beside it, in a SQLite file by
+default ([below](#schema)).
 
 ## Two ways to run it
 
@@ -20,11 +21,12 @@ The Docker setups run SurrealDB 3.2.4 on the `surrealkv` storage engine. Lens is
 engine and with SurrealDB 2.3 and 3.2 servers.
 
 !!! note "Embedded mode is for trying Lens and small archives"
-    The embedded engine (SurrealDB 2.x inside the Python SDK) is slow to write and its full-text index loses entries
-    when the database is closed and reopened. Lens repairs the index once per process, before the first search (the
-    API does it at startup, `lens` commands that don't search skip it), which takes about 30 ms per transcript
-    segment: seconds for a few thousand segments, but about ten minutes for 20,000. Beyond a few thousand segments,
-    run a SurrealDB server; the Docker setups already do.
+    The embedded engine (SurrealDB 2.x inside the Python SDK) is slow to write. With `search.engine: surrealdb`, its
+    full-text index also loses entries when the database is closed and reopened. Lens then repairs the index once per
+    process, before the first search (the API does it at startup, `lens` commands that don't search skip it), which
+    takes about 30 ms per transcript segment: seconds for a few thousand segments, but about ten minutes for 20,000.
+    The default word index (SQLite) isn't affected. Beyond a few thousand segments, run a SurrealDB server; the Docker
+    setups already do.
 
 !!! warning "Use `surrealkv` (or RocksDB/TiKV) for servers, not `memory`"
     Under concurrent writes, SurrealDB 3.2.4's `memory` engine occasionally loses updates (we reproduced duplicate ids
@@ -54,8 +56,8 @@ With `opensearch`, the same words go to an OpenSearch (or Elasticsearch-compatib
 an index per table named `lens-<namespace>-<database>-<table>`, with the `english` analyser unless
 `search.stemming: none`. It suits an archive too big for one small machine, with OpenSearch on another one. When the
 cluster can't be reached, searches read the tables instead and say nothing is wrong, so Lens keeps working. The
-`opensearch` compose profile (`docker compose --profile opensearch up`) runs one next to Lens at
-`http://opensearch:9200`.
+`opensearch` profile of the production compose file (`docker compose -f docker-compose.prod.yml --profile opensearch
+up -d`) runs one next to Lens at `http://opensearch:9200`.
 
 With `surrealdb`, full-text indexes use 3.x's `FULLTEXT` syntax and fall back to 2.x's `SEARCH`, with a
 `snowball(english)` analyser unless `search.stemming: none`. After changing either setting, run `lens reindex` (or
@@ -113,9 +115,10 @@ single statements and for `run()` transactions (which roll back as a whole, so r
   deletes only the extras.
 * **Multi-statement `query()` in the Python SDK** only checks the first statement's result. Transactions go through
   `DB.run()`, which checks every statement.
-* **The embedded full-text index loses postings on reopen** (the `SEARCH` index of the 2.x embedded engine): after a
-  restart, searches silently missed most segments. `DB.ready_fulltext()` rebuilds it once per process before the
-  first full-text query; `tests/domain/test_search_index.py` covers it. Servers (3.x `FULLTEXT`) are unaffected.
+* **The embedded full-text index loses postings on reopen** (the `SEARCH` index of the 2.x embedded engine, used only
+  with `search.engine: surrealdb`): after a restart, searches silently missed most segments. `DB.ready_fulltext()`
+  rebuilds it once per process before the first full-text query; `tests/domain/test_search_index.py` covers it.
+  Servers (3.x `FULLTEXT`) are unaffected.
 * **The embedded engine's vector index drops filtered rows**: with a KNN operator plus a parenthesised or OR
   filter, or sometimes just two filters, 2.x returned nothing. On the embedded engine, search by meaning compares the
   query with every passage that passes the filter (`vector::similarity::cosine`) instead, which is exact and takes

diff --git a/docs/encryption.md b/docs/encryption.md
@@ -46,9 +46,12 @@ How encrypted files are used:
 Not encrypted yet, and planned: moving a recording's files to its new namespace's key (until then a moved file still
 opens with the key of the namespace it came from).
 
-What isn't encrypted by Lens: the SurrealDB database itself, including transcripts and the indexes that full-text
-and semantic search need, and your own folders that Lens scans (it only reads them). Put the data volume on an
-encrypted disk (LUKS, FileVault, BitLocker or an encrypted ZFS dataset) to cover these.
+What isn't encrypted by Lens: the SurrealDB database itself, including transcripts and the vectors that search by
+meaning needs; the word search index, which by default is a SQLite file in `<data_dir>/search/` (SurrealDB's own index
+with `search.engine: surrealdb`, or an OpenSearch cluster with `opensearch`); and your own folders that Lens scans (it
+only reads them). Put the data volume on an
+encrypted disk (LUKS, FileVault, BitLocker or an encrypted ZFS dataset) to cover these, and the OpenSearch cluster's
+disk when you use one.
 
 ## Vaults
 

diff --git a/docs/processing.md b/docs/processing.md
@@ -154,9 +154,11 @@ about them come back as not found.
 ## Search
 
 All words must appear, matched after English stemming ("exploit" also finds exploits and exploiting); "quoted phrases"
-must appear as written; OR separates alternatives. Filter by namespace, speaker, emotion or recording. For archives that
-aren't in English set `search.stemming: none` and run `lens reindex`. Prefix search (`expl*`) from the SQLite
-version is gone; stemming covers most of what it was used for.
+must appear as written; OR separates alternatives. Filter by namespace, speaker, emotion or recording. The words are
+looked up in the index of `search.engine`: a SQLite FTS5 file by default, SurrealDB's own index, or OpenSearch
+([configuration](configuration.md#word-search)). For archives that aren't in English set `search.stemming: none` and
+run `lens reindex`. There is no prefix search: a trailing `*` (`expl*`) is ignored and the word is searched as it is;
+stemming covers most of what prefixes were used for.
 
 ### Search by meaning
 

diff --git a/docs/storage.md b/docs/storage.md
@@ -12,7 +12,7 @@ right there (the same dialog as Sources) and is checked before it's saved.
 | Setting | |
 |---|---|
 | `store` | `local`: `data_dir/objects` on this machine (the default). `connection`: a storage connection |
-| `connection` | the id of a storage connection (Settings → Sources): S3 or S3-compatible (AWS, MinIO, Backblaze, R2 ...), Google Drive, Dropbox, OneDrive, SFTP, SMB, WebDAV, or a folder on this machine inside `sources.local_roots` |
+| `connection` | the id of a storage connection (on the Sources page, `/sources`): S3 or S3-compatible (AWS, MinIO, Backblaze, R2 ...), Google Drive, Dropbox, OneDrive, SFTP, SMB, WebDAV, or a folder on this machine inside `sources.local_roots` |
 | `folder` | where on it: a path; for S3, the bucket and a path (`my-bucket/lens`); for a folder on this machine, its full path |
 | `crypt` | also wrap it in an [rclone crypt](https://rclone.org/crypt/) remote, which hides the names and sizes of what is kept there |
 

diff --git a/docs/technology-selection.md b/docs/technology-selection.md
@@ -2,16 +2,27 @@
 
 - [FastAPI](https://fastapi.tiangolo.com/) and [Pydantic](https://docs.pydantic.dev/): the HTTP API, request and
   response validation, and the OpenAPI schema the frontend client is generated from
-- [SurrealDB](https://surrealdb.com/): documents, graph edges and full-text search in one database, embedded or as a
-  server ([why](architecture.md#design-decisions))
-- [PyJWT](https://pyjwt.readthedocs.io/): short-lived access tokens; scrypt (standard library) for passwords;
-  [cryptography](https://cryptography.io/) (AES-GCM) for credentials stored in the database
+- [SurrealDB](https://surrealdb.com/): documents, graph edges and vectors in one database, embedded or as a server
+  ([why](architecture.md#design-decisions))
+- Word search: [SQLite FTS5](https://www.sqlite.org/fts5.html) (standard library) by default, or an optional
+  [OpenSearch](https://opensearch.org/) cluster ([Database](database.md#schema))
+- [py_webauthn](https://github.com/duo-labs/py_webauthn): passkeys, the default way to sign in (passwords are off on
+  fresh installs); [PyJWT](https://pyjwt.readthedocs.io/): short-lived access tokens; scrypt (standard library) for
+  passwords where they're on; [cryptography](https://cryptography.io/) (AES-GCM) for credentials stored in the database
+- [rdflib](https://rdflib.readthedocs.io/): the archive as RDF (Dublin Core) for export, import and SPARQL
+  ([RDF](rdf.md))
+- [OpenTelemetry](https://opentelemetry.io/): opt-in traces and metrics over OTLP/HTTP ([Telemetry](telemetry.md))
+- [llama.cpp](https://github.com/ggml-org/llama.cpp): `llama-server` runs a GGUF chat model on the server itself
+  ([Local models](local-models.md))
 - [Next.js](https://nextjs.org/) (App Router) with [Auth.js / NextAuth v5](https://authjs.dev/) for the browser session
 - [@hey-api/openapi-ts](https://heyapi.dev/): the typed API client; [Zod](https://zod.dev/) for form validation
 - [shadcn/ui](https://ui.shadcn.com/) and [Tailwind CSS](https://tailwindcss.com/)
+- [BlockSuite](https://blocksuite.io/): the notes editor; [@xyflow/react](https://reactflow.dev/): the canvas
+  pipelines and workflows are drawn on
 - Processing: ffmpeg, [SenseVoice](https://github.com/FunAudioLLM/SenseVoice) / [faster-whisper](https://github.com/SYSTRAN/faster-whisper) /
   [mlx-whisper](https://github.com/ml-explore/mlx-examples), [SpeechBrain](https://speechbrain.github.io/) ECAPA voiceprints,
   optional [pyannote](https://github.com/pyannote/pyannote-audio), Tesseract / Apple Vision / RapidOCR, OpenCV YuNet + SFace,
+  ONNX models on [ONNX Runtime](https://onnxruntime.ai/) (YOLOX objects, RapidOCR),
   [rclone](https://rclone.org/) for remote storage, Jinja2 (sandboxed) for templates
 - [uv](https://docs.astral.sh/uv/), [pnpm](https://pnpm.io/), [Ruff](https://docs.astral.sh/ruff/), mypy, ESLint,
   pytest, Jest, Docker Compose, pre-commit, [mkdocs-material](https://squidfunk.github.io/mkdocs-material/)