adeelahmad · Oct 14, 2026 · Oct 10, 2026 · Oct 10, 2026 · Oct 10, 2026 · Oct 10, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,10 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
 
 ## Unreleased
 
+- **Apache Tika.** Lens can use an Apache Tika server (`--profile tika` in the compose files, or one of your own, set
+  in Settings → Documents). Documents nothing here can make into a PDF, such as Word files on a lean server or a
+  Raspberry Pi, are read by Tika as text, and every document's own title, authors, language and dates are read as
+  Dublin Core terms. Off until a Tika server is set ([Configuration](docs/configuration.md#apache-tika)).
 - **New pipelines, workflows, templates and API tokens come with a name.** The web app guesses one from what you're
   making ("Transcribe to report", "Recording workflow", "Prompt template 2", "Read only token"), as it already did for
   routines, so making one takes no typing; type a name to choose your own. The pencil next to the name renames it

diff --git a/ci/clicks/run.sh b/ci/clicks/run.sh
@@ -19,7 +19,7 @@ stop() {
     wait "$p" 2>/dev/null || true
     kill "$w" 2>/dev/null || true
   done
-  rm -rf "$tmp"
+  rm -rf "$tmp" 2>/dev/null || { sleep 2; rm -rf "$tmp" || true; }
 }
 trap stop EXIT INT TERM
 

diff --git a/docker-compose.prod.yml b/docker-compose.prod.yml
@@ -164,6 +164,21 @@ services:
     restart: unless-stopped
     networks: [lens]
 
+  # Apache Tika (docs/configuration.md#apache-tika): reads the text and description of about a thousand kinds of file where Lens can't make
+  # them into PDFs itself, and what every document says about itself. Opt-in, about 250 MB of memory:
+  # `docker compose --profile tika up -d` (or COMPOSE_PROFILES=tika in .env), then Settings → Documents → Tika server:
+  # http://tika:9998. Reachable only inside Lens's network; it has no sign-in of its own.
+  tika:
+    image: apache/tika:${TIKA_VERSION:-4.0.0-1}
+    profiles: [tika]
+    security_opt: ["no-new-privileges:true"]
+    read_only: true
+    tmpfs: [/tmp]
+    environment:
+      JAVA_TOOL_OPTIONS: -Xmx${TIKA_HEAP:-512m}
+    restart: unless-stopped
+    networks: [lens]
+
   # The MCP runner (mcp-runner/runner.py, docs/assistant.md): MCP servers that run as a program (stdio, usually npx or
   # uvx), which admins add as extensions. Opt-in: `docker compose --profile mcp up -d`, or COMPOSE_PROFILES=mcp in .env.
   # The sandbox: none of Lens's data or keys, no capabilities, a read-only image, and a network shared only with the

diff --git a/docker-compose.yml b/docker-compose.yml
@@ -145,6 +145,20 @@ services:
       - matterbridge-data:/matterbridge
     networks: [lens]
 
+  # Apache Tika (docs/configuration.md#apache-tika): reads the text and description of about a thousand kinds of file where Lens can't make
+  # them into PDFs itself, and what every document says about itself. Opt-in, about 250 MB of memory:
+  # `docker compose --profile tika up -d` (or COMPOSE_PROFILES=tika in .env), then Settings → Documents → Tika server:
+  # http://tika:9998. Reachable only inside Lens's network; it has no sign-in of its own.
+  tika:
+    image: apache/tika:${TIKA_VERSION:-4.0.0-1}
+    profiles: [tika]
+    security_opt: ["no-new-privileges:true"]
+    read_only: true
+    tmpfs: [/tmp]
+    environment:
+      JAVA_TOOL_OPTIONS: -Xmx${TIKA_HEAP:-512m}
+    networks: [lens]
+
   # The MCP runner (mcp-runner/runner.py, docs/assistant.md): MCP servers that run as a program (stdio, usually npx or
   # uvx), which admins add as extensions. Opt-in: `docker compose --profile mcp up -d`, or COMPOSE_PROFILES=mcp in .env.
   # The sandbox: none of Lens's data or keys, no capabilities, a read-only image, and a network shared only with the

diff --git a/docs/api.md b/docs/api.md
@@ -509,7 +509,10 @@ made, unless someone named or dated it. Its attachments are kept as its files (r
 read become resources of their own beside it, each with `attached_to: {resource, file, title}` (the email, and its
 title when the caller may see it); an attachment file's `resource` is the resource it became, or the one it was already
 in the namespace (`documents.attachment_resources`). Transcribing an email again keeps and makes nothing twice.
-`GET /uploads/limits` says what the server can convert (`convert: {office, pages, msg}`); an upload it can't read is
+`GET /uploads/limits` says what the server can convert (`convert: {office, pages, msg, tika}`); with an
+[Apache Tika](configuration.md#apache-tika) server set, what nothing else can convert is read by Tika as text without
+pages (`engine` `text:tika`), and every document has `described`, what it says about itself as Dublin Core terms
+(`POST /settings/documents/tika/test` checks the server). An upload it can't read is
 refused, saying what it needs, and the files that can be transcripts too (PDFs, Word, text and Markdown) can be imported
 as transcripts instead (`POST /import`). Its pipeline's read step (its transcribe step, for a document or an image:
 no speech model is fetched for one) draws its pages and reads their text: a PDF's own text, block by block with where each block is on its page (poppler's

diff --git a/docs/configuration.md b/docs/configuration.md
@@ -237,6 +237,9 @@ each the first time it's needed; without them, PDFs and images are read as befor
 | `documents.converter` | `auto` | who makes the PDFs: `auto` uses [anytopdf](#anytopdf) only for what LibreOffice and Chromium can't do here, `anytopdf` uses it for every document and image (and fetches it), `lens` never uses it |
 | `documents.anytopdf_url` | none | a conversion node: another machine running `anytopdf queue serve` and `anytopdf queue work`, such as `https://convert.home:8640`. Set, it converts instead of the program here |
 | `documents.anytopdf_token` | none | the node's bearer token (its `ANYTOPDF_QUEUE_TOKEN`), kept encrypted |
+| `documents.tika_url` | none (off) | an [Apache Tika](#apache-tika) server, such as `http://tika:9998` for the compose files' `tika` service |
+| `documents.tika_text` | true | with a Tika server: documents nothing here can make into a PDF are read by Tika, as text without pages |
+| `documents.tika_metadata` | true | with a Tika server: what every document says about itself (title, authors, language, dates) is read by Tika |
 
 Set at startup only (the config file; the web app can't choose what the server runs):
 
@@ -247,6 +250,37 @@ Set at startup only (the config file; the web app can't choose what the server r
 | `documents.anytopdf` | the one Lens downloaded, else `anytopdf` on PATH | the anytopdf program |
 | `documents.web_networks` | `[]` | networks (CIDR, such as `10.20.0.0/16`) that web pages and calendar feeds may be fetched from besides the public internet: for an intranet, or a calendar server at home; loopback and cloud metadata addresses stay out unless listed. `LENS_WEB_NETWORKS` in `.env` (comma-separated) adds to it, for Docker and the packages |
 
+### Apache Tika
+
+[Apache Tika](https://tika.apache.org) (Apache-2.0) reads the text and the description of about a thousand kinds of
+file. Lens can use a Tika server; it's off until `documents.tika_url` is set. The compose files have one ready, as an
+opt-in profile (about 250 MB of memory, so leave it off on a small Raspberry Pi):
+
+```sh
+docker compose -f docker-compose.prod.yml --profile tika up -d
+```
+
+(or add `COMPOSE_PROFILES=tika` to Lens's `.env` and run the install command again). Then, in Settings → Documents,
+set the Tika server to `http://tika:9998` and press Test. A Tika server elsewhere works the same way.
+
+- **Reading what nothing else can** (`documents.tika_text`). When neither LibreOffice, Chromium nor
+  [anytopdf](#anytopdf) can make a document into a PDF here (a Word file on a lean server, say), or one of them fails
+  on it, Tika reads it instead. Uploads and folder scans then take every document type, Office files included. Such
+  a document has its text, which is searched, analyzed and asked about like any other's, but no pages to look at;
+  its `engine` is `text:tika`. An email or an archive read this way gives its text, including its attachments' and
+  files', but they aren't kept as files or resources of their own.
+- **What a document says about itself** (`documents.tika_metadata`). Every document is also described by Tika, as
+  Dublin Core terms (`GET /resources/{id}` → `described`: title, creators, subject, description, publisher,
+  language, created, modified, pages, words, format). Its title and creation date become the resource's while it's
+  still named and dated after its file (an email's own subject and date come first), and its authors and language
+  are its [metadata](rdf.md)'s until someone saves others. If Tika can't describe it, the document is read anyway.
+
+The document goes only to the address set: http(s) with no user name or password in it, no redirect followed, and
+sent as `document.<type>`, never by its own name. Tika has no sign-in of its own, so the compose service is reachable
+only inside Lens's network; put one elsewhere behind TLS on a network you trust, since every document is sent to it.
+Its answers are bounded (64 MB) before Lens reads them. The compose service runs read-only, as a user without
+privileges, with `TIKA_HEAP` (default `512m`) as its Java heap and `TIKA_VERSION` (default `4.0.0-1`) as its image.
+
 ### anytopdf
 
 [anytopdf](https://github.com/adeelahmad/anytopdf-rs) (MIT or Apache-2.0) is Lens's sister project: one static program

@@ -1,10 +1,12 @@
 # Deployment
 
 Lens runs as containers: SurrealDB, the API, one or more workers, and the Next.js app. `docker-compose.prod.yml`
-is a production-shaped starting point. Two more services are opt-in profiles: `--profile fedora` adds a Fedora 6
-repository to keep a copy of the archive in ([Fedora](fedora.md)), and `--profile opensearch` adds OpenSearch for word
+is a production-shaped starting point. More services are opt-in profiles: `--profile fedora` adds a Fedora 6
+repository to keep a copy of the archive in ([Fedora](fedora.md)), `--profile opensearch` adds OpenSearch for word
 search on a big archive (then choose it in Settings → Search and reindex; the default is a SQLite index in
-`data_dir/search`).
+`data_dir/search`), and `--profile tika` adds an Apache Tika server that reads documents nothing else here can, and
+what every document says about itself (then set it in Settings → Documents;
+[Configuration](configuration.md#apache-tika)).
 
 ```bash
 export ACCESS_SECRET_KEY=$(openssl rand -hex 32) ARCHIVE_SECRET_KEY=$(openssl rand -hex 32) \

@@ -39,6 +39,7 @@
     store,
     telegram,
     telemetry,
+    tika,
     tunnel,
 )
 from app.schemas.admin import (
@@ -73,6 +74,7 @@
     TelegramStatus,
     TelemetryStatus,
     TelemetryTestResult,
+    TikaTestResult,
     TunnelStatus,
 )
 from app.schemas.auth import AccountToken
@@ -281,6 +283,17 @@ def test_conversion_node(user: AdminWriter, cfg: Cfg) -> ConversionNodeTestResul
     return ConversionNodeTestResult(ok=error is None, error=error, ms=None if error else int((time.time() - t0) * 1000))
 
 
+@router.post("/settings/documents/tika/test")
+def test_tika(user: AdminWriter, cfg: Cfg) -> TikaTestResult:
+    """Check that a Tika server answers at documents.tika_url, and say which version it is."""
+    t0 = time.time()
+    try:
+        said = tika.version(cfg)
+    except ValueError as e:
+        return TikaTestResult(ok=False, error=str(e))
+    return TikaTestResult(ok=True, version=said, ms=int((time.time() - t0) * 1000))
+
+
 @router.get("/settings/decisions/status")
 def decision_status(user: AdminReader, cfg: Cfg) -> DecisionStatus:
     """Who takes routine decisions, and whether a local Laya model can here: on this machine (Apple Silicon, once

@@ -32,7 +32,7 @@
 from email.utils import getaddresses, parsedate_to_datetime
 from html.parser import HTMLParser
 
-from . import anytopdf, netguard, store
+from . import anytopdf, netguard, store, tika
 
 R = store.R
 OFFICE = {
@@ -187,19 +187,36 @@ def _has_msg():
     return True
 
 
+def _can(cfg):
+    """What this server can make into PDFs itself, or with anytopdf: {office, pages, msg}."""
+    by = None if anytopdf.mode(cfg) == "lens" else anytopdf.available(cfg)
+    lo, web = bool(soffice(cfg)) or fetchable("libreoffice"), web_capture(cfg)
+    office, pages = lo or by == "node", bool(web or lo or by)
+    return {"office": office, "pages": pages, "msg": pages and _has_msg()}
+
+
 def capabilities(cfg):
     """What this server can make into PDFs: Office files (LibreOffice), text, Markdown, saved web pages and emails
     (Chromium or LibreOffice), Outlook .msg emails (those, and the extract-msg package), and web pages captured from
     their address (Chromium). anytopdf does the same where Lens can't (anytopdf.py): emails, text and pages here, and
-    Office files too on a conversion node."""
-    by = None if anytopdf.mode(cfg) == "lens" else anytopdf.available(cfg)
-    lo, web = bool(soffice(cfg)) or fetchable("libreoffice"), web_capture(cfg)
-    office, pages = lo or by == "node", bool(web or lo or by)
-    return {"office": office, "pages": pages, "msg": pages and _has_msg(), "web": web}
+    Office files too on a conversion node. A Tika server reads all of those as text where nothing here can (tika.py)."""
+    got = {**_can(cfg), "web": web_capture(cfg), "tika": tika.reads_text(cfg)}
+    if got["tika"]:
+        got.update(office=True, pages=True, msg=True)
+    return got
 
 
 def unavailable(cfg, name):
-    """Why this server can't read this document (None when it can): what it would need."""
+    """Why this server can't read this document (None when it can): what it would need. A Tika server reads, as text,
+    any document Lens can't make into a PDF here."""
+    why = unconvertible(cfg, name)
+    if why and tika.reads_text(cfg) and ext_of(name) in store.DOCUMENT_EXT:
+        return None
+    return why
+
+
+def unconvertible(cfg, name):
+    """Why this server can't make this document into a PDF (None when it can): what it would need."""
     ext = ext_of(name)
     if ext in store.CAMERA_EXT:
         if anytopdf.mode(cfg) == "lens" or not anytopdf.available(cfg):
@@ -208,11 +225,13 @@ def unavailable(cfg, name):
         return None
     if ext not in store.DOCUMENT_EXT or ext == ".pdf":
         return None
-    can = capabilities(cfg)
+    can = _can(cfg)
     if ext in OFFICE and not can["office"]:
-        return f"converting {word(name)}s needs LibreOffice on the server (the lens:full image) or an anytopdf conversion node"
+        return (
+            f"converting {word(name)}s needs LibreOffice on the server (the lens:full image), an anytopdf conversion node or a Tika server"
+        )
     if ext not in OFFICE and not can["pages"]:
-        return f"converting {word(name)}s needs Chromium or LibreOffice on the server (the lens:full image)"
+        return f"converting {word(name)}s needs Chromium or LibreOffice on the server (the lens:full image), or a Tika server"
     if ext == ".msg" and not can["msg"]:
         return "reading Outlook .msg emails needs the extract-msg package on the server (pip install -e '.[msg]')"
     return None
@@ -270,7 +289,7 @@ def office_pdf(cfg, src, out):
     """An Office, OpenDocument or RTF file made into a PDF by LibreOffice, offline and with a profile of its own."""
     exe = soffice(cfg)
     if not exe:
-        raise Unavailable(unavailable(cfg, src) or "LibreOffice isn't installed")
+        raise Unavailable(unconvertible(cfg, src) or "LibreOffice isn't installed")
     with tempfile.TemporaryDirectory(prefix="lens-office-") as tmp:
         doc = pathlib.Path(tmp) / f"document{ext_of(src)}"
         shutil.copyfile(src, doc)
@@ -827,7 +846,7 @@ def to_pdf(cfg, src, out, say=None):
     """Make `out`, the PDF of the document at `src`. Returns what was learnt on the way: {by, title?, email?,
     attachments?}. Unavailable when the server can't; ValueError when the file can't be read. A program installed
     the first time it's needed (fetchable) is installed first, which `say` tells."""
-    why = unavailable(cfg, src)
+    why = unconvertible(cfg, src)
     if why:
         raise Unavailable(why)
     ext, path, via = ext_of(src), pathlib.Path(src), by_anytopdf(cfg, src)