adeelahmad · Oct 7, 2026 · Oct 7, 2026 · Oct 7, 2026 · Oct 7, 2026 · Oct 7, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,9 @@ The backend (`fastapi_backend`) and the frontend (`nextjs-frontend`) are version
 
 ## Unreleased
 
+- **anytopdf can find faces.** Settings › Video, OCR, faces and objects › Face engine › anytopdf: its plugins find
+  the faces in frames and pages and describe them with the same SFace model, so a server needs no OpenCV, and the
+  people a namespace already knows still match ([Video](docs/video.md)). Who someone is stays in Lens.
 - **Search photos by what they show.** Turn on Settings › Search › Search photos by what they show (off by default)
   and Lens downloads anytopdf's CLIP model (about 600 MB). Search then also shows the photos most like what you typed,
   such as "a bus on a city street", whatever words are on them ([anytopdf](docs/configuration.md#anytopdf)).

diff --git a/docs/components.md b/docs/components.md
@@ -15,6 +15,7 @@ archive that only uses cloud models never downloads them, which keeps a small ma
 | Whisper model | `transcribe.engine` is `whisper` | the faster-whisper model named in `transcribe.whisper.model` |
 | Voice IDs | speakers are separated by clustering (the default) | SpeechBrain and its ECAPA voiceprint model |
 | Face models | `video.face_engine` is `opencv` (the default) | OpenCV, YuNet and SFace |
+| Face models | `video.face_engine` is `anytopdf` | SFace, for anytopdf to describe faces with (anytopdf too) |
 | Object model | `video.object_engine` is `yolox` (the default) or `anytopdf` | YOLOX-s, and ONNX Runtime for `yolox` |
 | Chat model, embedding model | the LLM provider or embeddings server is Ollama | pulled on that server (`/api/pull`) |
 | Outlook .msg emails | listed in `components.also` | extract-msg (GPL-3.0, so only when asked for) |

diff --git a/docs/configuration.md b/docs/configuration.md
@@ -248,6 +248,10 @@ Lens can also ask anytopdf's face and object plugins what is in a picture (`anyt
 (`--plugin-sandbox strict`: no network, writing only to their own workspace, reading only the picture and the models
 Lens names), and the objects plugin runs only when it's given a model, such as Lens's YOLOX file.
 
+With `video.face_engine` set to `anytopdf`, the faces step uses these plugins too ([Video](video.md)): YuNet finds the
+faces and the face-id plugin describes them with Lens's SFace file. Lens reads each face's description from the face
+index of that run, then deletes it with the rest of the run.
+
 **Places and dates.** With `analysis.anytopdf` on (Settings › Analysis › Find places and dates with anytopdf; off by
 default), analysis asks anytopdf, run here with no network and no plugins, what a recording's text and a photo hold:
 

diff --git a/docs/video.md b/docs/video.md
@@ -18,8 +18,12 @@ soundtrack, and three more steps look at the picture. Each step skips itself for
   - When a face is on screen while one speaker talks, the app suggests they're the same person.
   - People can be renamed, merged (with undo), linked to a speaker, marked "not a face" or deleted.
   - Engines: OpenCV's YuNet + SFace (`pip install -e ".[faces]"`, then point `video.yunet_model` and
-    `video.sface_model` at the ONNX files from the OpenCV Zoo) or InsightFace, whose pretrained models are licensed for
-    non-commercial research only.
+    `video.sface_model` at the ONNX files from the OpenCV Zoo), InsightFace, whose pretrained models are licensed for
+    non-commercial research only, or `anytopdf`: [anytopdf](configuration.md#anytopdf)'s plugins find faces with
+    their built-in YuNet and describe them with the same SFace file, so it needs neither OpenCV nor Python packages
+    (Lens fetches anytopdf and SFace on first use). Its faces match ones OpenCV found, so switching keeps the people a
+    namespace knows. It looks at up to 50 frames or pages per run, sandboxed and offline; anytopdf's own face index
+    is thrown away after each run, since who someone is stays Lens's.
 
 Faces are biometric data, so they are off by default. A namespace owner chooses:
 

diff --git a/fastapi_backend/app/domain/anytopdf.py b/fastapi_backend/app/domain/anytopdf.py
@@ -22,11 +22,13 @@
 What it answers is bounded (MAX_JSON for a job, MAX_PDF for its PDF, which must start like one) before Lens reads it.
 
 analyze() asks its face and object plugins, kept beside the program, what is in a picture: sandboxed, with no network,
-reading only the picture and the models Lens names.
+reading only the picture and the models Lens names. faces() also has its face-id plugin describe each face, for
+video.face_engine `anytopdf`.
 """
 
 from __future__ import annotations
 
+import contextlib
 import json
 import os
 import pathlib
@@ -61,6 +63,8 @@
 JOB_ID = re.compile(r"[A-Za-z0-9_.:-]{1,128}")
 # the plugins Lens keeps beside the program, for analyze(): the rest of the release's plugins stay behind
 PLUGINS = ("anytopdf-plugin-faces", "anytopdf-plugin-face-id", "anytopdf-plugin-objects", "anytopdf-plugin-clip")
+# what finds faces (YuNet, built in) and what describes them (an ArcFace-style ONNX model), for faces()
+FACE_PLUGINS = PLUGINS[:2]
 
 
 class Unavailable(RuntimeError):
@@ -366,13 +370,10 @@ def check(cfg):
         return f"it can't be reached ({getattr(e, 'reason', e)})"
 
 
-def analyze(cfg, picture, models=None, seconds=None):
-    """What anytopdf's plugins find in a picture, run here: [{"kind": "face" | "object", "label", "box": [x, y, w, h]
-    fractions, "score", "attributes"}], one per thing found where it is (their summaries left out).
-
-    `models` are the plugins' settings, such as {"ANYTOPDF_OBJECTS_MODEL": "/models/yolox_s.onnx"}: a plugin that
-    needs a model runs only when it's given one, and may read nothing else (the plugins run sandboxed, with no network
-    and no config file). Faces are found by the bundled YuNet whatever is given."""
+@contextlib.contextmanager
+def _looked_at(cfg, pictures, models, seconds, args=()):
+    """anytopdf run here over copies of the pictures with its plugins, sandboxed with no network: (its folder, its
+    graph, {copy: picture}) while the block runs, then nothing of it is left."""
     from . import convert
 
     exe = binary(cfg)
@@ -381,20 +382,34 @@ def analyze(cfg, picture, models=None, seconds=None):
     models = {k: str(v) for k, v in (models or {}).items() if k.startswith("ANYTOPDF_") and v}
     seconds = seconds or _opts(cfg).get("convert_seconds") or 300
     with tempfile.TemporaryDirectory(prefix="lens-anytopdf-") as tmp:
-        pic = pathlib.Path(tmp) / f"picture{pathlib.Path(picture).suffix.lower() or '.jpg'}"
-        shutil.copyfile(picture, pic)
+        names = {}
+        for n, picture in enumerate(pictures):
+            pic = pathlib.Path(tmp) / f"picture-{n}{pathlib.Path(picture).suffix.lower() or '.jpg'}"
+            shutil.copyfile(picture, pic)
+            names[str(pic)] = picture
         env = netguard.nowhere_env({**os.environ, "HOME": tmp, "TMPDIR": tmp})
         env = {k: v for k, v in env.items() if not k.startswith("ANYTOPDF_")}
         env.update(models, ANYTOPDF_DATA_DIR=f"{tmp}/data")
         reads = [a for v in models.values() if os.path.exists(v) for a in ("--plugin-sandbox-allow-read", v)]
         graph = pathlib.Path(tmp) / "graph.json"
         argv = [exe, "--no-config", "--plugin-sandbox", "strict", *reads, "--plugin-timeout", str(int(seconds))]
-        argv += ["convert", str(pic), "-o", f"{tmp}/out.pdf", "--dump-graph", str(graph), "--ocr", "off", *QUIET]
+        argv += ["convert", *names, "-o", f"{tmp}/out.pdf", "--dump-graph", str(graph), "--ocr", "off", *QUIET, *args]
         r = convert._run(argv, seconds, env=env, cwd=tmp)
         if r.returncode != 0 or not graph.is_file():
             said = convert._last_said(r.stderr or r.stdout)
             raise ValueError(f"anytopdf couldn't look at it ({said or f'exit {r.returncode}'})")
-        found = json.loads(graph.read_text())
+        yield pathlib.Path(tmp), json.loads(graph.read_text()), names
+
+
+def analyze(cfg, picture, models=None, seconds=None):
+    """What anytopdf's plugins find in a picture, run here: [{"kind": "face" | "object", "label", "box": [x, y, w, h]
+    fractions, "score", "attributes"}], one per thing found where it is (their summaries left out).
+
+    `models` are the plugins' settings, such as {"ANYTOPDF_OBJECTS_MODEL": "/models/yolox_s.onnx"}: a plugin that
+    needs a model runs only when it's given one, and may read nothing else (the plugins run sandboxed, with no network
+    and no config file). Faces are found by the bundled YuNet whatever is given."""
+    with _looked_at(cfg, [picture], models, seconds) as (_, found, _names):
+        pass
     out = []
     for unit in found.get("units") or []:
         for a in unit.get("annotations") or []:
@@ -414,6 +429,57 @@ def analyze(cfg, picture, models=None, seconds=None):
     return out
 
 
+def face_plugins(cfg):
+    """Whether the plugins that find and describe faces are beside the program (fetch() keeps them in plugins/)."""
+    exe = binary(cfg)
+    found = [pathlib.Path(exe).resolve().parent / "plugins" / n for n in FACE_PLUGINS] if exe else []
+    return bool(found) and all(p.is_file() and os.access(p, os.X_OK) for p in found)
+
+
+def faces(cfg, pictures, model, seconds=None):
+    """The faces on each picture, found by anytopdf's YuNet and described by its face-id plugin with `model` (the SFace
+    file Lens fetches, so they match faces OpenCV found), in one run: {picture: [{"box": [x, y, w, h] fractions,
+    "score", "embedding": unit vector}]}. The face index it builds is thrown away with the run: who someone is stays
+    Lens's (faces.py)."""
+    models = {"ANYTOPDF_FACE_EMBED_MODEL": model}
+    with _looked_at(cfg, pictures, models, seconds, ["--recognize-faces", "--face-index", "faces.sqlite"]) as (tmp, graph, names):
+        found = faces_found(graph, tmp / "faces.sqlite")
+    return {picture: found.get(copy, []) for copy, picture in names.items()}
+
+
+def faces_found(graph, index):
+    """The faces in the graph of a run, by the file each is on: {path: [{"box", "score", "embedding"}]}. The graph has
+    each face's box and confidence but never its embedding; that is in the run's face index (faces.sqlite, its
+    sightings), at the same box."""
+    import sqlite3
+
+    import numpy as np
+
+    def key(path, box):
+        return str(path), *(round(float(v), 3) for v in box)
+
+    seen = {}
+    if pathlib.Path(index).is_file():
+        with contextlib.closing(sqlite3.connect(f"file:{index}?mode=ro", uri=True)) as db:
+            for path, vec, *box in db.execute("SELECT source_path, vector, x, y, w, h FROM sightings ORDER BY id"):
+                seen[key(path, box)] = np.frombuffer(vec, dtype="<f4").astype(np.float64)
+    paths = {s.get("id"): s.get("path") for s in graph.get("sources") or []}
+    out = {}
+    for unit in graph.get("units") or []:
+        path = paths.get(unit.get("source_id"))
+        for a in unit.get("annotations") or []:
+            r = a.get("region")
+            if a.get("kind") != "face" or not r:  # the one without a box is the picture's count ("2 faces")
+                continue
+            box = [float(r["x"]), float(r["y"]), float(r["width"]), float(r["height"])]
+            emb = seen.get(key(path, box))
+            if emb is None:
+                continue
+            face = {"box": box, "score": float(a.get("confidence") or 0.0), "embedding": emb / (np.linalg.norm(emb) + 1e-9)}
+            out.setdefault(path, []).append(face)
+    return out
+
+
 def page_words(pdf):
     """The words anytopdf read on each page of a PDF it made, from the chunks it embeds in it: {page (from 0): {"words":
     [{text, box}] in reading order, box [x, y, w, h] as fractions of the page, "ocr": whether OCR read them}}. Only pages

diff --git a/fastapi_backend/app/domain/components.py b/fastapi_backend/app/domain/components.py
@@ -373,17 +373,26 @@ def fetch(self, cfg, m, say):
 
 
 class Faces(Component):
+    """OpenCV with YuNet and SFace; for anytopdf (video.face_engine `anytopdf`) only SFace, which its face-id plugin
+    describes faces with, since it finds them itself."""
+
+    def _files(self, cfg):
+        return FACE_FILES[1:] if cfg["video"].get("face_engine") == "anytopdf" else FACE_FILES
+
     def needed(self, cfg, m):
         v = cfg["video"]
+        if v.get("face_engine") == "anytopdf":
+            return not v.get("sface_model")
         return v.get("face_engine") == "opencv" and not (v.get("yunet_model") and v.get("sface_model") and importable("cv2"))
 
     def present(self, cfg):
-        return importable("cv2") and all(model_file(cfg, n) for n in FACE_FILES)
+        opencv = cfg["video"].get("face_engine") == "anytopdf" or importable("cv2")
+        return opencv and all(model_file(cfg, n) for n in self._files(cfg))
 
     def fetch(self, cfg, m, say):
-        if not importable("cv2"):
+        if cfg["video"].get("face_engine") != "anytopdf" and not importable("cv2"):
             pip_install(cfg, "faces", say)
-        for n in FACE_FILES:
+        for n in self._files(cfg):
             fetch_file(cfg, n, say)
 
 
@@ -405,15 +414,17 @@ def fetch(self, cfg, m, say):
 
 class Anytopdf(Component):
     """The anytopdf program (anytopdf.py), fetched when Settings → Documents asks for it to run here, or Settings →
-    Analysis asks it for places and dates (enrich.py), or Settings → Video for objects (objects.py)."""
+    Analysis asks it for places and dates (enrich.py), or Settings → Video for objects (objects.py) or faces."""
 
     def _serves(self, cfg):
         from . import anytopdf
 
         converts = anytopdf.mode(cfg) == "anytopdf" and not anytopdf.node(cfg)
         detects = cfg["video"].get("object_engine") == "anytopdf"  # its objects plugin runs here, node or not
+        faces = cfg["video"].get("face_engine") == "anytopdf"
         enriches = bool((cfg.get("analysis") or {}).get("anytopdf"))
-        return ({"transcribe"} if converts else set()) | ({"objects"} if detects else set()) | ({"analyze"} if enriches else set())
+        steps = {"transcribe": converts, "objects": detects, "faces": faces, "analyze": enriches}
+        return {step for step, wanted in steps.items() if wanted}
 
     def needed(self, cfg, m):
         from . import anytopdf
@@ -582,8 +593,8 @@ def detail(self, cfg):
     Anytopdf(
         "anytopdf",
         "anytopdf",
-        "makes documents and photographed pages into searchable PDFs (documents.converter), finds places and dates, and objects (video.object_engine)",
-        steps={"transcribe", "analyze", "objects"},
+        "makes documents and photographed pages into searchable PDFs (documents.converter), finds places and dates, and objects and faces (video.object_engine, video.face_engine)",
+        steps={"transcribe", "analyze", "objects", "faces"},
         size_mb=50,
         license="MIT OR Apache-2.0",
     ),

diff --git a/fastapi_backend/app/domain/settings.py b/fastapi_backend/app/domain/settings.py
@@ -128,7 +128,7 @@
     ("search", "engine"): set(store.SEARCH_ENGINES),
     ("reports", "audio"): {"link", "embed", "none"},
     ("video", "ocr_engine"): {"auto", "tesseract", "apple-vision", "rapidocr", "doctr", "none"},
-    ("video", "face_engine"): {"opencv", "insightface", "none"},
+    ("video", "face_engine"): {"opencv", "insightface", "anytopdf", "none"},
     ("video", "object_engine"): {"yolox", "ultralytics", "anytopdf", "off"},
     ("decisions", "engine"): {"auto", "jev", "laya", "llm", "off"},
     ("voice", "input"): {"auto", "server", "browser"},

diff --git a/fastapi_backend/app/domain/video.py b/fastapi_backend/app/domain/video.py
@@ -3,12 +3,14 @@
 ffmpeg finds scene changes and samples frames. An OCR engine reads text from the frames: Tesseract (anywhere), Apple
 Vision (on a Mac, through pyobjc), RapidOCR (onnxruntime) or docTR (PyTorch). A face engine detects faces and, where a namespace allows
 it, describes them for the per-namespace registry in faces.py: OpenCV's YuNet + SFace (Apache-2.0/MIT models from the
-OpenCV Zoo; set video.yunet_model and video.sface_model) or InsightFace (its pretrained models are licensed for
-non-commercial research only). Frames and face crops live in data_dir/frames/<recording>/.
+OpenCV Zoo; set video.yunet_model and video.sface_model), InsightFace (its pretrained models are licensed for
+non-commercial research only) or anytopdf, whose plugins find faces and describe them with the same SFace model.
+Frames and face crops live in data_dir/frames/<recording>/.
 """
 
 from __future__ import annotations
 
+import contextlib
 import functools
 import json
 import pathlib
@@ -23,6 +25,7 @@
 from . import ingest, jobs, keyring, store
 
 R = store.R
+FACE_BATCH = 50  # pictures per run of a face engine that reads many at once
 VIDEO_TYPES = {
     ".mp4": "video/mp4",
     ".m4v": "video/mp4",
@@ -442,9 +445,38 @@ def faces(self, path):
         ]
 
 
+class AnytopdfFaces:
+    """anytopdf (anytopdf.py) finds faces with the YuNet built into its faces plugin and describes them with the same
+    SFace model OpenCVFaces uses, through its face-id plugin, so neither OpenCV nor Python packages are needed and its
+    faces match the ones OpenCV found. Who they are stays Lens's: the namespace's registry in faces.py."""
+
+    name = "anytopdf"
+
+    def __init__(self, cfg):
+        from . import anytopdf, components
+
+        self.cfg = cfg
+        self.model = cfg["video"].get("sface_model") or components.model_file(cfg, components.FACE_FILES[1])
+        if not (self.model and pathlib.Path(self.model).exists()):
+            raise RuntimeError("set video.sface_model to the OpenCV Zoo SFace ONNX file")
+        if not anytopdf.face_plugins(cfg):
+            raise RuntimeError("anytopdf isn't here with its faces and face-id plugins")
+
+    def faces(self, path):
+        return self.faces_on([path])[str(path)]
+
+    def faces_on(self, paths):
+        """The faces on many pictures in one run of anytopdf: {path: faces}."""
+        from . import anytopdf
+
+        found = anytopdf.faces(self.cfg, [str(p) for p in paths], self.model)
+        return {str(p): found[str(p)] for p in paths}
+
+
 def face_engine(cfg):
     try:
-        return {"opencv": OpenCVFaces, "insightface": InsightFaces}[cfg["video"].get("face_engine") or "opencv"](cfg)
+        engines = {"opencv": OpenCVFaces, "insightface": InsightFaces, "anytopdf": AnytopdfFaces}
+        return engines[cfg["video"].get("face_engine") or "opencv"](cfg)
     except (ImportError, RuntimeError, KeyError):
         return None
 
@@ -576,7 +608,15 @@ def step_faces(db, cfg, rid, say):
     else:
         frames, step = rec.get("samples") or [], rec.get("sample_ms") or 5000
     d, dets = frames_dir(cfg, rid), []
-    for t, name in frames:
-        with keyring.plain_picture(db, cfg, d / name) as pic:
-            dets += [{"t": t, "frame": name, **f} for f in engine.faces(pic)]
+    if hasattr(engine, "faces_on"):  # many pictures at a time, where starting the engine costs more than a picture
+        for n in range(0, len(frames), FACE_BATCH):
+            batch = frames[n : n + FACE_BATCH]
+            with contextlib.ExitStack() as open_:
+                pics = [str(open_.enter_context(keyring.plain_picture(db, cfg, d / name))) for _, name in batch]
+                found = engine.faces_on(pics)
+            dets += [{"t": t, "frame": name, **f} for (t, name), pic in zip(batch, pics) for f in found[pic]]
+    else:
+        for t, name in frames:
+            with keyring.plain_picture(db, cfg, d / name) as pic:
+                dets += [{"t": t, "frame": name, **f} for f in engine.faces(pic)]
     faces.store_tracks(db, cfg, rid, rec["space"], dets, mode, step, say, paged)
diff --git a/fastapi_backend/archive.example.yaml b/fastapi_backend/archive.example.yaml
@@ -132,7 +132,7 @@ video:                              # everything here except the model paths can
   ocr_engine: auto                  # auto (Apple Vision on a Mac, else Tesseract, else RapidOCR) | tesseract | apple-vision | rapidocr | none
   ocr_languages: [eng]
   ocr_min_confidence: 60
-  face_engine: opencv               # opencv (YuNet + SFace from the OpenCV Zoo) | insightface (non-commercial models) | none
+  face_engine: opencv               # opencv (YuNet + SFace from the OpenCV Zoo) | insightface (non-commercial models) | anytopdf (its plugins, with SFace) | none
   yunet_model: null                 # path to face_detection_yunet_2023mar.onnx (bootstrap-only)
   sface_model: null                 # path to face_recognition_sface_2021dec.onnx (bootstrap-only)
   face_cluster_threshold: 0.6