Repository navigation
Add live, fail-closed image search over Wikimedia Commons - #607
Closed
blueheron786 wants to merge 13 commits into
Closed
blueheron786 wants to merge 13 commits into
blueheron786 wants to merge 13 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Collaborator
Author
|
Coverage: the missing lines were all in the model wrappers, which only ran in tests that are skipped without the real models. They now run in CI around a tiny tokenizer and fake ONNX sessions ( |
GET /api/v2/image-search/ searches Commons live and returns only images that pass every safety layer: a query gate, deterministic deny rules over title, description and categories, a self-hosted DistilBERT text classifier, and NudeNet run on a thumbnail that is fetched, scored and discarded in memory. Anything uncertain, missing or failing is excluded; a Commons outage is a 503, never an unchecked list. Nothing is stored except approved results in the cache for ten minutes. GET /api/v2/image-search/top/ returns one approved image for a strong title match, for showing alongside web results. Wikimedia's 429/503 Retry-After is honoured through a shared back-off, requests carry an identifying User-Agent, and concurrency is bounded. Models are built by scripts/export_image_safety_models.py into IMAGE_SEARCH_MODEL_DIR. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- A thumbnail fetch that fails because Wikimedia is unreachable or rate-limiting now raises instead of excluding the image, so it becomes a 503 and is never cached as a final empty result. - The plain-search fallback also triggers on an API error body; a failing fallback or a non-JSON body is ImageSourceUnavailable, not a 500. - Entity-encoded markup in Commons metadata is decoded before tags are stripped, so it cannot be revived; only https licence links and commons.wikimedia.org page links pass. - The text classifier judges long texts chunk by chunk (worst chunk wins) and treats texts too long to check, and empty texts, as uncertain, so categories are no longer beyond its 128-token window. - Deny rules fold accents, width variants and invisible characters. - CirrusSearch syntax in the query is removed before it reaches Commons. - The query gate refuses only a confident NSFW verdict: the model scores bare words such as "cat" as uncertain, which blanked innocent searches. - Document that no layer detects CSAM and that approved thumbnail URLs are mutable. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A separate stage exports the text model to ONNX with CPU torch and fetches NudeNet's detector; the final stage copies only the exported files and sets IMAGE_SEARCH_MODEL_DIR, so the image ships onnxruntime and not torch. The stage depends on nothing but the export script, so it stays cached until the script changes. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CI has no models, so the wrappers' own code only ran in tests that are skipped there. Run it in CI around a tiny tokenizer and fake ONNX sessions: tokenizing, chunking, overflow, softmax, preprocessing, thresholds and undecodable or oversized images. Also cover a successful thumbnail fetch, top_image served from the cache, and the /top/ 503. mwmbl/imagesearch is now fully covered without the models. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The text model is trained on sentences and on a bare word it is noise: it calls "gorilla", "monkey" and "ape" NSFW, so those searches returned nothing. Judge the query framed as a caption instead, and add the explicit terms the deny rules lacked (onlyfans, camgirl, busty, horny, milf and similar), since the rules, not the model, are what catch explicit queries. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
An uncached image search fetches up to thirty thumbnails and runs two models, and the endpoints are public. Limit each client network (the proxy-seen address, IPv6 grouped by /64, as search_callers does) to IMAGE_SEARCH_REQUESTS_PER_MINUTE across both endpoints, with a 429 before any work is done. One client's burst would otherwise also trip Wikimedia's own limit and pause image search for everyone. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
blueheron786
force-pushed
the
wikicommons_image_search
branch
from
November 8, 2026 02:51
5893b23 to
e18d7ac
Compare
One search ran the text model over about 60 chunks and took 10-23s of CPU (2-6s of wall time) on a 12-core machine. Two things were wasting it: - Every chunk was padded to 128 tokens, though the average real length is about 72. The text model now runs in batches of 8 sorted by length, each trimmed to its longest chunk. Padding is masked, so a chunk's probability is unchanged: on 539 real chunks all verdicts were identical. - ONNX Runtime started a spinning thread per core for each model. Sessions now get a configured thread count (2 for text, 1 for NudeNet, which already runs in parallel across the thumbnail workers) and idle threads sleep. Measured per search, CPU time, before -> after: text model 10.4 / 16.4 / 23.5s -> 4.3 / 4.0 / 6.2s (wall time no worse) NudeNet ~1.1-1.7s per 12 images -> ~0.3s query gate 0.04-0.23s -> 0.015s Dynamic int8 quantization of the text model was also tried: about 2x faster again, but it moved verdicts (several chunks swung by 0.5+ in NSFW probability), so it is left for a labelled evaluation. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The same Commons images come up in many queries, but only whole result lists were cached (for 10 minutes), so every new query re-ran the text model and re-fetched and re-scored every thumbnail. Verdicts are now cached per image: - text verdicts by a hash of the text, so an edited description is rejudged; - pixel verdicts by the file's Commons SHA-1 and thumbnail width, so a replaced file is rejudged even though its thumbnail URL is unchanged. Keys also carry a fingerprint of the model files and the thresholds and chunking that decide a verdict, so retraining or retuning invalidates them. Only verdicts are stored, under hashed keys, never text or images. What is deliberately not cached: an UNCERTAIN pixel verdict (it may only mean a thumbnail failed to download or decode), and any pixel verdict for a file whose current version is under 24 hours old, because Wikimedia can keep serving the replaced file's thumbnail for a while and its verdict must not be filed under the new SHA-1. A SHA-1 collision between a harmless and an explicit file is an accepted risk, documented in verdict_cache. Redis rather than SQL: the verdicts are cheap to recompute, the deployed Redis is already a bounded allkeys-lru cache, and a Redis flush costs only a re-check. SQL would add a migration, model-version invalidation, and make Commons-derived data permanent; it is worth revisiting if images are indexed. Measured end to end against live Commons with the real models (CPU time per search): cold 4-7s; the same images under another query string 0.1-0.4s; a repeat once the result cache has expired 0.04s, with no thumbnail fetches. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
Author
|
Following up on the server-load discussion. I pushed two independent commits to this branch, one for each question: can we cut CPU per search, and should we cache the expensive part. The numbers below are from real searches against live Commons with the real models. The short answers:
Where the CPU was goingA search runs the DistilBERT text model over about 60 chunks (each image's title, description and categories), then NudeNet on up to 30 thumbnails. I measured CPU time against wall time, and the CPU was far higher than the wall time. Two causes:
I measured on a 12-core machine and only the model scoring stages, so it won't match the 0.8s Blue measured. The relative improvement is what carries over. Commit 1:
|
| CPU time per search | Before | After |
|---|---|---|
Text model (cat / mount fuji / paris) |
10.4 / 16.4 / 23.5s | 4.3 / 4.0 / 6.2s |
| NudeNet, per 12 images | about 1.1-1.7s | about 0.3s |
| Query gate | 0.04-0.23s | 0.015s |
Wall time for the text model is no worse (1.8-5.8s before across my runs, 2.0-3.2s after). Wall time is noisy because it depends on the machine and what else is running, which is why I quote CPU. The saving is in CPU, not latency, which is what matters for server load.
What I tried and did not ship: int8 quantization. It was about another 2x faster and the model file is 4x smaller (268MB to 67MB). It changed verdicts, though. On the same 539 chunks, 18 moved category. Individual chunks swung by 0.5 or more in NSFW probability (0.96 to 0.08 on a "beaches" text, and 0.04 to 0.90 on "Featured pictures of Assisi"). For a safety filter I don't think we should accept that without a labelled evaluation, so it's still available as a later step.
Commit 2: c0bea96 cache the verdicts
At the moment we only cache whole result lists for 10 minutes. The same images appear in many different queries (red panda, red pandas, panda, giant panda), and each new query re-ran the text model and re-fetched and re-scored every thumbnail. Now each image's verdict is cached.
What the keys are:
- Text verdicts: a hash of the text. If someone edits a description or category on Commons, it's a new key and gets judged again.
- Pixel verdicts: the file's Commons SHA-1 plus thumbnail width. A file replaced after approval gets a fresh check even though its thumbnail URL is unchanged. That is stronger than the current URL-based behaviour.
- Both keys include a fingerprint of the model files plus the thresholds and chunking settings. Retraining a model or moving a threshold makes every old entry unreachable, so a cached verdict can't go stale. The
v1prefix in the key is for a code change that alters verdicts. - Only verdicts are stored, under hashed keys. No text or images.
What is deliberately not cached:
- An
UNCERTAINpixel verdict. It may only mean a thumbnail failed to download or decode this time, so caching it would turn a transient failure into a permanent exclusion. - Any pixel verdict for a file whose current version is under 24 hours old (
IMAGE_SEARCH_VERDICT_MIN_FILE_AGE_SECONDS). I got this from a review: Wikimedia can keep serving the old file's thumbnail for a while after a replacement. Without the gate, a SAFE for the old image could be filed under the new file's SHA-1 for 30 days. The gate means fresh files are simply re-checked each time. - Rate-limit or outage errors raise before anything is written, so nothing is stored when a batch fails part-way. This is tested.
Measured end to end (live Commons, real models, CPU per search):
| Search | CPU |
|---|---|
Cold, red panda |
4.2s |
Same images under another query string (red pandas, panda red) |
0.44s, 0.10s |
Overlapping topic (giant panda, then panda) |
3.5s, then 0.9s |
Cold, eiffel tower |
7.2s |
eiffel tower again once the 10-minute result cache had expired |
0.04s, no thumbnail fetches |
This also speeds up the web results top-image lookup, since it goes through the same checks.
Why Redis and not permanent SQL
- The verdicts are cheap to recompute. A lost entry costs one re-check, never a wrong answer.
- Production Redis is already a bounded 2GB
allkeys-lrucache, so it evicts the least-used entries without any extra bookkeeping. - SQL would need a migration, invalidation when the model changes (which the fingerprint gives us for free here), and it would make Commons-derived data permanent.
- Where SQL becomes the right answer: Daoud's idea of storing images locally and building up an index. At that point we want durable per-image records, and it's worth designing together with the index. This commit doesn't block that, and the key design (content hash plus model fingerprint) carries over.
The cache only helps repeat images. Cold queries for new topics still cost the full model time (the first column of the table above), so the load is bounded by how many new images people search for, not by search volume. Warm-up will be slow on a fresh deploy.
Trade-offs and risks to be aware of
- SHA-1 collision. An attacker who could craft a SHA-1 collision between a harmless and an explicit file, and get both onto Commons, could get the explicit one approved from the cache. That takes real resources, and I've documented it as an accepted risk in
verdict_cache.py. If you'd rather close it, the alternative is keying on a hash of the thumbnail bytes, but that means fetching every thumbnail and loses most of the saving. - Memory. Each image can add two entries (text and pixel), each a short hash key plus a one-word value. My 6 test searches created 183 entries. I haven't measured the per-entry size in Redis, so treat any total as unknown, but LRU eviction under the existing 2GB cap bounds it either way.
- Cache poisoning. The cache can only contain verdicts our own models produced for the exact content, under keys that include the model fingerprint. I couldn't find a way for a user to influence a verdict for content they don't control.
- It doesn't fix the cold-search cost. For lo 8000 gged-in-only image search or more compute (as Daoud and Blue discussed), these changes just mean we need less of both.
Not changed
- Verdict logic, thresholds and the fail-closed behaviour are unchanged.
- No API changes. The one new request field is
sha1|timestampon the Commons call, which I checked the live API returns. - Tests: 1907 passed in the full suite, and
make checkpasses. New tests cover batching and ordering, the fingerprints, cache hits and misses, replaced files, fresh files, mixed cached/uncached batches and partial failures. - Both commits touch the shared settings, so they carry
No functional changes to the crawler.
Next steps, if useful
- Set
IMAGE_SEARCH_TEXT_THREADSandIMAGE_SEARCH_PIXEL_THREADSagainst the real server's core count and the load we see. - Evaluate int8 quantization on the labelled queries. It's the biggest remaining CPU win.
- Blue's idea of replacing DistilBERT with a lighter text model would multiply with all of this, and the fingerprint means a model swap invalidates the cache cleanly.
Compares 15 image models and the production text model against hand-verified art nudes and Commons photographs, in the production ONNX configuration. The pixel layer misses 71% of art nudes and 27% of nude photographs; zero-shot SigLIP2 with art-aware prompts is the only candidate that fixes this, and the per-image DistilBERT check adds no measurable recall. The report ends with prioritised, actionable recommendations and the limits of the evidence. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Blue-Eye (DINOv3 ViT-L at 512 px) is precise but narrow: 0 of 238 verified-safe images excluded at its default and 74% of nude photographs caught, yet it labels 42 of 52 art nudes safe (its card counts non-sexual art as safe) and costs 7.5 s per image under ONNX Runtime on one thread. Reaching 95% art recall would exclude about half of safe images. It is rejected; the recommendations are unchanged. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Measured against 52 hand-verified nudes (docs/image-search-safety-benchmark.md), the per-image DistilBERT check caught none of the 22 that the deny rules miss, excluded 4-8% of safe images, and was the largest CPU cost of a cold search (about 5 s of about 6 s of model time). An image's text is now judged by the deny rules alone. The classifier stays for the query gate, which runs it on one short caption. Its verdict cache and fingerprint, which only served the per-image check, go with it. Recall is unchanged; safe images excluded fall from about 8% to 4%. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Against 52 hand-verified art nudes NudeNet catches 29% and Falconsai 2%: dedicated NSFW classifiers are trained on porn and hentai, so a statue or a painting is outside what they learned. SigLIP2 scored against fixed prompts that name the art forms catches 96% of the same nudes, and 94% of 331 nude photographs with thresholds fitted on the art alone (see docs/image-search-safety-benchmark.md for method and limits). The layer is off by default (IMAGE_SEARCH_ZERO_SHOT = "off"); "enforce" makes an image need SAFE from every pixel layer. The threshold is the 99% art-recall point, 0.386, fitted down to 0.38. - ZeroShotSafetyClassifier runs the exported image tower under onnxruntime and compares the embedding with precomputed prompt embeddings, so the server needs no tokenizer or torch. Checked on 80 real thumbnails against the Hugging Face reference: max probability difference 0.015, no decision flips at 0.38. - scripts/export_image_safety_models.py exports the fp32 image tower (int8 cut recall from 94% to 75%), the prompt embeddings and the temperature. The Docker export stage runs it, so the image grows by about 370 MB. - Each pixel layer's verdicts are cached on their own, keyed by file and by that layer's fingerprint, so turning a layer on or off or retuning it leaves the other's cached verdicts valid, and the thumbnail is fetched once for all layers. NudeNet's cache keys are unchanged. Known limit, recorded in the report: blank, flat or abstract images score high (solid black 0.86) because none of the safe prompts describes them. A prompt for them is tuning against the evaluation and needs a held-out re-test. No functional changes to the crawler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
IMAGE_SEARCH_ZERO_SHOT now defaults to "enforce", so an image must be called
SAFE by both NudeNet and the SigLIP2 zero-shot layer. Tests keep it "off"
(the model is not built in CI) and switch it on with a fake model where they
exercise it. "off" restores the previous behaviour.
Needs the exported zero_shot model: the Docker export stage builds it, and a
local IMAGE_SEARCH_MODEL_DIR that predates it needs
scripts/export_image_safety_models.py re-run, or image search fails loudly
rather than returning unchecked images.
Live run with the real models against Commons: "greek statue" returns 8 images
instead of 23, ordinary queries lose 2-3 of 30 ("red panda" 30 -> 28, "eiffel
tower" 30 -> 27). A cold search costs about 12 s of CPU instead of 1 s at one
thread (cached verdicts make repeat images free); the text classifier's
removal in the earlier commit already cut about 5 s from the previous total.
No functional changes to the crawler.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
Author
Image search safety: what changed since the load discussion, and what to reviewYou were worried about server load. Benchmarking the models for that turned up a bigger problem: the pixel layer misses most art nudity and a quarter of nude photographs. I pushed 5 commits. The full report is docs/image-search-safety-benchmark.md (about 80 lines, with prioritised recommendations). The evidence, in four numbersMeasured on 52 hand-verified art nudes and 331 Commons nude photographs, 16 image models, same images for all (Blue-Eye on a 550-image subset):
Pre-built NSFW classifiers are trained on porn vs. everyday photos, so statues and paintings are out-of-distribution. The per-image DistilBERT text check caught 0 of the 22 nudes that the keyword rules miss. Commits
What you're signing up for
Where to look
Limits and open questions
|
Collaborator
Author
Performance after the SigLIP2 layer, and the case for scoring offlineWhere the load stands. Model CPU for a cold search of about 28 images (one thread per session, measured):
So the safety layer costs about 2x what we had just before it, and roughly the same as where we started. Wall time is similar (SigLIP2 over 4 thumbnail workers replaces DistilBERT's 2–3 s). Per image it is about 0.37 s of CPU (NudeNet 30 ms + SigLIP2 340 ms). Repeat images are free through the 30-day verdict cache, but we haven't measured the production hit rate, and that decides the real steady-state cost. Why scoring should move offline. The cost is per image, not per search, and the same images recur across queries. The cheapest step is not a bulk crawl: keep scoring images as searches surface them (Daoud's incremental idea), but make the result permanent.
A real bulk index of Commons is a different decision. SigLIP2 alone is about 94 CPU-hours per million images, and scoring ~15M files means fetching ~15M thumbnails from Wikimedia (a rough estimate of hundreds of GB). That should be coordinated with them, and I'd only do it on a GPU. Cheaper wins that don't need new infrastructure (not implemented):
Open decisions for review: what to do with an image that has no stored verdict yet (score inline, as now, or exclude until scored), and whether to move to Postgres now or after the hit-rate numbers. |
Collaborator
Author
|
Closing as discussed. Lots of good learning lessons. We're going to go with Jev instead, because it is quite performant and quite good at filtering. |
This was referenced Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Live image search over Wikimedia Commons, with a fail-closed safety pipeline.
GET /api/v2/image-search/?q=&limit=returns approved images only.GET /api/v2/image-search/top/?q=returns one approved image for a strong title match, for showing beside web results, ornull.Nothing from Commons is stored. Approved results sit in the cache for 10 minutes, and rejected metadata is never kept. Thumbnails are loaded by the browser straight from Wikimedia; Mwmbl never proxies or stores them.
Safety layers
An image is returned only if every layer says SAFE. Anything uncertain, missing or failing is excluded.
eliasalbouzidi/distilbert-nsfw-text-classifier(Apache-2.0), exported to ONNX and run in-process with the already-presentonnxruntimeandtokenizers. Long texts are judged chunk by chunk; texts too long to check are uncertain.upload.wikimedia.org/thumb.wikimedia.org, with no redirects and a size cap.A Commons outage or rate limit is a 503, never an empty or unchecked list. It is not cached.
Wikimedia etiquette
Identifying User-Agent, bounded concurrency, and
429/503+Retry-Afterhonoured through a shared cooldown in the cache. Ranking uses the profile and filters MediaSearch uses (filetype:bitmap -fileres:0,classic_noboostlinks) and falls back to plain file search. CirrusSearch syntax in the user's query is stripped.Deployment: models are required
The models are not in the repo or the Docker image. Build them once with
scripts/export_image_safety_models.py(needs torch; the server does not) and pointIMAGE_SEARCH_MODEL_DIRat the output (defaultdevdata/image_safety). Without them the endpoints fail loudly with a 500; they never return images.Limits, stated plainly
Verification
make checkpasses;make test: 1847 passed (the one failure,test_openapi's exact v2 path list, is updated here).test/test_image_search.py(74 tests) covers each layer vetoing, fail-closed paths (thumbnail fetch failure, outage, rate limit, not-cached), URL/host validation, markup injection, rate-limit cooldown, and chunked text classification. The real-model tests run when the models are built.red pandareturns images;nude womanreturns none; sunbathing photos are removed by rules and by pixels; Michelangelo's David is removed by pixels alone.Reviewed by a code-review, security and trust-and-safety pass; their findings are fixed in the second commit.
Front end: mwmbl/front-end (see the linked PR there).