Re-measure image safety, gate and minimum with Commons captions by daoudclarke · Pull Request #630 · mwmbl/mwmbl

daoudclarke · 2026-11-11T06:49:17Z

From Claude:

Stack: builds on #627 (seed-images-eval-harden), and merges #628 (seed-images-prod-pipeline) into it, so the diff against the base also shows #628's production changes. This PR's own commits add 456 lines.

What changed

#628 shows the judge each file's Commons caption (a Caption: line, capped at 250 characters). The thresholds (NSFW < 1.0, need >= 1.65, relevance >= 1.5) were measured without captions. This PR checks them again with captions.

Merge of Seed Search images: the request flow (not wired in) #628. The conflict resolution keeps the eval branch's ApprovedImage, takes production's SourceImage.text with the caption, and keeps both sides' image settings. collect.image_from_json loads candidate files collected before captions as caption "" and revised_at = uploaded_at.
mwmbl/rankeval/seed_images/captions.py:
- fetch: captions by title (prop=entityterms, English label, 50 titles per request, one request at a time, image search's User-Agent and back-off). It fetches 6,530 files: the benchmark plus the real and labelled queries' candidates. 37% have a caption.
- benchmark and wording: the safety benchmark in separate mode with the same batches, with captions and with the NSFW question reworded to name the caption.
- real: the 200 real queries' candidates.
- probes: injection.py's self and cross probes, with the payload as the caption.
- gate: the need and relevance request on the 150 labelled queries.
- report: prints the results.
gate_analysis --arm captions.
A "Captions" section in mwmbl/rankeval/seed-search-images-eval.md.

Results: the thresholds hold

Check	Hardened (no captions)	With captions
Benchmark art / photos / safe flagged at 1.0	96.2% / 98.5% / 4.1%	96.2% / 98.5% / 4.1%
AUC art / rule-evading art / photos	0.986 / 0.966 / 0.995	0.985 / 0.965 / 0.994
Real candidates passing (3,397 images)	95.6%	95.8%
Gated-in queries, k=4, min 1.5: images / good / wrong	2.34 / 91.8% / 0.9%	2.32 / 90.8% / 0.9%
Need on 150 labelled queries	—	shift +0.006, r 0.999, 3 cross 1.65

No nude benchmark image changes verdict.
Caption injection had no effect. None of the "ignore instructions", "volunteers checked" or fake-closing-tag captions passed any of the 383 nude images. One image's caption doesn't move the others in its batch.
8000 Rewording the NSFW question to "file name, caption, description and categories" gives the same benchmark table.

Production follow-up (not changed here)

NSFW_QUESTION and IMAGE_RELEVANCE_QUESTION in mwmbl/imagesearch/prompt.py still say "file name, description and categories". They should say "file name, caption, description and categories". The NSFW rewording was measured and changes nothing. The relevance rewording was not measured. Changing the wording changes nsfw_fingerprint, so the cached verdicts are re-asked.

Verification

make check passes, and so does make test (database mwmbl_b, redis db 14).
New tests cover caption parsing, including title normalisation and missing files; the request parameters; the reworded question; captions reaching the judge's text; and loading old candidate files.
One subagent did a combined code, safety and holistic review. Fixed after it:
- The probes had put the payload in both the description and the caption, because probe_batches() had already injected the description. They now use the caption only, and were re-run.
- The write-up now says the noise baseline is confounded and that label bias is a hypothesis.
- The write-up now states the untested attacks: a bland or false caption on an explicit file, and non-English captions. The 7-day minimum revision age is the mitigation.

Part of #610

🤖 Generated with Claude Code

…ommons mwmbl/imagesearch/seed_images.py runs the #610 request flow in two phases for Combined Search to overlap with Staan and with web ranking: start() gates the query, searches Commons and asks the NSFW question; finish() asks image need and relevance in one request once Staan has answered, then selects. It fails closed to `unavailable`, including when the images are not ready shortly after web ranking finishes. Not wired into an endpoint. The Commons request now also returns each file's English caption (MediaInfo, via prop=entityterms) and its latest revision time, and asks for English metadata explicitly. Files edited in the last week, and files on an operator denylist, are never judged or shown. The need and relevance answers are range-checked like the NSFW ones, web results are capped and kept to one line each, and image requests have their own timeout. No functional changes to the crawler. Part of #610 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…from review Image work gets its own thread pool so threads blocked on Commons or Redis after a missed deadline cannot hold up web search; the selection is shielded so a deadline doesn't cancel the shared phase-1 work. Files in deletion categories, and files with a recent file version, are never judged; the denylist is matched case-insensitively; captions are capped. No functional changes to the crawler. Part of #610 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Brings in #628's Commons captions and revision times, so the eval can measure the questions with captions. Conflicts: types.py keeps the eval branch's ApprovedImage and takes production's caption in SourceImage.text; settings keep both sides' image settings. No functional changes to the crawler. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…n fields The cached candidates were collected before #628 fetched captions and revision times, so their images have no caption, as when they were judged. No functional changes to the crawler. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

#628 shows the judge each file's caption. captions.py fetches the captions of the benchmark and the eval's candidates by title, then re-asks the safety benchmark, the real candidates, injection probes with the payload as the caption, and the need and relevance request on the labelled queries. The thresholds (NSFW 1.0, need 1.65, relevance 1.5) hold; the eval doc gains a Captions section. No functional changes to the crawler. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…'s gaps The review found probe_batches() had already put each payload at the head of the description. The probes now start from the benchmark's own images and were re-run. No functional changes to the crawler. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

codecov · 2026-11-11T06:54:10Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Commons captions are now part of each image's text (#628), so the questions say so. The caption eval (#630) measured the NSFW rewording on the safety benchmark: an identical table, and the 1.0 threshold holds. The relevance rewording was not measured. The NSFW question's fingerprint changes, so cached verdicts are asked again. No functional changes to the crawler. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The safety benchmark and injection probes held with captions (#630), so the kill switch no longer needs to default to off. Only requests sending images=true search images, and an instance without JEV_API_KEY still answers unavailable without calling Commons or the judge. No functional changes to the crawler. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

daoudclarke and others added 6 commits November 10, 2026 17:29

daoudclarke mentioned this pull request Nov 11, 2026

Show Seed Search's images on Combined Search with images=true #631

Merged

Navigation Menu

Repository navigation

Re-measure image safety, gate and minimum with Commons captions - #630

Re-measure image safety, gate and minimum with Commons captions#630
daoudclarke wants to merge 6 commits into
seed-images-eval-hardenfrom
seed-images-eval-captions

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

What changed

Results: the thresholds hold

Production follow-up (not changed here)

Verification

Uh oh!

Codecov Report

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant