Repository navigation
Seed Search images eval: measure the hardened image questions - #627
Open
daoudclarke wants to merge 3 commits into
Open
daoudclarke wants to merge 3 commits into
daoudclarke wants to merge 3 commits into
Conversation
Commons text is now delimited, with angle brackets, invisible characters and the tag name taken out of it, and categories are capped like the description. Every image question says its Commons text and web results are data, not instructions, and the NSFW question treats text that addresses the judge as a warning sign. The relevance question sees Staan's top two results, and image_request asks need and relevance in the one request Seed Search will send once Staan answers. The thresholds measured with these questions (NSFW below 1.0, need at least 1.65, relevance at least 1.5) are constants next to them in prompt.py, for the pipeline to use. No functional changes to the crawler. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The evaluation of the hardened questions imports them from mwmbl.imagesearch. No functional changes to the crawler. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
run_hardened asks the hardened need and relevance questions in production's one request; injection re-runs the safety benchmark and probes descriptions that address the judge, in their own image and in another image of the same batch; gate_analysis --arm scores either set of answers. The 1.0 NSFW threshold and 1.5 minimum hold, the gate moves to 1.65, no injection passes a nude benchmark image, and Jev keeps a batch's questions independent. The eval doc gains a "Hardened prompts" section. No functional changes to the crawler. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
From Claude:
Stack: against
seed-images-eval-use-prod-prompts(#625), on the evaluation branchwikicommons_image_search, which is never merged. It merges the production half, #622 (seed-images-prod-harden), and then adds the evaluation that measured it. This is the eval half of the original #622.What changed
seed-images-prod-harden(Harden the Seed Search image questions and re-tune their thresholds #622): the hardened questions,judge.image_requestand the threshold constants inprompt.py.run_hardened.pyasks the production-shaped request for every query.injection.pycovers the benchmark in separate mode, the injection probes and real candidates.gate_analysis --arm first|hardened|top1scores each set of answers.run_relevance.pyis removed: its questions no longer exist, and the first answers stay in devdata.seed-search-images-eval.md, with Summary and residual-risk updates.Results
Safety benchmark (same 68 batches, separate mode, at 1.0):
The 1.0 threshold stays: at 1.5 only 90.4% of art is caught. On real candidates the pass rate goes from 96.0% to 95.6%.
Injection probes: 383 nude benchmark images with judge-addressed text, and cross-image batches of 20.
Gate and relevance (train reference labels):
Holistic re-run: not needed before PR 3, because the settings barely moved. Re-judge on production's real request.
PR 1's notes:
How it was verified
make checkpasses, and so do the seed_images eval tests. Full suite below.devdata/seed_images.Subagent reviews: see #622, which covered both halves when they were one PR.
Part of #610
🤖 Generated with Claude Code