i build ai agents and the tools that check whether they actually did what they said.
- agent-run-reliability · reruns 24 agent tasks eight times on two models to see how many runs a ranking needs. with one run per task, two evaluations pick different winners 27.5% of the time.
- code-freeze-bench · rebuilds the 2025 incident where an agent wiped a production database during a code freeze, on a real postgres server, to find out which guardrails actually stop it.
- receipts · coding agents often say "tests pass" or "fixed it" without having checked. this reads an agent's session and flags the claims its own tool calls don't back up.
- benchmeasure · treats ai benchmarks as measurement instruments and asks how much a leaderboard ranking can be trusted. near the top of swe-bench, not much.
- emotion-label-ceiling · grades the same speech-emotion model against the emotion the actor was asked to play and against the one listeners heard. the two scores are 25.6 points apart.
- laneproof · runs many coding agents in parallel and only counts work that lands on main with its evidence re-run, so no agent's "done" is taken on trust.
also minecraft-sre (an agent on call for a minecraft server, graded from the world itself rather than its report), index-cards (a memory layer for coding agents over a folder of markdown notes), sayso (your own agent can answer your claude code sessions, and anything risky waits for a yes signed on your phone), because (preregistered experiments on what makes claude code follow its instructions), mcpify (any scriptable mac app as an mcp server), hill-climb (a skill for improving code in measured rounds) and perch (everything running on your mac, in the menu bar). write-ups and papers at abhaymettu.com.
economics at uw-madison, cs and data science minors, may 2026 · previously swe intern at martin (yc s23) · linkedin