Repository navigation
Web UI and deploy scripts: the Eval page reads /runs and shows status, error kind, metrics and withheld counts; the Fly smoke test PUTs runs - #43
Merged
Conversation
…, error kind, metrics and withheld counts; the Fly smoke test PUTs runs Stage 8 of #24. The UI stops reading runs from the Eval body, which no longer carries them, and reads GET /evals/{ns}/{name}/runs one page at a time: columns Run, Status, Error, Started, Ended, one per metric id present on the page, and one per judgement when a Card is chosen from the comparison view (cards=), with a line per Card giving runs_used and changed_since_card. The "runs in this version" caption goes, since runs belong to the record and the page reloads them on ns / name and not on ?v=; the header shows the envelope's runs summary. The query builder reads /schemas/eval-2 for Eval paths because /schemas/eval stays the 1.0 document this release. withheld, in the GET and query envelopes since 0.1.4 and counting run_results since stage 7, is now rendered on the Card and Eval pages and as a tag on query results. client.ts gains the run calls; schema.d.ts is unchanged (regenerated identical to the snapshot). deploy/fly/smoke.sh posts the 2.0 header, PUTs r1 from fixtures/run.json (with the uploaded calls / diff attachments on the run, where run.json declares them) and an r2 derived from it with status error, reads GET /runs, posts the Card from card-run-results.json against the seq the POST returned (a missing seq now stops the script instead of citing @1) and reads GET /runs?cards= for the judgements; the anonymous checks cover the new route and the Eval deep link. No credential reaches stdout or stderr. e2e/smoke.sh posts no Eval and is unchanged. One Playwright test seeds an Eval, two runs and a Card through the API and checks the page paints the run rows, the metric column and, after choosing the Card, the judgement column. Verified: just web-check (svelte-check 196 files, 0 errors; prettier clean); just e2e (release build, Docker Postgres, e2e/smoke.sh 18 checks passed, Playwright 6 passed incl. runs.spec.ts); the client regenerated from the OpenAPI snapshot and diffed identical; bash -n on both scripts; deploy/fly/smoke.sh reviewed by reading only, not run (it targets the hosted service); shellcheck not installed here. Refs #34 Signed-off-by: Yutaka Nishimura <ytk.nishimura@gmail.com>
This was referenced Sep 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage 8 of 9 of #24 (Runs as first-class rows), on top of stages 1–7 (#36–#42).
web/,deploy/fly/smoke.sh, a Playwright test. No Rust change. Mergeable on its own; the version bump is stage 9 (#35).What changed
web/src/lib/api/client.ts):listRuns,getRun,putRun,putRuns(batch),archiveRun(PATCH),deleteRun, with the DTO types.schema.d.tsis unchanged: regenerating it from the OpenAPI snapshot gives an identical file (checked twice, by the implementer and in review).web/src/routes/evals/[ns]/[name]/+page.svelte): the Runs table readsGET …/runsone page at a time (limit=50, cursor stack for Previous / Next asRecordListdoes), keyed on ns / name and not on?v=since runs belong to the record; columns Run, Status, Error (kind), Started, Ended, one per metric id present on the page (sorted union), and, when a Card is chosen from the comparison view's distinct Cards (#judge-card, one at a time →cards=), one judgement column per (Card, metric) with "not used" and "changed" marks and a line per Card withruns_usedandchanged_since_card. Non-live rows show a state tag. The "runs in this version" caption and the Outcome / Calls / Artifacts columns are gone; the header shows the envelope'srunssummary ("N runs: ok, error, skipped", plus archived / deleted when present). A 404 from/runsrenders as an error, not a crash. Norecord.runsoroutcomeremains inweb/src.Withheld.svelte(new): renders thewithheldcounts (relations,run_results) as arole="note", nothing at 0 / 0; used on the Card and Eval pages. Query result rows show a compact tag (RecordList.svelte); plain listings carry nowithheld.RecordList/QueryBuilderread/schemas/eval-2for Eval paths (/schemas/evalstays the 1.0 document this release).lib/paths.tsARRAY_PATHSunchanged; record queries do not gainruns.deploy/fly/smoke.sh: header POST (2.0,attachments = []) → PUTr1fromfixtures/run.jsonwith the uploadedcalls.jsonl/diff.patchon the run (hashes and sizes rewritten byjq, paths unchanged) → PUTr2derived from it (status: error,error.kind: timeout, no attachments) →GET …/runsasserts r1okand r2error/timeout→ Eval GET assertsruns.count >= 2→ Card fromcard-run-results.jsonwithrelations[0].toset to the seq the POST returned (a missing seq now stops the script instead of citing@1) andrun_results[].evalrewritten →GET …/runs?cards=assertsruns_used == 2, r1pass, r2fail→ after publishing, anonymous…/runs?cards=and the Eval deep link are 200. Every POST / PUT accepts 200 so re-runs pass. Token handling is unchanged (header file in a mode-700 temp dir, never on the command line or stdout); the new echo lines print status codes and the first 300 bytes of envelopes that carry no credential. Reviewed by reading only; not run.e2e/smoke.sh: unchanged (posts no Eval, reads noruns8000 ).web/tests/browser/runs.spec.ts): seeds an Eval, two runs and a Card through the API withEVALHUB_E2E_TOKEN(skips without it, asorgs.spec.ts), waits on the/runsresponse, and asserts the header text, thecore/tokens_outcolumn, r1 / r2 cells, then after choosing the Card thecore/passcolumn and cells, with no alert.Verified
corepack pnpm install --frozen-lockfilepnpm-lock.yamluntouchedjust web-checkschema.d.tsjust e2e(release build, Docker Postgres, Playwright)e2e: 18 passed, 0 failed; Playwright 6 passed incl.runs.spec.ts(run by the implementer and again in review)bash -n deploy/fly/smoke.sh e2e/smoke.shNot run:
deploy/fly/smoke.sh(targets the hosted service),shellcheck(not installed on this host), Rust crate tests (no crate change).Notes
card-run-results.json'sattrs.runsnamesr1andr2and the store requires each to exist.r1, whererun.jsondeclares them. The Card keeps itssamples.jsonlattachment, so the download checks still apply./runsrequests; a failed Next page hides the pager until the Card or page changes; after a?v=change the chosen Card may not be among the new options while the table stays joined with it.Refs #34