8000 Relations: a reader or writer learns nothing of a version it may not see (0.1.4) by ynishi · Pull Request #26 · ynishi/evalhub · GitHub
[go: up one dir, main page]

Skip to content

Relations: a reader or writer learns nothing of a version it may not see (0.1.4) - #26

Merged
ynishi merged 2 commits into
mainfrom
fix/card-private-leak
Sep 25, 2026
Merged

ynishi merged 2 commits into
mainfrom
fix/card-private-leak

Conversation

@ynishi
@ynishi ynishi commented Sep 25, 2026 •
Copy link
Copy Markdown
Owner

For #25. Releases 0.1.4.

evalhub_schema::eval promises that when a public Card points at a private Eval, a reader without access learns that the Eval exists, its version_id and its content_hash, and nothing else. The hub kept that promise only in GET …/relations. Three other paths gave the Eval away:

  1. Reads returned the Card body verbatim. GET /cards/… and POST /cards/query returned relations[].to, which is the private ns/name@seq as text.
  2. Query conditions matched every edge. A relations.to condition (exact or prefix) was evaluated against targets the caller could not see, so a caller could confirm a private name by whether a public Card was hit.
  3. Writes resolved targets regardless of visibility. Posting a Card or POST …/relations that named someone else's private version resolved it, so the response and the refs_resolved badge told the writer that the version existed.

Export was not a path: hf-model-index is rendered from results[] and carries no relation.

What changes

  • Reads withhold what the reader may not see. Every read that returns a body removes the relations[] elements whose target the reader may not see, and lists them in the envelope as the commitment:

    "withheld": { "relations": [ { "type": "core/uses_eval", "version_id": "…", "content_hash": "…" } ] }

    A body with elements withheld is not the stored body, so its content_hash does not match it. Verify the hash only on responses that have no withheld. A member of the Eval's namespace gets the body unchanged and no withheld.

  • Query conditions see only visible targets. A relations.to condition matches only edges whose target the caller may see. Negated, the hidden edge is just as absent.

  • A write learns no more than a read. At ingest, in POST …/relations and in the deferred resolver, a target the writer may not see is handled exactly like one that does not exist: it is stored unresolved, the response is the same and no refs_resolved badge is given. Once the target is visible to a reader, the edge resolves for them. The writer's own namespace and their organisations' namespaces are visible, so legitimate private references resolve as before.

  • One visibility predicate (evalhub_store::relations::visible_to) serves the relation DTO, body redaction and resolution.

Contract

  • withheld is new and optional on the version envelope and on query hits. The OpenAPI snapshot and the web/ client are regenerated.
  • The SDK crates' public API is unchanged: just semver-check reports no semver update required for evalhub-schema, -core and -query. The schema change is limited 8000 to the crate doc (eval § "Privacy across the relation").
  • .sqlx is regenerated for the changed queries.

Verification

  • crates/evalhub-server/tests/privacy.rs: five HTTP tests covering the body on GET, the body on query, the relations.to condition, the badge on POST, and POST …/relations. All five were first run against main and failed on the defect itself: alice/secret@1 appeared in the body, bob's condition hit, the badge differed from that of a nonexistent target, and the relation response revealed the target.
  • Store tests: relations_resolve_only_to_versions_the_writer_may_see, resolution_never_reveals_a_private_version and relation_filters_see_only_targets_the_caller_may_see were checked against this commit with the three visibility checks disabled, and each failed on the leak.
  • cargo fmt --check; cargo clippy --workspace --all-targets -- -D warnings; cargo test -p evalhub-schema / -core / -query / -store / -server (Docker); just sqlx-check; pnpm check; just e2e (18 passed); just package + package-gate; just semver-check.

Test image: MinIO from Chainguard

The first CI run of this PR failed in evalhub-store tests/objects.rs before any test code ran: quay.io now answers 401 to anonymous pulls of quay.io/minio/minio (and minio/mc), which passed on main the day before. The store and server test harnesses and compose.yml now use cgr.dev/chainguard/minio, pinned by digest (it publishes latest only). It ships mc and a shell, so the bucket setup is unchanged. The tests mount a tmpfs on /data and wait for the API: http banner, because on the overlay root MinIO first logs a rename error that also starts with API:.

…see (0.1.4)

A public Card citing a private Eval gave the Eval away three ways. GET
and POST .../query returned the Card body verbatim, so relations[].to
carried the private ns/name@seq. A relations.to condition in a query
matched every stored edge, so a caller could confirm a private name by
whether a public Card was hit. And writes resolved targets regardless of
visibility, so the response and the refs_resolved badge told the writer
whether a private version existed.

Reads now remove the relations[] elements whose target the reader may
not see and list them under withheld.relations as the commitment
(type, version_id, content_hash). Query conditions match only visible
targets. Ingest, POST .../relations and the deferred resolver treat an
invisible target exactly as a nonexistent one: stored unresolved, same
response, same badge; it resolves once the target becomes visible.

The crate docs (schema eval, store records/relations, server API) state
the rule. OpenAPI snapshot, web/ client and .sqlx are regenerated.

Verified: cargo fmt --check; cargo clippy --workspace --all-targets -D
warnings; cargo test -p evalhub-schema / -store / -server (Docker);
just sqlx-check; pnpm check; just e2e.

Refs #25

Signed-off-by: Yutaka Nishimura <ytk.nishimura@gmail.com>
CI for ba247f0 failed in evalhub-store tests/objects.rs before any test
code ran: pulling quay.io/minio/minio:RELEASE.2025-02-28T09-55-16Z
answered 401 unauthorized. The same image passed CI on 0c96b4a the day
before; quay.io now refuses anonymous manifest reads for minio/minio and
minio/mc, and Docker Hub's minio/minio was already unpullable.

cgr.dev/chainguard/minio is pullable anonymously and ships mc and a
shell, which the bucket setup needs. It publishes only `latest`, so it
is pinned by digest. /data is not a volume in that image: on the overlay
root MinIO logs a rename error whose first line is "API: ...", which
the old wait matched before the server was up. The tests mount a tmpfs
on /data and wait for the "API: http" banner instead. compose.yml uses
the same image for minio and, for its bundled mc, for minio-init.

Verified: cargo fmt --check; cargo clippy --workspace --all-targets -D
warnings; cargo test -p evalhub-store and -p evalhub-server (Docker).

Signed-off-by: Yutaka Nishimura <ytk.nishimura@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0