8000 fix(es-compat): fan _delete_by_query/_update_by_query out to every alias member (#450) by xerj-org · Pull Request #552 · xerj-org/xerj · GitHub
[go: up one dir, main page]

Skip to content

fix(es-compat): fan _delete_by_query/_update_by_query out to every alias member (#450) - #552

Merged
xerj-org merged 2 commits into
mainfrom
fix/450-by-query-alias-fanout
Aug 21, 2026
Merged

xerj-org merged 2 commits into
mainfrom
fix/450-by-query-alias-fanout

Conversation

@xerj-org
Copy link
Copy Markdown
Owner

Closes #450.

The bug (silent partial delete — data integrity)

_delete_by_query and _update_by_query both began with Engine::get_index(&index), which resolves an alias to aliased.first() — one member. A destructive by-query through a multi-index alias therefore touched only the first member and answered a clean {"deleted"/"updated": N, "failures": []}, indistinguishable from a complete run. Strictly worse than the #433/#449 read truncation: a short read is recoverable, a short delete leaves the caller believing the operation completed.

Reproduced: 3-member alias tri (12 docs), POST /tri/_delete_by_query {match_all} returned {"total":4,"deleted":4,"failures":[]} — idx-b/idx-c untouched.

The fix

resolve_by_query_targets() resolves the selector the way the read paths do (resolve_index_selector, #433): expand to every member, and — crucially for a destructive path — AND in any alias filter exactly as search_impl does, so a filtered alias deletes only its slice, not the whole backing index (the #524 hazard). Each member runs the by-query; aggregate_by_query_results() sums the counts and concatenates failures. A single member (plain index or one-member alias) is returned byte-for-byte unchanged, so the existing wire format never drifts — only genuine multi-member aliases aggregate. If any member errors, that error is surfaced rather than a clean aggregate.

Verification (rustc 1.98.0 — CI @stable)

…ias touches only the first member

Failing repro (WIP): a 3-member alias 'tri' with 12 docs; POST /tri/_delete_by_query
match_all returns {"total":4,"deleted":4,"failures":[]} — a silent partial success
that deletes only idx-a's share and leaves idx-b/idx-c intact. Both by-query write
handlers resolve the alias via Engine::get_index -> aliased.first(). Fix (fan out to
every member + apply alias filter) lands next.
…ias member (#450)

Both by-query write handlers began with Engine::get_index, which resolves an
alias to aliased.first() — one member. A destructive by-query through a
multi-index alias therefore touched only the first member and answered a clean
{deleted/updated: N} with failures:[], indistinguishable from a complete run: a
silent partial delete the caller cannot detect (strictly worse than the #433/#449
read truncation — a short delete is unrecoverable).

Resolve the selector the way the read paths do (resolve_index_selector, #433):
expand to every member, AND in any alias filter exactly as search_impl does so a
FILTERED alias deletes only its slice and not the whole backing index (the #524
hazard), then run the by-query on each member and aggregate. A single member (a
plain index or one-member alias) is returned byte-for-byte unchanged, so the
existing wire format never drifts; only genuine multi-member aliases aggregate.

Adds by_query_through_alias_touches_every_member.rs covering both verbs.
@cla-bot cla-bot Bot added the cla-signed label Aug 21, 2026
@xerj-org
Copy link
Copy Markdown
Owner Author

Independent 3-skeptic verification (each REFUTES, own worktree, rustc 1.98.0)

Verdict: 0 BLOCKING. correctness=nit, regression-risk=should-fix, evidence-holds=should-fix — all refuted=false.

correctness (nit): Fan-out + aggregation correct. aggregate_by_query_results sums took/total/deleted|updated/batches/version_conflicts/noops and concatenates failures correctly; error-detection via find(v.get("status").is_some()) is sound (success bodies never carry status; every error path does). Single-member path is byte-identical (len()==1 early-returns raw body; effective_query==user_query with no alias filter). No double-run (resolve_index_selector dedupes). Nit: multi-member _update_by_query aggregate omits the "deleted": 0 field the single-member update body includes (24887) — cosmetic asymmetry, not a wrong count.

regression-risk (should-fix): Full xerj-api suite green — 465 passed, 0 failed across 40 binaries; clippy -D warnings clean. Single-member wire format never drifts; a nonexistent exact index still 404s; a filtered alias is narrowed via bool{must:[query],filter:[alias_filter]} — can only under-delete, never over-delete. should-fix GAP: neither test exercises a filtered alias (0 filter occurrences), so the #524 filtered-slice path is unverified by execution. Minor note: a wildcard spec matching zero indices now returns 200 {deleted:0} instead of 404 (aligns with read-path semantics; not data-loss).

evidence-holds (should-fix): Core #450 claim holds. Fail-before real: get_index (engine.rs:3193-3205) → aliased.first(), so pre-fix delete drained only member-0 (4/12) and reported clean success. Fix expands to all members and runs the same query against each; both tests pass. The riskiest claim — filtered alias deletes only its slice (#524) — is CORRECT by inspection: filters gathered keyed on the written name (24387-24397), mirroring search_impl, ANDed as bool.filter into one effective_query applied to EVERY member. Over-delete is structurally impossible (a filter only narrows). should-fix: that path is verified only by reading, not execution — recommend a filtered-alias delete test.

Gate: (a) 17/17 green, none pending; (b) 0 BLOCKING (should-fix/nit tracked as fast-follow); (c) CLEAN. Merging. Follow-up filed for the filtered-alias test + update-aggregate nit.

@xerj-org
xerj-org merged commit f0ad444 into main Aug 21, 2026
18 checks passed
buger pushed a commit to probelabs/xerj that referenced this pull request Aug 22, 2026
Newest tag is now v1.0.0-rc.24 (includes xerj-org#552 fix xerj-org#450 data-loss + xerj-org#549 knn invariant tests).
Footer XERJ.AI stamps and llms*.txt 'Current release' bumped rc.23 -> rc.24 across landing/;
llms release date -> 2026-08-21. landing-constants-guard passes with VERSION=1.0.0-rc.24.
buger pushed a commit to probelabs/xerj that referenced this pull request Aug 22, 2026
…_by_query aggregate deleted:0 (xerj-org#553)

Fast-follow to xerj-org#450/xerj-org#552. The 3-skeptic verification confirmed the filtered-alias
over-delete safety by code inspection but flagged it was untested. This adds the
execution test: a match_all _delete_by_query through a filtered 2-member alias deletes
only the 2 in-slice (active) docs and leaves the 2 out-of-slice (archived) docs — the
xerj-org#524 hazard. Non-vacuous: without the filter, match_all would delete all 4 and empty
both members.

Also fixes the flagged nit: aggregate_by_query_results now includes deleted:0 for
_update_by_query, matching the single-member update body (2+-member updates dropped it).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_delete_by_query and _update_by_query through a multi-index alias silently touch only the first member and report success

1 participant

0