8000 fix(autoindex): remove one folder safely and refuse wildcard deletes (#2114) by xerj-org · Pull Request #2126 · xerj-org/xerj · GitHub
[go: up one dir, main page]

Skip to content {"props":{"docsUrl":"https://docs.github.com/get-started/accessibility/keyboard-shortcuts"}}

fix(autoindex): remove one folder safely and refuse wildcard deletes (#2114) - #2126

Merged
xerj-org merged 3 commits into
mainfrom
cursor/autoindex-safe-remove-da05
Oct 22, 2026
Merged

xerj-org merged 3 commits into
mainfrom
cursor/autoindex-safe-remove-da05

Conversation

@xerj-org
Copy link
Copy Markdown
Owner

What this changes, and why

A coding agent deleted a live ax-docs index after xerj autoindex refused a second folder under the default prefix ax (#2114, with #2115 and #2116). Three gaps made that easy:

  • The prefix-reuse refusal did not lead with a distinct --prefix, so "this prefix is taken" read as a cleanup task.
  • There was no command that listed one autoindexed folder and deleted only that folder. The path was a raw DELETE plus _delete_by_query on autoindex-catalog, with no preview and no confirmation.
  • DELETE of *, _all, or any wildcard was accepted, while closing an index by wildcard was already refused. ES 8 refuses those deletes by default (action.destructive_requires_name=true).

xerj autoindex remove --prefix P [--root DIR] reads autoindex-catalog first and prints the roots, indices, catalog rows, doc counts and store sizes. It deletes nothing unless a terminal answers y or the caller passes --yes. A non-TTY run (an agent) prints the plan as JSON and exits 4. Two folders under one prefix still require --root, and --yes does not choose a folder. An index another folder still names, a shared edges index, and autoindex-catalog are left in place. MCP xerj_autoindex_remove is the same plan and deletes only when confirm is true.

DELETE /{index} now returns 400 Wildcard expressions or all indices are not allowed when the selector is *, _all, or contains * or ?, unless action.destructive_requires_name is false. The ES-YAML runner turns that guard off for its DELETE /_all cleanup and puts the key back to null, so the next file sees the default again. indices.delete/10_basic.yml opts out for its expansion cases; 20_destructive_wildcard.yml asserts the refusal.

The prefix-reuse refusal now starts with "do not delete the other folder's indices" and a ready-to-copy xerj autoindex <folder> --prefix <name>. Agent docs (AGENTS.md, llms.txt, llms-full.txt, the MCP tool descriptions) say never DELETE an index this session did not create, and to use xerj autoindex remove.

Rebased onto origin/main after --walk-only (#2006). remove refuses --walk-only rather than ignoring it.

Evidence

cargo +1.97.1 fmt --all
cargo +1.97.1 clippy --locked -p xerj-autoindex -p xerj-api -p xerj-mcp \
  -p xerj-server -p xerj-console-api -p es-yaml-runner \
  --all-targets -- -D warnings
# exit 0 (autoindex rechecked after the rebase resolution; exit 0)

cargo +1.97.1 test --locked -p xerj-api -p xerj-autoindex \
  -p xerj-mcp -p xerj-server -p xerj-console-api -p es-yaml-runner -- --test-threads=16
# xerj-api: integration suites passed, including
#   destructive_requires_name 3 passed
#   corrupt_index_http 11 passed
# xerj-autoindex --lib: 1560 passed, 0 failed, 2 ignored
# xerj-mcp and xerj-server: passed (rerun after console-api aborted the first invocation)
# xerj-console-api --lib: SIGABRT, tokio worker stack overflow, on the default stack
#   (reproduced with --test-threads=8). RUST_MIN_STACK=33554432: 67 passed, 0 failed.
#   The only edit in that crate is the MCP tool count in a knowledge blurb (19 -> 20).

# debug node: ./target/debug/xerj --insecure --data-dir /tmp/xerj-yaml-data --port 9200
./target/debug/es-yaml-runner --url http://127.0.0.1:9200 --dir tests/es-compat-yaml/yaml
# 1438 passed · 0 failed · 10 skipped · 1448 total (227 files)
# indices.delete alone: 13 passed, 0 failed

# after rebasing onto origin/main (e1348b4d, --walk-only):
cargo +1.97.1 test --locked -p xerj-autoindex --lib -- \
  remove_requires walk_only_is_accepted an_index_another_root prefix_reuse \
  two_roots_without yes_with_a_root
# 12 passed

The acceptance checks covered by those runs: remove without --yes sends no DELETE and exits 4 with the root list; --yes without --root still deletes nothing when two folders share the prefix; --root A --yes deletes only A's index and catalog rows; an index both roots name (ax-docs) is kept; DELETE /ax-*, /_all and /* are 400 until action.destructive_requires_name is false, then /ax-* is 200 and does not delete a non-matching index; the prefix-reuse message names a distinct --prefix before --replace-prefix.

Checks

  • cargo fmt --all (CI runs rustfmt --check and clippy -D warnings)
  • Scoped release build of the crates touched: cargo build --release -j 32 -p <crate>
  • cargo test -p <crate> passes (cargo check is not sufficient — test code is interleaved with production code)
  • ES-YAML conformance suite at 0 failed (cargo run --release -p es-yaml-runner -- --dir tests/es-compat-yaml/yaml), or: not applicable because this change is docs/landing-only
  • New ES-compatible behaviour has a matching YAML case under engine/tests/es-compat-yaml/yaml/
  • Docs updated if user-visible behaviour changed
  • For non-trivial changes: the applicable audit in docs/CONTRIBUTION_REVIEW.md
  • For non-trivial changes: if this PR changes an operational document or runnable recipe, complete the operational audit in docs/CONTRIBUTION_REVIEW.md. Record a negative control that exits non-zero with no artifact extracted, installed, published, or served. Keep verification and its action in one block; rerun the control with the failing exit discarded and with the next documented block appended, and require that neither reaches an artifact or unsafe/privileged path. List tested variants and unsupported variants separately, and verify every publication copy. Otherwise mark this item N/A with a reason.

Not run: a release rebuild. The ES-YAML number above is from target/debug/xerj and target/debug/es-yaml-runner, not cargo run --release. Clippy was the touched crates, not cargo clippy --workspace. xerj-console-api passed only with RUST_MIN_STACK=33554432.

The recipe edit is one sentence inside the existing --prefix description in docs/recipes/zero-config-indexing.md. It adds no shell, TOML, or systemd block, so the operational-recipe negative-control audit is N/A. Failure atomicity for remove is covered by the partway-delete path: a failed delete stops, and the JSON remaining object is a fresh catalog read. That path is unit-tested for the planner and the no-delete confirmation; a mid-bulk HTTP failure against a live node was not simulated.

Open PRs #1909 and #1925 also edit landing/llms.txt. This branch adds one caveat bullet.

Provenance

  • Written by: AI agent (Cursor Grok 4.7), on branch cursor/autoindex-safe-remove-da05
  • Verified (commands run, output observed): the commands and counts in Evidence. Wildcard refusal and the opt-out were observed over HTTP in destructive_requires_name and in the YAML files. remove without --yes was observed against a local mock catalog server (no DELETE in the request log). The shared-index case (ax-docs named by both roots) passed in an_index_another_root_still_names_is_not_deleted.
  • Assumed (not tested, and what would falsify it): a release binary behaves like the debug binary on the YAML suite. A catalog larger than the 10,000-document result window is refused before any delete (the bail is in search_kind; it was not driven with 10,000 real documents). A release build that failed, or a YAML file that failed only under --release, would falsify the debug result.
Open in Web Open in Cursor 

A coding agent deleted a live ax-docs index after a prefix-collision
refusal (#2114). Every folder defaults to the prefix ax, the refusal
did not lead with a distinct --prefix, nothing listed one folder and
deleted only that, and DELETE /ax-* was accepted.

xerj autoindex remove --prefix P [--root DIR] reads the catalog, prints
indices, catalog rows, doc counts and store sizes, and deletes nothing
unless a terminal answers y or the caller passes --yes. Two folders
under one prefix still require --root, including with --yes. An index
another folder still names, a shared edges index, and autoindex-catalog
are left in place. MCP xerj_autoindex_remove is the same plan and
deletes only when confirm is true.

DELETE of *, _all, or any wildcard is a 400 unless
action.destructive_requires_name is false, matching ES 8. The YAML
runner turns the guard off for DELETE /_all cleanup and puts the key
back to null. The prefix-reuse refusal now leads with a ready-to-copy
xerj autoindex <folder> --prefix <name> and says not to delete the
other folder's indices.

Verified locally with rustc 1.97.1, debug builds:
- cargo fmt --all
- cargo clippy --locked -p xerj-autoindex -p xerj-api -p xerj-mcp
  -p xerj-server -p xerj-console-api -p es-yaml-runner --all-targets
  -- -D warnings (exit 0)
- cargo test --locked of those crates: xerj-api integration tests
  passed, including destructive_requires_name (3) and
  corrupt_index_http (11); xerj-autoindex --lib 1560 passed, 2 ignored;
  xerj-mcp and xerj-server passed. xerj-console-api --lib aborted on
  the default thread stack (SIGABRT, stack overflow in a tokio worker)
  and passed 67/67 with RUST_MIN_STACK=33554432.
- es-yaml-runner against a debug --insecure node on :9200:
  1438 passed, 0 failed, 10 skipped (227 files).

Not run: a release rebuild of the touched crates.

Co-authored-by: xerj-org <xerj-org@users.noreply.github.com>
A dataset row matches `ax` when its index name starts with `ax-` and its
slug is the rest of the name. A legacy row with no slug only needs the
`ax-` start, so `ax-2-docs` (prefix ax-2) counted as ax's. Rows whose
run_id names a run of another prefix, and slug-less rows under a longer
sibling prefix the catalog knows about, are now never in the delete set.
@cla-bot
cla-bot Bot commented Oct 22, 2026
Copy link
Copy Markdown

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Xerj Merger.
This is most likely caused by a git client misconfiguration; please make sure to:

  1. check if your git client is configured with an email to sign commits git config --list | grep email
  2. If not, set it up using git config --global user.email email@example.com
  3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

@cla-bot
cla-bot Bot commented Oct 22, 2026
Copy link
Copy Markdown

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Xerj Merger.
This is most likely caused by a git client misconfiguration; please make sure to:

  1. check if your git client is configured with an email to sign commits git config --list | grep email
  2. If not, set it up using git config --global user.email email@example.com
  3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

@xerj-org
xerj-org marked this pull request as ready for review October 22, 2026 03:06
@xerj-org
Copy link
Copy Markdown
Owner Author

Xerj Merger made the fix: xerj autoindex remove could claim a sibling prefix's datasets (7d307a3).

Removing ax matched a legacy dataset row with no slug whose index name only started with ax-, such as ax-2-docs from prefix ax-2. Rows whose run_id names a run of another prefix, and slug-less rows under a longer sibling prefix the catalog knows, are now never in the delete set. Test: a_sibling_prefix_dataset_is_never_ours.

Destructive-path review:

  • Nothing is deleted without --yes, a y at a terminal, or MCP confirm: true. Without one of those, a non-terminal run exits 4 and prints the plan.
  • Two roots under one prefix need --root.
  • An index another folder still names is kept, and so is an edges index another run still names.
  • autoindex-catalog and wildcard names are refused at plan and at apply.
  • Catalog paging bails out with "Nothing was deleted" rather than acting on a partial view.
  • DELETE of *, _all or a wildcard is now 400 unless action.destructive_requires_name is false (ES 8 default). _close stays stricter.

Notes:

  • MCP confirm: true can be passed on the first call; the tool description says to read the plan first. A plan token would make that binding. That is a possible follow-up, not a blocker.
  • xerj-console-api did not need RUST_MIN_STACK in my runs: its tests pass on main and on main plus this PR with the default stack.

Local CI passed (fmt, clippy -D warnings, ci-test workspace, ES-YAML gate). Fixes the #2114 product gap.

Sorry, something went wrong.

@xerj-org
xerj-org merged commit 8aeb836 into main Oct 22, 2026
7 of 10 checks passed
xerj-org added a commit that referenced this pull request Oct 22, 2026
…r sweep (#2136)

* release(rc.98): version stamp, CHANGELOG, ROADMAP roll, landing footer sweep

Window since v1.0.0-rc.97 (about 120 PR merges on main's first-parent line,
every one cited in [1.0.0-rc.98]; the release-notes gate passes locally).

Stamp: workspace 1.0.0-rc.97 -> 1.0.0-rc.98 (Cargo.toml + Cargo.lock).
CHANGELOG: [Unreleased] moves into [1.0.0-rc.98], plus the window's uncited
merges grouped by area; the rc.97 heading gets its tag date (2026-10-19).
ROADMAP review line + roll: open defects stay at five. Landing footer sweep
RC.97 -> RC.98, the data-latest-tag fallback, and the regenerated sitemap.

* release(rc.98): cite #2124, #2126 and #2127

* Merge main into release/rc98

* release(rc.98): one #2124 entry

* release(rc.98): intro names #2128 and #2126

* release(rc.98): regenerate sitemap after the merges

---------

Co-authored-by: Xerj Merger <merger@xerj.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Uh oh!

There was an error while loading. Please reload this page.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0