Repository navigation
fix(api,engine): data-stream rollover honours dry_run, conditions and write routing - #2900
Merged
xerj-org merged 2 commits intoNov 3, 2026
Merged
Conversation
`POST /{data_stream}/_rollover` ignored its body. `dry_run:true` created a
real backing index and answered dry_run:false, `conditions` never blocked a
rollover, and old_index reported the data-stream name instead of the current
backing index. The engine also left the alias with two members and no
is_write_index flag, so the first write after any rollover answered 400
"no write index is defined".
The handler now parses body and ?dry_run, evaluates max_docs/max_age (ORed,
reported in an ES-shaped conditions map) before touching the engine, and
returns the candidate old/new backing names even when it does not roll. A dry
run or unmet condition creates nothing; an existing candidate is a 409. Size
conditions and alias-rollover body fields are refused rather than ignored.
rollover_data_stream now marks the new backing is_write_index:true and every
older member false, preserving filter/routing metadata, so the stream stays
writable.
Evidence (rc.100, real node):
- before: dry_run:true -> rolled_over:true, generation 2; max_docs:100 with
one doc -> rolled; old_index "ds"; after rollover a write -> 400 no write index
- after: dry_run -> rolled_over:false/dry_run:true, generation 1;
max_docs:2 with one doc -> false/no roll; with two docs -> true/roll,
old_index .ds-ds-000001/new .ds-ds-000002; write after roll -> .ds-ds-000002;
max_size -> 400; write after plain rollover succeeds
- fail-before: all 7 new tests fail on the old handler
- xerj-api --lib 411/0; cluster_metadata_survives_restart 13; ilm_safety_rails 7;
fmt/clippy clean; quick build.
Scope: alias rollover remains unsupported; max_size/max_primary_shard_*
conditions and new_index/settings/mappings/aliases are refused.
# Conflicts: # CHANGELOG.md
Owner
|
Xerj Merger made the fix: merged. Data-stream |
5 of 9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes, and why
POST /{data_stream}/_rolloveraccepted a body but never read it. Measured on rc.100:The body fields are not cosmetic:
dry_runpromises no mutation, andconditionsis how ILM/data-stream maintenance decides whether a new backing is needed. The response also named the data stream (ds) instead of the old backing index.While writing the post-rollover test, a second defect surfaced: the engine added the new backing to the alias but never marked it
is_write_index: true. With two members and no flag, the alias was ambiguous, so the first write after any rollover answered 400no write index is defined for alias [ds].The fix
?dry_run) instead of ignoring it.max_docsandmax_agebefore touching the engine. Conditions are ORed, as ES defines them, and each result is reported as"[max_docs: N]": true|false/"[max_age: 1h]": true|false.old_index(current backing) andnew_index(next generated backing) even when the conditions are unmet, matching ES's response.new_index,settings,mappings,aliases) are refused with a named 400 rather than silently ignored.Engine::rollover_data_streammarks the new backingis_write_index: trueand clears every older member's flag, preserving each member's filter/routing. The stream is writable immediately after the rollover.Evidence
Real node, rc.100; stream
dsstarts with one backing and one document:{"dry_run": true}rolled_over:true,dry_run:false, generation 2, backing createdrolled_over:false,dry_run:true, generation 1, candidate.ds-ds-000002not created{"conditions":{"max_docs":2}}with 1 docrolled_over:false,conditions["[max_docs: 2]"]:false, generation 1rolled_over:true, conditions true,old_index:".ds-ds-000001",new_index:".ds-ds-000002"no write index is defined for alias [ds]_index:".ds-ds-000002"old_index:"ds"old_index:".ds-ds-000001"{"conditions":{"max_size":"1gb"}}max_size, nothing createdFail-before / pass-after (old handler + old engine, new tests kept):
Checks
cargo fmton every touched file (rustfmt --checkclean)cargo build --release -j 32 -p xerj-api -p xerj-engine— not run; aquickbuild ofxerj-serversucceeded on the final head. Left to CI.cargo test -p xerj-api --lib: 411 passed, 0 failed (final headb220fc30+ this diff)data_stream_rollover_conditions: 7 passed;cluster_metadata_survives_restart13,ilm_safety_rails7indices.rolloverYAML fixtures.CHANGELOG.mdUnreleased/Fixed states dry-run/conditions, the write-index fix, and the refused fields.docs/CONTRIBUTION_REVIEW.md— state correctness: dry-run and unmet conditions have no engine call; the target-exists check precedes conditions; the write flag is set before the stream returns to service.Not run: scoped release build and a current-runner ES-YAML pass; both left to CI.
Provenance
max_size;xerj-api --lib411/0, the stream/rollover suites above,rustfmt --check, clippy clean on the changed files,quickbuild.[max_docs: 2]/[max_age: 1h]; upstreamindices.rolloverYAML pins both, includingrolled_over:falsewith candidate names.max_docsis not met); this handler returns the same class of conflict.max_ageuses the persistedcreated_at.jsontime for the backing index; an index missing that record falls back to "now", so it would not meet a positive age condition until it ages. If ES used the index's settings timestamp instead, the measured boundary could differ.