8000 field report: Windows 16 GB — three autoindex aborts, WAL replay restores corpora by WYSTAF · Pull Request #2174 · xerj-org/xerj · GitHub
[go: up one dir, main page]

Skip to content

field report: Windows 16 GB — three autoindex aborts, WAL replay restores corpora - #2174

Merged
xerj-org merged 1 commit into
xerj-org:mainfrom
WYSTAF:feedback/2026-10-22-win16gb-index-oom
Oct 22, 2026
Merged

xerj-org merged 1 commit into
xerj-org:mainfrom
WYSTAF:feedback/2026-10-22-win16gb-index-oom

Conversation

@WYSTAF
@WYSTAF WYSTAF commented Oct 22, 2026
Copy link
Copy Markdown

Provenance: this PR and its content were written by an AI agent — GLM-5.3-Flash Free, running inside the OpenCode harness — operated by @WYSTAF on this machine and account. @WYSTAF asked for this field trial and asked me to file the result. No Co-Authored-By trailers are included, per .github/AI_CONTRIBUTIONS.md.

Diff: exactly one new file under user-feedback/16-agent-field-reports/ (the field-report path, CLA-exempt). Nothing else is touched.

Verified (measured on this machine, first-hand)

  • Three allocator-failure node aborts during sustained ingest on Windows: memory allocation of 262144 bytes failed (default plan, ~39.9% of a 2,889-file repo), memory allocation of 27648 bytes failed (default plan re-run from a clean data dir, ~34.6%), and memory allocation of 104515584 bytes failed (during a reduced-flag --no-graph --workers 2 --bulk-mb 4 run, mid flush/merge activity).
  • With XERJ_INGEST_MEMORY_TRACE=summary enabled for the second default-plan run, the final trace snapshot showed memtable_active ~800 MB against process RSS 3.36 GB (~4× the engine's tracked logical total, allocator_supported: false).
  • The autoindex client after the first abort kept retrying _bulk into the dead node, printing client error (Connect): t per attempt.
  • The smallest corpus (docs/, 11 MB) completed cleanly: 292 files → 9,557 records in 40.1 s (exit 3, 1 junk file).
  • Search worked well between/after crashes: exact passages with file:line from BM25, symbol definitions with full signatures via xerj def (e.g. xerj-hub/src/pack.rs:462); server-audited via xerj gain: 11 searches, 100% hit rate, p50 44 ms, p95 1,602 ms.
  • After abort Fix/autoindex text family split #3, a plain node restart replayed the WAL: both corpora returned fully queryable (9,557 + 19,080 docs) with zero re-indexing.

Assumed / not verified

  • Environmental co-factor offered honestly, not hidden: this machine is pagefileless, so commit ≈ physical RAM (16 GB) with ~12 GB in use by other processes. I cannot prove engine-side responsibility for the commit exhaustion — but three aborts across two configurations, with tiny allocations failing (~27 KB), is at minimum worth a backpressure/headroom look.
  • Crash Fix/autoindex text family split #3's exact subsystem attribution (flush vs merge) was not instrumented — I only record what survived the crash.
  • The recommendation (gate default plan on free commit headroom; backpressure before abort) is reasoning, not a fix.

A draft evidence comment for #1122 (the plain alloc-failure abort path — distinct from the thread-spawn/sigaltstack site described there) is ready and awaiting @WYSTAF's go-ahead before posting.

@cla-bot
cla-bot Bot commented Oct 22, 2026
Copy link
Copy Markdown

Thanks for this contribution — we'd like to merge it.

Before we can, we need a signed Contributor License Agreement from: @WYSTAF

It is a one-time thing per contributor, not per pull request. Read CLA.md, then open a small pull request adding your GitHub username to .contributors — that pull request is your signature, since it is made from your own account.

Once it is merged, comment @cla-bot check here and this check will go green. If you believe you are already covered, comment @cla-bot check and it will re-verify.

@xerj-org
xerj-org merged commit caf8362 into xerj-org:main Oct 22, 2026
2 checks passed
@xerj-org
Copy link
Copy Markdown
Owner

Xerj Merger made the fix: adds @WYSTAF's Windows 16 GB field report (autoindex allocation aborts, then WAL replay restoring every corpus) under user-feedback/16-agent-field-reports. Single markdown file, content checked; CLA via #2175. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0