Repository navigation
docs(roadmap): enterprise sources, clusters/sizing, on-prem model providers; add SIZING_AND_SLA.md - #2994
Open
xerj-org wants to merge 1 commit into
Open
docs(roadmap): enterprise sources, clusters/sizing, on-prem model providers; add SIZING_AND_SLA.md#2994xerj-org wants to merge 1 commit into
xerj-org wants to merge 1 commit into
Conversation
…viders; add SIZING_AND_SLA.md Motivation: enterprise users running XERJ on their own hardware ask for search on top of more than files (databases, vector stores, archives), for a cluster they can size and rely on, and for the model providers they already run (Cohere among them). The roadmap said nothing about sources beyond files, carried clustering as one line under "Beyond 1.0", and no page said how to size a node. ROADMAP.md, three new sections before "Beyond 1.0 - themes" (structure, the review line and the Open defects shortlist are unchanged): - Enterprise data sources beyond files: archives (#2982, merged after the rc.101 tag), live Postgres logical-replication sync with status/lag (#2984, open), MySQL binlog (planned, no code), rows to facts for XERJ Query-1 (#2988, open), vector-DB importers for Qdrant/Chroma/pgvector/Pinecone (planned, no code). - Clusters, scaling and sizing: what the 2026-11-05 rc.101 (dfbab65) lab runs measured (0 of 1,404,558 acked ids lost after kill -9; election 170-405 ms; one dead node blocks writes to 2 of 3 shards, 40 of 60 got 503; _count 93/104/103 vs 300; _mget 2 of 6; health counts dead nodes; 17-25% of acked docs 404 after a 4-shard index), the open fixes (#2992 per-index shard routing 994/6,000 -> 0/6,000 missed GETs; #2993 in-sync shrink 26,802-doc replica gap -> 0; #2987 leader no-op commit; #2991 xerj index exit 1 when docs are lost, rc.101 dropped 127,138 docs and exited 0), and nine ordered deferred items (promotion, PreVote/CheckQuorum, read fan-out, honest health, scale-out, ops, memory/ingest, a real multi-process CI job plus a measured cluster SLA run, dead-code cleanup). - Enterprise on-prem integrations: model providers. Supported embedders and rerankers, then Cohere as user-requested: Rerank (#2985, on main, not released), Embed (#2983, open; native /v2/embed planned), Command/North tool use (#2983 tools + #2990 page, open; Query-1 tool and North connector planned). Off by default, no live Cohere run yet, no affiliation. "Distributed clustering maturity" now points at the new section. docs/SIZING_AND_SLA.md (new): states there is no formal SLA, the hardware (32-core x86_64, 119 GB, shared host) and date, the single-node sizing table (1 GB raw: 4 cores / 8-10 GB RAM / 2.5 GB disk; 5 GB: 8 / 16 / 12; 10 GB: 8-16 / 24 / 25), search (5 GB, 8 cores: 196 qps p99 8.4 ms at 1 client, 747 qps p99 20.4 ms at 8; ~110 qps per busy core), ingest (_bulk 8,415 docs/s, p99 11.6 s), recovery (kill -9 with 726 MB WAL: 24.0 s, 200,000 of 200,000 docs), cluster works / does-not-work tables, and how to size today. It flags that the operations page's capacity-planning block predates these runs. Checks: release-notes gate check 1a logic run locally: shortlist #1122 open, #1170 open. cargo test -p xerj-engine --test docs_capability_lists: 15 passed, 0 failed. No landing page touched (docs/*.md has no site generator), so no SEO registration.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds three planned-work sections to ROADMAP.md and a new sizing page, docs/SIZING_AND_SLA.md. Enterprise users running XERJ on their own hardware ask for search over more than files, for a cluster they can size and rely on, and for the model providers they already run.
ROADMAP.md
The review line, section structure and Open defects shortlist are unchanged. The shortlist still lists #1122 and #1170, and both are open on the tracker. The three new sections sit before Beyond 1.0 — themes:
/v2/embedprovider is planned.docs/SIZING_AND_SLA.md
dfbab650d.landing/docs/operations.htmlpredates these runs.Checks
cargo test --release -p xerj-engine --test docs_capability_lists: 15 passed, 0 failed.Caveats
🤖 Generated with Claude Code