Repository navigation
feat(rerank): Cohere Rerank as a second provider on _search - #2985
Merged
Merged
Conversation
A search can now say `"rerank": {"provider": "cohere"}` and have its top
hits reordered by Cohere Rerank (/v2/rerank) instead of the default
TypeSafe judge. Enterprise on-prem users asked for it: Cohere as the
second stage over XERJ's first stage, with the endpoint configurable so
document text can stay on a private Cohere deployment.
What it does
- Config: `[rerank.cohere] api_key, endpoint` with env fallback
COHERE_API_KEY / COHERE_ENDPOINT (config wins, same rule as the
TypeSafe pair), endpoint validated at load and at boot (no userinfo,
absolute http(s)), key redacted in Debug. Default endpoint
https://api.cohere.com/v2/rerank. `[rerank] enabled = false` forbids
this provider too.
- xerj-rerank: the HTTP machinery (retry/back-off, deadline, response
cap, key redaction, degrade policy) is now shared by JevProvider and the
new CohereProvider through one HttpProvider with a wire enum; only the
request body and the parse differ. Cohere request: model, query,
documents (title. text, clipped like the TypeSafe request), top_n =
batch, max_tokens_per_doc = the char ceiling. Cohere parse is strict:
an index outside the batch, an index answered twice, or a score that is
not a finite number in 0..=1 makes the batch Malformed (502), never a
half-trusted reorder. Per-provider batch ceiling: 100 for Cohere, 30
for TypeSafe unchanged.
- Request validation: model allow-list rerank-v5.0-fast (default),
rerank-v5.0-pro, rerank-v3.5; `instructions` and `min_score` with
provider cohere are a 400 before anything is sent (Cohere takes no
instruction; its scores are not calibrated on our data, unmeasured, and
pruning changes hits.total).
- Usage: `_rerank.usage.search_units` from Cohere's
meta.billed_units.search_units (0 for TypeSafe; token counts 0 for
Cohere).
- GET /_xerj/rerank: `providers: ["jev","cohere"]`, `configured` true if
either provider has a key, new `by_provider.{jev,cohere}` blocks (key
set + source, endpoint + source, max_docs_per_call; Cohere models,
default model, unsupported fields). The top-level api_key/endpoint stay
the TypeSafe pair so existing readers keep working.
- MCP rerank argument description names the provider option; the
published schemas (mcp-tools.json regenerated with
scripts/mcp-schema-check.sh --write; openai/anthropic copies carry the
same sentence).
- docs/RERANK.md: setup, request with expected output, limits, and a new
row in "Every way data leaves a XERJ node". xerj.default.toml shows
the commented [rerank.cohere] block. Settings count 137 -> 139 in every
place the count test names.
Tests (no Cohere key exists here: stub server only, no live numbers)
- xerj-rerank lib: 62 passed (7 new: v2 wire format, index->ordinal
mapping + search_units, 8 malformed shapes, config defaults/clamp/
refusals, settings precedence + key isolation, 429 retried then
reordered, 401 surfaced once with the key redacted).
- xerj-common: 3 new (endpoint checks from file and env, Debug never
prints either key, TOML parse + unknown key refused); count test
updated.
- xerj-api rerank_stage_http: 61 passed (7 new: reorder + bearer + model
+ top_n + search_units, 503 naming COHERE_API_KEY with a TypeSafe key
only, three 400s that send nothing, 401 -> 502 with no key echo, slow
provider degrades to engine order, status by_provider, env names pinned
across crates).
- egress_inventory 4/4, published_schema_drift 4/4, clippy -D warnings
and fmt clean on the touched crates.
- ES-YAML against the release build on a private port: 1448 passed,
0 failed, 10 skipped.
- Pre-existing, not from this change: xerj-common
resource::tests::process_memory_does_not_grow_with_a_touched_file_mapping
fails on unmodified main on this host too.
Reference: the provider shape follows the existing JevProvider in this
crate; its back-off already cites Meilisearch's Cohere client
(personalization/mod.rs:308-321, MIT). Cohere's request/response shape
from docs.cohere.com/reference/rerank (v2).
Not in this change (roadmap): min_score for Cohere after calibration is
measured, Context Focus spans, native /v2/embed.
xerj-org
added a commit
that referenced
this pull request
Nov 6, 2026
…site Merges main, which now carries #2982, #2983, #2984, #2985 and #2988. The Cohere integration page and docs/COHERE.md move those rows from "rc.102, in review" to "on main, ships in rc.102". #2984 added xerj_sources, so the published mcp-tools.json lists 24 tools (21 in a build with no hub key; 25 with XERJ Query-1 on). Swept the counts on for-agents, product, docs/cli, llms.txt, llms-full.txt, the compare and answers pages (sources in content/, regenerated) and the tool lists in README.md and AGENTS.md (xerj_sources added). Dated captures (rc.97, 2026-10-28) keep the number they recorded. Checks: SEO writers + every --check, seo_lint 0 gate violations, landing constants guard, test_article_schema, test_llms_field_report, test_verify_agent_discovery, gen_cohere_tools --check.
b0ril
pushed a commit
to b0ril/xerj
that referenced
this pull request
Nov 6, 2026
…, COHERE nav item Enterprise users who run search on their own hardware keep asking for Cohere models on top of their own files and databases. This adds: - landing/integrations/cohere.html: what plugs in where (Rerank provider, Embed via proxy, Command tools + RAG cookbook, MCP), an on-prem architecture diagram, config snippets, what leaves the network, and a status table: shipped (rc.101) / rc.102 in review (xerj-org#2983 xerj-org#2984 xerj-org#2985 xerj-org#2988) / roadmap. States plainly that XERJ is not affiliated with Cohere. - landing/use-cases/enterprise-on-prem-cohere.html: a worked example we ran on public data, modelled on an insurer: 13 public-domain NIST and CMS PDFs plus a synthetic 200,000-row Postgres claims table. Measured on one x86_64 machine (build: main 7458a22 + xerj-org#2983 xerj-org#2984 xerj-org#2985 xerj-org#2988; facts on rc.101 + A_s47): autoindex 22.9 MB / 14 files -> 3,033 records in 7.9 s xerj connect: snapshot ~47,100 rows/s; 60 updates commit->searchable p50 19.7 ms, p95 21.7 ms; delete and insert reflected filter + sum over 200k rows: 6,665 / $33,459,315, equal to Postgres XERJ Query-1 on 1,000 claims as facts: 8 questions, 5 answered and correct, 3 UNKNOWN, 0 wrong Cohere Rerank stage + Command RAG with file/page citations: run against the mock Cohere server from xerj-org#2983/xerj-org#2985, labelled as such. Includes a "what we did not test" box and a one-line agent prompt. - docs/COHERE.md: config keys, env vars, models allow-list, egress table, verify lines, limits, and the list of things a live key still has to confirm. docs/examples/usecase-cohere/: RESULTS.md, corpus SOURCES.txt, the commit-to-searchable script. - Nav: COHERE after XERJ QUERY-1 (set_nav.py), pages registered in pagedata.py, use-cases index card + subnav link on every use-case page. No customer, quotes, logos or Cohere quality/latency/cost figures are claimed. Checks: set_nav/fix_heads/fix_links/gen_sitemap/mk_og_card/build_articles/ build_skills --check ok, seo_lint 0 gate violations, landing constants guard ok, test_article_schema ok.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Cohere Rerank (
/v2/rerank) as a second reranking provider, selected per search with"rerank": {"provider": "cohere"}. Requested by enterprise on-prem users who want Cohere as the second stage over XERJ's first stage, with the endpoint configurable so document text can stay on a private Cohere deployment. Off until an operator sets a key.Config:
[rerank.cohere] api_key, endpoint; envCOHERE_API_KEY/COHERE_ENDPOINT; config wins. The endpoint is validated at load and at boot. The key is never echoed.[rerank] enabled = falseforbids this provider too.Behaviour:
rerank-v5.0-fast(default),rerank-v5.0-pro,rerank-v3.5. Any other name is a 400.instructionsandmin_scorewith providercohereare a 400, and nothing is sent. Cohere takes no instruction, and its scores are not calibrated on our data._rerank.usage.search_unitsreports Cohere's billed search units.GET /_xerj/rerankaddsproviders: ["jev","cohere"]and aby_providerblock. The top-level fields are unchanged.Docs:
docs/RERANK.md: setup, a request with its expected output, limits, and a new row in the data-egress table.xerj.default.toml: the commented[rerank.cohere]block.mcp-tools.jsonregenerated; the openai and anthropic copies carry the same sentence.Tests: no Cohere key exists here, so everything runs against a local stub of Cohere's documented request and response shape. This PR makes no live-quality, latency or cost claim.
xerj-reranklib: 62 passed (7 new).xerj-common: 3 new config tests.xerj-apirerank_stage_http: 61 passed (7 new).egress_inventory4/4,published_schema_drift4/4.-D warningsand fmt clean on the touched crates.Pre-existing and not from this change:
xerj-commonprocess_memory_does_not_grow_with_a_touched_file_mappingalso fails on unmodified main on the build host.Roadmap, not in this PR:
min_scorefor Cohere once calibration is measured, Context Focus spans, native/v2/embed.🤖 Generated with Claude Code