Repository navigation
No working result-limiting/pagination mechanism for plain (non-recursive) :find queries #306
Description
Summary
There is no way to bound the result size of a plain [:find ...] Datalog query. The only documented capping mechanism, :max-results N, is explicitly scoped to "inside the rule evaluator" and the reference states outright that "for non-recursive queries results are not truncated by this value." In practice this means any :find query over a large graph can return an unbounded, arbitrarily huge result set with no way for the caller to prevent it.
This is a real, repeatedly-hit problem on production-scale graphs, not a theoretical one.
Reproduction
Graph under test: ~6,300+ commit entities ingested from the arangodb repo (52,665 commits total, ingestion in progress — this is a small fraction of the eventual graph size).
A simple, reasonable query:
[:find ?e :where [?e :entity-type :type/commit]]
returns a 267,775-character single-line result — already large enough to exceed the calling client's token budget on this partial graph. The graph is only ~12% ingested at time of testing; the fully-ingested graph will be significantly larger.
Attempting to bound it with the documented :max-results 5 clause has no effect (see companion issue #305 — the clause is silently parsed-and-discarded for this query shape rather than applied or rejected):
[:find ?e :where [?e :entity-type :type/commit]] :max-results 5
Result: 267,943 characters — no meaningful reduction.
There is no :limit, :offset, cursor, or streaming mechanism documented or discoverable for plain :find queries.
Impact
- Any caller working with a moderately large graph must either already know the exact narrow shape of every query in advance (to avoid ever selecting a high-cardinality relation), or risk queries that fail entirely due to oversized responses.
- This makes exploratory/ad-hoc querying — one of the core use cases of a queryable graph memory — impractical once a graph reaches realistic production scale (tens of thousands of entities, as seen here).
- Callers have no graceful degradation path: a query either fits, or hard-fails with no partial results.
Suggested fix
Add a genuine result-limiting mechanism for plain :find queries — e.g. a :limit N (and ideally :offset N or a cursor) clause honored directly by the top-level query executor, independent of the rule evaluator's :max-results. This is table-stakes for any query interface expected to work against non-trivial data volumes.
Related
Companion issue #305: the parser silently discards trailing clauses/tokens after a valid query form instead of erroring, which is part of why this gap is easy to hit without realizing it (a caller may reasonably believe :max-results N is working when it silently isn't).
Activity
adityamukho commentedon Oct 14, 2026
#310 has been closed as a duplicate of this issue. It added two points that belong here:
:limit Nshould stop the engine early instead of truncating a fully built answer. That needs the streaming operator pipeline (enhancement: streaming, pull-based operator pipeline with bounded memory behind the cursor API (follow-up to #432) #473), which is why this issue comes after it.temporal_reasoningneeds it for memory retrieval, so it can rank and truncate a bounded candidate set instead of fetching every match.Scope stays
:limit Nwith optional:offset N, plus the 10M-fact:limitacceptance test moved here from #473.Merge pull request #586 from project-minigraf/docs/310-duplicate-refs
adityamukho commentedon Oct 14, 2026
Note from #544: the
Limit/Offsetoperators (#473 spec §6.1) are built here, with the syntax, rather than in #544.Merge pull request #728 from project-minigraf/docs/306-v3-readme-enve…
docs(spec): #306 slots, verifier, early-stop scope (Codex review)
docs(plan): #306 :limit, :offset and :order-by
Merge pull request #729 from project-minigraf/feat/306-limit
docs(roadmap): move v3.0.0-milestone features into v3.0.0 scope (#733)
docs(spec): #181 review: parse once, plan per execution; #306 codes
docs: :limit in CHANGELOG/README/ROADMAP/CLAUDE.md, test counts (#306)
docs(benchmarks): :limit envelope and acceptance numbers (#306)
Merge pull request #745 from project-minigraf/feat/306-limit-op
Merge pull request #755 from project-minigraf/feat/306-order-by
adityamukho commentedon Oct 25, 2026
Done on v3: PR 1 (:limit) #745, PR 2 (:order-by, :offset, top-k Sort) #755 (09266d8), docs site project-minigraf/minigraf-docs#69. Ships in v3.0.0.