8000 fix(or): seed an or branch whose expression reads an outer variable (#620) by adityamukho · Pull Request #622 · project-minigraf/minigraf · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
8000
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Rule queries with a literal argument return every row** (#608). Magic-sets rewriting restricts a rule to the facts its callers demand, but it took each rule's bound arguments from the first call it saw and assumed every other call matched. A call that did not — the same rule called with free arguments elsewhere in the query, a call inside `or` or `not`, a call from a rule the query calls with free arguments, or a rule-body call whose argument nothing bound yet — either failed with `INT-022` (`(chain ?x :c)` with `(chain ?x ?y) :- (link ?x ?m) (chain ?m ?y)`) or silently returned too few rows (`(link ?x :c) (link ?p ?q)` returned only the `:c` edges for `?p ?q`). A query with both arguments literal next to another rule call also lost its rows. The demand on each rule is now the one every call site supplies, and a rule some call site reads in full is evaluated in full. Present since v1.2.0.
- **A `:db/valid-at` pattern in a `not-join` body binds the query's valid time on every path** (#615). When the body also held an expression, sat in an `or` branch, or ran in a query that invokes a rule, the body bound `nil`, so `(not-join [?e] [?e :db/valid-at ?t] [(> ?t 0)])` never matched and excluded nothing; without the expression the same body excluded every entity. It now binds the query's `:valid-at` value (or the current time) as the other paths do. Present since v0.13.1; v2.x is affected too.
- **A `:db/valid-at` pattern in a rule definition binds the query's valid time** (#617). The rule evaluators never received it, so in a rule body the pattern bound `nil` and the query failed with `INT-022`; in a rule's `not` / `not-join` body it bound `nil`, so the negation never matched on it; and in a rule's `or` branch it bound the current time instead of the query's `:valid-at`. Rule bodies, their negation bodies and `or` branches now bind the query's `:valid-at` value, the current time without one, and `nil` under `:any-valid-time`, as patterns outside rules do. v2.x is affected too.
- **An `or` / `or-join` branch whose expression reads an outer variable matches** (#620). A branch such as `[(> ?v 1)]` in `[?e :n ?v] (or [(> ?v 1)] [(< ?v 0)])` was evaluated on its own from an empty binding and joined back afterwards, so its expression saw `?v` unbound and the branch matched nothing: the `or` silently dropped rows it should keep, in queries and in rule bodies. A branch whose expression reads a variable the branch does not bind itself is now seeded with each incoming row, as branches with `not` / `not-join` already were.
- **An `or` that shares no variable with the earlier clauses returns its cross product** (#610). Such an `or` — `[?x :edge ?z] (or [?p :edge :b] [?p :edge :c])` — returned no rows when no branch had `not`, `not-join` or a pseudo-attribute, because the incoming rows were replaced by the branch results. Each incoming row now joins every branch row, in queries and rule bodies. `or-join` still refuses a join variable no earlier clause binds (INT-018): it filters, it does not bind.
- **A prepared query keeps a literal `:as-of`, `:valid-at` or `:any-valid-time`** (#603). Binding a prepared query's slots dropped any `:as-of` or `:valid-at` that was not itself a slot, so `PreparedQuery::execute` and `PreparedQuery::query` ran at the current transaction time and valid now: `(query [:find ?v :as-of 1 :where [:a :n ?v]])` prepared returned the current rows, not those at transaction 1, and a prepared per-fact pseudo-attribute query with `:any-valid-time` failed with `INT-026`. Slot forms (`:as-of $tx`, `:valid-at $t`) were not affected. Present since 2026-04-04; fixed on v2.x too.
- **The wasm32 builds are tested again, and the browser layer no longer panics or drops pages.** The repo split (#231) moved the browser and WASI CI jobs to minigraf-wasm, which only builds the package, so core's own wasm tests stopped running and the wasm32 test build broke unnoticed. A new `wasm.yml` workflow runs the `wasm-bindgen` tests in headles 8000 s Firefox and Chrome, runs the unit and integration tests on `wasm32-wasip1` under Wasmtime, and lints both targets with `clippy -D warnings`. Fixes found on the way: `BrowserDb` skipped any dirty page it could not read when flushing to IndexedDB and still reported success, and now returns the error (also fixed in v2.0.4, #470). The IndexedDB upgrade callback, the IndexedDB backend handle and the page-buffer lock no longer `unwrap()` (on wasm32 a panic aborts the whole instance). IndexedDB keys that are not whole page numbers are rejected instead of truncated. The nine `BrowserBufferBackend` tests used `#[test]`, which the wasm test runner never runs, and now run. File-only modules (`wal`, `storage::verify`, `storage::dir_sync`) are no longer compiled for wasm32, so the wasm32 builds have no warnings.
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ Auto-migrates v7 → v8 on open (spec §9, with a backup meta page). v1–v6 are

## Test Coverage

**1564 tests** (1554 passing, 10 ignored; unit + integration + doc).
**1565 tests** (1555 passing, 10 ignored; unit + integration + doc).
See `docs/TEST_COVERAGE.md` for the full per-file breakdown.

**Testing conventions** — see the Testing Conventions section below before writing any tests.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ let r2 = pq.execute(&[("tx", BindValue::TxCount(2)), ("entity", BindValue::Entit

```bash
cargo run # interactive Datalog REPL
cargo test # run 1564 tests
cargo test # run 1565 tests
cargo run < demos/demo_recursive.txt # recursive rules demo
```

Expand Down
4 changes: 2 additions & 2 deletions docs/TEST_COVERAGE.md

Large diffs are not rendered by default.

96 changes: 93 additions & 3 deletions src/query/datalog/executor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1451,6 +1451,49 @@ pub(crate) fn reads_metadata(clauses: &[WhereClause]) -> bool {
})
}

/// The variables an `or` / `or-join` branch's expressions read that the branch
/// itself never binds — by a pattern, a rule invocation or an expression output,
/// its own or an enclosing nested branch's. Only the outer row can bind them, so
/// a branch with any must be seeded with that row, not evaluated from an empty
/// binding and joined afterwards (#620).
pub(crate) fn branch_outer_reads(branch: &[WhereClause]) -> std::collections::HashSet<String> {
let mut bound = std::collections::HashSet::new();
for c in branch {
match c {
WhereClause::Pattern(p) => bound.extend(optimizer::pattern_bound_vars(p)),
WhereClause::RuleInvocation { args, .. } => {
for a in args {
if let EdnValue::Symbol(s) = a
&& s.starts_with('?')
{
bound.insert(s.clone());
}
}
}
WhereClause::Expr {
binding: Some(b), ..
} => {
bound.insert(b.clone());
}
_ => {}
}
}
let mut reads = std::collections::HashSet::new();
for c in branch {
match c {
WhereClause::Expr { expr, .. } => reads.extend(optimizer::expr_vars(expr)),
WhereClause::Or(branches) | WhereClause::OrJoin { branches, .. } => {
for b in branches {
reads.extend(branch_outer_reads(b));
}
}
_ => {}
}
}
reads.retain(|v| !bound.contains(v));
reads
}

/// Filter `bindings` through a single `(not-join [...] ...)` body for the rules query
/// path — no exclusion-set fast path here (matches the pre-#248 post-filter, which
/// never had one for this path either).
Expand Down Expand Up @@ -2129,11 +2172,13 @@ pub(crate) fn apply_or_clauses(
// from the outer scope to evaluate correctly), fall back to the classic
// seeded-branch evaluation to preserve correctness.
// So does a branch whose pseudo-attribute may read an outer
// source's metadata (#594).
// source's metadata (#594), or whose expression reads a variable
// only the outer row binds (#620).
let any_branch_has_not = sorted_or_branches.iter().any(|b| {
b.iter()
.any(|c| matches!(c, WhereClause::Not(_) | WhereClause::NotJoin { .. }))
|| reads_metadata(b)
|| !branch_outer_reads(b).is_empty()
});

if any_branch_has_not {
Expand Down Expand Up @@ -2355,8 +2400,13 @@ pub(crate) fn apply_or_clauses(
// metadata (#594) needs it from the outer row: evaluate the
// branches per row, seeded with its join variables only (the
// others are fresh inside), and keep the distinct rows some
// branch matches, as the joined path below does.
if sorted_oj_branches.iter().any(|b| reads_metadata(b)) {
// branch matches, as the joined path below does. So does a
// branch whose expression reads a join variable it does not
// bind itself (#620).
if sorted_oj_branches
.iter()
.any(|b| reads_metadata(b) || !branch_outer_reads(b).is_empty())
{
let mut seen: std::collections::HashSet<Vec<(String, Value)>> =
std::collections::HashSet::new();
let mut result: Vec<Binding> = Vec::new();
Expand Down Expand Up @@ -5530,6 +5580,46 @@ mod expr_eval_tests {

// ── evaluate_branch / apply_or_clauses edge cases ────────────────────────

/// The branches of the query's last `or` / `or-join` clause.
fn last_or_branches(q: &str) -> Vec<Vec<WhereClause>> {
let DatalogCommand::Query(query) = parse_datalog_command(q).expect("parse failed") else {
panic!("expected a query");
};
match query.where_clauses.last() {
Some(WhereClause::Or(b)) | Some(WhereClause::OrJoin { branches: b, .. }) => b.clone(),
_ => panic!("expected an or clause last"),
}
}

#[test]
fn test_branch_outer_reads() {
let b = last_or_branches(
"(query [:find ?v :where [?e :n ?v] (or [(> ?v 1)] (and [?e :n ?v] [(< ?v 0)]))])",
);
assert!(branch_outer_reads(&b[0]).contains("?v"), "reads outer ?v");
assert!(branch_outer_reads(&b[1]).is_empty(), "binds ?v itself");

// A nested branch's private variable is not an outer read; the
// enclosing branch's variable is bound for the nested one.
let b = last_or_branches(
"(query [:find ?v :where [?e :n ?v] (or (and [?e :n ?v] (or-join [?v] (and [?x :n ?y] [(> ?y ?v)]))) [?e :n 5])])",
);
assert!(branch_outer_reads(&b[0]).is_empty(), "nested reads bound");
let b = last_or_branches(
"(query [:find ?v :where [?e :n ?v] (or (or [(> ?v 1)] [(< ?v 0)]) [?e :n 5])])",
);
assert!(
branch_outer_reads(&b[0]).contains("?v"),
"nested reads outer"
);

// An expression output is bound by the branch.
let b = last_or_branches(
"(query [:find ?w :where [?e :n ?v] (or (and [?e :n ?v] [(+ ?v 1) ?w] [(> ?w 1)]) [?e :n ?w])])",
);
assert!(branch_outer_reads(&b[0]).is_empty(), "?w bound by output");
}

#[test]
fn test_evaluate_branch_with_timestamp_valid_at() {
// Exercises executor.rs lines 930-931: evaluate_branch with Timestamp/AnyValidTime
Expand Down
4 changes: 2 additions & 2 deletions src/query/datalog/optimizer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ pub fn select_index(p: &Pattern) -> IndexHint {
}

/// Collect all logic-variable names (`?foo`) referenced in an Expr tree.
fn expr_vars(expr: &Expr) -> Vec<String> {
pub(crate) fn expr_vars(expr: &Expr) -> Vec<String> {
match expr {
Expr::Var(s) => vec![s.clone()],
Expr::Lit(_) | Expr::Slot(_) => vec![],
Expand All @@ -134,7 +134,7 @@ fn expr_vars(expr: &Expr) -> Vec<String> {

/// Collect the logic-variable names bound (output) by a Pattern.
/// Only Symbol values starting with `?` count — literals never bind.
fn pattern_bound_vars(p: &Pattern) -> Vec<String> {
pub(crate) fn pattern_bound_vars(p: &Pattern) -> Vec<String> {
let mut vars = Vec::new();
if is_variable(&p.entity)
&& let EdnValue::Symbol(s) = &p.entity
Expand Down
101 changes: 101 additions & 0 deletions tests/plan_independence_test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -294,3 +294,104 @@ fn unify_negative_zero_equals_zero() {
);
assert_eq!(r.len(), 1, "0.0 unifies with -0.0");
}

// ── #620: an `or` branch whose expression reads an outer variable ────────────

fn issue_620_data(db: &Minigraf) {
db.execute("(transact [[:a :n 1] [:b :n 2]])").unwrap();
}

fn ints(rows: &[Vec<String>]) -> Vec<String> {
let mut v: Vec<String> = rows.iter().map(|r| r.join(",")).collect();
v.sort();
v
}

#[test]
fn or_branch_expression_reads_outer_variable() {
let db = Minigraf::in_memory().unwrap();
issue_620_data(&db);
let r = same(
&db,
"(query [:find ?v :where [?e :n ?v] (or [(> ?v 1)] [(< ?v 0)])])",
);
assert_eq!(ints(&r), vec!["i:2"], "only ?v = 2 is over 1");
let r = same(
&db,
"(query [:find ?w :where [?e :n ?v] (or (and [(= ?v 1)] [?e :n ?w]) (and [(= ?v 2)] [?e :n ?w]))])",
);
assert_eq!(ints(&r), vec!["i:1", "i:2"], "each row matches one branch");
// One branch binds ?v itself, the other only reads it.
let r = same(
&db,
"(query [:find ?v :where [?e :n ?v] (or (and [?e :n ?v] [(> ?v 1)]) [(< ?v 0)])])",
);
assert_eq!(ints(&r), vec!["i:2"], "mixed branches");
}

#[test]
fn or_branch_expression_binds_from_outer_variable() {
let db = Minigraf::in_memory().unwrap();
issue_620_data(&db);
let r = same(
&db,
"(query [:find ?v ?w :where [?e :n ?v] (or [(+ ?v 10) ?w] [(- ?v 10) ?w])])",
);
assert_eq!(
ints(&r),
vec!["i:1,i:-9", "i:1,i:11", "i:2,i:-8", "i:2,i:12"],
"both branches bind ?w from ?v"
);
}

#[test]
fn nested_or_expression_reads_variable_of_enclosing_branch() {
let db = Minigraf::in_memory().unwrap();
issue_620_data(&db);
let r = same(
&db,
"(query [:find ?v :where [?e :n ?v] (or (and [?e :n ?v] (or [(> ?v 1)] [(< ?v 0)])) [?e :n 5])])",
);
assert_eq!(ints(&r), vec!["i:2"], "the inner or sees ?v");
let r = same(
&db,
"(query [:find ?v :where [?e :n ?v] (or (or [(> ?v 1)] [(< ?v 0)]) [?e :n 5])])",
);
assert_eq!(ints(&r), vec!["i:2"], "the inner or sees the outer ?v");
}

#[test]
fn or_join_branch_expression_reads_join_variable() {
let db = Minigraf::in_memory().unwrap();
issue_620_data(&db);
let r = same(
8695 &db,
"(query [:find ?v :where [?e :n ?v] (or-join [?v] [(> ?v 1)] [(< ?v 0)])])",
);
assert_eq!(ints(&r), vec!["i:2"], "only ?v = 2 is over 1");
let r = same(
&db,
"(query [:find ?v :where [?e :n ?v] (or-join [?v] (and [?x :n ?y] [(> ?v ?y)]) [(< ?v 0)])])",
);
assert_eq!(ints(&r), vec!["i:2"], "2 is over some ?y, 1 is not");
}

#[test]
fn or_branch_expression_reads_outer_variable_in_rule_body() {
let db = Minigraf::in_memory().unwrap();
issue_620_data(&db);
db.execute("(rule [(big ?e) [?e :n ?v] (or [(> ?v 1)] [(< ?v 0)])])")
.unwrap();
db.execute("(rule [(big2 ?e) [?e :n ?v] (or-join [?v] [(> ?v 1)] [(< ?v 0)])])")
.unwrap();
db.execute(
"(rule [(pick ?e ?w) [?e :n ?v] (or (and [(= ?v 1)] [?e :n ?w]) (and [(= ?v 2)] [?e :n ?w]))])",
)
.unwrap();
let r = same(&db, "(query [:find ?v :where (big ?e) [?e :n ?v]])");
assert_eq!(ints(&r), vec!["i:2"], "or in a rule body");
let r = same(&db, "(query [:find ?v :where (big2 ?e) [?e :n ?v]])");
assert_eq!(ints(&r), vec!["i:2"], "or-join in a rule body");
let r = same(&db, "(query [:find ?w :where (pick ?e ?w)])");
assert_eq!(ints(&r), vec!["i:1", "i:2"], "or-and in a rule body");
}
Loading
0