Repository navigation
feat(udf): fallible UDFs, QRY-014, registry snapshot and re-entry (#180 PR 1) - #746
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015NBT2Yx9JUQKYvv2CvC9py
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015NBT2Yx9JUQKYvv2CvC9py
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015NBT2Yx9JUQKYvv2CvC9py
…#180) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015NBT2Yx9JUQKYvv2CvC9py
A query snapshots the function registry when it opens and holds no lock while a user function runs; registration swaps in a new registry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015NBT2Yx9JUQKYvv2CvC9py
… §5 (#180) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015NBT2Yx9JUQKYvv2CvC9py
) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015NBT2Yx9JUQKYvv2CvC9py
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015NBT2Yx9JUQKYvv2CvC9py
# Conflicts: # ROADMAP.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015NBT2Yx9JUQKYvv2CvC9py
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015NBT2Yx9JUQKYvv2CvC9py
# Conflicts: # CLAUDE.md # docs/ERROR_REFERENCE.md # docs/TEST_COVERAGE.md # src/error.rs
# Conflicts: # CLAUDE.md # docs/TEST_COVERAGE.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Urda7FD9CYYoxdeGGJa11x
# Conflicts: # CHANGELOG.md # CLAUDE.md # docs/TEST_COVERAGE.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Urda7FD9CYYoxdeGGJa11x
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Urda7FD9CYYoxdeGGJa11x
10 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#180 PR 1 (plan steps 5–13; spec
docs/superpowers/specs/2026-10-24-udf-uniffi-design.md§3–§5, §7, §8). Core only: the binding PRs follow. #180 stays open until the last binding PR.What changes
try_register_predicate/try_register_aggregatetake closures that returnResult<_, UdfError>.UdfErroris exported at the crate root.register_predicate/register_aggregatekeep their signatures: they wrap their closures inOk, so both kinds of registration share one failure path.QRY-014user function `<name>` failed: <message>is raised at the first failing call: a predicate (in:where, rule bodies,notbodies,orbranches), or an aggregate'sinit,steporfinalise(as a:findaggregate or an:overwindow). No later call of any UDF is made for that query.eval_exprnow returnsEvalError::{NoValue, Udf}:NoValuedrops the row asErr(())did before, andUdffails the query. Before this PR, QRY-014 was unclaimed on every branch, feat(query): add set membership predicates #316's included.Functions(RwLock<Arc<FunctionRegistry>>) replacesArc<RwLock<FunctionRegistry>>for every holder.open_querytakes one snapshot, and theQRY-007/QRY-010/QRY-011checks, the stream and rule evaluation all read it.Stream::pullandopen_rulesno longer lock. Registration copies the registry, inserts the function and swaps the copy in. Before, a UDF that registered a function deadlocked.WriteTransactioneach call returnsINT-001except registration (checkpoint/verify/rebuild_indexes on a thread holding a WriteTransaction deadlock instead of INT-001 #734).Deviation from the plan
ERROR_REFERENCE.mdand the docs site describe the behaviour as it is.Tests
tests/udf_test.rs: every existing UDF test runs through both kinds of registration. New: a failing predicate in four positions; through a cursor (rows first, then the error, thenNone); through a prepared query; a failing aggregate in each of the three phases, as an aggregate and as a window. Each checks that no call follows the failing one.tests/udf_reentry_test.rs(new, file-backed): registration from inside a predicate, an aggregate and a rule body, each under a 30 s timeout (they deadlocked before); a cursor keeps the functions it opened with; one test per row of spec §5's table, in both columns, on a file database with committed and pending facts.UdfError, andQRY-014's code, category and text.Bench machine (merged with v3 at 8f2d51f)
cargo test: 1,790 passing, 13 ignored.--features test-hooks: 1,808 passing, 14 ignored. No failures.nightly-2026-10-01,cargo llvm-cov --branch): all pass. Crate 95.37% lines / 85.59% branches.cargo clippy --all-features -- -D warningsand the wasm32browserclippy: clean.TEST_COVERAGE.mdwill be updated from this PR'swasm.ymlrun (no Wasmtime on the bench).Lock audit (plan step 11)
I checked every
.read(),.write()and.lock()undersrc/query,src/graph/storage.rs,src/storage/cache.rsandsrc/storage/reader.rs, plus the other guards a read reaches (btree.rs,dict.rs,pins.rs,db.rs). Line numbers are at this PR's head. UDFs are called fromengine/filter.rseval(predicates) and fromaggregate.rsAgg/Running/windows/finish_groupsthroughengine/aggregate.rsandengine/window.rs(aggregates and windows). Every one of those calls runs insideStream::pullor insiderecursive::evaluateinopen_rules. No guard below is alive at any of them. Test-module guards are left out.src/query/datalog/functions.rs:210Functions::snapshot(read)Arc; the query keeps the clone. It replaces the read locks thatStream::pullheld for each batch andopen_rulesheld throughrecursive::evaluate.src/query/datalog/functions.rs:220Functions::register(write)src/query/datalog/executor.rs:362open_rule_query,rules.read()let plan = { … }block (magic-sets rewrite, planning), dropped beforeopen_rulesevaluates anything.src/query/datalog/executor.rs:441execute_rule,rules.write()src/graph/storage.rs:406Snapshot::read_pendingtx_bound,extend_with_pending,live_facts_of_scans(test only),pending_recordsandpending_live. Each copies the facts out and drops the guard on return, before the scan passes its batch on.src/graph/storage.rs:807/811write_pending,826/830restore_tx_counter,940set_committed,962/966/967publishsrc/graph/storage.rs:895/900FactStorage::snapshot(data,pending)snapshot()returns. The query opens on the result.src/graph/storage.rs:885/886, 981, 989/990, 1269/1270, 1297/1298, 1318/1319, 1378/1385(get_pending_facts,generation,pending_index_counts,get_facts_by_*,fact_count)src/storage/cache.rs:112/126get_or_load,153put_dirty,187flush,201invalidate,210evict,219cached_page_count,230capacityArcof the page.src/storage/reader.rssrc/storage/btree.rs:885/900/906/911MutexStorageBackendsrc/storage/dict.rs:47/55shared_get/shared_putsrc/storage/pins.rs:54Pins::lockpinwhen a snapshot is taken and byreleasewhen it drops, not during a pull.src/db.rs:765(execute_innerquery branch),979query_inner,prepared.rsexecute/query,fact_logsrc/db.rs:774(rule branch),808execute_statement,1110checkpoint_inner,1148verify_inner,1197rebuild_indexes_inner,454drop,1239generation,1249kill(write_lock)src/db.rs:1052begin_write→WriteTransactionwrite_lockis held for the transaction's whole lifetime, so it is held across a UDF inWriteTransaction::execute. Every call that could wait on it refuses on that thread withINT-001(#734); the write-transaction column ofudf_reentry_test.rschecks this.Nothing needed a fix, so the audit adds no re-entry test.
Docs
db.rsdoc comments on all four registration methods;CHANGELOG.md(Added);ERROR_REFERENCE.md(QRY-014, plus API-008 and QRY-008, whose causes no longer include a panicking UDF);ROADMAP.md,CLAUDE.md,docs/TEST_COVERAGE.md. Docs-site PR: project-minigraf/minigraf-docs#65, to merge after this one.Codex review: the two findings above (API-008/QRY-008 docs fixed; cursor wording made exact). It found no query path that reads the registry after open, and no swallowed failure.
🤖 Generated with Claude Code
https://claude.ai/code/session_015NBT2Yx9JUQKYvv2CvC9py