Repository navigation
Description
Goal
Ship Minigraf as an npm package (@minigraf/core) that runs natively in browser environments using WebAssembly, with full TypeScript types and a page-granular IndexedDB storage backend.
Background
Phase 7 (Datalog Completeness) is complete. The query engine is now production-grade. Phase 8.1a brings it to web apps and browser-hosted AI agent frameworks (LangChain.js, etc.).
The wasm32-unknown-unknown target has no filesystem access, so FileBackend cannot be used. Storage must go through IndexedDB. The optimizer.rs module is already gated behind a wasm feature flag — do not remove that gate.
Architecture: Page-granular IndexedDB backend
Do not implement a single-blob approach. Storing the entire .graph file as one IndexedDB blob causes unacceptable write amplification:
| Scale | File size | Single-blob cost per checkpoint |
|---|---|---|
| 10K facts | ~1.6 MB | 1.6 MB |
| 100K facts | ~16 MB | 16 MB |
| 1M facts | ~160 MB | 160 MB |
The correct design maps directly onto the existing StorageBackend trait:
IndexedDB object store: { key: page_id (u64), value: page_bytes (4 KB Uint8Array) }
read_page(id)→ IndexedDBget(page_id)— fetches one 4 KB recordwrite_page(id, bytes)→ IndexedDBput(page_id, bytes)— only dirty pages written- The existing LRU page cache sits in front of
StorageBackend; hot pages never reach IndexedDB
This is not a violation of the single-file philosophy. Logically it is still one database. Physically, IndexedDB is a key-value store, not a filesystem — storing pages as records is the right abstraction.
Tasks
1. IndexedDbBackend implementation (src/storage/backend/indexeddb.rs)
- Create
src/storage/backend/indexeddb.rs(the file does not yet exist — the roadmap anticipated it but it was never written) - Implement
StorageBackendforIndexedDbBackendusingweb-sysandwasm-bindgen - All IndexedDB operations are async; bridge them correctly into Rust (use
wasm-bindgen-futures) - The object store name should default to
"minigraf"but be configurable viaOpenOptions(new field:db_name: Option<String>) - Gate the entire file behind
#[cfg(target_arch = "wasm32")]and abrowserfeature flag
2. Public API annotations
- Annotate
Minigraf,WriteTransaction,OpenOptions, andexecute/prepare/checkpointwith#[wasm_bindgen] - Gate all
#[wasm_bindgen]annotations behind thebrowserfeature flag — do not pollute native builds - Map
anyhow::ErrortoJsValuefor the WASM boundary; do not panic across FFI - All public methods that return
Resultmust returnResult<T, JsValue>at the WASM boundary
3. Export / import API
Add two #[wasm_bindgen] functions (not methods — standalone JS exports):
/// Serialise all IndexedDB pages into a portable .graph blob (Uint8Array).
/// Reads all page_id records in order and concatenates.
pub async fn export_graph(db_name: &str) -> Result<Vec<u8>, JsValue>;
/// Load a .graph blob into IndexedDB, splitting it into 4 KB pages.
/// Overwrites existing pages.
pub async fn import_graph(db_name: &str, data: &[u8]) -> Result<(), JsValue>;4. Feature flags and Cargo.toml
Add to Cargo.toml:
[features]
browser = ["wasm-bindgen", "web-sys", "wasm-bindgen-futures", "js-sys"]
[target.'cfg(target_arch = "wasm32")'.dependencies]
wasm-bindgen = "0.2"
wasm-bindgen-futures = "0.4"
web-sys = { version = "0.3", features = ["IdbDatabase", "IdbFactory", "IdbIndex", "IdbObjectStore", "IdbOpenDbRequest", "IdbRequest", "IdbTransaction", "IdbTransactionMode", "Window"] }
js-sys = "0.3"Do not add these as unconditional dependencies — they must not affect native binary size.
Binary size budget: target < 1 MB gzipped for the .wasm artifact. Run wasm-opt (via wasm-pack) and audit any dependency that bloats size.
5. Threading
Standard browser WASM has no threads. Audit every use of Arc<Mutex<...>>, RwLock, and thread_local! in the code paths reachable from WASM. Either:
- Replace with single-threaded equivalents (
Rc<RefCell<...>>) under#[cfg(target_arch = "wasm32")], or - Confirm the existing code is already safe (no
Sendbounds crossing the WASM boundary)
6. Build toolchain and CI
Install wasm-pack and add a CI job:
wasm-browser:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown
- run: cargo install wasm-pack
- run: wasm-pack build --target web --features browser
- run: wasm-pack test --headless --chrome --features browser7. JS/TS integration example
Add examples/browser/ containing:
index.html— minimal page that loads the WASM moduleapp.ts— imports@minigraf/core, opens a database, runs atransact+query, logs resultsREADME.md— how to build and serve locally
8. Tests
- Unit tests for
IndexedDbBackendusingwasm-bindgen-testand a headless Chrome runner - Test: open → transact → query → close → reopen → query (persistence round-trip)
- Test: export → import round-trip (binary identity of the
.graphblob) - Test: concurrent reads (single-threaded browser model — ensure no deadlock)
Constraints
- No filesystem access; all storage via IndexedDB
- No
std::thread— browser WASM is single-threaded - Must compile cleanly with
wasm-pack build --target web --features browser - Must not break native (
cargo test) or WASI (cargo build --target wasm32-wasip1) builds - TypeScript
.d.tsis auto-generated bywasm-pack— do not write it manually; instead ensure the Rust API surface is clean enough that the generated types are ergonomic
Deliverable
wasm-pack build --target web --features browser produces a pkg/ directory containing:
minigraf_bg.wasm(< 1 MB gzipped)minigraf.js(JS glue)minigraf.d.ts(TypeScript definitions)package.jsonwith name@minigraf/core
Testing sign-off
The reviewer (repo maintainer) will:
- Run
wasm-pack build --target web --features browserlocally and inspect binary size - Open
examples/browser/index.htmlin Chrome and confirm the demo works - Confirm
cargo test(native) still passes with no regressions - Confirm CI passes (wasm-browser job green)
Activity