8000 Phase 8.1a: Browser WASM support (wasm32-unknown-unknown + wasm-bindgen) · Issue #129 · project-minigraf/minigraf · GitHub
[go: up one dir, main page]

Skip to content

Phase 8.1a: Browser WASM support (wasm32-unknown-unknown + wasm-bindgen) #129

Closed

Description

@adityamukho

Goal

Ship Minigraf as an npm package (@minigraf/core) that runs natively in browser environments using WebAssembly, with full TypeScript types and a page-granular IndexedDB storage backend.

Background

Phase 7 (Datalog Completeness) is complete. The query engine is now production-grade. Phase 8.1a brings it to web apps and browser-hosted AI agent frameworks (LangChain.js, etc.).

The wasm32-unknown-unknown target has no filesystem access, so FileBackend cannot be used. Storage must go through IndexedDB. The optimizer.rs module is already gated behind a wasm feature flag — do not remove that gate.

Architecture: Page-granular IndexedDB backend

Do not implement a single-blob approach. Storing the entire .graph file as one IndexedDB blob causes unacceptable write amplification:

Scale File size Single-blob cost per checkpoint
10K facts ~1.6 MB 1.6 MB
100K facts ~16 MB 16 MB
1M facts ~160 MB 160 MB

The correct design maps directly onto the existing StorageBackend trait:

IndexedDB object store: { key: page_id (u64), value: page_bytes (4 KB Uint8Array) }
  • read_page(id) → IndexedDB get(page_id) — fetches one 4 KB record
  • write_page(id, bytes) → IndexedDB put(page_id, bytes) — only dirty pages written
  • The existing LRU page cache sits in front of StorageBackend; hot pages never reach IndexedDB

This is not a violation of the single-file philosophy. Logically it is still one database. Physically, IndexedDB is a key-value store, not a filesystem — storing pages as records is the right abstraction.

Tasks

1. IndexedDbBackend implementation (src/storage/backend/indexeddb.rs)

  • Create src/storage/backend/indexeddb.rs (the file does not yet exist — the roadmap anticipated it but it was never written)
  • Implement StorageBackend for IndexedDbBackend using web-sys and wasm-bindgen
  • All IndexedDB operations are async; bridge them correctly into Rust (use wasm-bindgen-futures)
  • The object store name should default to "minigraf" but be configurable via OpenOptions (new field: db_name: Option<String>)
  • Gate the entire file behind #[cfg(target_arch = "wasm32")] and a browser feature flag

2. Public API annotations

  • Annotate Minigraf, WriteTransaction, OpenOptions, and execute/prepare/checkpoint with #[wasm_bindgen]
  • Gate all #[wasm_bindgen] annotations behind the browser feature flag — do not pollute native builds
  • Map anyhow::Error to JsValue for the WASM boundary; do not panic across FFI
  • All public methods that return Result must return Result<T, JsValue> at the WASM boundary

3. Export / import API

Add two #[wasm_bindgen] functions (not methods — standalone JS exports):

/// Serialise all IndexedDB pages into a portable .graph blob (Uint8Array).
/// Reads all page_id records in order and concatenates.
pub async fn export_graph(db_name: &str) -> Result<Vec<u8>, JsValue>;

/// Load a .graph blob into IndexedDB, splitting it into 4 KB pages.
/// Overwrites existing pages.
pub async fn import_graph(db_name: &str, data: &[u8]) -> Result<(), JsValue>;

4. Feature flags and Cargo.toml

Add to Cargo.toml:

[features]
browser = ["wasm-bindgen", "web-sys", "wasm-bindgen-futures", "js-sys"]

[target.'cfg(target_arch = "wasm32")'.dependencies]
wasm-bindgen = "0.2"
wasm-bindgen-futures = "0.4"
web-sys = { version = "0.3", features = ["IdbDatabase", "IdbFactory", "IdbIndex", "IdbObjectStore", "IdbOpenDbRequest", "IdbRequest", "IdbTransaction", "IdbTransactionMode", "Window"] }
js-sys = "0.3"

Do not add these as unconditional dependencies — they must not affect native binary size.

Binary size budget: target < 1 MB gzipped for the .wasm artifact. Run wasm-opt (via wasm-pack) and audit any dependency that bloats size.

5. Threading

Standard browser WASM has no threads. Audit every use of Arc<Mutex<...>>, RwLock, and thread_local! in the code paths reachable from WASM. Either:

  • Replace with single-threaded equivalents (Rc<RefCell<...>>) under #[cfg(target_arch = "wasm32")], or
  • Confirm the existing code is already safe (no Send bounds crossing the WASM boundary)

6. Build toolchain and CI

Install wasm-pack and add a CI job:

wasm-browser:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - uses: dtolnay/rust-toolchain@stable
      with:
        targets: wasm32-unknown-unknown
    - run: cargo install wasm-pack
    - run: wasm-pack build --target web --features browser
    - run: wasm-pack test --headless --chrome --features browser

7. JS/TS integration example

Add examples/browser/ containing:

  • index.html — minimal page that loads the WASM module
  • app.ts — imports @minigraf/core, opens a database, runs a transact + query, logs results
  • README.md — how to build and serve locally

8. Tests

  • Unit tests for IndexedDbBackend using wasm-bindgen-test and a headless Chrome runner
  • Test: open → transact → query → close → reopen → query (persistence round-trip)
  • Test: export → import round-trip (binary identity of the .graph blob)
  • Test: concurrent reads (single-threaded browser model — ensure no deadlock)

Constraints

  • No filesystem access; all storage via IndexedDB
  • No std::thread — browser WASM is single-threaded
  • Must compile cleanly with wasm-pack build --target web --features browser
  • Must not break native (cargo test) or WASI (cargo build --target wasm32-wasip1) builds
  • TypeScript .d.ts is auto-generated by wasm-pack — do not write it manually; instead ensure the Rust API surface is clean enough that the generated types are ergonomic

Deliverable

wasm-pack build --target web --features browser produces a pkg/ directory containing:

  • minigraf_bg.wasm (< 1 MB gzipped)
  • minigraf.js (JS glue)
  • minigraf.d.ts (TypeScript definitions)
  • package.json with name @minigraf/core

Testing sign-off

The reviewer (repo maintainer) will:

  1. Run wasm-pack build --target web --features browser locally and inspect binary size
  2. Open examples/browser/index.html in Chrome and confirm the demo works
  3. Confirm cargo test (native) still passes with no regressions
  4. Confirm CI passes (wasm-browser job green)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestrustPull requests that update rust code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0