[go: up one dir, main page]

Metadata control plane for agent workspaces

Durable agent workspaces.

NoKV publishes crash-consistent, versioned workspace state over object storage, with path-sharded metadata for small-file scale-out.

The navigation tax

Serverless agent workers still need durable boundaries.

Workers restart, fan out, and share infrastructure. Enterprise workspaces need tenant isolation, authenticated access, and stable, version-pinned reads while other agents write.

Today: pinned snapshot views, shard-local crash-atomic publication, and CoW fork-to-restore.

Next: enforced tenant identity, policy, and live-workspace freeze.

Animated demo: a small robot agent works away, emitting scattered run state — a config.yaml, params.json, two object-store keys, two database rows, and an outputs folder — which then converges into one NoKV namespace under /workspace/runs. The namespace then stays live: run 0143 publishes new artifacts as typed events, a snapshot pins a frozen view, and an agent answers find and grep calls with line-numbered evidence.

Scattered

state scatters across folders, JSON files, object-store keys, and database rows

  • yaml ~/experiments/run_0142/config.yaml
  • json params.json {"lr": 3e-4, "batch": 256}
  • s3 s3://ml-artifacts/ckpt/run0142/best_model_1.0_1.pt
  • s3 s3://ml-artifacts/logs/run0142/stdout.txt
  • dbrow runs(id=0142, status='completed', val_loss=0.418)
  • dbrow metrics(run=0142, step=12000, val_loss=0.418)
  • folder outputs/2026-05-30/metrics.csv
One namespace

NoKV gives that state one address

/workspace/runs/0142
├── config.yaml
├── metadata.json
└── artifacts/
    ├── stdout.txt
    ├── stderr.txt
    └── ckpt/best_model_1.0_1.pt
Alive

indexed, queryable, watchable, snapshot-able

  • PublishArtifact /workspace/runs/0143/artifacts/ckpt/best_model_1.1_0.pt
  • Create /workspace/runs/0143/metadata.json
  • Snapshot pinned /workspace/runs (id: 7)

What NoKV is

A filesystem to your agents. A metadata engine underneath.

To your tools and agents, NoKV looks like a filesystem: paths, folders, and files you can mount, list, and read. Underneath, file bodies live as immutable blocks in your S3-compatible object store. NoKV's built-in metadata engine keeps what exists, where, and in which version transactional, queryable, and snapshot-able. No separate database to run.

agents FUSE SDK CLI NoKV namespace metadata truth, snapshots, events (built-in engine) your object store immutable blocks (S3 / MinIO / RustFS / Ceph)

Shard the namespace. Keep each commit crash-atomic.

NoKV routes path ranges to independent Holt-backed owners, so small-file metadata work can scale in parallel without turning Holt into a distributed database.

Path-routed shards
Longest-prefix routing splits hot workspace trees
Embedded Holt
Independent metadata work inside each shard
Crash-atomic publish
One durable metadata command per shard
Explore Holt
Small-file scale-out Parallel metadata paths for RAGFS
RAGFS / agents / SDK filesystem-shaped small-file requests
NoKV path router longest-prefix route to shard owner
/workspaces/a/* Namespace shard 01
embedded Holt crash-atomic commit
/workspaces/b/* Namespace shard 02
embedded Holt crash-atomic commit
/runs/* Namespace shard 03
embedded Holt crash-atomic commit
immutable file bodies S3-compatible object storage
Qualification boundary

Current main supports path sharding with one active writer per shard. This is horizontal sharding, not consensus replication. Multi-machine hardening and enterprise small-file throughput qualification against partner workloads remain in progress.

Write-path primitives

Built for how AI work actually writes.

Publication is crash-atomic.

Each shard applies a publication as one durable metadata command. Pin a snapshot when a workflow needs a stable multi-read view.

Snapshots are time travel.

Pin a frozen view of any subtree and keep reading it while jobs write. GC never deletes what a snapshot still needs.

Changes are events, not polls.

Every create, rename, and publish lands as a typed, replayable event with a cursor.

Quickstart

Run it in five minutes.

bash
# 0. Prereqs: Rust toolchain (1.88+); RustFS (or any S3-compatible store); macFUSE on macOS for mount
# 1. Build
cargo build --release -p nokv --bin nokv

# 2. Local S3 (RustFS) + create the default bucket
mkdir -p /tmp/rustfs-data
RUSTFS_ACCESS_KEY=rustfsadmin RUSTFS_SECRET_KEY=rustfsadmin \
  rustfs server --address 127.0.0.1:9000 /tmp/rustfs-data &
AWS_ACCESS_KEY_ID=rustfsadmin AWS_SECRET_ACCESS_KEY=rustfsadmin \
  aws --endpoint-url http://127.0.0.1:9000 s3api create-bucket --bucket nokv

# 3. Start the metadata server (every client command talks to 127.0.0.1:7777)
cargo run --release -p nokv --bin nokv -- serve &

# 4. Use it
cargo run --release -p nokv --bin nokv -- init
cargo run --release -p nokv --bin nokv -- mkdir /runs
cargo run --release -p nokv --bin nokv -- put-artifact /runs/1/ckpt.bin ./ckpt.bin
cargo run --release -p nokv --bin nokv -- cat /runs/1/ckpt.bin > restored.bin
cargo run --release -p nokv --bin nokv -- snapshot /runs   # time travel starts here

Status: single-node remains the default. Experimental multi-shard support is available on current main; multi-machine production hardening and the POSIX surface remain in progress.