A recoverable filesystem effect engine: the crash-safe substrate
A change to many files on disk ends finished, undone or halted with evidence kept, never silently half-done, though readers may see it partway. It records its plan before touching anything, so after a crash, a kill or an outside writer the disk alone shows which changes were its own. Domain-neutral, for programs that keep records as files: one host, cooperating processes, certified filesystems.
Authority: docs/records.md, docs/protocol.md, docs/durability.md
Part of verifiably: verifiably builds understanding of the world from small, reproducible data analyses: each one a precise question asked of real data, recorded so anyone can run it again, with the data leading and unverified claims kept out. Overview
atoms · nodes · refs · beliefs · science · autonomy
pip install verifiably-atomsLinux is complete; macOS fills in capability by capability (the macOS backend is not yet built). Every
mount is probed for the capabilities it supports (atomic_exchange,
noclobber_transfer, durable_publish, …) and only the effects it cannot satisfy are
refused; the crash-certified configurations are the CERTIFIED_ALLOWLIST, and every
other tuple fails closed.
atoms knows nothing about records, claims or science. Its input is a TransactionSpec:
the files a change expects to find and the files it leaves, over five filesystem effects.
Any Python program that keeps its state as files can submit one. It is not ACID: the
files of one change are not made visible to other readers at once, and a reader can see
some of them changed and others not (docs/protocol.md). What the engine guarantees is the
outcome after an interruption: recovery finishes a change that durably committed, undoes
one that had not, and halts without overwriting when the disk holds state it cannot
attribute to itself. A halt can leave a change partly applied; the engine keeps the
evidence and refuses further work on that root until the state is resolved.
nodes is the logical side of the family: a corpus of typed markdown records and the write plans that change it. It decides what a write is and runs each plan through an executor it is handed; its own default executor writes best-effort. beliefs is where the two meet: its composition root depends on both and hands nodes an executor that compiles each write plan into one atoms transaction. Each arrow is a package dependency:
beliefs
/ \
v v
nodes atoms
nodes and atoms do not depend on each other, and nodes keeps its TypeScript implementation, which atoms has none of. A shared or cross-language way to reach the engine (docs/durability.md) would answer a consumer that needs one; nodes does not need it to be useful.
atoms.coordinator.commands is the public surface. A project root is registered once;
run_transaction then executes a compiled spec against it and converges after a crash
in a fresh process. The engine serializes the processes on one host that go through it;
a sync client such as Dropbox is an outside writer it does not coordinate: where the
disk shows its change, the transaction refuses or halts rather than overwrite it, and
two hosts changing one synced tree is unsupported (docs/protocol.md). Run
this from a directory on a certified ext4 volume (any other configuration fails closed
with CapabilityUnavailable, by design):
import hashlib, io, tempfile
from pathlib import Path
from atoms.coordinator.commands import register_root, run_transaction
from atoms.core.effects import CreateFileNoClobber
from atoms.core.fingerprint import ABSENT, FileState
from atoms.core.spec import build_spec
from atoms.fs.platform import select_backend
from atoms.fs.volume import StorageProfile
def sha(b): return "sha256:" + hashlib.sha256(b).hexdigest()
class Payloads: # a PayloadSource: a fresh stream per call, KeyError when unknown
def __init__(self, m): self.m = m
def open(self, digest): return io.BytesIO(self.m[digest])
work = Path(tempfile.mkdtemp(dir=Path.cwd())).resolve()
project, metadata = work / "project", str(work / "project.metadata")
(project / "d").mkdir(parents=True)
backend, storage = select_backend(), StorageProfile(profile_id="flush-honoring-disk.v1")
register_root(backend, str(project), metadata, storage, b"root", ())
body = b"after"; post = FileState(content_hash=sha(body), mode=0o644, byte_len=len(body))
spec = build_spec(consumer_tag="example", intent_digest=sha(b"example intent"),
initial_surface={"d/f.txt": ABSENT}, final_surface={"d/f.txt": post},
effects=[CreateFileNoClobber(effect_id="e1", path="d/f.txt", post=post)])
out = run_transaction(backend, str(project), metadata, storage, spec, Payloads({sha(body): body}))
print(out.outcome, (project / "d/f.txt").read_bytes()) # ChainOutcome.COMMITTED b'after'read_preimage(backend, project, metadata, storage, txid, path, max_bytes=...) then
returns the bytes a registered path held before a settled transaction.
python/— the package:atoms.core(the pure model),atoms.fs(capability backend, volume binding, project approval),atoms.store(the SQLite-in-WAL metadata store),atoms.chain(the tamper-evident project-local history),atoms.coordinator(lease, admission, preparation, executors, and the publiccommands), andpython/tools/certify(the QEMU crash-certification harness).docs/— the public reference documentation (records.md,protocol.md,durability.md,development.md).
- Records and On-Disk Formats —
TransactionSpec,PathState,.#~scratch naming,.#~chain/entries, and theatoms.dbSQLite WAL schema. - Transaction and Recovery Protocol — root lifecycle, leases,
run_transaction, automatic crash recovery,read_preimage, and error contracts. - Durability and Crash-Certification Standard — the eight filesystem capabilities, volume allowlist, SQLite-WAL probe, and QEMU power-cut matrix.
- Development and Verification Guide — setup, test execution policy, test volume configuration, hosted CI, and recertification workflow.
MIT, see LICENSE.