FUSE filesystem exposing a virtual disk as virtual.dat. Backing files grow as data
is written and can be split to fit FAT32.
dynfilefs -f storage_file -m mount_dir [-s size_MiB] [-p split_MiB] [-d] [--restore NAME] [--protect ID] [--no-auto-snapshots]-f: backing metadata file; data parts use.0,.1, etc.-m: empty mount directory.-s: virtual size, required for new storage. Larger values grow existing storage;+Nadds N MiB. Smaller sizes are ignored.-p: maximum data-part size, including indexes. Defaults to 4088 MiB for FAT32; existing storage keeps its original setting.-d: foreground debug mode. See--helpfor details.-i,--info: inspect saved geometry and reusable capacity without mounting or changing it.--snapshots: inspect the saved history tree before mounting.--json: JSON output for--infoor--snapshots.--tsv: read-only--snapshotsoutput for a selectable tree. Each UTF-8 row has five tab-separated fields: stable ID, name, UTC date, compact lane number and nearest visible parent ID (empty for a root). Parents precede children; linear histories stay in one lane and side branches are indented. The current branch comes last. Deleted ancestors are collapsed while their descendants remain visible.--hide ID:NAME(repeatable, with--tsv): collapse only a matching ID/name pair. Missing or renamed points are unaffected. This changes presentation only. Names containing tabs or line breaks are rejected by this format. JSON and ordinary tree output remain complete and unchanged.--restore NAMEor-o restore=NAME: start a new mounted session from a named snapshot.--no-auto-snapshots: start with automatic startup/end points disabled. Explicit snapshots and normal write synchronization remain available; enable automatic end points later by writingautosnapshot ontosnapshot.--protect IDor-o protect=ID(repeatable): retain an exact live 16-digit lowercase hexadecimal snapshot ID throughout this mount, including recovery, automatic pruning and clean unmount. Deletion returnsEBUSY. Missing/deleted IDs fail instead of selecting a reused name. This does not restore the image or persist a pin: a caller with a durable pending request must supply the option again on subsequent mounts. The storage format is unchanged. While mounted, writeprotect IDorunprotect IDtosnapshotto change protection. Multiple IDs can be protected: protect a new target before committing a changed selection, then unprotect the old target only after the caller's metadata is durable. This closes the selection-update crash window without requiring a clean shutdown hook. Runtime protection survives the caller's exit; it ends when DynFileFS itself closes the store. Live catalog entries exposeprotected; offline entries report false.
dynfilefs -i -f /var/lib/changes.datPrints key:value lines: version, size, sizeMiB, split, splitMiB,
blockSize, reusableBytes, currentParent, automaticSnapshots and currentMatchesParent.
Sizes without MiB are exact bytes;
MiB values are integers rounded down. currentParent is a stable 16-digit hexadecimal
snapshot ID, or none; JSON uses a string or null. Inspection includes the latest
complete checkpoint/log records without publishing recovery or creating a lock.
automaticSnapshots and currentMatchesParent are runtime values, so offline inspection returns null
in JSON or none in text output.
Live currentMatchesParent is true only when the synchronized current mapping,
geometry and live allocation bitmap equal the immutable parent. Unsynchronized
writes report false. This compares metadata, not payload contents, and does not
flush writes or recover an inner filesystem.
An actively mounted store returns EBUSY: use its live stats and snapshots.json
files instead. Errors go to stderr with a nonzero exit status.
Create a new 1 GiB disk:
mkdir -p /mnt/changes /mnt/data
dynfilefs -f /var/lib/changes.dat -m /mnt/changes -s 1024
mke2fs -F /mnt/changes/virtual.dat # Format only when creating a new disk.
mount -o loop /mnt/changes/virtual.dat /mnt/dataTo reuse it, run the mount commands without formatting. Unmount the inner filesystem first:
umount /mnt/data
umount /mnt/changesEquivalent /etc/fstab entries:
/var/lib/changes.dat /mnt/changes dynfilefs size=1024,split=4088 0 0
/mnt/changes/virtual.dat /mnt/data auto loop 0 0
With -f /var/lib/changes.dat, files in /var/lib can include:
changes.dat Metadata
changes.dat.0 Data and index pages, up to the configured split size
changes.dat.1 Further parts, created as needed
changes.dat.snapshots/ Named snapshot metadata, sharing the numbered data parts
0000000000000001.snap
0000000000000002.snap
0000000000000003.snap
changes.dat.lock Mount lock; removed after clean unmount
changes.dat.log Metadata log, up to 32 MiB
changes.dat.work Compacted checkpoint
changes.dat.migrate Temporary conversion marker
changes.dat.txn Checksummed metadata redo record; retain with the store
changes.dat.txn.lock Permanent metadata-transaction lock
Only needed files are created. Clean unmount leaves the main metadata file, numbered data parts, snapshot directory and permanent transaction companions. Published metadata and numbered parts retain their lengths and allocation for reuse. Other crash-recovery files remain until a successful next mount.
Keep the main metadata file, all numbered parts, the snapshot directory and .txn
together. A committed redo can be the only intact copy of the latest metadata.
Snapshots share backing blocks and are not independent backups.
Mounted stores retain directory handles instead of relying on their original absolute
paths. Writes, snapshot creation and clean unmount therefore keep working across
pivot_root, including when the store supplies a live system's writable layer.
Automatic filesystem freezing still requires accessible /proc, /sys and /dev
in the daemon's current root and an unchanged mount/device topology during creation.
There is one current state and a history of immutable snapshots. A normal mount continues the current state. At startup, dynfilefs ensures that state has a saved point; at clean shutdown it saves another point only if the state changed. The end of one session is therefore reused as the beginning of the next. Empty sessions do not create duplicate points. Unmount the inner filesystem before dynfilefs so a clean-end point includes the filesystem's clean shutdown.
Automatic names have the reserved auto- prefix, a UTC timestamp and a unique ID.
The ten newest automatic live points are retained; open points and the current
parent are protected, so the limit may temporarily be exceeded. Manually named
points remain until explicitly deleted. Automatically removed points retain small
history records but no references to their former data blocks.
Automatic clean-end points can be disabled for the current mount:
echo 'autosnapshot off' > /mnt/changes/snapshot
cat /mnt/changes/stats
echo 'autosnapshot on' > /mnt/changes/snapshotLive stats reports the boolean automaticSnapshots. It defaults to true unless
the mount uses --no-auto-snapshots. Disabling it suppresses the automatic clean-end point and automatic
pruning at shutdown; explicit named creation remains available. Working writes
still reach their normal durable checkpoints and clean shutdown still flushes
and publishes the current state. This switch does not erase changes or implement
discard by itself: an ordinary later mount continues that working state and can
save its startup point. A caller implementing discard must persist its own
baseline choice and explicitly restore that point on the next mount, including
after a crash. The runtime switch does not change the storage format or the
normal ten-point retention policy when automatic point creation is enabled.
Applications recovering a filesystem before preserving a pending restore can
mount with --no-auto-snapshots, recover and cleanly unmount the inner filesystem,
then use currentMatchesParent to reuse an identical point or create a new one.
After interruption, the current state recovers the last successful fsync. No
clean-end point is created by the interrupted process. Startup can save the recovered
block state, which may still need the inner filesystem's ordinary journal replay.
Such a point is crash-consistent, not necessarily suitable for a no-recovery read-only
filesystem mount. To recover its filesystem, restore it as the current writable disk
and mount the inner filesystem normally; the saved point itself remains unchanged.
Before mounting, inspect the tree or get records for an application:
dynfilefs -f /var/lib/changes.dat --snapshots
dynfilefs -f /var/lib/changes.dat --snapshots --jsonJSON contains currentParent and a snapshots array. Each node has id, parent,
name, created (Unix seconds), deleted, automatic, protected and reclaimableBytes. IDs are strings so they
remain exact in JavaScript. The tree marks the snapshot on which the current state
is based; it does not mean that unsaved current writes equal that snapshot.
Restoring B and creating D preserves C and records a new branch:
A
└── B
├── C
└── D
Deleting B keeps its stable ID, parent and display name as a history-only node. C and D keep their original ancestry. B's saved content becomes unavailable, and its exclusive blocks can be reused. The display name B can later be used for a different point with a new ID. Historical nodes do not pin data blocks.
echo 'create before-update' > /mnt/changes/snapshot
cat /mnt/changes/snapshot
ls /mnt/changes/snapshots/Reading snapshot returns one saved name per line. Each write must contain one complete
create NAME, rename ID NAME, delete NAME, protect ID, unprotect ID,
autosnapshot on or autosnapshot off command, with an optional final newline. Names may contain
international text and spaces; shell-quote them as needed. Names cannot end in a dot or
space, contain control characters, path separators or FAT-incompatible punctuation,
or exceed 250 UTF-8 bytes. The auto- prefix is reserved for system-created points.
Creation refuses an existing live name. Each saved point has its own complete mapping
and allocation metadata at
changes.dat.snapshots/ID.snap; its UTF-8 name is stored inside that checksummed
record. Unchanged data blocks are shared. Deleted points have a logical 4096-byte
history record with no mapping or allocation bitmap; their metadata file keeps its
previous allocation to avoid freeing a published FAT chain during an interrupted update.
rename ID NAME changes a live point's display name without changing its stable
16-digit lowercase hexadecimal ID, creation time, ancestry or saved content.
Names follow the same rules as creation and cannot replace another live name.
Image paths use stable IDs, so open descriptors and mounted views remain valid
while a point is renamed. Rename does not freeze the inner filesystem.
Giving an automatic point a user name makes it a manual point, retained until
explicitly deleted. Existing protection remains attached to its unchanged ID.
Manual creation automatically freezes a directly loop-mounted ext4 or XFS filesystem using
virtual.dat, saves the durable snapshot and resumes the filesystem. Detection uses
the backing file's device/inode identity and mounts visible in dynfilefs's mount
namespace. It requires permission to freeze the filesystem, normally root. The helper
also resumes its owned freezes if dynfilefs exits unexpectedly. A loop with partitions,
device-mapper users, an unsupported filesystem or no accessible direct mount makes
creation fail. Keep mount and device topology unchanged during creation.
With no matching loop device, a raw virtual-disk snapshot is allowed;
the caller must coordinate any other writers or users not covered by direct-loop
detection. Concurrent snapshot commands return EBUSY while another command runs.
To return to a saved point, first unmount the inner filesystem and dynfilefs, then:
dynfilefs -f /var/lib/changes.dat -m /mnt/changes --restore before-update
mount -o loop /mnt/changes/virtual.dat /mnt/dataThe saved point remains unchanged; new writes form the current state starting from it.
Other named points remain available; restoring never deletes newer points. Restore
selects the saved virtual size, including a smaller size than the previous current
state; -s can grow it as usual. Current
changes that must remain available should be saved as another named point first.
Saved images are available only at snapshots/ID.dat, using each point's exact
16-digit lowercase hexadecimal ID from snapshots.json. Human-readable names
are catalog metadata, not image paths. IDs stay fixed through rename, while the current disk stays
writable. These files are immutable virtual views, not additional physical disk copies.
For a snapshot created while the inner filesystem was frozen (replace the sample ID):
mkdir -p /mnt/old-state
mount -t xfs -o loop,ro,nouuid,norecovery \
/mnt/changes/snapshots/0000000000000002.dat /mnt/old-state
# For ext4, use: mount -t ext4 -o loop,ro,noload ...
umount /mnt/old-state
echo 'delete before-update' > /mnt/changes/snapshotAn open snapshot image, including one held by a loop mount, cannot be deleted (EBUSY).
After closing it, deletion releases blocks needed only by that point for future writes
from any restored state. Blocks still used by the current state, another snapshot,
a durable crash checkpoint or outstanding descriptor-backed reads remain protected.
Deletion does not freeze the inner filesystem. There is no data compaction, hole
punching or physical space reclamation
of data parts; freed slots are reused within the existing numbered files.
dynfilefs -f /var/lib/changes.dat --info --json
cat /mnt/changes/stats
cat /mnt/changes/snapshots.jsonreusableBytes counts physical 4096-byte slots in the existing numbered files that
the allocator can immediately use. It excludes every block still needed by the
current state, live snapshots, the durable crash checkpoint or a pending
descriptor-backed read. Live stats reflects unsynced allocations too; offline
inspection requires an unmounted store and reports capacity available for its next
mount. Legacy format-500 inspection reports zero until conversion.
Live snapsh
73AB
ots.json includes reclaimableBytes for each point: the additional
allocator capacity made reusable by deleting that point alone. Shared blocks and
blocks still needed by current writes, crash recovery or descriptor-backed reads
are excluded. Protected or open points may still show a value, but cannot be
deleted until their protection or open handles are released. Deleted points show
zero. Offline inspection and unavailable measurements report null. A failed
measurement is retried after the next point is created or deleted.
The first JSON history read after mounting, creating or deleting a point reads snapshot metadata twice and caches sparse ownership masks. Later reads count those masks in RAM, without scanning data files or freezing the inner filesystem. The value can change as current writes or checkpoints release old blocks. It measures reusable data-part slots, not physical file shrinking or metadata-file allocation.
The allocator uses existing free slots before extending a data part or creating a
new one. Index pages and copy-on-write replacements consume the same reserve, so
reusableBytes is not an exact allowance for new files inside XFS/ext4. Main metadata,
the log and new snapshot records may allocate space independently. Sparse or
copy-on-write host filesystems can also allocate storage while overwriting an existing
part range. No data compaction, hole punching or physical shrinking is performed.
Metadata updates flush a complete checksummed intent to the permanent .txn file
before overwriting the canonical file. The next writer finishes a valid intent;
offline readers expose it without changing the medium. A torn new intent leaves
the previous canonical record available. Published metadata is never replaced by
rename or truncated. Ordinary data synchronization still appends incremental log records.
Recover the backing filesystem before resuming writes after power loss. FAT has no
journal: an interrupted allocation can leave a chain longer than its directory
entry and make a later extension fail or turn the volume read-only. For an unmounted
FAT volume, use its normal fsck.fat recovery first. Metadata redo does not replace
backing-filesystem repair or make unreliable hardware flushes safe.
Integrators can use the same protocol for small policy files:
dynfilefs --metadata /path/to/.policy write state.txt first-line second-line
dynfilefs --metadata /path/to/.policy read state.txt
dynfilefs --metadata /path/to/.policy clear state.txt
dynfilefs --metadata /path/to/.policy recoverNames are relative to the pinned containing directory. Every writer and reader
must use the same prefix; do not edit these files through shell redirection.
mark NAME TIMESTAMP records an empty timestamp marker. Keep policy prefixes
separate from the virtual-disk prefix.
Format 500 converts automatically on its first mount, without moving payload data. Legacy split sizes must be an even number of MiB. The resulting format 600 records stable snapshot identities and parent links and cannot be opened by older versions. Version 6.03 keeps the format-600 payload but adds redo recovery and padded canonical records. Do not reopen a 6.03-managed store with 6.02 or an older binary.
Requires libfuse 3.8+ development headers, pkg-config and autotools:
./autogen.sh
./configure
makeFor a static Linux i686 binary at static/dynfilefs:
sh static/build.shThe static build requires an x86 Linux host, make, curl, tar, gzip, xz, sha256sum,
meson, ninja and pkg-config. JOBS=2 limits parallel builds; PACK=0 skips UPX.
Run the snapshot integration checks on Linux with FUSE, loop-mount privileges and e2fsprogs; XFS checks also require xfsprogs:
python3 tests/snapshots.py /absolute/path/to/dynfilefsUse --skip-loop to run only the raw-image and catalog checks.
Test directory renames and real root transitions with a C compiler, unshare and both filesystem toolsets (the test isolates its mounts and processes):
python3 tests/pivot-root.py /absolute/path/to/dynfilefsTests and benchmarks · Stable releases
Development version: 6.05, storage format 600.