Repository navigation
Flush credential stores to disk before returning - #207
Merged
Merged
Conversation
The account and auth token stores persist with mkstemp, write, and os.replace. That is atomic against a crash mid-write, but not durable: the rename can reach the directory entry while the contents are still in the page cache, so power loss or an abrupt host termination can leave a truncated or empty users.json on an instance that still holds accounts. Losing the credential store locks every user out permanently, reopens first-run setup on populated storage, and lets the next caller claim the instance. Add app/services/durable_write.py, which fsyncs the temp file before the replace and the containing directory after it, and route both credential stores through it. Directory fsync is POSIX only: Windows cannot open a directory and some filesystems reject the fsync, so that flush degrades silently rather than failing a write that has already landed. Locking, atomicity, and the on-disk format are unchanged. The added cost is a near-constant 0.06 to 0.22ms per save, independent of file size. At 4,471 records a save goes from 24.9ms to 25.1ms, 0.9%. Signed-off-by: Jason Madigan <jason@jasonmadigan.com>
get_auth_secret writes {storage}/auth_secret once, at first boot, and never
again. It has the same durability hole as the credential stores: without an
fsync the file can be lost to power loss, and every stored second factor
becomes undecryptable because the key that derives from it is gone.
Different write shape from the stores, so it does not use
write_json_atomically, but it reuses the same directory flush. The O_EXCL
create race that makes concurrent first boots safe is untouched.
Signed-off-by: Jason Madigan <jason@jasonmadigan.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The account store, the auth token store and the auth secret file were written with an atomic rename but never flushed. That is safe against a crash mid-write, but not against power loss or an abrupt host termination: the rename can reach the directory entry while the contents are still in the page cache.
Losing the account store locks every user out of an instance that still holds their data, and reopens first-run setup on it. Losing the auth secret makes every stored second factor undecryptable. Both are worth a flush.
The temp file is now flushed before the rename and the containing directory after it, since the rename itself can otherwise be lost to the same failure. The directory flush is POSIX-only and degrades quietly where it is unavailable rather than failing a write that has already landed.
The ordinary data stores use the same pattern and should probably follow, but as a separate change: the cost is identical, the consequence is not. A lost tool store costs re-tracing; a lost credential store costs the instance.
Measured cost is near-constant and independent of file size: +0.221ms on a save that takes 24.9ms with 4,471 records, about 0.9%.
742 tests, up from 728. The new ones prove the flush by inode identity rather than by counting calls: the recorded descriptors are matched against the file's post-rename inode and the storage directory's.