Repository navigation
[BUG] DBFS move silently overwrites an existing destination file #34
Description
Description
Moving onto an existing file succeeds and replaces its contents even though the DBFS API requires a RESOURCE_ALREADY_EXISTS error.
Reproduced on MiniLake 1.9.0, commit fe8837dcfac9df621dfabc440abc07ebb4cdbcbb, using the repository's Docker test image, a real HTTP server, and fresh disposable state. Image Python sources were verified byte-for-byte against this commit. No mocks or production data were used.
Affected Components
Steps to Reproduce
Use a disposable instance and data directory. Paths below are relative to its HTTP base URL; bodies are JSON unless explicitly described as raw bytes. IDs shown as W/J/R/Q/C/HANDLE are replaced with the IDs returned by prior calls.
POST /api/2.0/dbfs/putwith{"path":"/source","contents":"bmV3","overwrite":true}.- PUT via the same endpoint
{"path":"/destination","contents":"b3JpZ2luYWw=","overwrite":true}. POST /api/2.0/dbfs/movewith{"source_path":"/source","destination_path":"/destination"}.- Read
/destinationusingGET /api/2.0/dbfs/read?path=/destinationand decode its base64 content.
Expected Behavior
The move fails with RESOURCE_ALREADY_EXISTS. Source remains new, destination remains original.
Actual Behavior
The move returns 200, destination becomes new, and source no longer exists.
Root Cause Analysis
move_path calls Path.rename(destination) without a destination-exists guard. On the reproduced Linux filesystem, rename replaces an existing file.
Impact
Existing user files are lost through a normal API call with no overwrite option or explicit consent.
Severity is assessed within MiniLake's documented local development/CI use; it does not assume a shared production Databricks deployment.
Suggested Fix
Reject existing destination paths and use a no-replace move strategy that remains safe against interleaving writes. Preserve the source on failure.
Test Coverage Gaps
test_dbfs_move covers only a nonexistent destination. Add existing file and directory destinations and assert both sides remain unchanged.
Severity
High
Confidence
High — reproduced through the real HTTP API and traced to the source paths above.
Recommended Labels
bug, high-priority
Existing Issue Matches
No reasonable duplicate found. Reviewed all open and closed issues and open pull requests in dmux/minilake, then searched component/function/symptom terms for this finding. The only open PR at review time, #29, changes the virtualenv lock entry and does not address this defect. The documented intentional limitations were checked before reporting.
API Reference
DBFS move contract specifies RESOURCE_ALREADY_EXISTS for an existing destination.
Activity