Repository navigation
[BUG] Full reset leaves tables and data in the default DuckDB database #39
Description
Description
POST /_minilake/reset?full=true reports a successful wipe but leaves unqualified SQL tables and their rows in the shared Unity Catalog database.
Reproduced on MiniLake 1.9.0, commit fe8837dcfac9df621dfabc440abc07ebb4cdbcbb, using the repository's Docker test image, a real HTTP server, and fresh disposable state. Image Python sources were verified byte-for-byte against this commit. No mocks or production data were used.
Affected Components
- src/minilake/admin.py —
reset_state - src/minilake/admin.py —
_wipe_data_dirs - src/minilake/duckdb_pool.py —
reset_all
Steps to Reproduce
Use a disposable instance and data directory. Paths below are relative to its HTTP base URL; bodies are JSON unless explicitly described as raw bytes. IDs shown as W/J/R/Q/C/HANDLE are replaced with the IDs returned by prior calls.
- Create warehouse W.
- POST a statement with W and
CREATE TABLE surviving_reset AS SELECT 777 AS secret. - POST
/_minilake/reset?full=true. - Create a fresh warehouse W2.
- Execute
SELECT * FROM surviving_reseton W2.
Expected Behavior
A full reset removes user tables/data, so the final query fails because the table does not exist.
Actual Behavior
Reset returns 200 with State reset successfully, wiped 5 data directories; the final SELECT still returns [[777]].
Root Cause Analysis
Unqualified SQL writes to unity_catalog.duckdb. Full reset wipes selected subdirectories but not that file, and DuckDBPool.reset_all closes only warehouse connections, retaining the shared UC connection and database.
Impact
Test fixtures and UI reset workflows can reuse stale data despite requesting a full wipe, yielding misleading test results and retaining data users intended to remove.
Severity is assessed within MiniLake's documented local development/CI use; it does not assume a shared production Databricks deployment.
Suggested Fix
On full reset, close the shared connection, remove/recreate its database and WAL, and reinitialize required engine state after catalog detachment. Keep non-full reset semantics explicit.
Test Coverage Gaps
Admin reset tests exercise registries/directories rather than a table created through unqualified SQL. Add a before/after query and restart after full reset.
Severity
High
Confidence
High — reproduced through the real HTTP API and traced to the source paths above.
Recommended Labels
bug, high-priority
Existing Issue Matches
No reasonable duplicate found. Reviewed all open and closed issues and open pull requests in dmux/minilake, then searched component/function/symptom terms for this finding. The only open PR at review time, #29, changes the virtualenv lock entry and does not address this defect. The documented intentional limitations were checked before reporting.
Activity