Repository navigation
[BUG] Deleting schemas leaves orphaned functions and volumes #48
Description
Description
Schema deletion removes table registries but retains function/volume records. Those children remain retrievable after their parent and physical SQL schema are gone; catalog deletion also omits functions.
Reproduced on MiniLake 1.9.0, commit fe8837dcfac9df621dfabc440abc07ebb4cdbcbb, using the repository's Docker test image, a real HTTP server, and fresh disposable state. Image Python sources were verified byte-for-byte against this commit. No mocks or production data were used.
Affected Components
- src/minilake/services/unity_catalog.py —
delete_schema - src/minilake/services/unity_catalog.py —
delete_catalog - src/minilake/services/unity_catalog.py —
get_function - src/minilake/services/unity_catalog.py —
get_volume
Steps to Reproduce
Use a disposable instance and data directory. Paths below are relative to its HTTP base URL; bodies are JSON unless explicitly described as raw bytes. IDs shown as W/J/R/Q/C/HANDLE are replaced with the IDs returned by prior calls.
- Create catalog
orphans. - POST
/api/2.1/unity-catalog/volumeswith{"catalog_name":"orphans","schema_name":"default","name":"v","volume_type":"MANAGED"}. - POST
/api/2.1/unity-catalog/functionswith{"function_info":{"name":"inc","catalog_name":"orphans","schema_name":"default","routine_body":"SQL","routine_definition":"x + 1","data_type":"INT","full_data_type":"INT","input_params":{"parameters":[{"name":"x","type_text":"INT","type_name":"INT"}]}}}. - DELETE
/api/2.1/unity-catalog/schemas/orphans.default?force=true. - GET
/api/2.1/unity-catalog/volumes/orphans.default.vand/api/2.1/unity-catalog/functions/orphans.default.inc.
Expected Behavior
Forced deletion consistently removes child registrations and applies each child type's storage cleanup rules. Deleted children cannot block recreation.
Actual Behavior
Both GETs return 200 with the old child records. The schema has been dropped, so the function metadata no longer corresponds to a callable macro.
Root Cause Analysis
The schema cascade iterates only tables/external_tables. The catalog cascade iterates schemas/tables/external_tables/volumes but omits functions. Volume directories are also adopted without verifying that their parent still exists.
Impact
Catalog listings and execution disagree; recreating a schema/function can collide with stale records. Managed volume data remains associated with a nonexistent parent.
Severity is assessed within MiniLake's documented local development/CI use; it does not assume a shared production Databricks deployment.
Suggested Fix
Centralize child cleanup for forced parent deletion, including functions, volume lifecycle, and related grants as appropriate. Ensure disk adoption cannot resurrect children beneath deleted parents.
Test Coverage Gaps
Deletion tests primarily cover table cleanup. Add populated schemas/catalogs containing functions and volumes, verify get/list absence, and recreate the same names. This is separate from the missing force safeguard.
Severity
Medium
Confidence
High — reproduced through the real HTTP API and traced to the source paths above.
Recommended Labels
bug
Existing Issue Matches
No reasonable duplicate found. Reviewed all open and closed issues and open pull requests in dmux/minilake, then searched component/function/symptom terms for this finding. The only open PR at review time, #29, changes the virtualenv lock entry and does not address this defect. The documented intentional limitations were checked before reporting.
Activity