Repository navigation
[BUG] Cluster edits rejected for missing policy still change configuration #49
Description
Description
Editing a cluster with an invalid policy ID returns an error after overwriting other settings.
Reproduced on MiniLake 1.9.0, commit fe8837dcfac9df621dfabc440abc07ebb4cdbcbb, using the repository's Docker test image, a real HTTP server, and fresh disposable state. Image Python sources were verified byte-for-byte against this commit. No mocks or production data were used.
Affected Components
- src/minilake/services/clusters.py —
edit_cluster - src/minilake/services/clusters.py —
_validate_references
Steps to Reproduce
Use a disposable instance and data directory. Paths below are relative to its HTTP base URL; bodies are JSON unless explicitly described as raw bytes. IDs shown as W/J/R/Q/C/HANDLE are replaced with the IDs returned by prior calls.
- POST
/api/2.1/clusters/createwith{"cluster_name":"original","spark_version":"13.3.x-scala2.12","num_workers":1}; capture C. - POST
/api/2.1/clusters/editwith{"cluster_id":"C","cluster_name":"changed","spark_version":"14.3.x-scala2.12","num_workers":9,"policy_id":"missing"}. - GET
/api/2.1/clusters/get?cluster_id=C.
Expected Behavior
The invalid reference causes an atomic rejection, leaving the original name, Spark version and worker count.
Actual Behavior
Edit returns HTTP 400 (Cluster policy 'missing' not found), but the cluster is renamed to changed, has 9 workers, and its version has changed.
Root Cause Analysis
edit_cluster assigns configuration fields directly to the stored dictionary before calling _validate_references.
Impact
Clients see a failed operation while their cluster metadata has changed, causing drift and incorrect retry/reconciliation behavior. No real compute is involved.
Severity is assessed within MiniLake's documented local development/CI use; it does not assume a shared production Databricks deployment.
Suggested Fix
Validate references and the complete proposed configuration before replacing the stored fields. Keep the original record intact on any error.
Test Coverage Gaps
Policy validation tests cover create, but do not assert an existing cluster is unchanged after an invalid edit. Add failed edit cases for missing policy/pool and compare the full before/after record.
Severity
Medium
Confidence
High — reproduced through the real HTTP API and traced to the source paths above.
Recommended Labels
bug
Existing Issue Matches
No reasonable duplicate found. Reviewed all open and closed issues and open pull requests in dmux/minilake, then searched component/function/symptom terms for this finding. The only open PR at review time, #29, changes the virtualenv lock entry and does not address this defect. The documented intentional limitations were checked before reporting.
Activity