Repository navigation
[BUG] SQL jobs ignore the saved query catalog and schema #50
Description
Description
Running a saved query through sql_task.query discards its stored default catalog/schema, so unqualified SQL that works with its saved namespace fails or resolves elsewhere. Alert evaluation uses the same text-only resolution.
Reproduced on MiniLake 1.9.0, commit fe8837dcfac9df621dfabc440abc07ebb4cdbcbb, using the repository's Docker test image, a real HTTP server, and fresh disposable state. Image Python sources were verified byte-for-byte against this commit. No mocks or production data were used.
Affected Components
- src/minilake/services/jobs.py —
_resolve_sql_task_text - src/minilake/services/jobs.py —
_execute_sql_task - src/minilake/services/saved_queries.py —
resolve_query_text - src/minilake/services/alerts.py —
evaluate_alert
Steps to Reproduce
Use a disposable instance and data directory. Paths below are relative to its HTTP base URL; bodies are JSON unless explicitly described as raw bytes. IDs shown as W/J/R/Q/C/HANDLE are replaced with the IDs returned by prior calls.
- Create warehouse W and catalog
namespace; executeCREATE TABLE namespace."default".source AS SELECT 42 AS id. - POST
/api/2.0/sql/querieswith{"query":{"display_name":"namespace query","query_text":"SELECT * FROM source","catalog":"namespace","schema":"default","warehouse_id":"W"}}; capture query Q. - Execute
SELECT * FROM sourcethrough Statement Execution with W,catalog:"namespace",schema:"default"; it returns 42. - Submit
/api/2.2/jobs/runs/submitwith{"tasks":[{"task_key":"query","sql_task":{"warehouse_id":"W","query":{"query_id":"Q"}}}]}and poll the run.
Expected Behavior
The job executes with the saved query namespace and returns the same row as direct execution.
Actual Behavior
The job fails with Table with name source does not exist! Did you mean "namespace.default.source"?.
Root Cause Analysis
resolve_query_text returns only the SQL text. Both job execution and alert evaluation call _execute_sql_real without the saved catalog/schema arguments that the engine already supports.
Impact
Saved queries behave differently in jobs and direct execution. If another unqualified table exists in the default database, the code can resolve the wrong object instead of producing the reproduced missing-table error.
Severity is assessed within MiniLake's documented local development/CI use; it does not assume a shared production Databricks deployment.
Suggested Fix
Resolve the saved query execution context along with its SQL text, and pass its catalog/schema through job and alert execution while respecting explicit supported overrides.
Test Coverage Gaps
SQL-task tests use fully qualified or default-database tables. Add an unqualified saved query with a nondefault catalog/schema and a conflicting table in the base namespace.
Severity
Medium
Confidence
High — reproduced through the real HTTP API and traced to the source paths above.
Recommended Labels
bug
Existing Issue Matches
No reasonable duplicate found. Reviewed all open and closed issues and open pull requests in dmux/minilake, then searched component/function/symptom terms for this finding. The only open PR at review time, #29, changes the virtualenv lock entry and does not address this defect. The documented intentional limitations were checked before reporting.
Activity