Repository navigation
Expand Databricks API emulation: 13.3% -> 17.8% coverage - #17
Merged
Merged
Conversation
Adds a measured coverage map and closes the highest-value gaps it found. Endpoints answered: 155 -> 208 of the 1168 the databricks-sdk calls. CLI command groups working end-to-end: 10 -> 21. Coverage tooling: - scripts/cli_coverage.py drives the real `databricks` CLI against a running minilake and probes every (method, path) the installed SDK calls, recording which fall through to the catchall 501. The unit is the method+path pair, not the path: probing everything with GET scores POST-only endpoints as missing, since FastAPI falls through when no method matches. - docs/CLI_COVERAGE.md (generated) and docs/EMULATION_ROADMAP.md (curated). - Databricks CLI pinned into Dockerfile.test so the map is reproducible. Features that already had an engine and only lacked an endpoint: - sql_task.query now resolves a saved query's text and runs it on the real SQL engine. No new endpoints -- the Queries API had existed all along, and the "no Queries API" note in FEATURES.md was stale. - POST /jobs/runs/submit: one-shot runs, the path `bundle run` and most CI code take. Reuses the existing DAG scheduler; a submitted run carries its tasks inline instead of reading them from a job. Honours idempotency_token. - Alerts (/api/2.0/sql/alerts) plus sql_task.alert, which evaluates the condition against real query results rather than storing a state. Compatibility gaps that stopped clients during setup: - UC current-metastore-assignment, metastore_summary, metastores list/get. - SCIM Users/Groups/ServicePrincipals: full CRUD, the PATCH forms Terraform sends for group membership, and `attribute eq "value"` filtering. The identity user is seeded so Me and Users agree. - Tokens, secret scope ACLs, and clusters update/pin/unpin. New functionality: - UC functions are backed by real DuckDB macros, so a function registered through the UC API is callable from the Statement Execution API by its three-part name -- not metadata-only. - Cluster policies and instance pools. Neither is enforced (there is no compute to constrain), but both resolve; clusters retain and validate policy_id/instance_pool_id, and one in use cannot be deleted. Identities, tokens and ACLs remain records rather than credentials: minilake accepts any bearer token by design, so none of this authenticates anything. Tests: 338 passing (up from 267), all driven through the real databricks-sdk against the Dockerized stack.
Owner
Author
|
LGTM! 🚀 |
9569
This was referenced Sep 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
minilake's API coverage had never been measured.
FEATURES.mdlisted what wasimplemented and kept a hand-written "out of scope" table, but nothing answered
"what exactly breaks when someone points the CLI, the SDK or Terraform at this?"
This adds a reproducible answer, then closes the highest-value gaps it found.
Endpoints answered: 155 → 208 of the 1168 the
databricks-sdkcalls (13.3% → 17.8%).CLI command groups working end-to-end: 10 → 21 — newly working:
alerts,cluster-policies,groups,groups-v2,instance-pools,metastores,service-principals,service-principals-v2,tokens,users,users-v2.The coverage harness
scripts/cli_coverage.pypoints the realdatabricksCLI at a disposable minilakeand probes every endpoint the installed SDK is known to call, via
databricks api <verb> <path>. An endpoint counts as missing only when minilakeanswers with the
catchall501.The unit of measurement is the
(method, path)pair, not the path: probingeverything with GET scores POST-only endpoints as missing, because FastAPI falls
through to the catchall when no method matches. That distinction alone moved the
measured baseline from 10.0% to 13.3%.
Outputs
docs/CLI_COVERAGE.md(generated, regenerate freely) anddocs/EMULATION_ROADMAP.md(hand-written, ranks what to close next). The CLI ispinned into
Dockerfile.testso numbers stay comparable across runs.What was implemented
Things that already had an engine and only lacked a door — the best value here
by a wide margin:
sql_task.query— zero new endpoints.saved_queries.pyhad storedquery_textall along; the task just never looked. The "no Queries API" note inFEATURES.mdwas simply stale.POST /jobs/runs/submit— the one-shot run pathdatabricks bundle runandmost CI code take. Reuses the existing DAG scheduler wholesale; a submitted run
carries its tasks on the run instead of reading them from a job. Honours
idempotency_token.sql_task.alert, which evaluates the conditionagainst real query results rather than storing a constant.
Compatibility walls — small endpoints that stopped clients before they reached
anything that worked:
current-metastore-assignment(+metastore_summary,metastoreslist/get).One endpoint returning one synthetic id, and clients were giving up on it.
Terraform sends for group membership and
attribute eq "value"filtering. Theidentity.pyuser is seeded soMeandUsersnever disagree.update/pin/unpin.New functionality:
through the UC API is then callable from the Statement Execution API by its
three-part name —
SELECT cat.sch.add_tax(100)returns110.0. Not metadata-only.compute to constrain), but both resolve. Clusters now retain and validate
policy_id/instance_pool_id, and a policy or pool in use cannot be deleted.Scope notes
Identities, tokens and ACLs are records, not credentials. minilake accepts any
bearer token by design, so creating a user makes no way to log in and revoking a
token locks nobody out. These exist because tooling creates them as a setup step
and stops when that step fails.
One deliberate change to existing behaviour:
CreateClusterRequestusedextra="ignore"and droppedpolicy_id(its docstring said so explicitly). Nowthat policies and pools are real resources, a cluster that forgot the policy it was
created with would show as permanent drift in Terraform, so those two fields are
kept and validated. Everything else it ignores, it still ignores.
Also fixed along the way
services/catchall.pyregisters only GET/POST/PUT/PATCH/DELETE, so aHEAD or
OPTIONSrequest to an unrouted path misses it and gets Starlette's bare 404instead of minilake's 501 — contradicting the project's "fail loudly" rule. Nothing
is broken today (both
HEADendpoints the SDK calls are implemented), so it isdocumented in the roadmap rather than changed here.
Testing
338 passing, up from 267 — 71 new tests across 9 files, all driven through the
real
databricks-sdkagainst the Dockerized stack per the project's testing rules.Two are the ones that would fail if a feature ever quietly degraded to metadata:
test_sql_function_is_callable_by_three_part_nameandtest_sql_task_alert_triggers_on_real_data.To regenerate the coverage map: