A local Databricks API emulator — real SQL, real Delta Lake, real Spark job execution, on your machine, with no account and no cloud bill.
Start at Getting Started if you just want it running. If you already know what you need, use the map below.
| Page | What's in it |
|---|---|
| Getting Started | Install (pip, Docker, source), first catalog and query, the internal health/reset endpoints |
| Configuration | Every environment variable, persistence, service filtering, job execution modes, HTTPS/TLS |
| Page | What's in it |
|---|---|
| Web UI | The built-in SQL workspace at /ui — query editor, data catalog, saved queries, history |
| Databricks SDK | Python WorkspaceClient against minilake: Unity Catalog, warehouses, SQL, jobs |
| Terraform & Asset Bundles | The databricks provider, and bundle deploy / bundle run end to end |
| Spark & Delta Lake | EXTERNAL Delta tables, the write-with-Spark/read-with-SQL loop, spark.table() by name, JupyterLab |
Exposing minilake to LLM agents is the part of this project you are most likely to want. It has its own section:
| Page | What's in it |
|---|---|
| MCP overview | What it is, enabling it, registering with an agent, security |
| Tool reference | All 67 tools, grouped, with the four composites called out |
| Resources & prompts | The context an agent should load before acting |
| Examples | Complete walkthroughs, from a seeded table to a real Spark job |
| Troubleshooting | The mistakes agents actually make here, and the errors they produce |
| Page | What's in it |
|---|---|
| Testing & development | Running the suite (Docker is not optional), adding a feature |
| Releases & CI/CD | How a tag becomes a published image |
| Feature status | Endpoint-by-endpoint status and design rationale |
| API coverage map | Generated: every SDK endpoint probed through the real databricks CLI, answered vs 501 |
| Emulation roadmap | What to build next and why, with measured gap counts |
| Contributing | Project structure and the PR checklist |
Worth reading before you invest in it. These are deliberate, not a backlog:
- No authentication. Any token is accepted, and there is exactly one user.
- No access control. The Permissions API is real CRUD and always allows everything, so a test that passes here says nothing about grants in a real workspace.
- No Spark compute for Clusters. The cluster lifecycle is a state machine; real Spark happens through Jobs, in sibling containers.
- One process. DuckDB is single-writer, so concurrent load serializes on a lock.
See Known Gaps for the full list.