Repository navigation
feat: embedded web UI and JupyterLab (1.7.0) - #4
Merged
Merged
Conversation
# Conflicts: # Dockerfile
Adds the Athena-style SQL workspace at /ui and a real JupyterLab running inside minilake, reverse-proxied at /jupyter and framed by the UI. Serving Jupyter same-origin is load-bearing rather than cosmetic: jupyter-server sends `Content-Security-Policy: frame-ancestors 'self'`, so a notebook server on its own port cannot be framed by the UI. The proxy relays WebSockets so kernels work, and binds Jupyter to loopback so it is exposed exactly as far as minilake is. No pyspark ships in the image — notebooks reach Spark through the Jobs API, in the same sibling containers a job already uses. Also widens UI coverage of the API (Unity Catalog mutations, job create/edit/delete with per-task run detail, workspace browser, clusters, secrets), adds the Query History and Saved Queries APIs, and makes `POST /_minilake/reset?full=true` honour `full` — it was accepted and ignored, so the UI's two reset buttons did the same thing. See CHANGELOG.md for the full list.
The release adds functionality — the embedded web UI and JupyterLab, the Query History and Saved Queries APIs, and the optional `notebook` extra — so semver calls for a minor, not a patch.
The root .gitignore carried an unanchored `lib/` among the Python build artifacts, which also matched `ui/src/lib/` — so the UI's entire API client and helpers (22 files) were never committed. Local builds passed because the files were on disk; CI built from a clean checkout and failed with `Module not found: Can't resolve '@/lib/...'` across every page. Anchors the Python artifact patterns to the repo root, so `/lib/` no longer reaches into subdirectories, and adds the missing files. `build/`, `dist/` and `var/` had the same latent trap.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gives minilake a face: an Athena-style SQL workspace served at
/ui, and a realJupyterLab running inside the image. Releases as 1.7.0 (see CHANGELOG.md).
Embedded JupyterLab
A real notebook server running as a child process of minilake, reverse-proxied at
/jupyterand framed by the UI at/ui/notebooks. On by default;MINILAKE_NOTEBOOK=0disables it.
Serving it same-origin is load-bearing, not cosmetic. jupyter-server sends
Content-Security-Policy: frame-ancestors 'self', so a notebook server on its own portcannot be framed by the UI at all — the browser blocks it. Proxying it under
/jupytersatisfies
'self'without weakening the policy, and keeps minilake's one-port,one-origin property.
Without that the Lab UI loads and then hangs at "Connecting to kernel".
127.0.0.1and is reachable only through the proxy, so it is exposedexactly as far as minilake is and no further.
same sibling containers a job already uses. The bundled quickstart carries the helper
that stages a script, runs it and returns its output.
minilake[notebook], bundled in the image. Image goes from1.52 GB to 1.82 GB — the Jupyter dependency tree, not Spark.
Web UI
The UI previously covered ~31 of ~109 routes, with six services having no surface at
all. This closes most of that gap:
tables and volumes. Drops cascade on the server and the confirmation says so.
offers only the task types minilake actually executes (
notebook_task,spark_python_task,sql_task.file) — the rest runSKIPPED, and a form offeringthem would promise work that never happens.
/Workspace, clusters, secrets, notebooks.destructive action without it.
Also ships the Query History and Saved Queries APIs that back the corresponding
views.
Bug fix worth calling out
POST /_minilake/reset?full=trueacceptedfulland ignored it — the UI's "Fullreset" and "Reset state" buttons did the same thing. A full reset now deletes the on-disk
data directories (per-catalog DuckDB databases, volume directories, warehouse files) that
no service
reset()touches. Covered by two tests that assert the difference.Verification
and 2 for the reset distinction.
nbconvert --execute: Spark wrotea Delta table, minilake SQL read the same files, Spark read them back.
assets streaming,
/ui/notebooksserving.Deliberately out of scope
permissionsanddbfshave no UI. Permissions are stored but never enforced(single-user allow-all), so a screen would imply access control that does not exist.
DBFS shares its on-disk root with the Files API, which the Files page already browses.
SDK.
--profile notebookcontainer is left untouched. It offers something thebuiltin does not (a local
SparkSessionin the kernel), so it is not a duplicate —but it is worth deciding whether to keep it.
Note
--allow-rootis passed to jupyter-server because the image runs as root, where itotherwise refuses to start. The only symptom was a 503 from the proxy, with nothing
pointing at the cause; it is commented in the code.