Repository navigation
fix: VS Code extension login, run status, and workspace file system against minilake - #14
Merged
Merged
Conversation
<
8000
/div>
GET /api/2.0/workspace/export always returned the base64-JSON envelope
({content, file_type}), ignoring the direct_download query param. The
real API streams raw bytes when direct_download=true, and that's how the
Databricks CLI reads bundle state files (state/resources.json) during
`bundle deploy`/`bundle run` — the JSON-wrapped response doesn't parse as
the expected state file, so the CLI silently falls back to "no remote
state" (serial=0, lineage=""). Every deploy/run after the first then fails
with "lineage mismatch in state files", even though the previous deploy
succeeded and the state file is right there in the workspace.
file_type is still hardcoded to "py" on the non-direct_download path,
unchanged from before — no caller in this codebase depends on it being
accurate yet.
Claude-Session: https://claude.ai/code/session_01JMs8RavvkXJPrqA2jALKnp
RunInfo never set run_page_url, so it was always absent from the API response. The Databricks CLI prints this field as `Run URL: %s` right after triggering a run, and both `bundle run`'s log tailer and the VS Code extension's Bundle Resource Explorer regex-match a run id out of that printed URL (`/runs?/(\d+)/`) to start polling runs/get for status. With no URL, the CLI prints "Run URL: " with nothing after it, the regex never matches, and the extension's run status monitor sits at "unknown" until its 60s watchdog trips — showing "Timeout while fetching run status" even though the run already finished successfully. Built via a new _run_page_url() helper using request.base_url, the same pattern sql_statements.py already uses for external_link URLs. Claude-Session: https://claude.ai/code/session_01JMs8RavvkXJPrqA2jALKnp
Two related gaps in the Workspace API that broke the VS Code extension's "Workspace File System" browser with "Can't fetch details for /Users/minilake-user": 1. `/Users/<user>` never existed. A real workspace provisions this the moment the user exists; minilake only ever created `/Workspace/Users/<user>/...`, as a side effect of a bundle deploy importing files there. Fixed with a new `ensure_user_home()`, called once from the app startup lifespan (idempotent, so it's harmless against the persisted `minilake_data` volume too). 2. Even once a directory exists, `get-status`/`list` returned it without an `object_id`. The extension's WorkspaceFsEntity treats object_id, object_type and path as required on every fs object and throws when one is missing, so a directory response was silently treated as "can't fetch details" — masked when using the Go CLI directly, since Go unmarshals a missing int field to 0 instead of leaving it unset. Fixed with a new `_directory_object_id()` that lazily assigns and caches an id per directory path, the same lazy-assignment pattern already used for files. Claude-Session: https://claude.ai/code/session_01JMs8RavvkXJPrqA2jALKnp
/workspace/import rejected anything but format=SOURCE with a 501, but the VS Code extension's "New Notebook"/"New File" actions in the Workspace File System browser always send format="AUTO" unconditionally — never SOURCE. That made creating any new item there fail outright with "Import format 'AUTO' is not implemented (only SOURCE is supported)". The real API's AUTO infers a format from the path/content; minilake only ever stores plain Python source regardless of what's requested, so treating AUTO the same as SOURCE is accurate, not just a workaround. Claude-Session: https://claude.ai/code/session_01JMs8RavvkXJPrqA2jALKnp
Fixes only, targeting the Databricks VS Code extension and bundle CLI against a minilake workspace: workspace/export now honors direct_download, runs/get and runs/list populate run_page_url, /Users/<user> is created on startup with directories carrying an object_id, and workspace/import accepts AUTO as an alias for SOURCE. No API changes beyond those fixes. Full suite: same 27 pre-existing environment-dependent failures as on main (Docker mount path issues unrelated to this release), 232 passed (231 on main + 1 new AUTO-format regression test). Claude-Session: https://claude.ai/code/session_01JMs8RavvkXJPrqA2jALKnp
Two issues the lint CI job caught: the run_page_url docstring uses \d in a regular string (needs the r"""..."""" raw-string prefix), and the new identity import in workspace.py wasn't isort-ordered (minilake.services sorts after minilake.models). ruff --fix / ruff format applied both; no behavior change. Claude-Session: https://claude.ai/code/session_01JMs8RavvkXJPrqA2jALKnp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes four Workspace/Jobs API gaps in minilake, all discovered while getting
the Databricks VS Code extension and
databricks bundleworking end-to-endagainst a local minilake instance (see the
databricks-cli-example-minilakesample project). Ships as 1.7.5.
workspace/exportignoreddirect_download, always returning thebase64-JSON envelope instead of raw bytes. The Databricks CLI relies on
direct_download=trueto read bundle state files (state/resources.json),so every
bundle deploy/runafter the first failed withlineage mismatch in state files, even though the previous deploy had succeeded.runs/get/runs/listnever populatedrun_page_url. The CLI printsthis as
Run URL: %sright after triggering a run, and bothbundle run's log tailer and the VS Code extension's Bundle Resource Explorerregex-match a run id out of that URL to start polling for status. With no
URL, the extension's run-status monitor sat at "unknown" until its 60s
watchdog gave up with "Timeout while fetching run status" — for runs that
had already finished successfully.
/Users/<user>never existed, and directories had noobject_id.minilake only ever created
/Workspace/Users/<user>/...as a side effectof a bundle deploy. The extension's Workspace File System browser starts
at
/Users/<user>and requiresobject_idon every object it fetches, soit failed outright with "Can't fetch details for /Users/".
workspace/importrejectedformat=AUTOwith a 501, but theextension's "New Notebook"/"New File" actions always send
AUTO, neverSOURCE— so creating any new item in the Workspace File System browserfailed with "Import format 'AUTO' is not implemented".
Also bumps the version to 1.7.5 (
pyproject.toml,uv.lock,FEATURES.md,the GitHub Pages version badge in
site/index.html) and adds a CHANGELOGentry, following the project's existing release-commit pattern.
Test plan
pytest tests/— same 27 pre-existing, environment-dependent failuresas on
main(Docker volume mount path issues, unrelated to this PR),232 passed (231 on
main+ 1 new regression test for the AUTO importformat fix)
to end, against the
databricks-cli-example-minilakesample bundle:databricks bundle deploy,bundle run(Run URL populated, nolineage mismatch),
workspace get-status /Users/minilake-user(directory exists with an
object_id), and creating a new item withformat=AUTOvia a direct API call matching what the extension sends