Repository navigation
perf(docker): fetch speech and object models at run time, not at build time - #183
Merged
Merged
Conversation
uv gave up on big wheels (ctranslate2 on arm64) after its default 30 s read timeout, failing the one-command install. Allow 300 s with 5 retries, and give the image's uv sync steps a BuildKit cache mount so a rerun resumes from what was already downloaded (the cache stays out of the image). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GP566GcZrYdzeWsGQeDCYA
Light by default: the worker already fetches the configured speech engine on first use (components.py), and cloud speech needs none, so baking faster-whisper in (ctranslate2, onnxruntime, av: about 225 MB) only cost download time and disk. EXTRAS=whisper bakes it back in. The no-engine error now points at Settings → Components, cloud speech and EXTRAS. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GP566GcZrYdzeWsGQeDCYA
…ll image Same as faster-whisper: components.py already fetches the objects extra and YOLOX-s into the data folder the first time a job needs them, so the full image no longer bakes them in at build time. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GP566GcZrYdzeWsGQeDCYA
Owner
Author
|
Generated by Claude Code |
# Conflicts: # fastapi_backend/Dockerfile
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Adeel · project thread
Stacked on #182 (same Dockerfile lines); retarget to main once that merges.
Before: the backend image baked in faster-whisper (ctranslate2, onnxruntime, av: about 225 MB), and the full image also baked in ONNX Runtime and downloaded YOLOX-s, all at build time. A slow link failed the install on these downloads (ctranslate2 on arm64 in Adeel's case), though the worker already fetches each of them on first use.
After: none of them are in the images. A worker fetches what a feature needs in the background, at run time, the first time a job needs it or when it's turned on in Settings → Components (
components.py, unchanged).EXTRAS=whisper(orobjects) in.envstill bakes one in. Withcomponents.autooff and no speech engine, the transcribe error points at Settings → Components, Settings → Speech andEXTRAS. Models already in/opt/lens/modelsare still found.How: dropped
--extra whisperfrom theuv syncsteps and the whole objects/YOLOX step from thefullstage infastapi_backend/Dockerfile; rewordedingest.get_engine's docstring and error and its test,objects.py's comments, anddocs/deployment.md/docs/configuration.md. Cloudron and Proxmox packages are unchanged.🤖 Generated with Claude Code
https://claude.ai/code/session_01GP566GcZrYdzeWsGQeDCYA