Repository navigation
feat(enhance): enhance low quality images before they're read - #479
Merged
Merged
Conversation
The new enhance step, first in the standard pipeline, measures each image
(size, sharpness, noise, contrast, and face size where the namespace finds
faces) and keeps an enhanced copy of a low quality one that read, faces,
objects and descriptions then work from. The original stays as it is.
Off by default (Settings → Enhance). The classic engine (Pillow) needs
nothing; Real-ESRGAN general x4v3 (BSD-3-Clause, 5 MB ONNX) is fetched the
first time it's chosen. The copy is encrypted like the namespace's files,
re-keyed and deleted with them, and downloadable by members only.
Agents: POST /resources/{rid}/enhance, the MCP tool enhance and the enhance
automation action enhance an image on request. The web app has the settings
section, the pipeline step and an Enhance row in the Details tab.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m72U9mPLCbuH8rSAZGSuZ
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m72U9mPLCbuH8rSAZGSuZ
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m72U9mPLCbuH8rSAZGSuZ
Owner
Author
|
Upgrade from the last release failed on 643e1d7: the check read Monday and Friday as gone after the upgrade. That isn't this PR's doing: since #395 the Generated by Claude Code |
…-hnzflg # Conflicts: # README.md # threat-model.yaml
…-hnzflg # Conflicts: # README.md
…-hnzflg # Conflicts: # threat-model.yaml
…-hnzflg # Conflicts: # CHANGELOG.md # threat-model.yaml
…-hnzflg # Conflicts: # CHANGELOG.md # docs/mcp.md # fastapi_backend/app/domain/pipelines.py # threat-model.yaml
…-hnzflg # Conflicts: # CHANGELOG.md # README.md # docs/components.md # docs/mcp.md # nextjs-frontend/components/pipelines/pipeline-canvas.tsx # threat-model.md # threat-model.yaml
The threat model's YAML now changes with the prose, as the validator asks since main's same-commit scheme. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m72U9mPLCbuH8rSAZGSuZ
…-hnzflg # Conflicts: # CHANGELOG.md # README.md # docs/mcp.md
…-hnzflg # Conflicts: # threat-model.yaml
adeelahmad
pushed a commit
that referenced
this pull request
Oct 18, 2026
…enhance step in #479 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014WcBxU5ARBoLWMYZU46St6
…-hnzflg # Conflicts: # CHANGELOG.md
#400 is released, so the requirements check on main was red; its In progress line goes, and its checks join the built list. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m72U9mPLCbuH8rSAZGSuZ
…-hnzflg # Conflicts: # fastapi_backend/app/domain/deletion.py # fastapi_backend/app/domain/pipelines.py # nextjs-frontend/components/pipelines/pipeline-canvas.tsx
…-hnzflg # Conflicts: # docs/configuration.md # docs/mcp.md
Owner
Author
|
Clicks and keystrokes per feature failed on 9bb0ba9: the phone survey's Generated by Claude Code |
…-hnzflg # Conflicts: # README.md # docs/requirements.md
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m72U9mPLCbuH8rSAZGSuZ
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m72U9mPLCbuH8rSAZGSuZ
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m72U9mPLCbuH8rSAZGSuZ
21 tasks done
adeelahmad
added a commit
that referenced
this pull request
Oct 18, 2026
…#569) * feat(pipelines): clean up a recording's sound before it's transcribed A new pipeline step, enhance (Clean up audio), runs a recording through WebRTC's audio processing with pywebrtc-audio (Apache-2.0, ~0.5 MB, fetched the first time the step runs): noise suppression, automatic gain, a high-pass filter, echo cancellation for two-channel call recordings, and voice detection that leaves pauses out of what's transcribed. The transcript's times are mapped back onto the recording's own timeline. The step keeps its options and what it measured on the recording (audio_cleanup); transcribing then hears the cleaned audio. The file is never changed and no copy is stored. No pipeline has the step until one adds it, so defaults are unchanged. Settings > Transcription > Audio cleanup holds the options (transcribe.cleanup). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014WcBxU5ARBoLWMYZU46St6 * refactor(pipelines): name the step clean_audio, apart from the image enhance step in #479 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014WcBxU5ARBoLWMYZU46St6 * style(pipelines): format the canvas's asset steps --------- Co-authored-by: Adeel Ahmad <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Before: a small, blurry, noisy or dull image (a phone snap of a whiteboard, a thumbnail, a dark scan) was read as it was, so OCR, faces, objects and descriptions worked from a poor picture.
After: a new enhance step, first in the standard pipeline, measures each image (size, sharpness, noise, contrast, and the size of its faces where the namespace finds faces) and, when it looks low quality, keeps an enhanced copy that
readand the steps after it work from. The original stays as it is. It's off by default: Settings → Enhance.classic(Pillow: median filter, Lanczos, autocontrast, unsharp mask; nothing to fetch, the default) andrealesrgan(Real-ESRGAN general x4v3, BSD-3-Clause, 5 MB ONNX at a pinned revision with SHA-256, run in 128 px tiles on ONNX Runtime; fetched the first time it's chosen, and pictures over 1.5 MP fall back to classic).enhance.only_faceslimits it to pictures with a face.renditions/<id>-enhanced.jpg) is encrypted like the namespace's files, re-keyed and deleted with the resource, and downloadable by members only (GET /resources/{rid}/enhanced, a signed link inenhanced.download).POST /resources/{rid}/enhance, the MCP toolenhance, and theenhanceautomation action. Each makes the copy and runs the rest of the pipeline again.Motivation and Context
Adeel asked for a pipeline step that detects low quality images and enhances them automatically (thread "detect and auto enhance for better quality as a step"). Audio cleanup before transcription follows in its own PR.
Types of changes
!in the title)Checklist:
make hookschecks it on commit).docs/accordingly.## UnreleasedinCHANGELOG.md.Done questions (AGENTS.md)
POST /resources/{rid}/enhance, the MCP toolenhance, theenhanceautomation action;get_recordingsays what was found.enhance.*settings in the web app with defaults; nothing in the setup wizard.recording.enhanced) and a new settings section with defaults; nothing to migrate.tests/api/test_enhance.py,__tests__/enhance.test.tsx), docs and the generated client (make openapi).Local CI (
ci/local-ci.shon c5b013a): FastAPI embedded and server, Next.js, pre-commit and PR title all pass.🤖 Generated with Claude Code
https://claude.ai/code/session_012m72U9mPLCbuH8rSAZGSuZ