8000 feat(pipelines): convert steps that make a video's soundtrack an audio asset and a document's pages image assets by adeelahmad · Pull Request #568 · adeelahmad/lense · GitHub
[go: up one dir, main page]
Skip to content

feat(pipelines): convert steps that make a video's soundtrack an audio asset and a document's pages image assets - #568

Merged
adeelahmad merged 3 commits into
mainfrom
claude/project-thread-ax3yr7
Oct 18, 2026
Merged

feat(pipelines): convert steps that make a video's soundtrack an audio asset and a document's pages image assets#568
adeelahmad merged 3 commits into
mainfrom
claude/project-thread-ax3yr7

Conversation

@adeelahmad
@adeelahmad adeelahmad commented Oct 18, 2026
Copy link
Copy Markdown
Owner

Requested by Adeel · project thread

Description

Before: a pipeline's steps only made derived data of an asset (transcript, shots, pages). There was no way to turn a video into an audio asset, or a document's pages into image assets.

After: a new Convert step (step library and canvas, {"type": "convert", "to": "audio" | "images"}):

  • To audio: a video's soundtrack becomes an audio asset, "<title> (audio)", as m4a (default), mp3, opus or flac. AAC sound in m4a (or MP3 in mp3) is copied as it is, not encoded again. Audio, documents and silent videos are skipped with the reason.
  • To images: each page the Read step drew becomes an image asset, "<title>, page N" (at most 200). Without drawn pages it skips and says to put a Read step first. A multi-page TIFF's frames work too.

New assets stay in the same namespace and collection, are encrypted where its files are, and say what they were made from (converted_from on the recording API, and a "Soundtrack of / Page N of" link in the asset header). They then run the pipeline their content type gives them, as an upload does; pipeline names another one, or "none" keeps them as they are. Re-running keeps what was made; force remakes the files (an old file in another format is removed). A converted asset is never converted again, so a pipeline can't feed itself; deleting it deletes its file. Default pipelines are unchanged.

How: app/domain/conversions.py (ffmpeg for audio, the existing page JPEGs for images, the attachment-resource pattern for new assets); pipelines.validate_steps checks the settings (only a convert step takes to, format, pipeline); workers that transcribe or read run it too, so older workers.steps lists don't leave it waiting.

Motivation and Context

Parked todo "Asset converter steps" (DEVLOG: video → audio, document → images), started by Adeel's "Start all".

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change; ! in the title)

Checklist:

  • The title is a Conventional Commit.
  • My code follows the code style of this project (make hooks checks it on commit).
  • I added or updated tests (tests/api/test_conversions.py, __tests__/pipeline-convert.test.ts).
  • My change requires documentation updates, and I have updated docs/ accordingly (processing.md "Converting assets", requirements.md, README built list).
  • Users need more than the title to understand it: n/a, the title says it.

Done questions (AGENTS.md)

  • Goal: delivers what was asked; merges on green CI (pending).
  • Agents: pipelines with convert steps are made and run over the REST API (POST /api/v1/pipelines, /run) and by automations' Run pipeline; converted_from is in the recording API.
  • Zero setup: ffmpeg and poppler are already in the install; no new prerequisite.
  • Fewest clicks: one click adds a ready-to-run Convert step (to audio); images is one select away.
  • Configurable: format and what runs next are step settings with defaults, in the editor.
  • Light: off unless a pipeline adds it; AAC is copied, not re-encoded; images reuse the pages already drawn.
  • Private and secure: new files stay under data_dir, encrypted per namespace; no new network surface (threat model n/a: ffmpeg already reads the same media).
  • Cost: no model calls; new assets' pipelines go through budgets as any run does, or "none".
  • User decides: default pipelines unchanged; nothing converts until someone adds the step.
  • Upgrades: a new optional field (converted_from) only; no stored-data change needs an upgrade step.
  • Honest: README lists it as built with its checks.
  • Tests, docs and generated files updated (make openapi).
  • Open standards; no new dependencies.
  • Reported in the project thread. Refine later: copy the video's transcript to its audio asset instead of transcribing again; PNG or higher-resolution page images; moving a converted asset to another namespace keeps its file under the old namespace's uploads folder (as attachments do today).
  • Reuse: ffmpeg/ffprobe helpers, the Read step's page images, keyring, and the attachment-resource pattern.

Local CI (ci/local-ci.sh, backend, frontend, pre-commit, title): all pass on ed001e1.

Generated by Claude Code

@adeelahmad adeelahmad self-assigned this Oct 18, 2026
@adeelahmad
adeelahmad merged commit 623ba87 into main Oct 18, 2026
11 checks passed
@adeelahmad
adeelahmad deleted the claude/project-thread-ax3yr7 branch October 18, 2026 07:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@adeelahmad adeelahmad

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@adeelahmad @claude
0