8000 Purge generated exports on a configurable retention by jasonmadigan · Pull Request #203 · tracefinity/tracefinity · GitHub
[go: up one dir, main page]

Skip to content

Purge generated exports on a configurable retention - #203

Merged
jasonmadigan merged 1 commit into
mainfrom
176-stl-retention
Aug 29, 2026
Merged

jasonmadigan merged 1 commit into
mainfrom
176-stl-retention

Conversation

@jasonmadigan
Copy link
Copy Markdown
Collaborator

Summary

  • Generated exports (STL, 3MF, zip, hash sidecars) are regenerable from stored polygons and config, so there is no reason to keep them indefinitely. A background sweep now purges files in {storage}/*/outputs/ older than STL_RETENTION_HOURS (default 24; 0 keeps everything for self-hosters who want that).
  • Sweep-only rather than delete-on-download: the 3D preview streams the same file the download buttons use through an unhookable StaticFiles mount, so deleting on first fetch would break the preview. Cache hits touch mtimes, so a bin in daily use never expires.
  • Deletion scope is deliberately narrow: fixed-depth glob, suffix allowlist, no recursion. Photos, polygons, projects and stores are structurally out of reach.
  • The stale-tab case (open past retention, no edits) is handled end to end: expired downloads now 404 with a regenerate hint instead of crashing mid-response, and the bin page's export buttons fetch, regenerate on 404 and retry once, delivering via blob with the server filename. Sibling 3MF/zip/insert endpoints distinguish "expired" from "never generated".
  • Docs updated (README env table, docs/stl-generation.md, docs/api.md), including the honest caveat that only bin pages self-heal; session-flow exports stay purged until generation is re-requested.

Test evidence

backend/venv/bin/python -m pytest 370 passed (356 baseline + 14 new)
pnpm test 151 passed / 18 files (145 baseline + 6 new)
make lint exit 0, 0 errors, 9 pre-existing warnings in untouched files
npx tsc --noEmit clean

Retention loop is tested for exception survival (a raising sweep is logged and the loop continues). Scope tests include decoy files at storage root, wrong suffixes, nested dirs and sibling stores, all asserted to survive. The 404-then-regenerate download path is unit-tested including the no-regenerate-on-500 and throw-on-double-404 branches.

Closes #176

Generated STL/3MF/zip exports are regenerable from stored polygons and
bin config, so a background sweep now deletes them once older than
STL_RETENTION_HOURS (default 24; 0 keeps them forever). Scope is limited
to export suffixes directly inside each user's outputs/ dir; photos,
traces, tools, bins, and session data are never touched, and the sweep
tolerates files vanishing under it and survives a failing iteration.

Delete-on-download was rejected: BinPreview3D streams the same
/storage/.../outputs file the download links point at, and /storage is a
StaticFiles mount with no per-request hook, so deleting on first fetch
would break the preview. The sweep covers both never-downloaded and
downloaded leftovers instead.

The 24h default never breaks a same-day open page: the frontend re-posts
/generate on page load, and a cache hit now refreshes artefact mtimes so
files a live page references keep their retention clock running.

A stale bin page whose exports were purged no longer opens a dead JSON
tab: the export buttons fetch the file, regenerate from saved state on a
404, retry once, and deliver via a blob anchor click. All export
endpoints now distinguish expiry (record shows a prior generation) from
never generated in their 404 detail, and Content-Disposition is
CORS-exposed so blob downloads keep server filenames in dev.

Signed-off-by: Jason Madigan <jason@jasonmadigan.com>
@jasonmadigan
jasonmadigan marked this pull request as ready for review August 29, 2026 16:56
@jasonmadigan
jasonmadigan merged commit fbb1ad1 into main Aug 29, 2026
6 checks passed
@jasonmadigan
jasonmadigan deleted the 176-stl-retention branch August 29, 2026 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

retention for stl

1 participant

0