Repository navigation
feat(shares): import files added over S3 and WebDAV - #408
Merged
Merged
Conversation
A file written into the file tree over S3 or WebDAV is imported as a web upload is (uploads.take): same type and size checks, the namespace's pipeline, fingerprint dedupe. Where it lands decides where it goes: a collection's folder, a new folder under Collections (a new collection), or the default collection. It needs the import permission or an editor's role on the collection, and never replaces a file Lens made. S3 takes PutObject (plain, chunked, aws-chunked with signed chunks and checksums), multipart uploads, folder markers, rclone's self-copy to set times, and deletes of files kept aside. WebDAV takes PUT, MKCOL, MOVE and DELETE of files kept aside, and answers LOCK/UNLOCK/PROPPATCH as class 2 so Finder and Explorer mount it writable. Expect: 100-continue is held until the write is known to be allowed. shares.add_files (on) turns it off from Settings, File shares. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013sC1p4dJXjteytbxDXnk5g
…-cmjahx # Conflicts: # CHANGELOG.md # threat-model.yaml
…-cmjahx # Conflicts: # threat-model.yaml
…-cmjahx # Conflicts: # CHANGELOG.md # threat-model.yaml
Closing a connection with part of a refused body unread reset it, so a client still sending (a chunked body over the size limit) got a broken pipe instead of 413. The S3 and WebDAV servers now read and drop what follows for up to two seconds before closing (RFC 9112, 9.6). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013sC1p4dJXjteytbxDXnk5g
…-cmjahx # Conflicts: # CHANGELOG.md
Merged
16 of 22 tasks
adeelahmad
added a commit
that referenced
this pull request
Oct 15, 2026
…pages and notes as built (#424) Move usage analytics (#274) and assets and people (#273) out of the README's In progress list, and add what merged since #383: a namespace's own MCP server, chat completions and one-namespace API keys; editing and merging entities and topics with undo, from agents too; entity pages at wiki addresses; Connect with Google and rclone sign-in on the server; the notes tree, today's note, pins, search and Recently deleted; backups that keep Lens's files; and grouped, searchable settings. In progress now lists the open feature PRs (#398, #400, #402, #408, #419), and the docs index links usage analytics. Claude-Session: https://claude.ai/code/session_01HDx7PPUauyjMn1LZs8JAsS Co-authored-by: Adeel Ahmad <noreply@anthropic.com>
…-cmjahx # Conflicts: # CHANGELOG.md # threat-model.md # threat-model.yaml
…-cmjahx # Conflicts: # CHANGELOG.md # threat-model.yaml
…-cmjahx # Conflicts: # CHANGELOG.md # threat-model.yaml
…-cmjahx # Conflicts: # docs/file-tree.md # fastapi_backend/app/domain/s3.py # fastapi_backend/app/domain/settings.py # fastapi_backend/app/domain/store.py # fastapi_backend/app/domain/webdav.py # threat-model.json # threat-model.md # threat-model.yaml
…-cmjahx # Conflicts: # CHANGELOG.md
…ecks Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013sC1p4dJXjteytbxDXnk5g
Since #395 the entity-types upgrade step types Terms such as Monday as Dates, which the entities list leaves out unless asked for, so the check read them as gone. It now lists the quiet types too: an entity may change type in an upgrade, but none may go. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013sC1p4dJXjteytbxDXnk5g
8 of 11 tasks
…-cmjahx # Conflicts: # CHANGELOG.md # threat-model.yaml
…-cmjahx # Conflicts: # threat-model.yaml
…-cmjahx # Conflicts: # threat-model.yaml
This was referenced Oct 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Adeel · project thread
Description
Before: the S3 and WebDAV file shares were read-only. Copying a file into them answered AccessDenied or 403.
After: a file copied into the tree over S3 or WebDAV is imported, as a web upload is. Put it in a collection's folder and it goes to that collection; make a new folder under
Collectionsand it's a new collection; put it anywhere else in a namespace and it goes to the default collection.rclone copy ./talks lens:pods/Collections/Talks, Finder and Windows Explorer all work, and a secondrclone copycopies nothing. This also works at the web app's address (/s3,/dav, from #402) once an admin turns that on.How: a new
share_import.pyplaces each write (namespace, collection, new collections to make), checks it like an upload, and hands the file touploads.take(), so it gets the same type and size checks, the namespace's pipeline and fingerprint dedupe. The name the client gave stays in its folder, for that person, for 10 minutes (with the client's MD5 and mtime) so clients find what they just wrote. Files an OS or app writes for itself (.DS_Store,._*,Thumbs.db, Office lock files,.part/.tmp) and empty files are kept aside and never imported; renaming one to a real name imports it.http_body.pystreams bodies to disk with a size cap (Content-Length, chunked, and S3's aws-chunked with signed chunks and trailing checksums), and closes a refused upload gracefully so the client reads the 413 instead of a reset. S3 gets PutObject, multipart uploads, folder markers, rclone's self-copy to set times, and deletes of files kept aside. WebDAV gets PUT, MKCOL, MOVE/DELETE of files kept aside, and LOCK/UNLOCK/PROPPATCH answered as class 2, which Finder and Explorer need before they mount a share writable.Expect: 100-continueis held until the write is known to be allowed, so refused files are never sent.Motivation and Context
Adeel asked that any file added through Lens's file protocols be imported automatically. This PR covers S3 and WebDAV. SMB and NFS follow in their own PRs.
Steps to reproduce (if appropriate):
Turn on File shares, make an access key, then
rclone copy ./some-pdfs lens:<namespace>/Collections/Newand look at the namespace: the collection exists and the PDFs are being processed.Types of changes
!in the title)Checklist:
make hookschecks it on commit).test_s3_writes.py(9),test_webdav_writes.py(6, including writes through the web app's relay),test_webdav.pyupdated.ci/local-ci.shon this branch merged with main: backend, Next.js and pre-commit pass.docs/accordingly (docs/file-tree.md, Adding files).## UnreleasedinCHANGELOG.md.Done questions (AGENTS.md)
GET /api/v1/shares/s3reportsadd_files.rclone copy. Collections are made from folder names.shares.add_files(on) is a switch in Settings, File shares; not in the wizard.shares.remoteon as well. Threat model (P-SHARES, entry points,shares.enabled) updated in md, yaml and json.add_filesdefaults on because importing what's added is what was asked; one switch makes the shares read-only again.docs/requirements.mdnow name the checks (and fold in feat(shares): reach S3 and WebDAV through remote access, at /s3 and /dav #402's line). Finder and Explorer flows are tested from their request sequences; rclone 1.75.1 was run for real (copy, multipart, check, re-copy, touch).make openapiregeneratedopenapi.jsonand the client.docs/file-tree.md: deleting, renaming and moving assets through shares; S3 copies; read-only keys; keeping added names longer than 10 minutes and across restarts.uploads.take()and the shares' existing servers, keys and file tree; no new library.🤖 Generated with Claude Code
https://claude.ai/code/session_013sC1p4dJXjteytbxDXnk5g