Repository navigation
fix(sources): keep working when Google Drive rate limits - #288
Merged
Merged
Conversation
A throttled Drive account showed as a broken connection, a scan marked the file it couldn't fetch as an error for good, and Lens's own files failed to save once Drive's daily upload limit was hit, though reading still worked. Now rate limits are told apart from failures: the check shows the connection working (limited), rclone gets Drive's default 10 low-level retries, throttled scan files wait for the next scan, and new files are kept on this machine while Drive refuses uploads. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G2TAAG5yQ66RzoXMi3MZdV
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Adeel · project thread
Description
Before: when Google Drive throttled an account, Lens treated the connection as broken. The connection showed a red error, a watched folder marked the file it couldn't fetch as an error for good (never retried), and once Drive's daily upload limit was hit, saving Lens's own files failed, even though reading from Drive still worked.
After: a throttled connection shows as working, slowly (◆ "Works, slowly · …"). Files a scan couldn't fetch wait for the next scan, and the rest of that scan waits too rather than hammering the account. While Drive refuses uploads, new files are kept on this machine (each records where it went) and files already on Drive are still read from there. Settings → Storage → Check it says "It works, but it's rate limited."
How: rclone's errors are classified: rate limit / quota / 429 messages raise
sources.RateLimited. Drive gets rclone's default 10 low-level retries (it was 2, so a single throttle burst failed the call), and file-store copies pass--drive-stop-on-upload-limitso the daily upload limit fails fast instead of retrying for hours. Health and the file-store check carry a newlimitedfield.Motivation and Context
Reported in the project thread: Drive showed an error on a rate limit even though it could still download/read.
Types of changes
Checklist:
tests/api/test_rate_limits.py, with a fake rclone that throttles on demand; the real-rclone file store tests pass with the new flag.)docs/accordingly.🤖 Generated with Claude Code
https://claude.ai/code/session_01G2TAAG5yQ66RzoXMi3MZdV
Generated by Claude Code