8000 Link health checks: models, checker and background task by sincere-arjun · Pull Request #1 · sincere-arjun/linkding · GitHub
[go: up one dir, main page]

Skip to content

Link health checks: models, checker and background task - #1

Open
sincere-arjun wants to merge 4 commits into
masterfrom
moneypenny/39a0b659
Open

sincere-arjun wants to merge 4 commits into
masterfrom
moneypenny/39a0b659

Conversation

@sincere-arjun
@sincere-arjun sincere-arjun commented Nov 2, 2026 •
Copy link
Copy Markdown
Owner

Refs: upstream linkding issue 68 (sissbruecker/linkding, not linked on purpose). This is the backend for periodic link health checks on bookmarks.

What it adds

  • Bookmark fields (migration 0055_link_health):
    • link_status_code, link_check_error, link_last_checked, link_failure_count
    • link_broken, which is indexed
  • UserProfile fields: enable_link_health_checks (off by default) and link_health_check_interval (6h / 12h / daily / weekly, default daily).
  • Settings: LD_LINK_CHECK_TIMEOUT (default 10s) and LD_LINK_CHECK_HOST_DELAY (default 1.0s). Both can be set by environment variable and are documented in docs/src/content/docs/options.md.
  • Checker (bookmarks/services/link_health.py):
    • Sends a HEAD request first and retries once with a streamed GET if that fails. Every response is closed.
    • All requests go through http_client, so the SSRF guard still applies.
    • Uses the configured timeout. A timeout is recorded as "timeout", any other request error as "connection error".
    • Waits between bookmarks on the same host within a batch.
  • 3-failure rule: a bookmark is marked broken after 3 consecutive failures (a status of 400 or more, a timeout or a connection error). A successful check resets it.
  • Background tasks (bookmarks/services/tasks.py):
    • A huey task runs every 15 minutes. For each user with checks enabled, it queues their bookmarks that are due, including archived ones, in batches of 50.
    • Each batch skips bookmarks that were checked after it was queued, so duplicate batches don't send requests again.
    • Nothing runs when LD_DISABLE_BACKGROUND_TASKS is set.
  • Check now: check_bookmark_links(bookmark_ids, current_user) only checks the user's own bookmarks and accepts ids sent as strings. It runs even when periodic checks are off.
  • Search filter: a new broken param on BookmarkSearch. It accepts off / yes, isn't saved as a preference, and yes shows only bookmarks with link_broken=True. BookmarkSearchForm gets the matching broken field.

Review fixes

  • record_result now writes with one database update instead of save(update_fields=...), which has two effects:
    • Deleted bookmarks: if a bookmark is deleted while its link is being checked, the rest of the batch still runs. Before, the error made huey retry the batch, and "Check now" could count the same failure twice.
    • Concurrent checks: the failure count is increased in the database (F()), so two runs checking the same bookmark at once can't lose a failure. link_broken is set in the same update.
  • The LD_LINK_CHECK_HOST_DELAY docs now describe what the delay actually covers.
  • New test that huey registers the scheduler to run at :00, :15, :30 and :45.
  • Shared bookmarks ignore broken: the public shared bookmarks API (/api/bookmarks/shared/, open to anonymous users) used to apply ?broken=yes, so anyone could list which shared links are dead. query_shared_bookmarks now ignores the filter; the owner's 8000 own active and archived lists still apply it. New API tests (anonymous and another signed-in user) and a queries test failed before the fix.

Tests

  • make test: 1413 passed. make lint and ruff format --check: clean.
  • New bookmarks/tests/test_link_health_tasks.py covers:
    • users with checks enabled vs disabled
    • bookmarks due vs not due for each interval
    • bookmarks never checked
    • archived bookmarks
    • LD_DISABLE_BACKGROUND_TASKS
    • "Check now" with periodic checks off
    • batching and skipping duplicate batches
    • the crontab registration
    • a bookmark deleted during a batch
    • the failure count staying correct with concurrent checks
  • New cases in test_bookmarks_service.py, test_bookmark_search_model.py, test_queries.py and test_bookmarks_api.py.
  • Every test mocks HTTP, so none of them make real network requests.

Merge order

The Frontend PR (settings section, Broken filter, badge, "Check links" bulk action) and the Tests PR (checker unit tests and the e2e filter test) build on this one. Please merge this PR first.

🤖 Generated with Claude Code

kashyaparjun and others added 4 commits November 2, 2026 14:24
Adds the shared contract for periodic link health checks: Bookmark link_*
fields and UserProfile settings (migration 0055), LD_LINK_CHECK_TIMEOUT and
LD_LINK_CHECK_HOST_DELAY settings, the link_health checker service, huey
tasks including the periodic scheduler, check_bookmark_links and the
BookmarkSearch "broken" filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scheduled link health check tasks now carry a not_checked_since timestamp,
so bookmarks that were checked after being scheduled are skipped instead of
being checked again when the queue is backed up. Adds tests for the
scheduler and task enqueueing, check_bookmark_links, the BookmarkSearch
broken param and the broken query filter, and documents the
LD_LINK_CHECK_TIMEOUT and LD_LINK_CHECK_HOST_DELAY options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
record_result now writes with a queryset update instead of
save(update_fields=...), so a bookmark deleted while its link is being
checked no longer raises, aborts the batch and makes huey retry it (which
counted failures twice for Check now). The failure count is incremented
atomically with F(), so concurrent checks of the same bookmark don't lose
failures.

Also test that the scheduler is registered as a periodic huey task every
15 minutes, and clarify what LD_LINK_CHECK_HOST_DELAY actually limits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The public shared bookmarks API (AllowAny) built its search from the
request, so ?broken=yes let anyone list which shared links are dead.
query_shared_bookmarks now skips the broken filter; the owner's own
active and archived queries still apply it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sincere-arjun sincere-arjun changed the title Link health checks: models, checker and background task (#68) Link health checks: models, checker and background task Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0