Repository navigation
feat(sources): RSS and Atom news feeds as a source - #453
Merged
Merged
Conversation
A news feed is a new non-storage source type, "rss". Its address (a feed, or a site's page that names one) is sealed like an iCal link; each post is listed as its own Atom entry file (.atom), watched like any source and imported as a text resource with its link, author and dates, so answers cite the post. Feeds are fetched through the SSRF guard with conditional GET, parsed with feedparser (BSD) with external entities off, and post HTML is cleaned before it is kept. Adds the feed_post content type, docs, the threat model entry point and the web app's source form. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SWH8a26M9wkAPq2eRrfWZK
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SWH8a26M9wkAPq2eRrfWZK
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SWH8a26M9wkAPq2eRrfWZK
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SWH8a26M9wkAPq2eRrfWZK
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SWH8a26M9wkAPq2eRrfWZK
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SWH8a26M9wkAPq2eRrfWZK
…-8xn6n8 # Conflicts: # CHANGELOG.md # threat-model.yaml
…-8xn6n8 # Conflicts: # CHANGELOG.md # threat-model.yaml
16 of 22 tasks
adeelahmad
added a commit
that referenced
this pull request
Oct 16, 2026
Automations merged (#432 to #464): the stream now holds every kind of activity, the Timeline page reviews it, and automations act on it with triggers, conditions, actions, approvals, limits and budgets, made by agents over MCP and in the assistant or drafted from a sentence. Say so under Automate it, with the studio labelled planned as docs/automations.md does; add the open RSS feeds (#453) and condition operators (#434) to In progress; link the timeline from the docs index. Claude-Session: https://claude.ai/code/session_01HDx7PPUauyjMn1LZs8JAsS Co-authored-by: Adeel Ahmad <noreply@anthropic.com>
…-8xn6n8 # Conflicts: # CHANGELOG.md # threat-model.json # threat-model.md # threat-model.yaml
…-8xn6n8 # Conflicts: # CHANGELOG.md # threat-model.yaml
…-8xn6n8 # Conflicts: # CHANGELOG.md # threat-model.yaml
…-8xn6n8 # Conflicts: # CHANGELOG.md # threat-model.yaml
The threat model's news feed entries move to their own pull request, so this one stops conflicting with every other change to the threat model while its CI runs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SWH8a26M9wkAPq2eRrfWZK
…-8xn6n8 # Conflicts: # README.md
19 of 23 tasks
adeelahmad
added a commit
that referenced
this pull request
Oct 17, 2026
News feeds (#453) are fetched like iCal feeds: through netguard, with their address and password sealed. The threat model now says so: a News feed sources entry point, and news feeds beside iCal in P-SECRETS, P-ADMIN-GUARD, N-EGRESS, the outbound domains and the web networks hazard. Claude-Session: https://claude.ai/code/session_01SWH8a26M9wkAPq2eRrfWZK Co-authored-by: Adeel Ahmad <noreply@anthropic.com>
This was referenced Oct 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Adeel · project thread
Description
Before: Lens had no way to follow a blog or news site. RSS and Atom feeds could not be added as a source.
After: on the Sources page, Add → News feed (RSS or Atom) takes a feed's address, or a site's page that names its feed (
<link rel="alternate">). Each post shows as a file (2026-09-30 New buoys.atom), and a watch imports new and edited posts as text resources with the post's link, author, feed and dates, so answers cite the post. The same works through the API (POST /api/v1/sourceswithtype: "rss").How:
rss.pyis a non-storage source module beside iCal and Google Calendar (feeds.OWN). The address is sealed like an iCal link. Feeds are fetched through the existing SSRF guard (netguard, public addresses ordocuments.web_networks), with conditional GET (ETag / Last-Modified) so an unchanged feed costs a tiny request and no parsing. The iCal download code moved into a sharedfeeds.getboth use. Feeds are parsed with feedparser (BSD-2) from bytes, with external entities off; post HTML is sanitized and links made absolute. Each post is stored as an Atom entry document (RFC 4287), read as Markdown text for indexing. A feed's posts leave memory five minutes after a read; its listing stays. A newfeed_postcontent type recognises.atomfiles, and.atomcan be imported directly. The README lists news feeds as built, with their checks indocs/requirements.md.The threat model's news feed entries (a "News feed sources" entry point, and news feeds beside iCal in P-SECRETS, P-ADMIN-GUARD and N-EGRESS) follow in their own small pull request right after this one: every other change to the threat model conflicted with this pull request while its CI ran, five times.
Motivation and Context
Adeel asked "Do we allow rss feeds?" and said yes to adding them.
Steps to reproduce (if appropriate):
https://example.org/blog.Types of changes
!in the title)Checklist:
make hookschecks it on commit).tests/api/test_news_feed_source.py,test_content_types.py,__tests__/sources-model.test.ts)docs/accordingly. (processing.mdNews feeds,sensors.md,configuration.md,requirements.md, README)## UnreleasedinCHANGELOG.md.Done questions (AGENTS.md)
/api/v1/sources,/watches); dedicated MCP/assistant tools follow in a later PR.make openapi,make requirements).docs/processing.md(JSON Feed, full-article fetch, OPML import).feeds.get), netguard and the source/watch machinery.🤖 Generated with Claude Code
https://claude.ai/code/session_01SWH8a26M9wkAPq2eRrfWZK