Repository navigation
Widgets (#317) PR 5: tannerd.net provider and the widget indexing command - #657
Merged
Merged
Conversation
daoudclarke
force-pushed
the
claude/issue-317-pr5
branch
from
November 13, 2026 22:07
481d55c to
53dc839
Compare
Five reviewed calculator specs for tannerd.net (percentage change, days between dates, salary to hourly, trip cost, electricity cost), an OpenAPI importer that drafts a spec from an operation and one sample answer, and `manage.py index_widgets`, which skips (and removes the documents of) sites whose robots.txt does not allow Mwmbl, and removes the documents of specs that changed or went. No functional changes to the crawler. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DTbu4ortjqC5Z7Kw9377LE
Remove an unused local that failed make check, and save the widget keys that a failed index_widgets run may have written so the next run can remove them. No functional changes to the crawler. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DTbu4ortjqC5Z7Kw9377LE
daoudclarke
force-pushed
the
claude/issue-317-pr5
branch
from
November 13, 2026 22:40
53dc839 to
d801980
Compare
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a su
314F
bset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
mwmbl/widgets/providers/tannerd-*.json: five reviewed specs for tannerd.net calculators (percentage change, days between dates, salary to hourly, trip cost, electricity cost). Written from the real OpenAPI (/api/openapi.json) and real sample calls; each spec's query patterns were also run against the live API once, and all five answered 200 with the expected fields. tannerd.net's robots.txt allows everything but/admin/.mwmbl/widgets/openapi_import.py:draft_spec(openapi, path, sample)drafts a spec from an OpenAPI GET operation (query parameters become inputs, examples become defaults, scalars underresultbecome outputs), plus a small CLI. Drafts are for a person to finish (triggers and query patterns).manage.py index_widgets: checks robots.txt for each spec's page and API (an unreadable robots.txt counts as blocked), deletes the widget documents of blocked sites and of specs that changed or went (keys remembered inDATA_PATH/widget-keys.json), then writes one document per allowed spec throughindex_widget_documents.test/test_widgets_tannerd.py: no network. Saved real answers, trigger and non-trigger queries, ambiguous values, sync/purge, robots, the command, and the importer.Why / decisions
{mpg, trip, cost, :n}), so bare "trip cost" or "salary" queries do not fire. Patterns refuse ambiguous values (1,000,60k,18 cents per kwh, monthly salary) and the widget is then hidden.page_urlishttps://tannerd.net/tools/<slug>/?ref=mwmbl, with no placeholders. The endpoint takes a value from the page URL before the query, so a share_url with example values filled in would make the widget always show the example. Therefparameter keeps the URL from equalling the real page's URL. Downside: the link back does not carry the user's values.purge_documents, which removes by URL across the token pages and could delete a crawled page with the same URL.Verification
make checkpasses;test_widgets_tannerd,test_widgets_lookup,test_widgets_apiandtest_index_batchespass (102 tests) 8000 . 500 added lines.Part of #317. Stacked on #317 PR 4 (
claude/issue-317-pr4).Generated by Claude Code