Repository navigation
feat: re-align News/Article parsers with current MAL markup (jikan#629) - #630
Merged
Merged
Conversation
getNewsList and getArticle were the last two uncovered parser endpoints: the
Markup they targeted no longer exists on myanimelist.net.
- ResourceNewsListParser now reads div.news-unit items on the news index
(exact-class selector + ad-unit filter) and the real ?p= pager; the stale
js-scrollfix-bottom-rel/.clearfix layout selector matched nothing.
- ResourceNewsListItemParser + NewsListItemParser read the news-unit fields
(title link, image-link img src, information author/comments/tags).
- ArticleParser::getViews() resolves to null when the page carries no count
(type now ?int); featured pages keep their real counts. getDate() no longer
mutates the shared DOM via Parser::removeChildNodes, so image/info getters
compose.
- ResourceArticle:: is now ?int.
- ArticleRequest is unchanged: featured/{id} still serves classic articles
(only modern news ids 404 on it).
- NewsListItemParserTest re-pointed at the recorded news index and un-skipped
(8 cases previously 'Implementation incomplete').
Tests: ResourceNewsListParserTest, ArticleParserTest (new), NewsListItemParserTest
(re-aligned), all hermetic against fixture PR jikan-fixtures#9. Full suite now
495 tests / 0 skipped (News is no longer skipped). CHANGELOG + DECISIONS
updated. endpoint catalog regenerated: coverage 4 -> 2 (only the raw JSON
list readers getUserAnimeList/getUserMangaList remain, by design).
This was referenced Oct 11, 2026
News/Article parsers no longer match current MAL markup (coverage gap: getNewsList, getArticle)
#629
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is i
3241
nvalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What / why
Closes the last two uncovered parser endpoints (coverage 4 → 2, only the raw JSON list readers
getUserAnimeList/getUserMangaListremain, by design).getNewsListandgetArticleparsed markup that no longer exists on the live site.Live markup surveyed (2026-10-12)
/news): items arediv.news-unit(exact class;news-unit-rightmust not match) — the oldjs-scrollfix-bottom-rel > .clearfixselector matched nothing. MAL injects ad placements insidediv.news-unit; a unit counts only when it carries a title link. Pagination =?p=range links./featured/{id}): still alive — classic article pages with the view count in the first.informationblock (<b>9,893</b>). Only modern news-id pages 404 on/featured/.ArticleRequesttherefore staysfeatured/{malId}.Parser changes
ResourceNewsListParsernews-unit+ ad filter;getHasNextPage()→ real?p=2+pager detectionResourceNewsListItemParsernews-unitfields (title link, imagesrc, info author/comments/forum, excerpt)NewsListItemParsergetDate()no longer strips nodes from the live DOM (Parser::removeChildNodesmutated the crawler and broke later getters)ArticleParsergetViews(): ?int— scoped to the article's own info block, returnsnullwhen no count (no fabricated numbers);getDate()non-mutatingResourceArticle$viewsnow?intTests / fixtures
ResourceNewsListParserTest(endpoint) +ArticleParserTest(endpoint) against recordednewsindex andfeatured/123.NewsListItemParserTestre-pointed at thenewsindex and un-skipped — the 8 previously "Implementation incomplete" cases now pass against real recorded markup (the date assertion is plausibility-based because the index serves relative dates like "11 hours ago").Fixed(3 consumer-visible fixes) + DECISIONS.md "News and featured-article re-alignment" (what drifted, what was re-aligned, what's deliberately out of scope).Verification
composer test— 495 tests / 1,067 assertions / 0 skipped / 0 failures (News no longer skipped)php-cs-fixer --dry-runclean; grumphp pre-commit + backend-suite gates cleancomposer endpoints:check— manifest OK, coverage 2 (raw JSON readers only)Deliberately out of scope (documented in DECISIONS.md)
ArticleListItemParser::getViews()staysint(related-article blocks on featured pages still serve counts; not under endpoint tests).news/{id}blurb pages (a separate layout) and the News search/tags parsers were not re-aligned —getArticle's source is the featured stream.