8000 feat: re-align News/Article parsers with current MAL markup (jikan#629) by irfan-dahir · Pull Request #630 · jikan-me/jikan · GitHub
[go: up one dir, main page]

Skip to content

feat: re-align News/Article parsers with current MAL markup (jikan#629) - #630

Merged
irfan-dahir merged 1 commit into
masterfrom
feat/news-article-realignment-629
Oct 11, 2026
Merged

irfan-dahir merged 1 commit into
masterfrom
feat/news-article-realignment-629

Conversation

@irfan-dahir
Copy link
Copy Markdown
Collaborator

What / why

Closes the last two uncovered parser endpoints (coverage 4 → 2, only the raw JSON list readers getUserAnimeList/getUserMangaList remain, by design). getNewsList and getArticle parsed markup that no longer exists on the live site.

Live markup surveyed (2026-10-12)

  • News index (/news): items are div.news-unit (exact class; news-unit-right must not match) — the old js-scrollfix-bottom-rel > .clearfix selector matched nothing. MAL injects ad placements inside div.news-unit; a unit counts only when it carries a title link. Pagination = ?p= range links.
  • Featured articles (/featured/{id}): still alive — classic article pages with the view count in the first .information block (<b>9,893</b>). Only modern news-id pages 404 on /featured/. ArticleRequest therefore stays featured/{malId}.

Parser changes

File Change
ResourceNewsListParser results → exact-class news-unit + ad filter; getHasNextPage() → real ?p=2+ pager detection
ResourceNewsListItemParser re-aligned 8 getters to news-unit fields (title link, image src, info author/comments/forum, excerpt)
NewsListItemParser getDate() no longer strips nodes from the live DOM (Parser::removeChildNodes mutated the crawler and broke later getters)
ArticleParser getViews(): ?int — scoped to the article's own info block, returns null when no count (no fabricated numbers); getDate() non-mutating
ResourceArticle $views now ?int

Tests / fixtures

  • New ResourceNewsListParserTest (endpoint) + ArticleParserTest (endpoint) against recorded news index and featured/123.
  • NewsListItemParserTest re-pointed at the news index and un-skipped — the 8 previously "Implementation incomplete" cases now pass against real recorded markup (the date assertion is plausibility-based because the index serves relative dates like "11 hours ago").
  • Fixtures: jikan-fixtures PR More Anime/Manga stuff can be parsed #9.
  • DOCS: CHANGELOG Fixed (3 consumer-visible fixes) + DECISIONS.md "News and featured-article re-alignment" (what drifted, what was re-aligned, what's deliberately out of scope).

Verification

  • composer test — 495 tests / 1,067 assertions / 0 skipped / 0 failures (News no longer skipped)
  • php-cs-fixer --dry-run clean; grumphp pre-commit + backend-suite gates clean
  • composer endpoints:check — manifest OK, coverage 2 (raw JSON readers only)

Deliberately out of scope (documented in DECISIONS.md)

  • ArticleListItemParser::getViews() stays int (related-article blocks on featured pages still serve counts; not under endpoint tests).
  • news/{id} blurb pages (a separate layout) and the News search/tags parsers were not re-aligned — getArticle's source is the featured stream.

getNewsList and getArticle were the last two uncovered parser endpoints: the
Markup they targeted no longer exists on myanimelist.net.

- ResourceNewsListParser now reads div.news-unit items on the news index
  (exact-class selector + ad-unit filter) and the real ?p= pager; the stale
  js-scrollfix-bottom-rel/.clearfix layout selector matched nothing.
- ResourceNewsListItemParser + NewsListItemParser read the news-unit fields
  (title link, image-link img src, information author/comments/tags).
- ArticleParser::getViews() resolves to null when the page carries no count
  (type now ?int); featured pages keep their real counts. getDate() no longer
  mutates the shared DOM via Parser::removeChildNodes, so image/info getters
  compose.
- ResourceArticle:: is now ?int.
- ArticleRequest is unchanged: featured/{id} still serves classic articles
  (only modern news ids 404 on it).
- NewsListItemParserTest re-pointed at the recorded news index and un-skipped
  (8 cases previously 'Implementation incomplete').

Tests: ResourceNewsListParserTest, ArticleParserTest (new), NewsListItemParserTest
(re-aligned), all hermetic against fixture PR jikan-fixtures#9. Full suite now
495 tests / 0 skipped (News is no longer skipped). CHANGELOG + DECISIONS
updated. endpoint catalog regenerated: coverage 4 -> 2 (only the raw JSON
list readers getUserAnimeList/getUserMangaList remain, by design).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0