Repository navigation
bug(parser): HTMLParser ignores flatten_media_to_text configuration #22007
Description
Description
The ingestion pipeline templates and defaultSetups() define flatten_media_to_text: false, allowing users to flatten structured table and image elements into plain text items.
While PDFParser, MarkdownParser, and DOCXParser support flatten_media_to_text, HTMLParser (internal/parser/parser/html_parser.go) currently drops flatten_media_to_text in ConfigureFromSetup. As a result, HTML documents always emit doc_type_kwd: "table" with structured TableData and doc_type_kwd: "image", ignoring the flattening setting.
Expected Behavior
When flatten_media_to_text is enabled:
- Structured table items should render their HTML content into
item["text"], deleteitem["table"], and setitem["doc_type_kwd"] = "text". - Image items should have
item["doc_type_kwd"]rewritten to"text".
Reacted by dosubot
Activity