Repository navigation
[Bug]: DeepDOC (Go) stores the last page number for a cross-page table, so its thumbnail and highlight show the wrong page #22116
Description
Self Checks
- I have searched for existing issues search for existing issues, including closed ones.
- I confirm that I am using English to submit this report (Language Policy).
- Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
- Please do not modify this template :) and fill in all the required fields.
RAGFlow workspace code commit ID
14bb02e (tag v1.0.0-rc1). The code path below is unchanged on main at 3c6d225.
RAGFlow image version
v1.0.0-rc1
Other environment information
- Docker Compose, Elasticsearch doc engine, MinIO
- Ingestion pipeline: Parser (PDF, parse_method DeepDOC, output_format json) -> GeneralChunker
- Source file: 26-page A4 PDF printed from Chrome (Skia/PDF), with several tables that continue across a page breakActual behavior
If a table continues across a page break, its chunk gets the page number of the table's last page. The top and bottom values still describe the table's part on its first page.
Two tables from the same PDF, with the positions returned by GET /api/v1/datasets/{dataset_id}/documents/{document_id}/chunks:
| Table | Where it is in the PDF | Stored positions |
|---|---|---|
| A | page 7 from y≈520 to the page end, then page 8 down to y≈230 | [[8, 43, 550, 520, 772]] |
| B | page 21 from y≈746 (header row only), all of page 22, then page 23 down to y≈310 | [[23, 52, 543, 746, 766]] |
Table A is one page off. Table B spans three pages and is two pages off. Text chunks and tables that fit on one page are correct. In this PDF, about a third of the table chunks point to a later page. Every one we checked by hand crosses a page break.
Effects:
- The chunk image (the thumbnail under a chat answer) is cropped from the wrong page, so it shows unrelated content.
- Opening the citation shows the PDF on the wrong page and highlights an unrelated area.
- The chunk text (the table HTML) is correct, so retrieval and answers are not affected.
Expected behavior
The stored position points to the page that holds the table. At minimum the row uses the first page, because top and bottom belong to that page. For table A that is [[7, 43, 550, 520, 772]]. Ideally the chunk keeps one row per page, so the preview can highlight every page of the table (see #16735).
Steps to reproduce
1. Create a dataset whose ingestion pipeline is Parser (PDF -> DeepDOC, json) -> GeneralChunker.
2. Upload a PDF with a table that starts in the lower half of a page and continues on the next page.
3. Parse the document.
4. List its chunks: GET /api/v1/datasets/{dataset_id}/documents/{document_id}/chunks
5. Find the table chunk. Its positions row has the page number of the table's last page, with the top/bottom of its first page.
6. Ask a chat assistant on this dataset a question that the table answers.
7. The citation thumbnail and the PDF highlight show the wrong page.Additional information
The cause appears to be the 0-based to 1-based page conversion in the DeepDOC JSON path:
SectionsToJSONwrites each position's page cell as a list of every page the section covers: boxes_sections.go#L156-L171pdfParseResultToJSONWithOptionspasses_pdf_positionsthroughnormalizeDeepDocPositionsand also stores the result aspositions: pdf_parser_common.go#L577-L579normalizeDeepDocPageNumberreduces a list page cell to its last element: pdf_parser_common.go#L682-L703
case []any:
if len(v) == 0 {
return 0, false
}
return normalizeDeepDocPageNumber(v[len(v)-1])top and bottom are left as they are, so the page and the coordinates no longer match. The on-demand crop and the index both read this row, which is why the thumbnail and the highlight are wrong in the same way.
Possible fixes:
- Use
v[0], which matchestop. This is a one-line change and puts the highlight on the right page. - Or keep one row per page for spanning positions. fix(chunker): preserve list-form PDF positions in title chunkers #21258 already lets the chunker accept list page cells, and
PositionsFromMatrixalready decodes them.
Related, but different:
- [Bug]: The cross-page table chunk contains coordinates of multiple pages, but only the first page gets highlighted after clicking on the front end #16735: the frontend scrolls only to the first page of a cross-page chunk (v0.25.1, stored data correct).
- [Bug]: [Bug]: Cross-page table highlights only show on the first page with MinerU backend when clicking a chunk in the preview. #16883 / fix: preserve mineru cross-page table positions #17017: the MinerU backend stored only the first page of a cross-page table.
Activity