8000 [Bug]: Knowledge compilation Graph template always reports "0 relations" even when the LLM returns relationships · Issue #22340 · infiniflow/ragflow · GitHub
[go: up one dir, main page]

Skip to content

[Bug]: Knowledge compilation Graph template always reports "0 relations" even when the LLM returns relationships #22340

Open

Description

@mowentianji

[Bug]: Knowledge compilation Graph template always reports "0 relations" even when the LLM returns relationships

Environment

  • RAGFlow version: v1.0.0 (docker: infiniflow/ragflow:v1.0.0, slim/Go build)
  • Doc engine: Infinity v0.7.5-x64-v3
  • Compiler LLM: qwen3.8-max via OpenAI-API-Compatible (Aliyun MaaS)
  • Embedding: text-embedding-v3 (same provider)
  • Data: structured Excel exports (equipment ledger ~2400 rows, maintenance work orders ~580 rows), also tested as row-per-record plain text

Symptoms

Every extraction batch logs X entities, 0 relations, no matter how the Graph template is configured. The resulting knowledge graph is a star of isolated entity nodes (1900+ entities, 0 edges). The Artifacts → Graph view shows N / N entities with no connections.

Observed with both chunk shapes we tested:

  1. Wide Excel parsed via DeepDoc (chunks end up as single-column <table> fragments — a separate table-parsing concern)
  2. Row-per-record plain text (each line: 【record】field1=value1; field2=value2; ...) — chunks are clean, and entity extraction works well (30 entities/batch from 20 records)

Evidence that the LLM does return relationships

Direct call to the same LLM with the same row-per-record text (outside RAGFlow), asking for {"entities": [...], "relationships": [{"from_entity", "relation", "to_entity"}]}:

  • The model returns well-formed JSON with entities and 16+ relationships per small batch (workorder → equipment → spare part chains).
  • Inside RAGFlow, the same data through the Compiler always counts 0 relations.

What we tried (all left relations at 0)

  • Graph template group with global_rules explicitly requiring a relationships array (config saved, JSON valid, confirmed in compilation_template.config)
  • RelationSpecification.fields populated per the official docs (4 relation types with type/description/rule)
  • Both handlers are affected: list-template (table chunks) and hypergraph-template (after rules were set) — e.g. hypergraph-template: batch 1/3 done: 30 entities, 0 relations
  • Different chunk shapes (raw xlsx vs row-per-record text)
  • Tenant default models set (user_tenant.llm_id / embd_id), template llm_id set

Suspected cause

The compiler's response parser appears to look for a different key/shape than what the model produces (e.g. expecting a relations field or a different triple structure), so valid relationship arrays are silently dropped. Related reports: #18778 (kg operator output empty), #18779 (compiled artifacts not returned), #3273 (relationship extraction count 0).

Repro steps (minimal)

  1. Create a Graph template group (defaults + a global rule requesting relationships)
  2. Build an ingestion pipeline with Compiler → that template, bind a small dataset
  3. Upload a row-per-record text file (a few records per line, key=value; ...)
  4. Parse → check task logs: batches report N entities, 0 relations
  5. Call the same LLM directly with the same text and a relationships JSON prompt → relationships are returned

Question

Is there a specific output schema the Graph compiler expects from the LLM for relationships (exact field names / structure), or is relation extraction currently non-functional in v1.0.0? Any workaround short of building the graph outside RAGFlow?

Happy to share our row-per-record sample file and the exact template config JSON if useful.

Activity

  1. arifrhm commented on Nov 13, 2026

    @arifrhm

    Hi @mowentianji,

    I've investigated this issue directly in the Go compiler codebase (internal/ingestion/component/knowledge_compiler/structure) and identified the exact root causes behind why 0 relations is reported:

    Root Causes

    1. Strict Response Unwrapping in unwrapItems (structure/compile.go:94):
      unwrapItems strictly expects the canonical {"items": [...]} payload. When models (or custom prompt rules) return relationships formatted as {"relationships": [...]}, {"relations": [...]}, {"edges": [...]}, or a raw JSON array [...], unwrapItems silently returns nil, causing extractHypergraph to return 0 relations.

    2. Endpoint & Predicate Field Alias Gaps in relationEndpoint & buildRows (structure/compile.go:280, 350):
      When extracting endpoints, relationEndpoint currently only checks source, src, from and target, tgt, to, while the relation type only checks type. Common outputs such as from_entity/to_entity, source_entity/target_entity, head/tail, subject/object, and relation predicate names like relation, relationship, predicate evaluate to empty strings.

    3. Cascading Drop in FilterOrphanRelations (common/relations.go:26):
      Because from or to resolves to "" due to the above alias mismatch, FilterOrphanRelations treats them as malformed/orphan relations and discards 100% of them before persistence.

    4. Direct JSON Array Unmarshaling in jsonchat.go:239:
      tryUnmarshalJSONErr expects map[string]any, so top-level arrays [{...}] trigger unmarshal errors.


    I would like to take this issue! I am preparing a PR with:

    • Resilient unwrapItems supporting relationships, relations, edges, etc., and raw array wrapping.
    • Comprehensive alias resolution for from_entity/to_entity and relation predicate keys.
    • Regression unit tests covering these LLM output patterns.
  2. mowentianji commented on Nov 14, 2026

    @mowentianji
    Author

    Update: Root cause found (source-level) + verified workaround

    We kept digging after filing this. The bug is a key-naming mismatch between the UI editor and the Go compiler, and we've verified a workaround that restores relation extraction.

    Root cause

    The template editor saves the Graph template config with camelCase top-level keys into compilation_template.config:

    { "EntitySpecification": {...}, "RelationSpecification": {...}, "global_rules": "...", "kind": "graph" }

    But the compiler (internal/ingestion/component/knowledge_compiler/structure/prompt.go) reads snake_case keys:

    // HypergraphPrompts: reads parserConfig["entity"] / parserConfig["relation"]
    if len(relationsCfg) == 0 { return nodePrompt, "" }
    // empty edge prompt means "config declares no relations" → stage 2 (edge extraction) is skipped entirely

    So relationsCfg is always empty for UI-configured templates → the relation/edge extraction stage is silently skipped → every batch logs N entities, 0 relations. No error is surfaced anywhere; the UI shows the configured relation cards correctly (it reads its own camelCase keys), which makes this very hard to notice.

    Additional contract details we confirmed while fixing:

    • Relation endpoints must be source / target (not from/to or head/tail)
    • LLM output envelope is {"items": [...]} (already handled by unwrapItems)

    Verified workaround

    Rewrite compilation_template.config with snake_case keys (same content, renamed keys):

    {
      "kind": "graph",
      "entity":   {"description": "...", "fields": [{"type": "equipment", "description": "..."}]},
      "relation": {"description": "...", "fields": [{"type": "caused_by", "description": "source=equipment, target=event"}]},
      "global_rules": "..."
    }

    then restart the ragflow container (config is loaded at startup), re-publish the pipeline, and re-parse the documents.

    Results after the workaround (LLM: qwen3.8-max or deepseek-v4.1-flash, both verified):

    Source Entities Relations
    60-row work-order sample (txt) 56–123 78–165
    582-row work-order xlsx 172 151
    HSE policy text sample 35 31
    Single docx work order 27 25

    Related second issue (possibly separate)

    Some real-world xlsx files (exported by legacy ERP systems) lose all but the first column during parsing — both Go streaming readers and Python openpyxl(read_only=True) see 1 column, while DOM-mode readers see all 16. Re-saving the file (rewriting it with openpyxl DOM mode) fixes it. This may be worth a dedicated check in the Parser (e.g. fall back to a non-streaming read when the declared sheet dimension contradicts actual cells).

    Happy to provide our template config JSON and a minimal xlsx repro file if useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0