8000 cluster read table output cuts fields out of long struct values · Issue #95 · p0fi/matter-cli · GitHub
[go: up one dir, main page]

Skip to content

cluster read table output cuts fields out of long struct values #95

Closed

Description

@tobiasgraf

Problem Statement

When a user runs cluster read (or the <Cluster> read/subscribe shorthand, or cluster invoke) against an attribute or command response whose value is a TLV structure with more than a couple of fields — e.g. GeneralDiagnostics' NetworkInterfaces, a list of interface structs with Name, IsOperational, HardwareAddress, IPv4Addresses, IPv6Addresses, and Type fields — the human-readable --format table output silently drops named fields to keep the line under a fixed character budget:

✓ General Diagnostics/NetworkInterfaces = [{0: "en6", 1: true, 2: null, ..., 7: 2}, ...]

The fields the user actually wanted (HardwareAddress, the address lists) are the ones missing, with no indication anything was cut beyond the .... The user explicitly named this attribute; they should see everything it reports, not a partial view based on a display budget that made sense for a different use case.

Solution

cli/cluster.go's formatTLVContainer currently applies the same first/last-element "elision" truncation to both array elements (fine — often redundant, similar-shaped items) and struct fields (not fine — fields are distinct, named data; dropping the middle ones loses information the elision marker doesn't reveal). It also middle-truncates long scalar leaf values (strings, byte-array hex dumps) via truncateMiddle regardless of container type.

This formatting path (formatAttrValue → decodeTLVValue → formatTLVElement → formatTLVContainer) is shared by every human-facing renderer that turns a decoded TLV value into a display string: cluster read's single-attribute line and whole-cluster table, cluster subscribe's streamed records, cluster invoke's response line, and tree -L 4's per-attribute value inline in the tree.

Those consumers have different needs. cluster read, subscribe, and invoke are all "the user asked about this one specific thing" — they should show it completely, with no elision or truncation at any level (struct fields, array elements, or leaf scalars). tree -L 4 8000 is a "scan everything" overview that prints one line per attribute across every endpoint and cluster on a node; compactness there is a legitimate, different requirement, and its existing elided/truncated rendering is left unchanged.

The fix makes this an explicit choice at each call site instead of one policy serving both purposes: cluster read, subscribe, and invoke render with full fidelity (nothing ever dropped or truncated); tree -L 4 keeps today's compact, elided rendering exactly as it is now. --format json/--format yaml are already unaffected (they render the native decoded Value, never the display string), and stay that way.

User Stories

  1. As an operator debugging network connectivity, I want matter @1/0 GeneralDiagnostics read NetworkInterfaces --format table to show every field of every interface struct, so that I can see hardware addresses and IP address lists without switching to --format json.
  2. As an operator, I want matter @1/0 GeneralDiagnostics read (whole-cluster read) to show full struct values for every attribute in the resulting table, so that a table row is never less informative than reading that one attribute individually.
  3. As an operator, I want matter cluster subscribe streamed records to show full struct/array values in their table-format rows, so that live updates carry the same fidelity as an equivalent read.
  4. As an operator, I want matter <Cluster> <Command> @1/1's printed response to show the full decoded response value, so that a command response is never silently missing fields.
  5. As an operator, I want a long string or byte-array attribute (e.g. a hex-encoded hardware address or certificate) read via cluster read/subscribe/invoke to print in full, not middle-truncated, so that I can copy the exact value without falling back to JSON.
  6. As an operator running matter @1 tree -L 4 across a node with many endpoints and clusters, I want the per-attribute value preview to stay compact (first/last element, ellipsis, truncated scalars) exactly as it does today, so that the tree remains scannable and isn't overwhelmed by one attribute's large struct or array.
  7. As a maintainer, I want the fidelity choice (full vs. compact) to be an explicit parameter at the formatting seam rather than an implicit, shared 40-character budget, so that future consumers of this formatter must consciously choose their behavior instead of inheriting whichever policy happened to be in place.
  8. As a maintainer, I want array-of-struct elision (dropping middle elements of a long array, e.g. interface 2 and 3 of a 4-interface list) to also be removed under full fidelity, so that "full fidelity" means the whole value, not just "structs no longer lose named fields but arrays still lose elements."
  9. As a scripting user piping --format json/--format yaml output, I want no change in behavior, so that existing scripts relying on the already-untruncated native Value continue to work exactly as before.

Implementation Decisions

  • Introduce a new named type in cli/cluster.go (string-based, mirroring the existing output.FormatType pattern) representing the display fidelity policy, with two values: a compact/elided mode (today's behavior) and a full-fidelity mode (no elision, no truncation at any level).
  • Thread this fidelity value as an explicit parameter through the existing formatting call chain: formatAttrValue → decodeTLVValue → formatTLVElement → formatTLVContainer. No new parallel/duplicate function tree — the same functions handle both modes via the parameter, since the two policies share all the container-walking logic (tlvChildren) and differ only in whether the length-based elision/truncation branches run.
  • buildReadRecords (cli/cluster.go), the shared record-builder used by both cluster read and tree -L 4, takes the fidelity value as a new parameter and passes it through to formatAttrValue when populating ReadRecord.Display.
  • Call sites pass full fidelity: cluster read's single-attribute and whole-cluster paths (displayReadRecord, renderReadTable, both fed by buildReadRecords), cluster subscribe's per-record Display population (cli/subscribe.go), and cluster invoke's response-line rendering (the two decodeTLVValue call sites in cli/cluster.go).
  • Call sites pass compact fidelity: tree.go's call into buildReadRecords for -L 4 attribute values — this is the only remaining consumer of the elided/truncated rendering, and its behavior is unchanged by this work.
  • Under full fidelity: formatTLVContainer never applies first/last-element elision to structs (all fields always shown) or to arrays (all elements always shown); truncateMiddle is not invoked for scalar leaf values (long strings and byte-array hex dumps print in full).
  • Under compact fidelity: behavior is byte-for-byte identical to what exists today — maxValueLen (40 chars), struct/array first-last elision, and truncateMiddle for scalar leaves are all unchanged.
  • renderReadTable's column-alignment logic (cli/output/table.go's TableFormatter) is not modified. It already does no truncation of its own — it pads every cell to the widest cell in its column — so a full-fidelity struct value in one row will widen that whole column. This is an accepted visual tradeoff, not a defect to fix as part of this change; JSON/YAML output is unaffected and remains the recommended format for large/structured values.
  • Update the now-inaccurate doc comments that describe the old single-policy behavior: formatTLVContainer's doc (currently claims all long containers get first/last elision), maxValueLen's doc, truncateMiddle's doc, renderReadTable's doc ("middle-truncated to keep the columns aligned"), and ReadRecord.Display's doc in cli/output/read.go (currently says Display is "possibly truncated" unconditionally). Each should describe the two-mode reality: compact fidelity (tree) truncates/elides as before, full fidelity (read/subscribe/invoke) does not.
  • No change to the TLV decoding/wire-protocol layer, the native Value/decodeTLVNative path, or --format json/--format yaml output — this is purely a --format table / one-line-display concern.

Testing Decisions

  • Tests should assert on externally observable formatted output (the returned display string, or ReadRecord.Display), not on internal truncation flags or intermediate state — consistent with the existing tests in this area.
  • Primary seam: decodeTLVValue/formatTLVContainer/formatTLVElement, already covered by TestDecodeTLVValue_Containers (cli/cluster_test.go). Extend this with the fidelity parameter across its existing sub-tests (short array, struct keyed by tag number, and any long/truncating cases), each run under both compact and full fidelity, asserting: compact fidelity's output is unchanged from today (locks in existing behavior — this test already exists specifically to pin externally observable behavior per its own doc comment), and full fidelity never contains an elision marker and never drops a struct field, for both a long array of similar structs (array-of-struct case) and a struct with many fields whose full rendering exceeds today's 40-character budget.
  • Add a table-driven case at this seam for a long scalar leaf (long string, long byte-array) under both fidelity modes: compact fidelity middle-truncates as truncateMiddle does today; full fidelity prints the value in full.
  • Secondary seam: buildReadRecords, already covered by TestBuildReadRecords_* (cli/cluster_read_test.go), notably TestBuildReadRecords_TruncationIsDisplayOnly. This existing test asserts unconditional truncation and needs to be split or parameterized: one case passing compact fidelity (asserts truncation still happens, matching today), one passing full fidelity (asserts no truncation, and that Display contains the complete value) — while continuing to assert, in both cases, that the native Value was always the untruncated original (that guarantee predates this change and is unaffected by it).
  • No changes expected to tree.go's own tests — tree's call site passes compact fidelity, so its existing test expectations should hold unmodified; that's the regression check that tree's behavior is untouched.
  • No live-device verification needed for this change. It is a pure string-formatting bug with no protocol/wire-format risk, so synthetic TLV byte fixtures (via the existing tlv.NewWriter test helpers already used in TestDecodeTLVValue_Containers) exercise every branch (struct field count, length thresholds, nesting, array-of-struct) more precisely and reproducibly than a live device spot-check.

Out of Scope

  • Bounding or wrapping renderReadTable's column widths for full-fidelity values (e.g. per-column max width, multi-line cells). If wide tables from large-struct attributes prove to be a real usability problem in practice, that is a separate follow-up issue.
  • Any change to tree -L 4's rendering, elision, or truncation behavior — it is explicitly kept as-is.
  • The wire-protocol list-chunking issue tracked separately as Reassemble chunked list-attribute reports for cluster read (Path.ListIndex) #96 (AttributePathIB.ListIndex / TLV null-vs-absent handling) — unrelated root cause, different layer, no dependency either direction, per the prior triage on this issue.
  • Any change to --format json/--format yaml output, which already renders the untruncated native Value and was unaffected by the original bug.

Further Notes

  • Prior triage on this issue (see issue comments) already confirmed the JSON-format symptom as originally reported was fixed by feat(cli): read every attribute of a cluster in one request #88/Read All Attributes From a Specific Cluster #82 (native Value vs. display-only Display split), and narrowed this issue specifically to --format table's struct-field elision.
  • This issue's scope grew during design discussion beyond just structs: the same shared formatting path also serves subscribe, invoke, and tree -L 4, and the fix needed to make an explicit per-consumer choice (full fidelity vs. compact) rather than adjusting the one shared policy, since tree -L 4 has a genuine, different need for compactness that a blanket "never truncate" fix would have broken.

Activity

  1. changed the title [-]properly handle lists[/-] [+]cluster read table output cuts fields out of long struct values[/+] on Sep 16, 2026
  2. added
    bugSomething isn't working
    cliCobra commands, flags, help, output formatting
    on Sep 16, 2026
  3. p0fi commented on Sep 16, 2026

    @p0fi
    Owner

    Triaged this against the current feat/wildcard-cluster-read branch and cross-checked it against #96.

    The JSON symptom as originally reported is already fixed, by commit 32a2143 (#88/#82), which split attribute output into a native Value (used by --format json/--format yaml) and a separate, table-only Display string. formatAttrValue/formatTLVContainer (the 40-char-limited, ...-eliding formatter) is now only used for the Display field.

    Verified live against a matter.js virtual device with a genuinely large NetworkInterfaces list (4 interfaces, byte-array hardware addresses, IPv4/IPv6 address lists):

    $ matter @1/0 GeneralDiagnostics read NetworkInterfaces --format json
    [{"...","value":[{"0":"en6","1":true,"2":null,"3":null,"4":"0xe6c9f7031f6d","5":[],"6":["0xfe80..."],"7":2}, ...4 full interfaces...]}]
    

    Full data, no truncation, native types.

    --format table still truncates, and this is the narrower, real issue that remains:

    $ matter @1/0 GeneralDiagnostics read NetworkInterfaces --format table
    ✓ General Diagnostics/NetworkInterfaces = [{0: "en6", 1: true, 2: null, ..., 7: 2}, ..., {0: "llw0", 1: true, 2: null, ..., 7: 2}]
    

    formatTLVContainer (cli/cluster.go) applies the same first/last-element elision to struct fields that it applies to array elements. That's a reasonable compression for an array of similar/redundant items, but for a struct's named fields (Name, IsOperational, HardwareAddress, ...) it silently drops specific, non-redundant data — which is what made the original report look "unusable." I've retitled and relabeled this issue to track that narrower problem.

    Not related to #96. #96 is a wire-protocol issue (spec-level list chunking across multiple ReportData messages via AttributePathIB.ListIndex, needing a TLV null-vs-absent fix). This issue is purely a CLI display-formatting limitation in the table renderer — different root cause, different layer, no dependency either direction.

    No code changes made as part of this triage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcliCobra commands, flags, help, output formattingready-for-agentReady for implementation by an agent

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0