8000 Research: can we build a full offline HCL mode? (VCF depot JSON + guide CSV both insufficient) · Issue #55 · dominikzorgnotti/esx-hcl-check · GitHub
[go: up one dir, main page]

Skip to content

Research: can we build a full offline HCL mode? (VCF depot JSON + guide CSV both insufficient) #55

Open

Description

@dominikzorgnotti

Summary

Investigation into a "true full offline mode" (local HCL for all device classes, not just vSAN storage). Two candidate data sources were examined against what the tool actually needs. Conclusion: neither enables offline hardware HCL. Filing so the findings are visible and the research isn't repeated.

Background

Today -offline verifies only storage controllers / vSAN SSD-NVMe against the local vSAN offline DB (vsan-offline-hcl.json). Everything else (system chassis, CPU, NICs, other I/O, GPUs) is live-API-only and becomes SKIPPED offline. Full offline mode needs a local, authoritative source for those classes, keyed by the identifiers the tool collects (PCI VID/DID/SVID/SSID, CPUID, model).

Candidate 1 — VCF offline-depot VmwareCompatibilityData.json

From a VCF offline depot at …/PROD/metadata/Compatibility/v2/VmwareCompatibilityData.json (basic auth; would be a user-supplied depot URL + credentials). ~28 MB.

It is a product interoperability matrix, not a hardware HCL. Self-identifies as bundleType.name = "vcf-interop-bundle", type = "productsBundle", params.include = "interop,upgrade".

  • Structure: products[] → releases[] → { interops[], upgrades[] }. 11 top-level products (ESX, vCenter, NSX, VCF Operations, HCX, SDDC Manager, …), 768 releases, ~94.5k interop edges, ~7.1k upgrade edges.
  • The entire file contains only six array keys: products, releases, interops, upgrades, footnotes, and dlGroups (always empty). Nowhere for hardware data to live.
  • Exhaustive search for vid/did/svid/ssid/cpuid/gpu/driver/firmware/model/pciid/… → zero occurrences. No IO devices, CPUID, GPUs, server models, or SSD entries.

Verdict: closes none of the offline gaps (io/server/cpu/ssd/gpu). Useful for a different future feature: it's an authoritative version ↔ gaBuildNumber ↔ releaseApiVersion map and would enable an ESXi ↔ vCenter ↔ NSX interop / upgrade-path check — fully offline-able from this file, but that's interop, not HCL.

Candidate 2 — "IO Devices" guide CSV export

A manual CSV export of the I/O Devices compatibility guide (~1.2 MB, ~7.7k device rows).

It's a lossy projection that drops the join key. Columns: Brand Name, Model, Device Type, Supported Releases, _ (blank/Restricted).

  • No VID/DID/SVID/SSID, no driver, no firmware. No key to join the tool's collected PCI IDs against, and driver/firmware certification is simply absent (the tool produces both hw_certified and driver_certified).
  • 261 groups of rows are byte-for-byte identical across every exported column (e.g. Intel … PRO/1000 PF Dual Port ×4) — distinct HCL entries differing only in the dropped PCI-ID/driver columns.
  • Also not valid RFC 4180: a model name contains an unescaped inner quote (2.5") that crashes a strict CSV parser (needs LazyQuotes).

Verdict: supports at most "some device with this model string is listed on ESXi X" — never a PCI-ID-keyed, driver-aware verdict. The needed data lives on the guide's per-productId detail pages (which carry PCI IDs + per-release driver/firmware), not the search-results export.

Possible paths forward (none built)

  1. Check the guide's export for a column chooser / "export all fields" including VID/DID. Cheapest decisive test. If present, a PCI-ID-keyed offline DB from CSV becomes viable.
  2. Harvest the per-productId detail pages into a bundled DB — real scraping, with maintenance + terms-of-use questions. The only path seen that yields driver/firmware-aware offline data for io/gpu.
  3. GPUs are already close to offline-able: the GPU guide (program=sptg) is only ~63 rows and is fetched whole (as of v0.5.3). Persisting that fetch would give an offline GPU DB almost for free — a plausible first step.

Absent one of these, full offline HCL is not achievable from the files examined, and offline mode stays vSAN-DB-only by design.

Filed as a research/findings record, not a bug. No code change proposed here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0