Structural code representations for Python, C and C++. Struco extracts ASTs, statement-level control-flow graphs (CFG) and program dependence graphs (PDG) at one shared granularity, exports them as typed JSON, verifies them against reference graphs, and includes a pipeline for comparing representations on ML tasks. A pre-registered study built on it measures what program structure actually adds to learned models.
📖 Documentation: https://fnhirwa.github.io/struco/
- Comparable across languages: every CFG node is a source statement in all three
languages (
&&/||/?:are atomic predicates); LLVM basic-block CFGs are available separately for C/C++. - Documented, fail-closed semantics: CFG and PDG specifications; unsupported constructs raise an error naming them instead of producing a wrong graph.
- Verified: hand-derived reference graphs (108 CFG and 732 PDG edges) and a Python bytecode-trace check run in CI (testing).
- Works on real code: on 18,000 CodeNet submissions, PDG extraction succeeds for 97.7% of Python 3, 92.8% of C and 75.5% of C++ files; the rest are rejected with an explicit reason.
- ML pipeline: problem classification, code retrieval and cross-language retrieval over tokens, AST, CFG, PDG and hybrids, with leakage-aware splits, linear and neural models, chance levels and paired statistics (guide).
The study compares nine representations on 100 CodeNet problems (three tasks, linear and neural models, five seeds, 17 ablations), with its problems, splits, models, hyperparameters and statistics committed before any result (study overview):
- Data dependence adds to control flow. PDGs beat CFGs in problem classification for both model families in all three languages (+0.04 to +0.13 macro-F1).
- AST–PDG hybrids help a graph neural network, not a linear model, and the gain is no longer distinguishable from zero once model capacity is matched.
- Structure vs. tokens depends on the model. Identifier-free graphs beat identifier-free tokens for linear models but not for a token CNN.
- Names dominate. Raw tokens, which keep identifiers, are the best representation in all 14 task–language–model settings.
Full tables: results and ablations.
git clone https://github.com/fnhirwa/struco.git && cd struco
python -m pip install -e ".[dev]" # extras: ml (downstream pipeline), docs (this site)
export STRUCO_CC=clang-18 STRUCO_CXX=clang++-18 # only if clang/clang++ are not on PATHPython 3.10+; Clang 17–18 for C/C++ (installation guide).
python -m struco extract benchmarks/fixtures/binary_search.py --out outputs/python
python -m struco extract benchmarks/fixtures/binary_search.c --cfg-mode source --out outputs/c
python -m struco validate --manifest gold/pdg_manifest.json --out outputs/pdg-oracle --strictfrom struco.
767A
extraction import extract
for graph in extract("benchmarks/fixtures/binary_search.py", ("pdg",)):
print(graph.function, len(graph.nodes), "nodes", len(graph.edges), "edges")More in the quick start, the command-line reference and the ML pipeline guide.
| Path | Contents |
|---|---|
struco/ |
the package: front ends, CFG/PDG analyses, schema, oracles, struco.ml |
gold/ |
hand-derived reference graphs and the bytecode-trace manifest |
benchmarks/ |
fixtures and smoke-test manifests |
tests/ |
unit and oracle tests, one file per module |
docs/ |
the documentation site (MkDocs) |
study/ |
the pre-registered study: design and decision log, frozen experiment records, result reports, Slurm scripts (README) |
Every study result can be regenerated from the committed records in
study/experiments/ with CodeNet and Clang 18.1.8; see
reproducing the study. Linear
models reproduce exactly; the GIN is not bit-reproducible on multithreaded CPUs.
Struco never executes analysed code except in dyncheck, which runs only the
trusted reference fixtures. Clang still reads untrusted sources (60-second timeout, no
memory isolation): run large datasets in a resource-limited environment.
See the contributing guide: new constructs follow specification → hand-derived reference edges → implementation → tests.
MIT © 2024-2026 Felix Hirwa Nshuti