🦾 IRON: Unlocking the Full Potential of NPUs 🦾¶
A close-to-metal Python API for AMD Ryzenâ„¢ AI NPUs.
IRON is an open-source Python library for programming the AI Engine array in
AMD Ryzenâ„¢ AI NPUs,
from a single operator up to a whole language model. It is built on the
MLIR-AIE compiler and its aie.iron
bindings, and runs on NPU1 (Phoenix, Hawk Point) and NPU2 (Strix, Strix
Halo, Krackan).
Installation Set up the XDNA driver, XRT, and IRON itself on Ubuntu.
Models Language models compiled for the NPU, from Llama to Gemma 4.
Performance Checks Operator and model performance, as measured by CI.
Highlights¶
- Operators: GEMM, GEMV, attention, normalization, RoPE, activations and more, each one declared Python class, tested against a numpy reference.
- Graphs: a model's operators are recorded once per input shape and compiled into one image, so intermediate values stay on the device.
- Autotuning: each operator's tunables are searched and saved in a per-device profile.
- Models on the NPU: Llama 3.2, Granite 4, Gemma 4 (text, vision and audio) and EmbeddingGemma 2, every step of the model's graph on the NPU. See Models.
Quick start¶
After installing IRON with its models extra
(pip install -e '.[models]'), and given a Hugging Face token for
meta-llama's gated repository:
from iron.applications.llama import Llama
model = Llama.load("meta-llama/Llama-3.2-1B") # downloads, fits, compiles once
print(model.generate("The capital of France is", 8)) # greedy; pass an iron.lm.Sampler to sample
Llama.load also takes a local directory holding config.json, the
checkpoint and tokenizer.json.
Citing IRON¶
The IRON Python API for Ryzenâ„¢ AI NPUs is described in the following paper:
E. Hunhoff, J. Melber, K. Denolf, A. Bisca, S. Bayliss, S. Neuendorffer, J. Fifield, J. Lo, P. Vasireddy, P. James-Roxby, E. Keller. "Efficiency, Expressivity, and Extensibility in a Close-to-Metal NPU Programming Interface". In 33rd IEEE International Symposium On Field-Programmable Custom Computing Machines, May 2025.