Mount /nix into Kubernetes pods using the CSI ephemeral volumes or NRI(Node Resource Interface). Volumes share lifetime with Pods and are embedded into the Podspec.
Stick your pubkeys in ./keys and they will be imported into the module system then run the following command and you'll have nixkube deployed.
nix run --file . kubenixEval.deploymentScript -- --yes --pruneIf you'd rather mangle YAML yourself you can use
nix build --file . easykubenix.manifestYAMLFileand stuff the result into Kustomize, a blender or your Kubernetes cluster
nixkube supports two methods for injecting Nix stores into pods:
Request Nix stores explicitly via CSI volumeAttributes. Specify one or more:
storePath- Direct nix store path (highest priority)flakeRef- Flake reference to buildnixExpr- Nix expression to evaluate and build
apiVersion: v1
kind: Pod
metadata:
name: hello-csi
spec:
containers:
- name: hello
image: nixos/nix:latest
volumeMounts:
- name: nix
mountPath: /nix
volumes:
- name: nix
ephemeral:
volumeClaimTemplate:
spec:
accessModes: ["ReadOnlyMany"]
storageClassName: ephemeral-storage
resources:
requests:
storage: 1Gi
csi:
driver: nixkube
volumeAttributes:
x86_64-linux: /nix/store/hello-......
aarch64-linux: /nix/store/hello-......
flakeRef: github:nixos/nixpkgs/nixos-unstable#hello
nixExpr: |
let
nixpkgs = builtins.fetchTree {
type = "github";
owner = "nixos";
repo = "nixpkgs";
ref = "nixos-unstable";
};
pkgs = import nixpkgs { };
in
pkgs.helloThe first successful option by priority wins.
Tip: For command arrays and environment variables, use lib.getExe to reference executables without managing full store paths:
command = [ (lib.getExe pkgs.hello) ]
env = {
name = "HELLO_CONFIG";
value = pkgs.hello-config;
}Use pod annotations to automatically inject Nix stores without explicit volume requests:
apiVersion: v1
kind: Pod
metadata:
name: hello-nri
annotations:
nix-nri/pod: |
/nix/store/hello-......
nix-nri/pod@aarch64-linux: |
/nix/store/hello-aarch64-......
spec:
containers:
- name: hello
image: nixos/nix:latest
# /nix is automatically mounted by NRI pluginWhat each mount path gives a pod, by container runtime (CRI) and OCI runtime
(RuntimeClass). ro and rw are the NRI modes: nixkube/pod-rw asks for a
writable /nix.
| runc | crun | youki | gVisor (runsc) | Kata Containers | |
|---|---|---|---|---|---|
| containerd | CSI, NRI ro/rw | CSI, NRI ro/rw | CSI, NRI ro/rw | CSI; NRI refused¹ | CSI, NRI ro², ³ |
| CRI-O | CSI, NRI ro/rw | CSI, NRI ro/rw | CSI, NRI ro/rw | no container runs⁴ | CSI, NRI ro², ³ |
- gVisor keeps the container's rootfs out of the node's reach. The container fails with a message that says so and names CSI instead.
- A VM gets /nix as a read-only virtio-fs share, and waits in the VM until
the build is done. A writable /nix and
nixkube/pod-pathmounts are refused with the reason: the tree is hardlinks into the node's store. Setnixkube.nri.vmRuntimeHandlersif your Kata handler is notkata. The wait is a static busybox that NRI puts in front of the container's args, because Kata checks the command exists before any OCI hook runs;crictl inspectshows the changed args. A container that also mounts something under /nix fails with EROFS, because its mount point would be inside the read-only share. - Needs Kata 4.0.0 or later. Earlier guest kernels have a use-after-free in virtio-fs when a container exits (kata-containers#12589): the VM's agent dies and every container in the pod exits with 255. This happens without nixkube too. nixpkgs still packages 3.32.0; vivarium carries 4.2.0 until nixpkgs moves.
- Measured with CRI-O 1.36.5 and runsc 20260406, on a plain busybox pod without nixkube.
CRI-O with Kata needs skip_mount_home = "true" in containers-storage's
overlay options, or the VM gets an empty rootfs. CRI-O refuses a Kata pod on
the host network ("Host networking requested, not supported by runtime");
containerd accepts one. CRI-O creates a container when an NRI plugin returns
an error, without the plugin's changes.
A pod with hostUsers: false gets both mounts under CRI-O. Under
containerd (2.3.4 to 2.4.1 and main) no user-namespaced container starts
from an image with zero layers, such as a FROM scratch image that
nixkube fills: "mkdirat rootfs/proc: permission denied", with or without
nixkube. Lillecarl/containerd#1 has the cause and #2 a fix. An image with
one layer, even one without /proc, is not affected.
Tested by nix/uml on Kubernetes 1.37, containerd 2.3.4, CRI-O 1.36.5,
runc 1.4.3, crun 1.29.1, youki 0.7.0, gVisor 20260406 and Kata 4.2.0 on
nested QEMU. umlMatrix.<backend>-<cri>[-kata] is each combination as an
attribute, by hand and not in CI:
nix run --file . umlMatrix.uml-crio.driver -- --out ./o
Examples:
This project is made possible by