CLI · Test Case Syntax · CI · Troubleshooting · Contributing · Upgrading
OATs is a declarative acceptance-test framework for OpenTelemetry. You describe,
in YAML, the telemetry an instrumented app should produce — traces, logs,
metrics, profiles — and OATS drives the app (or seeds telemetry directly), then
asserts against a real observability stack (Grafana, Loki, Tempo, Prometheus,
Pyroscope) via gcx.
A case reads like the outcome you care about:
name: rolldice traces have a route attribute
fixture:
compose:
file: docker-compose.oats.yml # your app; OATS adds a Grafana LGTM stack by default
input:
- path: /rolldice?rolls=5 # drive the app (seed defaults to type: app)
expected:
traces:
- traceql: '{ span.http.route = "/rolldice" }'
match_spans:
- name: "GET /rolldice"-
Install mise.
-
Pin
oatsandgcxinto your repo — mise resolves the latest releases and locks them intomise.toml:mise use --pin aqua:grafana/oats aqua:grafana/gcx
For CI, see docs/ci.md.
Without mise:
- GitHub releases — download for your OS/arch from the oats and gcx release pages. Linux release archives are libc-independent (built with CGO disabled) and work on both glibc- and musl-based distributions, such as Alpine Linux.
- go install (oats from source) —
go install github.com/grafana/oats@latest.
oats drives assertions through gcx. Release and mise-built binaries contain
the minimum compatible gcx version. They use gcx from PATH when it meets
that minimum, and automatically download and cache the verified minimum release
when gcx is missing, too old, or does not report a parsable version. Use
--gcx-download never (or OATS_GCX_DOWNLOAD=never) to disable this fallback;
in that mode an absent or too-old PATH binary is an error. Mise-managed
environments default to disabled downloads when mise is detected, even outside
a mise task. You can explicitly trust a binary with --gcx <path> or select an
exact release with --gcx-version <version>.
For Compose fixtures, OATS prefers Podman when it is available and falls back
to Docker. Use --container-runtime docker (or
OATS_CONTAINER_RUNTIME=docker) to make the engine explicit. k3d fixtures
currently require Docker. The builtin Compose fixture uses the latest
docker.io/grafana/otel-lgtm image by default; select another image version with
--lgtm-version <version> (or OATS_LGTM_VERSION). To override the complete
image reference, set LGTM_IMAGE, for example
LGTM_IMAGE=registry.example.com/mirror/otel-lgtm:0.29.1.
The recommended starting point is examples/python/: a real,
instrumented Flask "rolldice" app tested end to end. With Docker running, copy it
and run:
cp -r examples/python my-oats-test && cd my-oats-test
oats # run itAn OATS project is two files. oats-config.yaml lists the cases to run;
oats finds it in the current directory or any parent:
meta:
version: 3
cases: ["oats-case.yaml"]oats-case.yaml is the test itself — the app to drive and the telemetry to
expect:
name: python rolldice
fixture:
compose:
file: docker-compose.oats.yml # your app; OATS adds a Grafana LGTM stack by default
app_service: python # so OATS can reach the app…
app_port: 8082 # ...on its container port
input:
- path: /rolldice # drive the app (seed defaults to type: app)
expected:
traces:
- traceql: '{ span.http.route = "/rolldice" }'
match_spans:
- name: GET /rolldice
metrics:
- promql: 'http_server_active_requests{http_method="GET"}'
value: '>= 0'
logs:
- logql: '{service_name="rolldice"} |~ `rolling the dice`'
regex: 'rolling the dice'oats boots the app next to a throwaway Grafana LGTM stack (the default fixture —
no template: lgtm needed), drives /rolldice once, and retries each telemetry
query until its assertion passes or times out.
HTTP inputs can opt into bounded retries for readiness races. Retries are disabled by default because every attempt repeats the request and its side effects; see the input reference.
One-shot CLIs and batch jobs can run directly as a service in the Compose fixture, without an HTTP shim:
input:
- compose:
service: app
command: [mise, run, hello]No app of your own yet? A case can seed telemetry directly with
seed: {type: inline-otlp} and skip the app entirely — see
docs/case-reference.md.
examples/python/— starter: a real Flask app + compose fixture, asserting traces, metrics, and logs.examples/smoke/— advanced: a remote fixture with inline-otlp / app / profile / custom-check cases.examples/fixtures/— advanced: compose and k3d fixtures side by side.
- docs/cli.md — every command and flag
- docs/case-reference.md — test case syntax: the full config + case shape (fixtures, seed modes, assertion vocabulary, custom checks)
- docs/ci.md — installing and running OATS in CI, plus result caching and its caveats
- CONTRIBUTING.md — contributor orientation: architecture, CLI/env/tooling map, CI/e2e flow, and common gotchas
- UPGRADING.md — migrating older (schema-2) repos to v3
- AGENTS.md — for contributors and coding agents working on OATS (build, layout, conventions)