This runner bootstraps the auth E2E environment on a fresh machine and executes focused end-to-end auth checks against an isolated local IronClaw instance.
Use scripts/live-canary/run.sh
as the top-level entrypoint for scheduled and manual lane dispatch. This file
documents the underlying executor for the auth-smoke, auth-full, and
auth-channels lanes.
It is for:
- scheduled auth canaries on fresh CI runners
- manual pre-release auth verification
- validating that a clean machine can still build, launch the gateway, open a browser, complete hosted OAuth flows, and make authenticated tool calls
It is not the live-provider canary yet. This runner uses the existing mock-backed auth matrix so failures point at our own auth/runtime regressions instead of third-party provider drift.
The default smoke profile runs:
- WASM tool OAuth round-trip through the HTTP chat/auth APIs
- MCP OAuth round-trip through the HTTP chat/auth APIs
- MCP OAuth round-trip through the browser UI
- multi-user MCP auth isolation through the browser UI
The full profile adds:
- provider and exchange failure paths
- chat-first and settings-first auth flows
- refresh-on-demand and refresh-on-start coverage
The channels profile runs:
- WASM channel OAuth round-trip through the HTTP auth APIs
smoke is the scheduled canary because it is the currently stable fresh-machine
signal. full and channels are kept as manual/diagnostic profiles until the
remaining flaky or broken cases are fixed.
- Rust toolchain with
cargo - Python 3.11+
- network access for
pip installand Playwright browser download
For local developer runs, playwright install chromium is usually enough.
For fresh Ubuntu CI machines, use --playwright-install with-deps.
From the repo root:
python3 scripts/auth_canary/run_canary.pyRun the full profile:
python3 scripts/auth_canary/run_canary.py --profile fullRun the channel-only diagnostic profile:
python3 scripts/auth_canary/run_canary.py --profile channelsCI-style fresh-machine install:
python3 scripts/auth_canary/run_canary.py --playwright-install with-depsReuse an existing venv and binary:
python3 scripts/auth_canary/run_canary.py \
--skip-python-bootstrap \
--skip-buildPass extra pytest flags through:
python3 scripts/auth_canary/run_canary.py \
--pytest-arg=-x \
--pytest-arg=--maxfail=1List the exact tests for a profile:
python3 scripts/auth_canary/run_canary.py --profile smoke --list-testsBy default the runner writes JUnit output to:
artifacts/auth-canary/auth-canary-junit.xml
Override with:
python3 scripts/auth_canary/run_canary.py --output-dir /tmp/auth-canaryThe runner does this in order:
- create
tests/e2e/.venvif needed pip install -e tests/e2e- install Playwright Chromium
cargo build --no-default-features --features libsql- run the selected auth matrix tests
That makes it suitable for a clean CI VM or a brand-new dev box.