Browser-level end-to-end tests for the IronClaw web gateway using Python + Playwright.
- Python 3.11+
- Rust toolchain (for building ironclaw)
- Chromium (installed via Playwright)
cd tests/e2e
pip install -e .
playwright install chromiumThe tests need the ironclaw binary built with libsql support:
cargo build --no-default-features --features libsql# From repo root
pytest tests/e2e/ -v
# Run a single scenario
pytest tests/e2e/scenarios/test_chat.py -v
# With visible browser (not headless)
HEADED=1 pytest tests/e2e/scenarios/test_connection.py -vTests start two subprocesses:
- Mock LLM (
mock_llm.py) -- fake OpenAI-compat server with canned responses - IronClaw -- the real binary with gateway enabled, pointing to the mock LLM
Then Playwright drives a headless Chromium browser against the gateway, making DOM assertions.
| File | What it tests |
|---|---|
test_connection.py |
Auth, tab navigation, connection status |
test_chat.py |
Send message, SSE streaming, response rendering |
test_skills.py |
ClawHub search, skill install/remove |
test_tool_approval.py |
Tool approval overlay (approve, deny, always, params toggle) |
test_sse_reconnect.py |
SSE reconnection handling, keepalive comments, restart recovery, stale reconnect IDs, and connection-limit coverage |
test_html_injection.py |
HTML injection security |
test_extensions.py |
Extensions tab: install, remove, configure, OAuth, auth card, activate |
- Create
tests/e2e/scenarios/test_<name>.py - Use the
pagefixture for a fresh browser page - Use selectors from
helpers.py(updateSELdict if new elements are needed) - Keep tests deterministic -- use the mock LLM, not real providers
For the live 20+ turn persona workflows and recurring tool-misuse patterns seen
there, see LIVE_TOOL_FAILURES.md.
For tabs that depend on external data (extensions, jobs, memory, routines), use
Playwright's page.route() to intercept the browser's HTTP requests to the
ironclaw gateway and return deterministic fixture JSON. This avoids needing
real installed binaries, live external services, or complex database setup.
import json
async def test_something(page):
# 1. Set up route intercepts BEFORE navigation triggers the fetch
# Always use async def handlers — route.fulfill() is a coroutine and must be awaited.
async def handle_tools(route):
await route.fulfill(
status=200,
content_type="application/json",
body=json.dumps({"tools": [{"name": "echo", "description": "Echo"}]}),
)
await page.route("**/api/extensions/tools", handle_tools)
# 2. Navigate / interact to trigger the fetch
await page.locator('.tab-bar button[data-tab="extensions"]').click()
# 3. Assert on the rendered DOM
rows = page.locator("#tools-tbody tr")
assert await rows.count() == 1**/api/extensions matches http://host/api/extensions but NOT sub-paths
like http://host/api/extensions/install. For the bare list endpoint, add
a check inside the handler:
async def handle_ext_list(route):
path = route.request.url.split("?")[0]
if path.endswith("/api/extensions"):
await route.fulfill(json={"extensions": []})
else:
await route.continue_() # Let sub-paths through to the real server
await page.route("**/api/extensions*", handle_ext_list)async def handle_setup(route):
if route.request.method == "GET":
await route.fulfill(json={"secrets": [...]})
else: # POST
await route.fulfill(json={"success": True})
await page.route("**/api/extensions/my-ext/setup", handle_setup)calls = []
async def counting_handler(route):
calls.append(1)
await route.fulfill(json={"extensions": []})
await page.route("**/api/extensions", counting_handler)
# ... interact ...
assert len(calls) == 2 # called twice (initial + after some action)| Tab | Key API endpoints to mock |
|---|---|
| Jobs | /api/jobs, /api/jobs/{id}, /api/jobs/{id}/events |
| Memory | /api/memory/search, /api/memory/tree, /api/memory/read |
| Routines | /api/routines, /api/routines/{id}/runs |
For purely client-side UI (components rendered entirely in JS without API calls), call the JavaScript function directly to skip the network layer entirely:
# Show an approval card without needing a real tool execution
await page.evaluate("""
showApproval({
request_id: 'test-001',
thread_id: currentThreadId,
tool_name: 'shell',
description: 'Run something',
})
""")This is the pattern used in most of test_tool_approval.py and parts of
test_extensions.py (auth card, configure modal). The waiting-approval
regression in test_tool_approval.py uses a real tool call instead so it can
exercise backend approval state.