This repo exists for experimentation. Rather than starting from scratch every time you want to try something new, you can branch off a working baseline and focus on what you actually want to explore — saving hours of boilerplate setup.
The boilerplate here is built around two deliberate choices:
- LangGraph as the orchestration layer — flexible graph-based agent workflows that scale from simple routing to complex multi-agent systems.
- assistant-ui as the frontend — a high-quality, flexible React component library purpose-built for AI chat interfaces.
At the moment, you can experiment with LLM observability: each example/* branch drops in a different tracing tool (Langfuse, Arize Phoenix, Logfire) with a single commit, so you can compare them side-by-side without any noise. See the LLM Observability section for details.
Contributions of new experiment branches are very welcome!
Demo AI assistant: LangGraph intent routing + FastAPI SSE backend, and a Next.js UI built with assistant-ui (threads, search, reasoning display).
flowchart LR
UI[Next.js assistant-ui] -->|SSE POST /api/chat| API[FastAPI]
API --> LG[LangGraph]
LG --> C[Intent classifier]
C -->|math| M[Math agent]
C -->|general| G[General agent]
API --> DB[(SQLite)]
| Path | Description |
|---|---|
backend/ |
Python package assistant-service: LangGraph graph, Anthropic models, FastAPI, SQLite |
frontend/ |
Next.js app: useRemoteThreadListRuntime + useExternalStoreRuntime, SSE client |
- Python 3.11+ with uv
- Node.js 20+ with Yarn (or use
npx/npmas you prefer) - Anthropic API key
cd backend
cp .env.example .env
# Set ANTHROPIC_API_KEY in .env
uv sync --all-extras
uv run uvicorn assistant_service.main:app --reload --host 0.0.0.0 --port 8000- Health:
GET http://localhost:8000/health - SQLite file (default):
backend/data/assistant.db
cd backend
uv run pytest tests/ -vcd frontend
cp .env.example .env.local
# Optional: NEXT_PUBLIC_API_URL=http://localhost:8000
yarn install
yarn devOpen http://localhost:3000.
Base URL: http://localhost:8000 (set NEXT_PUBLIC_API_URL in the frontend).
| Method | Path | Description |
|---|---|---|
GET |
/health |
Liveness |
POST |
/api/threads |
Create thread; body optional {"id": "<uuid>"} for client-generated id |
GET |
/api/threads |
List threads; ?q= search, include_archived=true |
GET |
/api/threads/{id} |
Thread metadata |
PATCH |
/api/threads/{id} |
Body: title, archived |
DELETE |
/api/threads/{id} |
Delete thread and messages |
GET |
/api/threads/{id}/messages |
List messages (content, reasoning for assistant) |
POST |
/api/chat |
SSE stream; body {"thread_id", "message"} |
Each line is data: <json>:
{"type": "reasoning", "content": "...", "label": "..."}— a reasoning/thinking block;labelcontrols the collapsible section heading in the UI (e.g."Query intent","Reasoning"). Multiple labeled sections can appear per response — the frontend groups tokens by label.{"type": "token", "content": "..."}— assistant answer text{"type": "done"}— stream finished{"type": "error", "message": "..."}— error
The label field on reasoning events is set in _sse_chat() in backend/src/assistant_service/main.py. To change or add labels:
- Intent classification result — emitted once before the main agent streams, with
label: "Query intent". Change the string literal in theyieldcall afterclassify_intent_text()to rename it. - Main agent thinking — emitted for every
reasoning/thinkingblock from the LLM, withlabel: "Reasoning". Change the string literal in the twoyieldcalls inside theastreamloop. - New labeled section —
yieldany additional{"type": "reasoning", "content": "...", "label": "My label"}event at any point in_sse_chat(). The frontend will render it as a separate collapsible block with that heading.
Certain exact strings trigger special canned responses without calling the LLM:
| Query | What it does |
|---|---|
demo! |
Streams a rich display showcase: multiple labeled reasoning sections, headings, lists, tables, code blocks, math (KaTeX), emojis, and links — no API key needed. |
To customise the demo! response, edit DEMO_PARTS in backend/src/assistant_service/demo_response.py. Each entry in the list is one streamed part:
{
"type": "reasoning", # or "text"
"label": "My label", # reasoning only — sets the collapsible heading
"content": "...", # full markdown string, streamed token-by-token
}To add a new magic query, intercept it in _sse_chat() in backend/src/assistant_service/main.py before the classifier call, following the same pattern as the demo! check.
- Classifier (
claude-haiku-4-5) picksmathvsgeneral(see Anthropic models). - Subagents (
claude-sonnet-4-20250514) use Anthropic extended thinking; reasoning is streamed separately from the answer for the UI.
This repo ships a clean base with no observability on main. Use one of the example/* branches to add a tracing layer without touching any other code:
| Branch | Tool | Self-hosted? | Free tier? | Env vars required |
|---|---|---|---|---|
main |
None (base template) | — | — | — |
example/langfuse |
Langfuse | ✅ | ✅ | LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY |
example/logfire |
Logfire | ❌ | ✅ | LOGFIRE_TOKEN |
example/arize-phoenix |
Arize Phoenix | ✅ | ✅ | PHOENIX_COLLECTOR_ENDPOINT |
Each branch adds exactly one commit on top of main. To see what an integration requires:
git diff main example/langfuseTo start from an integration branch:
git checkout example/logfire
# Set the relevant env vars in backend/.env, then run normallyMIT — see LICENSE.