Quickstart · Connect your agent (video) · Website · Docs · Discord · Changelog
The open-source collaboration layer for teams building AI agents.
Open source · SaaS or self-hosted · UI, SDK, and MCP
Connect the agent you are building, share the link with your team, and get structured feedback from the people who know the right answers.
Most tools start with tests or traces. Rhesis starts one step earlier: the people who know what the agent should answer are not the people building it, and their feedback rarely arrives in a form you can act on.
- Feedback that stays attached. Every review sits on the test case and the agent version that produced it, not in a Slack thread
- One pass/fail bar. The whole team reviews against the same tests, and you measure against them
- UI for reviewers, SDK and MCP for builders. Same data, three ways in
- From feedback to CI. Recurring feedback becomes tests and metrics that run on every change
1. Connect the agent you are building. Via the SDK or a REST endpoint, no public URL needed (how, video).
2. Share the Rhesis link with your stakeholders. Domain experts, product managers, and reviewers open it in a browser. Nothing to install, no code.
3. They put the agent to work. In the playground they chat with the live agent and turn interesting conversations into tests. They create and run test sets, simulate multi-turn conversations, and leave structured feedback on what came back: pass/fail verdicts and comments, down to the individual metric or conversation turn.
4. Pull that feedback back into development. Read it from the SDK or the REST API, or work with it from Cursor, Claude Code, and other MCP clients through the Rhesis skill. Fix the agent, run the same tests again.
5. Agree on what the agent has to get right. Each cycle, feedback that arrived as prose becomes tests and metrics that check the same thing automatically. Reviews on a handful of cases end up as evals that run on every change.
| Role | How they use Rhesis |
|---|---|
| AI engineers | Connect the agent, pull feedback and reviewed test sets into the SDK, CI, and MCP while you build |
| Domain experts | Try the agent, review its answers, say what is wrong and what a correct answer looks like. UI, no code |
| Product managers | Turn scattered feedback into tests, and see whether the agent improves against the PRD. UI or MCP |
Nobody can review every case by hand. Once the first feedback is in, you can grow coverage from it:
- Architect: a chat agent that turns a PRD into requirements, metrics, and test sets, runs them, and charts the results
- Test generation from your requirements, a PRD, or an uploaded file
- Conversation simulation with Penelope; adversarial probing with Polyphemus and garak
- Requirement coverage: see which requirements pass or fail, and in which conversations
- Production tracing: OpenTelemetry traces with latency, tokens, and cost. Auto-instrumentation for LangChain, LangGraph, Microsoft Agent Framework, Pydantic AI, Google ADK, and Haystack
- Experiments: compare model, prompt, or retrieval changes on the same test set, by quality and cost
Generated tests are only as good as the requirements behind them. Instead of retyping a spec into a prompt, connect the tools your requirements already live in (Notion, GitHub, Jira, Confluence) and Rhesis writes tests from the real thing. See Tools.
| Who knows it | What they know | What Rhesis checks |
|---|---|---|
| Legal | "Bookings more than 24 h before departure are fully refundable." | The agent quotes the right refund terms for this booking's dates, across multi-turn conversations |
| Support lead | "Business accounts always escalate to a named contact." | Escalation is offered on the business tier, never on the consumer tier, and the contact is named |
| Product | "If the customer changes topic, the agent still finishes the cancellation." | Simulated users interrupt and rephrase; the agent keeps the booking reference and completes the task |
| Marketing | "We never promise delivery dates." | Adversarial prompts try to get a promise out of it; the brand-voice metric flags any that work |
None of these come from a metric library. Your team writes them once as requirements, and they run on every change.
Details: docs.rhesis.ai
app.rhesis.ai: managed service, connect your agent and invite your team.
git clone https://github.com/rhesis-ai/rhesis.git && cd rhesis && ./rh start./rh start pulls prebuilt images from GHCR. To build from the repo instead, use ./rh start --build (and ./rh restart --build after local Dockerfile changes).
Access: Frontend at localhost:3000, API at localhost:8080/docs
Commands: ./rh logs · ./rh stop · ./rh restart · ./rh delete
Local setup auto-logs you in. Not for production; see below.
Once the platform is running, connect your agent with the SDK:
pip install rhesis-sdkSee sdk/README.md.
| Option | Best for |
|---|---|
| Rhesis Cloud | Managed by us, nothing to run |
Local Docker (./rh start) |
Development and trying the platform |
| Docker Compose | A single server |
| Kubernetes (Helm) | Production clusters, via ArgoCD and the charts/rhesis chart we run ourselves |
| GCP (Terraform) | Provisioning a GKE cluster from scratch |
More: self-hosting docs.
Rhesis needs a way to invoke the agent under test. Two ways to do it: we recommend the SDK connector, but if your agent already has a public REST endpoint, that is the fastest way to get started.
Your process opens a persistent outbound WebSocket. Rhesis sends test inputs down that connection, so the agent needs no public URL and can stay on your laptop or inside your VPC. You write a function instead of describing a payload, and the same SDK carries tracing.
from rhesis.sdk.decorators import endpoint
@endpoint(name="my-chatbot")
def chat(message: str) -> str:
# Call your agent here
return my_agent(message)Run it, and the endpoint registers itself in Rhesis. See the SDK README for install, environments, and tracing.
Already serving HTTP on a reachable URL? Register it in the UI, no code and nothing to deploy. You supply auth headers plus request and response mapping, or let Rhesis derive the configuration from an OpenAPI spec or by exploring the endpoint. See Creating endpoints.
Either way, the next step is the same: share the link, and let your team start using the agent through the playground and test runs.
Feedback lands in Rhesis, but you do not have to leave your editor to act on it.
Use Rhesis from MCP-capable clients (Cursor, Claude Code, and others): build test sets, pull results, trigger runs, and read the feedback in the tools you already work in. Install with the skills CLI:
npx skills add rhesis-ai/rhesisSee the skills README.
Pull test runs, results, and the reviews attached to them from Python, or hit the API directly from CI in any language: OpenAPI spec.
The SDK also handles tracing. LLMs for generation and judges: Models.
| Path | What it covers |
|---|---|
sdk/ |
Python SDK: connector, synthesizers, metrics, tracing |
skills/rhesis/ |
Agent skill + MCP workflows for Cursor, Claude Code, and others |
apps/backend/ |
API and workers |
apps/frontend/ |
Web UI |
docs/ |
Documentation site source |
charts/rhesis/ |
Helm chart for Kubernetes deployments |
terraform/ |
Infrastructure as code for GCP |
MIT licensed. No plans to relicense core features. Enterprise features live in ee/ and remain separate.
We built Rhesis because the feedback that mattered most kept getting stuck outside the development loop. If you face the same problem, contributions are welcome.
See CONTRIBUTING.md for guidelines.
Ways to contribute: Fix bugs or add features · Contribute test sets for common failure modes · Improve documentation · Help others in Discord or GitHub discussions
- Documentation: Guides and API reference
- Discord: Community support
- GitHub Issues: Bug reports and feature requests
We take data security seriously. See our Privacy Policy for details.
Telemetry: Rhesis collects basic, anonymized usage statistics to improve the product. No sensitive data is collected or shared with third parties.
- Self-hosted: Opt out by setting
OTEL_RHESIS_TELEMETRY_ENABLED=false - Cloud: Telemetry enabled as part of Terms & Conditions
Made with in Potsdam, Germany 🇩🇪