Evaluate & Optimize LLM Pipelines

Assess with confidence how your system is performing, and make it better, faster and cheaper

Included with Ellf

Model and API integrationsConnect to any AI provider, API or local model, or add your own integrations

Auto-annotation with agentsAssign agents to your tasks and use LLMs to create and review evaluation data

Annotation platformCollect human feedback and gold-standard data with our battle-tested annotation tool

Repeatable workflowsSet up, re-run and compare experiments and evaluations whenever your system changes

Model distillationTurn LLM outputs into training data for small, fast and private components

Out-of-the-box infrastructureRun jobs in your own cluster in the cloud or locally, no DevOps experience required

Evaluate every part of your system

Ellf lets you run experiments on every component of your pipeline and its outputs, and build reliable evaluation data with human annotators and agents. Instead of relying on vibes, you get a clear picture of what works, what doesn’t and where to improve.

✳︎Claude Code

› I want to evaluate our support ticket pipeline. Can you set up a task so our team and an agent can annotate a sample, and compare the routing and extraction steps against the data?
⏺︎Sure! Let me set up the annotation task and an evaluation workflow for each component.

Ticket Routing

94%

Information Extraction

78%

OpenAI

Jev

Compare modelsCOMPLETE

Evaluate modelsevaluate

Distill large modelSTARTING

Human-in-the-loop distillationdistill

Replace costly LLM calls

Ellf helps you find out which parts of your system don’t need a large general-purpose model, and distill LLM behavior into smaller and faster components you can run privately and under your control.

Sam Taylor

Senior Analyst

Robin Lopez

Product Manager

Keep humans in the loop

Ellf makes it easy to get feedback from domain experts with efficient interfaces used by thousands of annotators over the past 10 years. Create the gold-standard data you need to trust your results.

cluster.example.com

Annotate documentsSTARTED

Text Classificationtextcat

Training experimentSTARTING

Train a modeltrain

Bring your own compute

Plug in a data-private cluster hosted on your own infrastructure to run evaluations, experiments and training jobs. Your data, prompts and models never have to leave your servers.

Get started with powerful project templates

Templates package up complete end-to-end workflows so you don't have to start from scratch. Point your coding agent at a template, adapt it to your use case and data, and press play, or export your own projects to share with your team.

✳︎Claude Code

› /ellf-template Set up this project using the sample data: https://github.com/explosion/ellf-templates/tree/main/llm-to-jev
⏺︎I've read the template's README. Let me set up the project, tasks and evaluation workflow.

Replacing an LLM Classifier with Jev

Text ClassificationEvaluation

2 tasks3 actions1 agent1 plan

github.com/explosion/ellf-templates/tree/main/llm-to-jev

Start task

Ready to get started?

Sign up for a free account to optimize and evaluate your LLM pipelines with Ellf, or read the docs to see how it all fits together.

Get Started Read the Docs

Frequently Asked Questions

Any other questions that are not covered here? Email us!

Which AI models and providers does Ellf support?

You can use Ellf with any AI provider, LLM API or local models. If you can do it in Python, you’ll be able to use it with Ellf! The built-in workflows include integrations with popular API providers like OpenAI, Google Gemini and TypeSafe, as well as local models via Hugging Face and spaCy.

You can access models via API requests with built-in secrets support for authentication, or run them directly from your data-private cluster. You and your coding assistant can also easily add custom workflows to connect to any internal or external service, and load, run or export custom models using any framework.

How does auto-annotation with agents work?

Ellf lets you create LLM-powered agents and assign them to annotation tasks, just like you would with human annotators. Agents support a range of different annotation types, like span annotation, text classification or binary feedback, and will automatically create annotated data, which is added to the dataset and available for review in the UI. You can also customize agents and add your own for different data types and using custom prompts and models.

How does my data stay private?

Ellf lets you plug in your own data-private cluster hosted locally or on your cloud infrastructure and under your control. The cluster provides all data, models and custom code you use and create, serves the annotation application and performs actions like model training or data analysis.

This means that no sensitive data ever leaves your servers or has to pass through our servers in the process. Whenever data from the cluster is used or shown in the web app or CLI, Ellf makes two separate requests: one to our server and one to your cluster.

The cluster can be set up in your cloud account (e.g. AWS, Google Cloud or Azure) or on a local workstation in a few simple steps, without requiring specific DevOps or infrastructure expertise. If you’ve connected Ellf to your coding assistant like Claude Code, it can also set up and launch your cluster for you.

Can I use Ellf for free?

Yes! The Solo plan is free for one person, with no time limit. Your cluster runs on your own machine or a machine on your home network, and you use Ellf from your devices on that network. You can choose Solo when you sign up, or switch to it from the subscription settings at any time.

To invite your team for collaboration and run your cluster in the cloud, you can subscribe to the Starter or Professional plan or start a free trial. For more details and feature comparison, see the plans overview.