Onnix is a lightweight and robust HTTP server framework designed specifically to serve ONNX Machine Learning models such as Embeddings and Reranking.
Built with Go, Onnix abstracts the complexity of model lifecycle management, concurrency, and error handling so developers can focus on business logic instead of infrastructure.
Onnix is ideal as a service layer for semantic search, RAG pipelines, vector search systems, and document ranking services.
Onnix automatically cancels ongoing inference when the client aborts the request (disconnect/timeout). This prevents unnecessary CPU and memory usage for heavy inference workloads.
The server will not crash if a panic occurs inside a handler. The error is logged internally and the client receives:
500 Internal Server Error
Onnix clearly separates:
- Client Errors (HttpError) → invalid user input
- Server Errors → internal failures
This prevents sensitive server information from leaking to clients.
The server follows a simple lifecycle pattern:
New() → Serve() → Start()
This makes initialization, testing, and maintenance straightforward.
All input and output communication uses JSON.
Your project should look like this:
project-root/
├── go.mod
├── main.go
├── onnix/
│ └── onnix.go
└── llm/
├── embedder.go
└── reranker.go
package main
import (
"log"
"project-root/onnix"
)
func main() {
// 1. Initialize (load ONNX models into memory)
app := onnix.New()
// 2. Graceful shutdown & resource cleanup
defer app.Stop()
// 3. Register endpoint handlers
app.Serve("/embedding", app.HandleEmbedding)
app.Serve("/reranking", app.HandleReranking)
// 4. Start the server
log.Println("Starting server on :12010")
if err := app.Start(":12010"); err != nil {
log.Fatalf("Server failed: %v", err)
}
}Default Base URL:
http://localhost:12010
Convert a list of text strings into numerical vector embeddings.
POST /embedding
Content-Type: application/json
{
"texts": ["first string", "second string"]
}curl -X POST http://localhost:12010/embedding \
-H "Content-Type: application/json" \
-d '{
"texts": [
"Go is an open source programming language",
"ONNX runtime for AI inference"
]
}'{
"results": [
[0.0123, -0.0456, 0.0789],
[0.0321, -0.0654, 0.0987]
]
}Each array element represents the embedding vector of the corresponding input text.
Sort a list of documents by relevance to a given query.
POST /reranking
Content-Type: application/json
{
"query": "search query text",
"docs": ["document A", "document B", "document C"]
}curl -X POST http://localhost:12010/reranking \
-H "Content-Type: application/json" \
-d '{
"query": "What is machine learning?",
"docs": [
"Machine learning is a branch of AI",
"Recipe for cooking rendang",
"Computers can learn from data"
]
}'Results are sorted by highest relevance score.
{
"results": [
{
"doc": "Machine learning is a branch of AI",
"score": 0.95
},
{
"doc": "Computers can learn from data",
"score": 0.82
},
{
"doc": "Recipe for cooking rendang",
"score": 0.05
}
]
}Onnix uses centralized error handling for consistency and security.
Occurs when the request input is invalid.
Examples:
- Invalid JSON
- Incorrect data type
- Missing required fields
Example response:
400 Bad Request
Invalid JSON body
Occurs when an internal failure happens, such as:
- Model inference failure
- Runtime panic
- Resource issues
The server returns:
500 Internal Server Error
Detailed error information is only logged on the server and not exposed to clients for security reasons.
curl -X POST http://localhost:12010/embedding \
-H "Content-Type: application/json" \
-d '{"texts": "this is a string, should be an array"}'HTTP/1.1 400 Bad Request
Invalid JSON body
Onnix can be used for:
- Semantic Search
- Retrieval-Augmented Generation (RAG)
- Vector Database Pipelines
- Document Ranking Systems
- AI Microservices
- Knowledge Search Engines
MIT License