One Go API for Anthropic, OpenAI, Google, and 20+ other providers — including local models through Ollama and vLLM. Switch providers without rewriting your request.
Zero external dependencies. Stdlib only.
Also available for TypeScript, Python, Rust, Swift, and Java.
go get github.com/aktagon/llmkit-go/v4c := llmkit.New("anthropic", os.Getenv("ANTHROPIC_API_KEY"))
resp, err := c.Text.System("You are helpful").Prompt(ctx, "Hello")
fmt.Println(resp.Text())c.Text, c.Image, c.Music, c.Video, c.Agent, and c.Upload are
pointer fields on *Client, not method calls. Chain methods clone the prototype and
return a fresh builder, so successive c.Text.System(...) calls each
yield a new *Text.
See examples/ for runnable single-file demos
(quickstart, agent, stream, upload, image-gen,
image-gen-openai, middleware). The shapes shown
below are exercised against mock HTTP servers by
example_test.go, so the documented call shapes
are guaranteed to match the public surface.
| Provider | Default Model | Env Var | Capability |
|---|---|---|---|
| ai21 | jamba-1.5-large | AI21_API_KEY | chat |
| anthropic | claude-sonnet-4-6 | ANTHROPIC_API_KEY | chat |
| cerebras | gpt-oss-120b | CEREBRAS_API_KEY | chat |
| cohere | command-r-plus | COHERE_API_KEY | chat |
| deepseek | deepseek-chat | DEEPSEEK_API_KEY | chat |
| doubao | doubao-1.5-pro-32k-250115 | ARK_API_KEY | chat |
| ernie | ernie-4.0-8k | QIANFAN_API_KEY | chat |
| fireworks | accounts/fireworks/models/gpt-oss-120b | FIREWORKS_API_KEY | chat |
| gemini-3.5-flash-lite | GOOGLE_API_KEY | chat | |
| grok | grok-3-fast | XAI_API_KEY | chat |
| groq | openai/gpt-oss-120b | GROQ_API_KEY | chat |
| jan | none - set with .Model() | JAN_API_KEY | chat |
| llamacpp | none - set with .Model() | LLAMACPP_API_KEY | chat |
| lmstudio | none - set with .Model() | LM_STUDIO_API_KEY | chat |
| minimax | MiniMax-Text-01 | MINIMAX_API_KEY | chat |
| mistral | mistral-large-latest | MISTRAL_API_KEY | chat |
| moonshot | kimi-k2.6 | MOONSHOT_API_KEY | chat |
| ollama | none - set with .Model() | OLLAMA_API_KEY | chat |
| openai | gpt-4o-2024-08-06 | OPENAI_API_KEY | chat |
| openrouter | openai/gpt-4o | OPENROUTER_API_KEY | chat |
| perplexity | sonar-pro | PERPLEXITY_API_KEY | chat |
| qwen | qwen-plus | DASHSCOPE_API_KEY | chat |
| sambanova | Meta-Llama-3.3-70B-Instruct | SAMBANOVA_API_KEY | chat |
| together | meta-llama/Llama-3.3-70B-Instruct-Turbo | TOGETHER_API_KEY | chat |
| vllm | none - set with .Model() | VLLM_API_KEY | chat |
| yi | yi-large | YI_API_KEY | chat |
| zhipu | glm-4-plus | ZHIPU_API_KEY | chat |
| recraft | recraftv3 | RECRAFT_API_TOKEN | image |
| inworld | inworld-tts-2 | INWORLD_API_KEY | speech |
| assemblyai | best | ASSEMBLYAI_API_KEY | transcription |
| pixverse | v4.5 | PIXVERSE_API_KEY | video |
| vidu | viduq3-pro | VIDU_API_KEY | video |
36 providers, 5 API shapes (AWS Bedrock Converse, Anthropic Messages, Google Generative AI, OpenAI Chat Completions, OpenAI Responses). The 32 above are usable with a provider name and an API key alone; azure, bedrock, vertex and workersai additionally need caller-supplied configuration (resource name, region, project, or account) and are listed in providers/providers.go.
One-shot request:
c := llmkit.New("anthropic", os.Getenv("ANTHROPIC_API_KEY"))
resp, err := c.Text.
System("You are helpful").
Temperature(0.7).
Prompt(ctx, "What is 2+2?")
fmt.Println(resp.Text()) // "4"
// Every Usage dimension is optional. nil means the provider did not report
// the value, which is NOT the same as reporting zero: a provider that says it
// used no cached tokens and one that never mentions caching are different
// facts, and a plain 0 cannot tell you which you have.
resp.Usage.Input // *int — prompt tokens
resp.Usage.Output // *int — completion tokens
resp.Usage.CacheRead // *int — tokens served from cache (all caching modes)
resp.Usage.CacheWrite // *int — tokens written to cache (Anthropic explicit caching)
resp.Usage.Reasoning // *int — internal reasoning tokens (OpenAI o-series, Gemini 2.5+ thinking)
resp.Usage.Cost // *float64 — provider-reported USD; nil is unreported, never "free"
if v := resp.Usage.CacheRead; v != nil {
fmt.Println(*v, "tokens served from cache")
}Capability-scoped fields (CacheRead, CacheWrite, Reasoning) are zero when the provider doesn't report them separately.
Streaming with a trailing-handle iterator. Stream returns a
*TextStream; range over Chunks() to consume deltas, then read
Response() for the accumulated text + token counts:
stream := c.Text.System("Be brief").Stream(context.Background(), "Tell me a one-line joke")
for chunk, err := range stream.Chunks() {
if err != nil {
log.Fatal(err)
}
fmt.Print(chunk)
}
fmt.Println()
final := stream.Response()
// Token counts are optional: nil means the provider did not report the
// value, which is not the same as reporting zero. The finish reason is one
// of five canonical values, empty when the provider reported none.
if final.Usage.Input != nil && final.Usage.Output != nil {
fmt.Printf("input=%d output=%d\n", *final.Usage.Input, *final.Usage.Output)
} else {
fmt.Println("this provider did not report token counts")
}
if final.FinishReason != "" {
fmt.Println("finish_reason:", final.FinishReason)
}Breaking the range loop cancels the producer goroutine cleanly.
Pass a JSON schema to get typed responses:
resp, err := c.Text.
Schema(`{"type":"object","properties":{"color":{"type":"string"}}}`).
Prompt(ctx, "The sky is blue")
// resp.Text() == `{"color":"blue"}`Multi-turn conversations with function calling. c.Agent is a
stateful builder — repeated Prompt calls on the same *Agent
accumulate conversation history. Any chain method (System, AddTool,
Temperature, ...) returns a forked clone with empty state.
agent.Reset() clears history without dropping the configured tools
or other chain state.
A Prompt call that fails leaves the history as it was and returns the
error that stopped it, such as an *APIError. agent.UnsavedTurns()
returns the turns that call made, tool results included. The next
Prompt call clears them when it starts, and so do agent.Reset() and
a chain method, whose clone starts with none.
agent := c.Agent.
System("You are a calculator").
AddTool(llmkit.Tool{
Name: "add",
Description: "Add two numbers",
Schema: map[string]any{"type": "object", "properties": map[string]any{
"a": map[string]any{"type": "number"},
"b": map[string]any{"type": "number"},
}},
Run: func(args map[string]any) (string, error) {
return fmt.Sprintf("%g", args["a"].(float64)+args["b"].(float64)), nil
},
}).
MaxToolIterations(5)
resp, err := agent.Prompt(ctx, "What is 2+3?")Upload files to a provider. Path and Bytes are mutually exclusive
on the same *Upload; Bytes requires Filename. The returned
File plugs into *Text.File(id):
file, err := c.Upload.Path("document.pdf").Run(ctx)
if err != nil {
return err
}
resp, err := c.Text.
File(file.ID).
Prompt(ctx, "Summarize this document")In-memory variant:
file, err := c.Upload.
Bytes(payload).
Filename("greeting.txt").
MimeType("text/plain").
Run(ctx)Attach an image to a text prompt with *Text.Image(mime, bytes); it is sent
as the provider's native image block (works on Anthropic, OpenAI, Google, and
Bedrock). Bytes-based, so no filesystem is required:
resp, err := c.Text.
Image("image/png", screenshotBytes).
Prompt(ctx, "Describe this screenshot in one sentence.")Generate images from text, optionally conditioned on reference images for
editing or composition. Use the typed-builder chain on c.Image:
c := llmkit.New(providers.Google, key)
resp, err := c.Image.Model("gemini-nano-banana-2.1").
AspectRatio("16:9").ImageSize("2K").
Generate(ctx, "A nano banana dish in a fancy restaurant")
os.WriteFile("out.png", resp.Images[0].Bytes, 0o644)For editing or compositional generation, accumulate text and image parts on the chain — the on-wire ordering matches the call order:
resp, err := c.Image.Model("gemini-nano-banana-2.1").
Text("Person:").Image("image/png", personBytes).
Text("Outfit:").Image("image/png", outfitBytes).
Generate(ctx, "Generate the person wearing the outfit.")The trailing Generate(ctx, msg) argument is desugared into a final
text Part appended to the chain — pass "" to omit it when every Part
is already supplied.
The table lists the aspect ratios and sizes each model documents. The SDK sends any value you set, listed or not, and the provider answers a value it does not accept. OpenAI takes any size within its documented bounds.
| Provider | Model | Aspect ratios | Sizes |
|---|---|---|---|
| Nano Banana 2.1 | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9, 1:4, 4:1, 1:8, 8:1 | 1K, 2K, 4K | |
| OpenAI | gpt-image-2 / 1.5 / 1 / 1-mini | n/a (size only) | any (e.g. 1024x1024, 1536x1024) |
| xAI | grok-imagine-image-quality | 1:1, 2:3, 3:2, 3:4, 4:3, 9:16, 16:9, 1:2, 2:1, 19.5:9, 9:19.5, 20:9, 9:20, auto | 1k, 2k |
OpenAI gpt-image-* models accept arbitrary sizes within documented
bounds (max edge ≤3840, both edges multiples of 16, ratio ≤3:1, total
pixels 655K–8.3M). They always return base64-encoded images, so
resp.Images[0].Bytes works the same on both providers.
Provider knobs are typed chain methods on *Image:
| Method | Provider support | Wire field |
|---|---|---|
Quality(s) |
OpenAI gpt-image-* | quality |
OutputFormat(s) |
OpenAI gpt-image-* | output_format |
Background(s) |
OpenAI gpt-image-* | background |
Count(n) |
OpenAI + xAI Grok | n |
Mask(mime, bytes) |
OpenAI gpt-image-* (edits) | multipart mask |
The chain validates per provider — calling Quality(...) on a Google
or xAI builder returns ValidationError immediately, without an HTTP
round-trip. Provider knobs that don't yet have typed methods (OpenAI:
output_compression, moderation) remain reachable via ExtraFields,
which is unvalidated and freeform.
c := llmkit.New(providers.OpenAI, key)
resp, err := c.Image.Model("gpt-image-2").
ImageSize("1024x1024").
Quality("high").
Count(4).
Generate(ctx, "A red circle on a white background")The dispatch is automatic: chains without image parts hit OpenAI's
/v1/images/generations (JSON); chains carrying one or more Image(...)
parts hit /v1/images/edits (multipart/form-data with one image[]
field per reference, in caller order).
OpenAI gpt-image-* models require organization verification — see platform.openai.com/docs/guides/your-data#organization-verification.
Up to 14 reference images per Google request, 16 per OpenAI request.
See examples/image-gen (Google) and examples/image-gen-openai (OpenAI)
for end-to-end runnable samples.
Generate audio from a text prompt. Use the typed-builder chain on
c.Music; the trailing Generate(ctx, prompt) argument is the prompt
text. Decoded audio bytes come back on resp.Audio[0].Bytes.
c := llmkit.Vertex(token).BaseURL(vertexBaseURL)
resp, err := c.Music.Model("lyria-002").
Generate(ctx, "a calm, slow instrumental with warm piano and soft strings")
os.WriteFile("out.wav", resp.Audio[0].Bytes, 0o644)Models that support vocals take lyrics via the .Lyrics(...) chain
method (use section tags like [verse] / [chorus]):
c := llmkit.New(providers.Google, key)
resp, err := c.Music.Model("lyria-3-pro-preview").
Lyrics("[verse] neon lights over the avenue").
Generate(ctx, "dream pop, 90 bpm")Instrumental-only models reject lyrics before the request is sent.
| Provider | Model(s) | Lyrics | Output |
|---|---|---|---|
| Vertex | lyria-002 (Lyria 2) |
no | WAV (~30s) |
lyria-3-pro-preview, lyria-3-clip-preview |
yes | MP3 | |
| MiniMax | music-2.6 |
yes | MP3 |
Vertex uses OAuth bearer auth instead of API keys. Pass an access token
(e.g. from gcloud auth print-access-token) and a base URL with your
{project_id} and {location} filled in:
https://{location}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{location}/publishers/google/models.
The SDK does not refresh the token.
See examples/music-gen for an end-to-end runnable sample.
Generate video from a text prompt. Video generation is asynchronous: Submit
returns a VideoHandle immediately; Wait polls until the job finishes. The
result carries a temporary hosted URL on resp.Videos[0].URL — download it
yourself (url delivery). The handle holds the request id and provider, so
Wait works across process boundaries.
c := llmkit.Grok(key)
h, err := c.Video.Model("grok-imagine-video").
Submit(ctx, "a slow cinematic drone shot over snow-capped alpine peaks")
resp, err := h.Wait(ctx)
v := resp.Videos[0]
fmt.Printf("url=%s duration=%ds mime=%s\n", v.URL, v.DurationSeconds, v.MimeType)| Provider | Model | Delivery |
|---|---|---|
| Grok | grok-imagine-video |
URL |
See examples/video-gen for an end-to-end runnable sample.
Control content filtering for Gemini providers. SafetySettings applies to text
generation, streaming, agents, and Gemini image generation.
import llmkit "github.com/aktagon/llmkit-go/v4"
// Gemini text or agent
resp, err := c.Text.
SafetySettings([]llmkit.SafetySetting{
{Category: llmkit.HarmCategoryDangerousContent, Threshold: llmkit.HarmBlockThresholdNone},
{Category: llmkit.HarmCategoryHarassment, Threshold: llmkit.HarmBlockThresholdHighOnly},
}).
Prompt(ctx, "Write a story")SafetySettings on a non-Gemini provider returns a *ValidationError. The
HarmCategory* and HarmBlockThreshold* constants cover all documented
values; raw strings also work.
c.Models and c.Providers cover model discovery. Every list comes live from the provider's /v1/models endpoint. Runnable counterpart at examples/catalogue/main.go.
// 1. Live list across configured providers.
live, err := c.Models.List(ctx) // LiveResult -- fan-out
chat, err := c.Models.WithCapability(llmkit.CapChatCompletion).List(ctx)
fresh, err := c.Models.Fresh().List(ctx) // skip the cache, refill it
// 2. Providers namespace.
c.Providers.List() // configured (credentials + /v1/models endpoint)
providers.List() // every provider the SDK ships with (static, keyless)
// 3. Scoped to one provider.
p := llmkit.Provider{Name: "anthropic", APIKey: "sk-..."}
info, err := c.Models.Provider(p).Get(ctx, "claude-opus-4-7") // single model
scoped, err := c.Models.Provider(p).List(ctx) // single-provider list
raw, err := c.Models.Provider(p).Raw().List(ctx) // ModelInfo.Raw populatedList(ctx) calls every configured provider's /v1/models and aggregates results into LiveResult.Models + a per-provider LiveResult.Errors map (partial success is the normal case). Each client keeps every provider's list in memory for one hour, so a repeat List or a Provider(p).Get for a listed ID makes no HTTP call. A failed call is not kept. Fresh() makes the call again and refills the cache. Provider(p).Raw().List(ctx) opts into populating ModelInfo.Raw with the provider-native record -- useful when you need fields the universal ModelInfo does not carry (Anthropic's capability matrix, Google's supportedGenerationMethods, etc.).
llmkit.KnownModels(name) returns the models this SDK version states facts for, as a []ModelInfo sorted by id. It needs no client, no API key and no network.
for _, m := range llmkit.KnownModels(providers.Google) {
if m.Speech != nil {
fmt.Println(m.ID, m.DisplayName, m.Speech.MaxSpeakers, m.Speech.Rejects)
}
}Each model carries its capabilities, its label as the display name, and the hints Image, Video, Speech and Music. Speech.Rejects names the request options the SDK rejects before send for that model, such as "style". Live fields (Description, ContextWindow, Raw) are empty, and the provider carries no API key. The list is not proof that the provider serves a model; use c.Models.Provider(p).List(ctx) for that. json.Marshal(m) writes the JSON form the Go, TypeScript and Rust SDKs share.
Sampling and decoding knobs are typed chain methods on *Text and
*Agent. They're all PascalCase and return a fresh builder:
c.Text.
Temperature(0.7).
TopP(0.9).
TopK(40).
MaxTokens(1000).
StopSequences("END").
Seed(42).
FrequencyPenalty(0.5).
PresencePenalty(0.5).
ThinkingBudget(2000).
ReasoningEffort("high").
Prompt(ctx, "...")*Agent exposes the same set plus MaxToolIterations(n). *Text
exposes History(...Message) for multi-turn replay; *Agent retains
history internally across Prompt calls instead.
A Message is a role plus an ordered list of parts. A user turn takes
text, image and file parts; an assistant turn takes text and tool call
parts; a tool turn takes tool result parts. Any turn can carry an image:
resp, err := c.Text.History(
llmkit.Message{Role: "user", Parts: []llmkit.Part{
{Text: "What is in this image?"},
{Image: &llmkit.MediaRef{MimeType: "image/png", Bytes: png}},
}},
llmkit.Message{Role: "assistant", Parts: []llmkit.Part{{Text: "A single white pixel."}}},
).Prompt(ctx, "What color is that pixel?")A tool call part's Input is the argument JSON as the provider sent it,
and the request and EncodeResponse send that text back as it is.
| Option | anthropic | openai | grok | |
|---|---|---|---|---|
| temperature | x | x | x | x |
| top_p | x | x | x | x |
| top_k | x | x | x | |
| max_tokens | x | x | x | x |
| stop_sequences | x | x | x | x |
| seed | x | x | x | |
| frequency_penalty | x | x | ||
| presence_penalty | x | x | ||
| thinking_budget | x | x | ||
| reasoning_effort | x | x |
BaseURL retargets the API host — any OpenAI-compatible server (vLLM, LM
Studio, Ollama, corporate gateways):
c := llmkit.OpenAI("anything").BaseURL("http://localhost:8080/v1")AddHeader attaches a custom HTTP header to every request — for example an
authenticated gateway that needs its own auth header alongside the provider
key. AddHeader is chainable and calls accumulate.
c := llmkit.Anthropic(apiKey).
BaseURL("https://gateway.example.com/anthropic").
AddHeader("cf-aig-authorization", "Bearer "+gatewayToken)The custom header is sent in addition to the provider's auth header; it cannot override the provider auth header or the required version header.
Register pre/post hooks around LLM requests, tool calls, cache creation, uploads, and batch submits. Pre-phase middleware can veto an operation by returning a non-nil error; post-phase runs for observation only.
import (
"context"
"fmt"
"github.com/aktagon/llmkit-go/v4"
"github.com/aktagon/llmkit-go/v4/providers"
)
// Observation: log token usage after every LLM request.
func logUsage(ctx context.Context, e providers.Event) error {
if e.Op == providers.OpLLMRequest && e.Phase == providers.PhasePost {
// Both dimensions are optional; a middleware that prints or bills on
// them must decide what an unreported turn means rather than treat it
// as zero.
if e.Usage.Input != nil && e.Usage.Output != nil {
fmt.Printf("%s/%s: %d in, %d out, took %s\n",
e.Provider, e.Model,
*e.Usage.Input, *e.Usage.Output, e.Duration)
}
}
return nil
}
// Veto: abort if a daily budget is exceeded (pre-phase).
func budgetGate(limit float64, spent *float64) providers.MiddlewareFn {
return func(ctx context.Context, e providers.Event) error {
if e.Op == providers.OpLLMRequest && e.Phase == providers.PhasePre && *spent >= limit {
return fmt.Errorf("daily budget $%.2f exceeded", limit)
}
return nil
}
}
c.Text.
AddMiddleware(budgetGate(5.00, &spent), logUsage).
Prompt(ctx, "Hello")See examples/middleware/ for a spend-cap implementation with a price
table and mutex-guarded accumulation. Middlewares fire in registration
order; the first pre-phase non-nil error aborts.
Streaming uses the same middleware shape: one pre-phase before the
request, one post-phase after the stream closes. Event.Usage
reflects the accumulated usage at stream close. Per-chunk observation
stays on the *TextStream.Chunks() range loop.
Opt-in OpenTelemetry. Attach a Telemetry and every call — success and
rejection alike — produces one OTEL GenAI span (operation, provider, model,
token usage, and error.type on failure) as standards-compliant OTLP/JSON
bytes. llmkit builds the span; you decide where the bytes go. Off unless
attached.
import "github.com/aktagon/llmkit-go/v4"
// Batteries: POST every span to an OTLP collector.
c := llmkit.New("openai", os.Getenv("OPENAI_API_KEY")).
AddTelemetry(llmkit.Telemetry{
Export: llmkit.HTTPExport("https://collector:4318", nil),
})
// Or bring your own transport — hand the bytes to your OTEL SDK:
c.AddTelemetry(llmkit.Telemetry{
Export: func(b []byte) { batchProcessor.Enqueue(b) },
})
resp, err := c.Text.Prompt(context.Background(), "Hello")HTTPExport is a synchronous, fail-open POST — convenient for low volume; for
high volume hand your own callback into your OTEL SDK's batch processor. The
same OTLP span shape is emitted byte-for-byte across all seven SDKs, so one
collector serves a polyglot fleet. A Telemetry with no Export is a
ValidationError.
# Install
go install github.com/aktagon/llmkit-go/v4/cmd/llmkit@latest
# Usage
llmkit -provider anthropic -system "You are helpful" -user "Hello"
llmkit -provider openai -stream -system "Count to 5" -user "Go"
llmkit -provider google -system "Extract color" -user "Sky is blue" \
-schema '{"type":"object","properties":{"color":{"type":"string"}}}'*Agent history persists across process boundaries through two paired
functions:
data, _ := bot.Save() // []byte
// ...later, fresh process...
bot, err := c.Agent.System("...").Tool(t).Load(data)
if errors.Is(err, llmkit.ErrUnsupportedWireVersion) { /* upgrade prompt */ }Or the free-function form for admin tooling:
data, _ := llmkit.SaveHistory(msgs) // []byte
msgs, _ := llmkit.LoadHistory(data) // []llmkit.MessageThe output is a JSON document with a _v integer envelope plus a
messages array. Each message is a role plus its parts, and image bytes
are inline base64. The version is tracked through
llmkit.WireSchemaVersion, currently 2. New optional fields may arrive
under one version and still load on older readers. A renamed, removed,
or retyped field requires a _v bump. LoadHistory rejects a version 1
document with ErrUnsupportedWireVersion. There is no migrator, so
rebuild such a history from your own records.
SaveHistory / LoadHistory are the ONLY guaranteed-stable
serialization path. Direct json.Marshal on a Message value
produces valid JSON but lacks the _v envelope, and LoadHistory
rejects it with ErrMissingWireVersion. Use the contract path for
anything that crosses a process boundary or a release.
This repo is a read-only mirror of a private source. File issues and feature requests here; patches should be submitted against the private source via christian@aktagon.com.
MIT