One TypeScript API for Anthropic, OpenAI, Google, and 20+ other providers — including local models through Ollama and vLLM. Switch providers without rewriting your request.
No runtime dependencies. Runs on Node ≥18, Bun, Deno, Cloudflare Workers, or any modern bundler (Vite, Next.js, esbuild, webpack 5+) — anywhere with fetch and Web Crypto.
Also available for Go, Python, Rust, Swift, and Java.
From npm:
bun add @aktagon/llmkit-ts
# or
npm install @aktagon/llmkit-tsFrom GitHub (skip the npm publish loop):
bun add github:aktagon/llmkit-ts#ts-v2.1.0
# or
npm install github:aktagon/llmkit-ts#ts-v2.1.0The package ships compiled ESM in dist/ (works in plain Node ESM, Workers, Deno) with its TypeScript declarations beside it. No build step required at the consumer.
import { responseText } from "@aktagon/llmkit-ts";
import { anthropic } from "@aktagon/llmkit-ts";
const c = anthropic(process.env.ANTHROPIC_API_KEY!);
const resp = await c.text
.system("You are concise.")
.prompt("Why is the sky blue?");
console.log(responseText(resp));
console.log(resp.usage.input, resp.usage.output);c.text, c.image, c.music, c.video, c.agent, and c.upload are fields on the Client — access them without parentheses. Chain methods (.system(...), .temperature(...)) clone the builder and return the clone, so a forked chain shares no state with its parent. The typed builder is the only public surface as of v1.0.0. One mental model — client.<capability>.<chain>.<terminal> — across every capability.
Runnable examples for each capability live in examples/; tests/examples.test.ts exercises every documented call shape against a mock HTTP server, so the snippets in this README cannot drift from the actual API surface.
| Provider | Default model | Env var |
|---|---|---|
| anthropic | claude-sonnet-4-6 | ANTHROPIC_API_KEY |
| openai | gpt-4o-2024-08-06 | OPENAI_API_KEY |
| gemini-2.5-flash | GOOGLE_API_KEY | |
| bedrock | anthropic.claude-sonnet-4-20250514-v1:0 | AWS_ACCESS_KEY_ID |
| grok | grok-3-fast | XAI_API_KEY |
| mistral | mistral-large-latest | MISTRAL_API_KEY |
| deepseek | deepseek-chat | DEEPSEEK_API_KEY |
| groq | openai/gpt-oss-120b | GROQ_API_KEY |
| together | meta-llama/Llama-3.3-70B-Instruct-Turbo | TOGETHER_API_KEY |
| cohere | command-r-plus | COHERE_API_KEY |
| ai21 | jamba-1.5-large | AI21_API_KEY |
| cerebras | gpt-oss-120b | CEREBRAS_API_KEY |
| ... | (full list in src/providers/providers.ts) |
36 providers, 4 API shapes (OpenAI-compatible, Anthropic Messages, Google Generative AI, AWS Bedrock Converse). Bedrock auth uses SigV4; other providers use API-key auth.
Per-provider factory functions: ai21, anthropic, assemblyai, azure, bedrock, cerebras, cohere, deepseek, doubao, ernie, fireworks, google, grok, groq, inworld, jan, llamacpp, lmstudio, minimax, mistral, moonshot, ollama, openai, openrouter, perplexity, pixverse, qwen, recraft, sambanova, together, vertex, vidu, vllm, workersai, yi, zhipu. Or use the generic newClient(name, key).
const resp = await c.text
.system("You are helpful")
.temperature(0.7)
.maxTokens(200)
.prompt("What is 2+2?");
console.log(responseText(resp)); // "4"
// Every Usage dimension is optional. undefined means the provider did not
// report the value, which is NOT the same as reporting zero: a provider that
// says it used no cached tokens and one that never mentions caching are
// different facts, and a plain 0 cannot tell you which you have.
resp.usage.input; // number | undefined — prompt tokens
resp.usage.output; // number | undefined — completion tokens
resp.usage.cacheRead; // number | undefined — tokens served from cache
resp.usage.cacheWrite; // number | undefined — tokens written to cache (Anthropic explicit)
resp.usage.reasoning; // number | undefined — internal reasoning tokens (OpenAI o-series, Gemini 2.5+)
resp.usage.cost; // number | undefined — provider-reported USD; undefined is unreported, never "free"
if (resp.usage.cacheRead !== undefined) {
console.log(resp.usage.cacheRead, "tokens served from cache");
}Capability-scoped fields (cacheRead, cacheWrite, reasoning) are zero when the provider doesn't report them separately.
const stream = client.text
.system("Be brief")
.stream("Tell me a one-line joke");
for await (const chunk of stream) {
process.stdout.write(chunk);
}
process.stdout.write("\n");
const final = stream.response();
if (final !== null) {
console.log(
`input=${final.usage.input} output=${final.usage.output} ` +
`finishReason=${final.finishReason ?? ""}`,
);
}TextStream implements AsyncIterable<string>. After iteration completes, stream.response() returns the final Response (with token counts) and stream.error() returns any terminal error. Handles both Anthropic-style typed events and OpenAI-style data-only frames internally.
import { responseText, type Tool } from "@aktagon/llmkit-ts";
const add: Tool = {
name: "add",
description: "Add two numbers",
schema: {
type: "object",
properties: {
a: { type: "number" },
b: { type: "number" },
},
},
run: ({ a, b }) => String(Number(a) + Number(b)),
};
const bot = c.agent
.system("You are a calculator.")
.addTool(add)
.maxToolIterations(5);
const resp = await bot.prompt("What is 2+3?");
console.log(responseText(resp));*Agent is stateful — repeated bot.prompt(...) calls accumulate history. Chain methods (.system(...), .addTool(...)) clone and reset state, so a forked builder gets a fresh conversation. bot.reset() clears state without dropping chained config.
A bot.prompt(...) call that fails leaves the history as it was and throws the error that stopped it, such as an APIError. bot.unsavedTurns returns the turns that call made, tool results included. The next bot.prompt(...) call clears them when it starts, and so do bot.reset() and a chain method, whose clone starts with none.
Tool dispatch covers Anthropic tool_use, OpenAI tool_calls, Google functionCall, and Bedrock Converse toolUse. Tool errors surface to the model as the result string verbatim — sanitise tool inputs at the source.
Attach an image to a text prompt with .image(mime, bytes); it is sent as the
provider's native image block (works on Anthropic, OpenAI, Google, and
Bedrock). Bytes-based, so it works with no filesystem (e.g. a browser
extension passing a screenshot straight through):
const resp = await c.text
.image("image/png", screenshotBytes)
.prompt("Describe this screenshot in one sentence.");import { google } from "@aktagon/llmkit-ts";
const c = google(process.env.GOOGLE_API_KEY!);
const img = await c.image
.model("gemini-nano-banana-2.1")
.aspectRatio("16:9")
.imageSize("2K")
.generate("A nano banana dish, studio lighting");
await Bun.write("out.png", img.images[0]!.bytes);For compositional editing, chain .text(...) and .image(mime, bytes) to interleave references with descriptions. The terminal msg is appended as a final text Part:
await c.image
.model("gemini-nano-banana-2.1")
.text("Person:")
.image("image/png", personBytes)
.text("Outfit:")
.image("image/png", outfitBytes)
.generate("Generate the person wearing the outfit.");The table lists the aspect ratios and sizes each model documents. The SDK sends any value you set, listed or not, and the provider answers a value it does not accept. OpenAI takes any size within its documented bounds.
| Provider | Model | Aspect ratios | Sizes |
|---|---|---|---|
| Nano Banana 2.1 | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9, 1:4, 4:1, 1:8, 8:1 | 1K, 2K, 4K | |
| OpenAI | gpt-image-2 / 1.5 / 1 / 1-mini | n/a (size only) | any (e.g. 1024x1024, 1536x1024) |
| xAI | grok-imagine-image-quality | 1:1, 2:3, 3:2, 3:4, 4:3, 9:16, 16:9, 1:2, 2:1, 19.5:9, 9:19.5, 20:9, 9:20, auto | 1k, 2k |
OpenAI gpt-image-* models accept arbitrary sizes within documented bounds (max edge ≤3840, both edges multiples of 16, ratio ≤3:1, total pixels 655K–8.3M). They always return base64-encoded images, so resp.images[0].bytes works the same on both providers.
Provider knobs are typed chain methods on the Image builder:
| Method | Provider support | Wire field |
|---|---|---|
.quality(s) |
OpenAI gpt-image-* | quality |
.outputFormat(s) |
OpenAI gpt-image-* | output_format |
.background(s) |
OpenAI gpt-image-* | background |
.count(n) |
OpenAI + xAI Grok | n |
.mask(mime, bytes) |
OpenAI gpt-image-* (edits) | multipart mask |
The chain validates per provider — calling .quality(...) on a Google or xAI builder rejects with ValidationError immediately, no HTTP round-trip. Knobs without typed methods (OpenAI: output_compression, moderation) remain reachable via .extraFields(...), which is unvalidated and freeform.
import { openai } from "@aktagon/llmkit-ts";
const c = openai(process.env.OPENAI_API_KEY!);
const resp = await c.image
.model("gpt-image-2")
.imageSize("1024x1024")
.quality("high")
.count(4)
.generate("A red circle on a white background");Dispatch is automatic: chains without image parts hit OpenAI's /v1/images/generations (JSON); chains carrying one or more .image(...) parts hit /v1/images/edits (multipart/form-data with one image[] field per reference, in caller order). gpt-image-* requires organization verification — see platform.openai.com/docs/guides/your-data#organization-verification.
Up to 14 reference images per Google request, 16 per OpenAI request.
Generate audio from a text prompt via the typed-builder chain on c.music. Decoded audio bytes come back on resp.audio[0].bytes. Models that support vocals take lyrics via .lyrics(...) (use section tags like [verse]); instrumental-only models reject lyrics before the request is sent.
const r = await client.music
.model("lyria-002")
.generate("a calm instrumental, warm piano and soft strings");
const first = r.audio[0];
if (!first) throw new Error("no audio returned");
await Bun.write("out.wav", first.bytes);Models with vocals take lyrics via .lyrics(...):
const song = await c.music.model("lyria-3-pro-preview").lyrics("[verse] neon lights").generate("dream pop, 90 bpm");| Provider | Model(s) | Lyrics | Output |
|---|---|---|---|
| Vertex | lyria-002 |
no | WAV (~30s) |
lyria-3-pro-preview, lyria-3-clip-preview |
yes | MP3 | |
| MiniMax | music-2.6 |
yes | MP3 |
Generate video from a text prompt. Video generation is asynchronous: submit returns a handle immediately, and handle.wait() polls until the job finishes. The result carries a temporary hosted URL on resp.videos[0].url — download it yourself.
const handle = await client.video
.model("grok-imagine-video")
.submit(
"a slow cinematic drone shot flying over snow-capped alpine peaks at golden hour",
);
const r = await handle.wait();
const v = r.videos[0];
if (!v) throw new Error("no video returned");
console.log(
`done: url=${v.url} duration=${v.durationSeconds}s mime=${v.mimeType}`,
);| Provider | Model | Delivery |
|---|---|---|
| Grok | grok-imagine-video |
URL |
Control content filtering for Gemini providers. safetySettings applies to text
generation, streaming, agents, and Gemini image generation. On a non-Gemini
provider it throws a ValidationError.
import { openai } from "@aktagon/llmkit-ts";
const c = openai(process.env.OPENAI_API_KEY!);
// from a path (Node/Bun only)
const file = await c.upload.path("./data.pdf").run();
// from bytes (works everywhere)
const file2 = await c.upload
.bytes(buf) // Uint8Array
.filename("report.pdf")
.mimeType("application/pdf")
.run();The .path() branch dynamically loads node:fs/promises and is unavailable in browsers / Cloudflare Workers / Deno without --allow-read. Use .bytes() for portable code.
const handle = await client.text
.system("Be brief")
.batch(
"Translate hello to French",
"Translate hello to Spanish",
"Translate hello to German",
);
const results = await handle.wait();
results.forEach((r) => console.log(responseText(r)));c.text.<config>.batch(...prompts) queues the batch and returns a BatchHandle you can persist. Call handle.wait() to block until completion, or handle.poll() to drive the loop yourself. The blocking one-liner is (await c.text.batch(...prompts)).wait(). Both inline (Anthropic) and file-reference (OpenAI two-hop) flows are handled internally.
// Anthropic — explicit cache_control wrap of the system prompt:
await c.text.system(longSysPrompt).caching().prompt("...");
// OpenAI — automatic server-side caching (caching() is a hint; reads
// surface in resp.usage.cacheRead regardless):
await c.text.system(longSysPrompt).caching().prompt("...");
// Google — pre-flight POST creates a cachedContents resource, then the
// main call references it. Google requires ~1k+ tokens of system prompt:
await c.text.system(bigSysPrompt).caching().prompt("...");The mode is provider-specific and inferred from the provider config. The default TTL comes from src/providers/caching.ts (Google: 3600s).
c.models and c.providers cover model discovery. Every list comes live from the provider's /v1/models endpoint. Runnable counterpart at examples/catalogue.ts.
import { Capabilities } from "@aktagon/llmkit-ts";
import type { Provider } from "@aktagon/llmkit-ts";
// 1. Live list across configured providers.
const live = await c.models.list(); // LiveResult — fan-out
const chat = await c.models.withCapability(Capabilities.ChatCompletion).list();
const fresh = await c.models.fresh().list(); // skip the cache, refill it
// 2. Providers namespace.
c.providers.list(); // configured (credentials + /v1/models endpoint)
providers.list(); // every provider the SDK ships with (static, keyless)
// 3. Scoped to one provider.
const p: Provider = { name: "anthropic", apiKey: "sk-..." };
const info = await c.models.provider(p).get("claude-opus-4-7"); // single model
const scoped = await c.models.provider(p).list(); // single-provider list
const raw = await c.models.provider(p).raw().list(); // ModelInfo.raw populatedlist() calls every configured provider's /v1/models in parallel and aggregates results into LiveResult.models + a per-provider LiveResult.errors map (partial success is the normal case). Each client keeps every provider's list in memory for one hour, so a repeat list() or a provider(p).get(id) for a listed id makes no HTTP call. A failed call is not kept. fresh() makes the call again and refills the cache. provider(p).raw().list() opts into populating ModelInfo.raw with the provider-native record — useful when you need fields the universal ModelInfo does not carry (Anthropic's capability matrix, Google's supportedGenerationMethods, etc.).
providers.knownModels(name) returns the models this SDK version states facts for, as a ModelInfo[] sorted by id. It needs no client, no API key and no network.
for (const m of providers.knownModels("google")) {
if (m.speech) console.log(m.id, m.displayName, m.speech.maxSpeakers, m.speech.rejects);
}Each model carries its capabilities, its label as the display name, and the hints image, video, speech and music. speech.rejects names the request options the SDK rejects before send for that model, such as "style". Live fields (description, contextWindow, raw) are empty, and the provider carries no API key. The list is not proof that the provider serves a model; use c.models.provider(p).list() for that. modelInfoToJSON(m) writes the JSON form the Go, TypeScript and Rust SDKs share.
Across every *Text / *Agent builder:
| Concept | Method | Notes |
|---|---|---|
| System prompt | .system(s) |
|
| Model override | .model(name) |
|
| Sampling | .temperature(t) |
|
| Token cap | .maxTokens(n) |
|
| Caching | .caching() |
|
| Conversation hist | .history(...msgs) |
*Text only. *Agent accumulates history across .prompt(...) calls on the same instance, so an explicit setter would shadow that semantics. |
| Structured output | .schema(json) |
OpenAI strict mode requires additionalProperties: false and required on object types. |
| Middleware hooks | .addMiddleware(...fns) |
See below. |
| Reasoning effort | .reasoningEffort(l) |
OpenAI o-series, Gemini 2.5+ |
| Thinking budget | .thinkingBudget(n) |
Anthropic, Gemini |
A Message is a role plus an ordered list of parts. A user turn takes text, image and file parts; an assistant turn takes text and tool call parts; a tool turn takes tool result parts. Any turn can carry an image:
const resp = await c.text
.history(
{
role: "user",
parts: [
{ text: "What is in this image?" },
{ image: { mimeType: "image/png", bytes: png } },
],
},
{ role: "assistant", parts: [{ text: "A single white pixel." }] },
)
.prompt("What color is that pixel?");A tool call part carries input, the arguments as the JSON text the provider sent; toolCallInput(call) parses it. The request, encodeResponse and saveHistory write that text as it is, so key order, escapes and an integer past 2^53 are not changed.
Sampling hyperparameters (.topP, .topK, .seed, .frequencyPenalty, .presencePenalty, .stopSequences) are validated per provider; unsupported options throw ValidationError rather than silently dropping.
The Image builder has a narrower set: .model, .aspectRatio, .imageSize, .includeText, .text, .image, .middleware. Upload: .path, .bytes, .filename, .mimeType, .middleware.
Register pre/post hooks around LLM requests, tool calls, cache creation, uploads, and batch submits. Pre-phase middleware can veto by returning a non-null Error; post-phase runs for observation only.
import type { Event, MiddlewareFn } from "@aktagon/llmkit-ts";
// Observation: log token usage after every LLM request.
const logUsage: MiddlewareFn = (e) => {
if (e.op === "llm_request" && e.phase === "post") {
console.log(
`${e.provider}/${e.model}: ${e.usage?.input} in, ${e.usage?.output} out, ${e.duration?.toFixed(1)}ms`,
);
}
return null;
};
// Veto: abort if a daily budget is exceeded.
const budgetGate =
(limit: number, spent: { value: number }): MiddlewareFn =>
(e) => {
if (e.op === "llm_request" && e.phase === "pre" && spent.value >= limit) {
return new Error(`daily budget $${limit.toFixed(2)} exceeded`);
}
return null;
};
await c.text.addMiddleware(budgetGate(5.0, spent), logUsage).prompt("...");A pre-phase veto throws MiddlewareVetoError so it can be discriminated from transport or provider errors. Middlewares fire in registration order; the first non-null pre-phase return aborts.
Wired at seven sites: Text.prompt, Text.stream, Agent LLM call, Agent tool execution (op=tool_call), Upload.run (op=upload), Text.batch (op=batch_submit), Google resource caching pre-flight (op=cache_create).
Opt-in OpenTelemetry. Attach a Telemetry and every call — success and rejection alike — produces one OTEL GenAI span (operation, provider, model, token usage, and error.type on failure) as standards-compliant OTLP/JSON bytes. llmkit builds the span; you decide where the bytes go. Off unless attached.
import { openai, httpExport } from "@aktagon/llmkit-ts";
// Batteries: POST every span to an OTLP collector.
const client = openai(process.env.OPENAI_API_KEY).addTelemetry({
export: httpExport("https://collector:4318"),
});
// Or bring your own transport — hand the bytes to your OTEL SDK:
client.addTelemetry({ export: (b) => batchProcessor.enqueue(b) });
const resp = await client.text.prompt("Hello");httpExport is a fail-open POST — convenient for low volume; for high volume hand your own callback into your OTEL SDK's batch processor. The same OTLP span shape is emitted byte-for-byte across all seven SDKs, so one collector serves a polyglot fleet. A telemetry config with no export throws a ValidationError.
import { openai } from "@aktagon/llmkit-ts";
const c = openai("anything").baseURL("http://localhost:8080/v1");Works for any OpenAI-compatible server (vLLM, LM Studio, Ollama, corporate gateways).
Attach a custom HTTP header to every request — for example an authenticated gateway that needs its own auth header alongside the provider key. addHeader is chainable and calls accumulate.
import { anthropic } from "@aktagon/llmkit-ts";
const c = anthropic(apiKey)
.baseURL("https://gateway.example.com/anthropic")
.addHeader("cf-aig-authorization", `Bearer ${gatewayToken}`);The custom header is sent in addition to the provider's auth header; it cannot override the provider auth header or the required version header.
*Agent history persists across process boundaries through two paired
functions:
const data = bot.save(); // string
// ...later, fresh process...
const bot = c.agent.system("...").tool(t).load(data);
// throws UnsupportedWireVersionError on mismatchOr the free-function form for admin tooling:
import { saveHistory, loadHistory } from "@aktagon/llmkit";
const data = saveHistory(msgs);
const msgs = loadHistory(data);The output is a JSON document with a _v integer envelope plus a
messages array. Each message is a role plus its parts, and image
bytes are inline base64. The version is tracked through
WIRE_SCHEMA_VERSION, currently 2. New optional fields may arrive
under one version and still load on older readers. A renamed, removed,
or retyped field requires a _v bump. loadHistory rejects a version
1 document with UnsupportedWireVersionError. There is no migrator, so
rebuild such a history from your own records.
saveHistory / loadHistory are the ONLY guaranteed-stable
serialization path. Direct JSON.stringify on a Message produces
valid JSON but lacks the _v envelope, and loadHistory rejects it
with MissingWireVersionError. Use the contract path for anything
that crosses a process boundary or a release.
This repo is a read-only mirror. File issues here; send code patches to christian@aktagon.com.
MIT