8000 [pull] main from can1357:main by pull[bot] · Pull Request #106 · ericrihm/oh-my-pi · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
8000
Diff view
Diff view
569 changes: 292 additions & 277 deletions MODULE.bazel.lock

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,7 @@ modelRoles:

### Four knobs that make routing useful

- **Custom providers** — Declare anything that speaks `openai-completions`, `openai-responses`, `openai-codex-responses`, `azure-openai-responses`, `anthropic-messages`, `bedrock-converse-stream`, `google-generative-ai`, `google-gemini-cli`, or `google-vertex` in `~/.omp/agent/models.yml`.
- **Custom providers** — Declare anything that speaks `openai-completions`, `openai-responses`, `openai-codex-responses`, `azure-openai-responses`, `anthropic-messages`, `bedrock-converse-stream`, `google-generative-ai`, `google-gemini-cli`, `google-vertex`, `typesafe`, or `openrouter-decisions` (the two judge APIs) in `~/.omp/agent/models.yml`.
- **Fallback chains** — Per-role or per-model chains under `retry.fallbackChains`. When the primary throws 429s or hits a quota wall, the next entry takes the rest of the turn — restored on cooldown.
- **Path-scoped models** — Scope `enabledModels` and `disabledProviders` entries to a `path:` prefix to pin a different model set on one repo without touching the global config. Scoped entries cover the path and everything under it.
- **Round-robin credentials** — Stack API keys per provider and the runtime rotates with session affinity and per-credential backoff. Useful when one key would burn its quota by lunch.
Expand Down
3 changes: 3 additions & 0 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 20 additions & 2 deletions docs/auth-broker-gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
The auth broker and auth gateway are two cooperating HTTP services that move OAuth refresh tokens and provider access tokens off developer laptops and into a single broker host.

- **`omp auth-broker serve`** holds the canonical SQLite credential vault, performs OAuth refreshes, and exposes snapshot, credential, block, usage, and health APIs under `/v1`.
- **`omp auth-gateway serve`** is a forward-proxy. It accepts OpenAI Chat Completions, Anthropic Messages, OpenAI Responses, and pi-native stream requests, resolves the broker-backed credential, and dispatches through `pi-ai` provider logic. Clients (containerised omp, llm-git, the macOS usage widget, …) never see the access token.
- **`omp auth-gateway serve`** is a forward-proxy. It accepts OpenAI Chat Completions, Anthropic Messages, OpenAI Responses, pi-native stream, TypeSafe System One judgment, and OpenAI/OpenRouter-style image, speech, transcription, embedding, rerank, and video requests, resolves the broker-backed credential, and dispatches through `pi-ai` provider logic. Clients (containerised omp, llm-git, the macOS usage widget, …) never see the access token.

Transport security between operator, broker, and gateway is delegated to the operator (Tailscale / Wireguard / reverse proxy + TLS). Every endpoint except `/v1/healthz` (broker) and `/healthz` (gateway) requires a bearer token.

Expand Down Expand Up @@ -151,8 +151,26 @@ omp auth-gateway check [--strict] [--json]
| `POST` | `/v1/messages` | bearer | Anthropic Messages wire format |
| `POST` | `/v1/responses` | bearer | OpenAI Responses wire format |
| `POST` | `/v1/pi/stream` | bearer | Native `pi-ai` stream wire format |
| `POST` | `/v1/systemone` | bearer | TypeSafe System One judgments (`judge` models, e.g. `typesafe/jev-latest`); `/alpha/decisions` is the OpenRouter Decisions alias |
| `POST` | `/v1/images/generations` | bearer | Image generation, OpenAI Images JSON wire; `/v1/images` is the OpenRouter alias (`image` models) |
| `POST` | `/v1/images/edits` | bearer | Image edits: OpenAI multipart or OpenRouter JSON input images |
| `POST` | `/v1/audio/speech` | bearer | Text-to-speech, OpenAI/OpenRouter JSON wire; answers raw audio bytes (`tts` models: `xai-tts`, `openai-speech`) |
| `POST` | `/v1/audio/transcriptions` | bearer | Speech-to-text, OpenAI multipart `file` or OpenRouter JSON `input_audio` base64 (`stt` models on `openai-transcriptions`; 25 MiB cap) |
| `POST` | `/v1/embeddings` | bearer | Embeddings, OpenAI wire (`embedding` models on `openai-embeddings`; OpenAI + OpenRouter; 8 MiB cap) |
| `POST` | `/v1/rerank` | bearer | Rerank, OpenRouter wire (`rerank` models on `openrouter-rerank`) |
| `POST` | `/v1/videos` | bearer | Submit a video generation job, OpenRouter wire (`video` models on `openrouter-video`); answers `202` with gateway-rewritten polling/content URLs |
| `GET` | `/v1/videos/:id` | bearer | Poll a video job. `:id` is gateway-issued and stateless: it encodes provider, model, and upstream job id |
| `GET` | `/v1/videos/:id/content` | bearer | Stream the finished video bytes with the upstream content type |

The model id is read from the top-level `model` field for foreign wire formats and from the pi-native request body for `/v1/pi/stream`. The gateway picks the first bundled `Model<Api>` matching that id, parses the inbound wire format into an omp `Context`, resolves the provider credential from broker-backed `AuthStorage`, dispatches through `streamSimple()`, and re-encodes the result to the inbound format (SSE for streamed responses).
The model id is read from the top-level `model` field for foreign wire formats and from the pi-native request body for `/v1/pi/stream`. It may be provider-qualified (`typesafe/jev-latest`) or bare (`jev-latest`). The gateway resolves it against the served catalog — every registry model of a kind the gateway has a route for (`chat`, `judge`, `image`, `tts`, `stt`, `embedding`, `rerank`, `video`), scoped to providers the broker holds credentials for — parses the inbound wire format, resolves the provider credential from broker-backed `AuthStorage`, dispatches through the matching `pi-ai` client (`streamSimple()` for chat, `TypeSafeJudge` for judgments, `generateImage` / `synthesizeSpeech` / `transcribeAudio` / `embed` / `rerank` / `submitVideo` for the modality routes), and re-encodes the result to the inbound format (SSE for streamed chat responses).

Chat routes reject non-chat models with a `400` that names the route to use instead (`Model typesafe/jev-latest is a judge model; use POST /v1/systemone`). `GET /v1/models` marks such rows with `kind` (`judge` | `image` | `tts` | `stt` | `embedding` | `rerank` | `video`); absent means chat.

Non-chat OpenRouter rosters are discovered live (`/embeddings/models`, `/videos/models`, rerank-flagged `/models` rows, 10BC8 TTS/STT via models.dev kinds) with small `bundle="always"` fallbacks in `providers/openrouter.kdl`; a provider maps each kind to its runner API through `kind-apis` in KDL. Per-search, per-second, and per-character billing have no catalog cost axis, so those rows carry zero token cost and the provider-reported `cost` in the response is authoritative.

Cost attribution is uniform: every route records observed usage against the caller's `x-omp-*` identity and carries the computed cost in `x-litellm-response-cost`. Upstreams that report tokens only (TypeSafe) are priced from the catalog model; the response body's own `cost` field (OpenRouter shape) is only present when the upstream billed one.

Pointing a TypeSafe SDK or omp's own `judge` role at the gateway means `TYPESAFE_BASE_URL=http://gateway:4000` with `TYPESAFE_API_KEY=<gateway token>`; OpenAI-SDK-style clients set their base URL to `http://gateway:4000/v1`.

There is no raw provider passthrough path. All supported routes go through `pi-ai` provider logic so credential-specific request shaping, OAuth refresh-on-auth-error, and provider quirks stay centralized.

Expand Down
4 changes: 4 additions & 0 deletions docs/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,10 @@ providers:
- `google-generative-ai`
- `google-gemini-cli`
- `google-vertex`
- `typesafe`
- `openrouter-decisions`

`typesafe` and `openrouter-decisions` are judgment APIs, not chat transports: a model declared with one answers System One judgment requests (`{baseUrl}/v1/systemone` and `{baseUrl}/decisions` respectively) and is selected by the `judge` model role. Its `headers` carry gateway routing or custom authentication headers for that traffic.

### Allowed auth/discovery values

Expand Down
20 changes: 20 additions & 0 deletions packages/ai/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,26 @@

## [Unreleased]

### Added

- Added `TypeSafeJudgeOptions.headers`, sent on every System One judgment request so proxy gateways can require routing or custom authentication headers.
- Added support for text embeddings
- Added support for document reranking
- Added support for video generation
- Added support for image generation across multiple providers
- Added support for audio speech synthesis
- Added support for audio transcription services
- Added System One judgment API route

### Changed

- Updated API response headers to report costs based on usage totals
- Added optional `kind` field to model list responses

### Fixed

- Fixed detection of Claude usage limit errors

## [18.2.7] - 2026-09-21

### Breaking Changes
Expand Down
24 changes: 24 additions & 0 deletions packages/ai/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,34 @@
"types": "./src/error/index.ts",
"import": "./src/error/index.ts"
},
"./embeddings": {
"types": "./src/embeddings/index.ts",
"import": "./src/embeddings/index.ts"
},
"./judgment": {
"types": "./src/judgment/index.ts",
"import": "./src/judgment/index.ts"
},
"./rerank": {
"types": "./src/rerank/index.ts",
"import": "./src/rerank/index.ts"
},
"./speech": {
"types": "./src/speech/index.ts",
"import": "./src/speech/index.ts"
},
"./images": {
"types": "./src/images/index.ts",
"import": "./src/images/index.ts"
},
"./transcription": {
"types": "./src/transcription/index.ts",
"import": "./src/transcription/index.ts"
},
"./video": {
"types": "./src/video/index.ts",
"import": "./src/video/index.ts"
},
"./*": {
"types": "./src/*.ts",
"import": "./src/*.ts"
Expand Down
Loading
Loading
0