Repository navigation
Support multiple model providers, custom endpoints, and Kubernetes Secret references for API keys #516
Description
I would like AX to support model providers beyond Google, including OpenAI, Anthropic, etc., and services exposing compatible APIs.
The provider configuration should support:
- Selecting the provider and model.
- Configuring a custom API endpoint or base URL, including OpenAI-compatible services and self-hosted model gateways.
- Reading the provider’s API key from a specified Kubernetes Secret.
AX already exposes a secretKey reference in its Model configuration. It would be useful to support an explicit Secret namespace and ensure the selected provider, endpoint, model, and credentials are used consistently throughout model execution, including workspace bootstrap.
For example, the following illustrates the desired configuration; the exact field names are open to discussion:
apiVersion: ax.io/v1alpha1
kind: Model
metadata:
name: default-model
atespace: default
spec:
provider: openai
model: my-model
baseURL: https://api.example.com/v1
secretKey:
namespace: kagent
name: kagent-my-provider
key: PROVIDER_API_KEY
The Kubernetes namespace containing the Secret should be independent of the AX/Substrate atespace. In this example, the Model belongs to atespace default, while its credentials come from the kagent Kubernetes namespace. Secret access should use the AX server’s Kubernetes identity and respect RBAC permissions.
Expected behavior:
- Existing Google provider configurations continue to work.
- OpenAI, Anthropic, and compatible custom endpoints can be configured through Model resources.
- API keys are resolved from Kubernetes Secrets without embedding them in manifests.
- Unsupported providers, invalid endpoints, and missing or inaccessible credentials produce actionable errors.
This would allow AX to work with existing enterprise model providers and credential management setups while keeping task execution configuration declarative.
Activity