FFFF Tasks fail on Substrate after v0.5.0: ActorTemplate.sandbox_config removed, resources and digest-pinned images required · Issue #498 · google/ax · GitHub
[go: up one dir, main page]

Skip to content

Tasks fail on Substrate after v0.5.0: ActorTemplate.sandbox_config removed, resources and digest-pinned images required #498

Open

Description

@maxtuzz

Summary

AX 8d2ccf3 (current main) cannot create tasks on Agent Substrate after v0.5.0. With agent-substrate/substrate@3d3b929 (2026-10-18), every Task fails at template creation. There is also no way to run tasks on a non-gVisor sandbox class such as microvm.

What changed in Substrate

In pkg/proto/ateapipb/ateapi.proto, ActorTemplate changed in four ways:

  1. sandbox_config (field 6) was removed. The WorkerPool now decides the sandbox class, and templates choose a pool through worker_selector.
  2. resources is required. cpu and memory limits are both mandatory.
  3. Container images must be pinned by digest.
  4. The Go module moved from github.com/agent-substrate/substrate to go.ate.dev/substrate. WakeupProbe and HttpGetAction were renamed.

Repro

  • GKE 1.36; Substrate 3d3b929 installed with hack/install-ate.sh --deploy-ate-system.
  • AX 8d2ccf3 deployed with make deploy.
  • Apply a minimal Task:
apiVersion: ax.io/v1alpha1
kind: Task
metadata:
  name: smoke1
  atespace: default
spec:
  resources:
    limits:
      cpu: "2"
      memory: "4Gi"

ax-server logs:

could not create custom ActorTemplate, falling back to default template
  error="creating actor template default/smoke1-tmpl-49b5e690: rpc error: code = InvalidArgument
  desc = actor_template: Invalid value: unknown field with protobuf tag 6"
direct reconcile error on create task task=smoke1
  error="ensuring actor: creating actor default/smoke1: rpc error: code = FailedPrecondition desc = actor template not found"

Once field 6 is gone, Substrate rejects the template again unless resources is set and the image has a @sha256: digest. The default gcr.io/ax-substrate/ate-images/ax-task-runner is tag-only.

Related

Possible fix

BuildActorTemplate in internal/substrate/client.go would:

  • drop SandboxConfig;
  • set Resources from the Task's limits;
  • set WorkerSelector from configuration, so an operator can route tasks to a pool of a given class (for example, ax.dev/pool=microvm).

We're running a minimal version of this against Substrate 3d3b929: gethamster@43fd8d5. It uses an AX_WORKER_SELECTOR env var, falls back to requests and then 2 CPU / 4Gi for limits, and passes go test ./....

We'd be glad to send a PR, either on top of #405/#484 or as a standalone change, once you confirm the direction. In particular: should the worker selector be a server flag, a per-Task field, or both?

Activity

  1. rakyll commented on Oct 19, 2026

    @rakyll
    Member
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0