8000 docs(migrations): document xmc-profile upgrades after component cutover by VLukiantsev · Pull Request #1606 · google/xmc · GitHub
[go: up one dir, main page]

Skip to content
8000

docs(migrations): document xmc-profile upgrades after component cutover - #1606

Merged
VLukiantsev merged 3 commits into
mainfrom
docs/components-migration-happy-path
Nov 6, 2026
Merged

VLukiantsev merged 3 commits into
mainfrom
docs/components-migration-happy-path

Conversation

@VLukiantsev
@VLukiantsev VLukiantsev commented Nov 6, 2026 •
Copy link
Copy Markdown
Collaborator

Summary

Closes two happy-path gaps in the component migration runbook (docs/migrations/components.md) ahead of 0.18.0. It documents how to keep xmc-profile up to date after the cutover, and replaces the elided gcluster command in Step 3 with the exact commands.

Motivation & Context

The migration targets existing installations on 0.18.x. After cutover, operators keep upgrading XMC, but nothing in the docs mentions that:

  • xmc-profile is a separate Helm release. Neither gcluster deploy nor an umbrella helm upgrade touches it.
  • migrate.py prepare pins the gateway image that was running at migration time: components.ingestion-gateway.imageTag, plus parameters.image when the umbrella sets ingestion_gateway_custom_image, as Cluster Toolkit does. These pinned values take precedence over the DEFAULT_INGESTION_GATEWAY_IMAGE that xmc-bridge would otherwise use (ResolveImage precedence). Without the new step, the gateway stays on the pre-migration image after later upgrades.

Scope is deliberately limited to the happy path. Rollback variants and troubleshooting are left out on purpose: the migration has a very small set of existing installations, which will be supported directly, and new installations start with components enabled.

Key Changes

  • docs/migrations/components.md:
    • New After Migration: Upgrading XMC section: keep enable_components: true, bump the pinned gateway image in profile-values.yaml, then re-run the Step 4 xmc-profile upgrade.
    • Step 3 Cluster Toolkit bullet now spells out the gcluster create / gcluster deploy --only xmc_app commands used in INSTALL.md.
  • INSTALL.md upgrade Step 6: links the new section.
  • INSTALL.md upgrade Step 4 (Option A, Cluster Toolkit): re-compile the blueprint (gcluster create … -w --force) before gcluster deploy --only xmc_app. Without it, deploy re-applies a stale ctk-out/ that lacks the new image_tag and chart templates.

Assumptions

  • Existing migrating installations follow the happy path, and failures are supported directly, so rollback and troubleshooting docs are out of scope for this PR. If that's wrong, operators on Cluster Toolkit would lack a documented rollback for the Terraform-managed umbrella release.
  • Recommending "bump the pinned tag" rather than "remove the pinned keys" is intentional: with --reset-then-reuse-values, deleting a key from the file does not clear the previously supplied value.

Verification & Testing

  • Commands Executed: ./.githooks/pre-commit (docs-only; linters pass, affected tests skipped). CI: 17/17 checks green.

Live Deployment & Manual Testing

  • Deployment Target: GKE dev cluster (pool slot) via dev_env.py gke-upgrade, built from this branch (code identical to main). Includes the stability-contract, vizier/hpo profile and bridge RBAC changes merged since feat(migrations): add one-time component migration script and runbook #1461.
  • Cluster Health: verify_xmc.py: 17/17 pods Ready, services, routes, TLS and log monitor all ✅ PASS.
  • Scenarios Executed:
    1. Legacy baseline:
      • Action: gke-upgrade --no-enable-components with in-cluster Redpanda/ClickHouse metrics addons.
      • Result: umbrella-owned ingestion-gateway-deployment 3/3, HPA, and <release>-ingestion-gateway-network-policy.
    2. Step 1 prepare:
      • Action: migrate.py prepare -n xmc -r <release> -o profile-values.yaml.
      • Result: schema-valid values. parameters.image was pinned to the running ingestion_gateway_custom_image (...:latest) and imageTag: latest. This confirms the pinning the new "After Migration" section documents.
    3. Step 2 handoff:
      • Action: migrate.py handoff -n xmc -r <release>.
      • Result: Deployment, Service and HPA transferred. All 3 pod UIDs unchanged, 0 restarts.
    4. Steps 3–4 (Cluster Toolkit under dev_env):
      • Action: gke-upgrade --enable-components --profile-values <abs-path> --only app.
      • Result: ran the same gcluster create … -w --force + gcluster deploy … --only xmc_app --auto-approve documented in Step 3, then helm upgrade --install --reset-then-reuse-values xmc-profile …. Gateway pods untouched until the bridge adopted the release.
    5. Step 5 finalize:
      • Action: migrate.py finalize -n xmc -r <release>.
      • Result: redpanda, clickhouse and ingestion-gateway COMPONENT_STATE_ENABLED (Ready/Synced); vizier/hpo DISABLED. xmc-cmp-ingestion-gateway rev 1 deployed, legacy NetworkPolicy replaced by ingestion-gateway-network-policy, /readyz OK. The bridge adoption did a rolling update of the gateway pods (0 restarts).
    6. After Migration: image bump:
      • Action: changed components.ingestion-gateway.parameters.image in profile-values.yaml to the digest form of the same image, then re-ran the documented helm upgrade --install --reset-then-reuse-values xmc-profile ….
      • Result: Component spec and gateway Deployment image updated to the new reference, xmc-cmp-ingestion-gateway rev 2 deployed, rolling update completed, Ready/Synced, /readyz OK.
Live Execution Output (Sanitized)
# handoff
{"command": "handoff", "undo": false, "transferred": ["deployment/ingestion-gateway-deployment", "service/ingestion-gateway-service", "horizontalpodautoscaler/ingestion-gateway-hpa"], "skipped": false}

# finalize
{"command": "finalize", "ingestion_gateway_enabled": true, "finalized": ["deployment/ingestion-gateway-deployment", "service/ingestion-gateway-service", "horizontalpodautoscaler/ingestion-gateway-hpa"]}
NAME                STATE                      READY   SYNCED   CLASS
clickhouse          COMPONENT_STATE_ENABLED    True    True     analytical-db
hpo                 COMPONENT_STATE_DISABLED   False   True     hyperparameter-optimization
ingestion-gateway   COMPONENT_STATE_ENABLED    True    True     metrics-ingestion
redpanda            COMPONENT_STATE_ENABLED    True    True     event-stream
vizier              COMPONENT_STATE_DISABLED   False   True     optimization-engine

# after-migration image bump
DEPLOY_IMAGE=<registry>/ingestion-gateway-<env>@sha256:1ebbba89…
deployment "ingestion-gateway-deployment" successfully rolled out
xmc-cmp-ingestion-gateway  REVISION 2  deployed
READYZ OK

Related Issues / Bugs

N/A

TAG=agy
CONV=5fd8cc79-e39e-4f23-8a80-e529a7076077

Add an "After Migration: Upgrading XMC" section to the component
migration runbook: xmc-profile is a separate Helm release that gcluster
and umbrella upgrades do not touch, and `migrate.py prepare` pins the
gateway imageTag (and parameters.image when a custom gateway image is
set), which takes precedence over the xmc-bridge default image. Replace
the elided gcluster command in Step 3 with the exact create/deploy
commands from INSTALL.md, and link the new section from INSTALL.md
upgrade Step 6.

TAG=agy
CONV=5fd8cc79-e39e-4f23-8a80-e529a7076077
@gemini-code-assist gemini-code-assist Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the documentation for migrating legacy metrics workloads to XMC components. It adds explicit gcluster commands for standalone cluster toolkit deployments and introduces a new section detailing the post-migration upgrade process for the separate xmc-profile Helm release. A review comment suggests using the <IMAGE_TAG> placeholder in the upgrade instructions to maintain terminology consistency with INSTALL.md.

Comment thread docs/migrations/components.md Resolved
Comment link: #1606 (comment)

TAG=agy
CONV=5fd8cc79-e39e-4f23-8a80-e529a7076077
@VLukiantsev
VLukiantsev marked this pull request as ready for review November 6, 2026 13:54
janzak-dev
janzak-dev previously approved these changes Nov 6, 2026
Comment thread INSTALL.md Resolved
Comment thread INSTALL.md Resolved
Comment link: #1606 (comment)

TAG=agy
CONV=5fd8cc79-e39e-4f23-8a80-e529a7076077
@github-actions github-actions Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Incremental PR Approval

PR author is a code owner and changes since last approval (3 lines) are within the 50-line grace delta.

Preserving prior approval by @janzak-dev on commit 7518a38.

@VLukiantsev
VLukiantsev added this pull request to the merge queue Nov 6, 2026
Merged via the queue into main with commit b103091 Nov 6, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

0