Repository navigation
test(e2e): add OSS 3P ClickHouse and Redpanda Kind E2E integration suite - #1105
Merged
Merged
Conversation
Implement the end-to-end integration test suite for OSS 3P ClickHouse and Redpanda components on ephemeral KinD clusters (`test/e2e/clickhouse_3p_test.go`, `test/e2e/redpanda_3p_test.go`, and `test/e2e/kind_3p_harness_test.go`): - Provision an isolated ephemeral KinD cluster with automatic cleanup (or reuse `KIND_CLUSTER_NAME` when provided via `dev_env.py`), install the production `components.xmc.dev` CRD and 3P operator CRDs (`clickhouseinstallations.clickhouse.altinity.com` and `clusters.cluster.redpanda.com`), and start `xmc-bridge` backed by an in-memory `ComponentManagerService` (CMS) and NodePort health prober. - Verify both Operator-Managed (`spec.wrapperRef`) and Operator-less (`stable-2026a` `spec.parameters`) reconciliation modes project `Ready=True`, `state=COMPONENT_STATE_ENABLED`, `observedGeneration`, and `.status.access.envVars` (`ENDPOINT`, `HOST`, `PORT`, `SECRET_NAME`). - Validate typed Go component resolvers (`componentconfig.ResolveOptions` and `ingestiongateway.ResolveRedpandaComponentConfig`) against live Kubernetes `Component` CRs and Secrets. - Spawn mock ML workload Pods injected with `.status.access.envVars` to execute SQL queries (`CREATE TABLE`, `INSERT`, `SELECT`) against live ClickHouse and publish/consume Kafka messages via `rpk` against live Redpanda. Fixes bug 548436596 TAG=agy CONV=6bc68b62-40f7-4652-8de6-f5a036d676b2
There was a problem hiding this comment.
Code Review
This pull request introduces standalone Go KinD integration test suites for ClickHouse and Redpanda 3P components, along with a shared test harness (kind_3p_harness_test.go) that provisions ephemeral KinD clusters, deploys mock workloads, and handles NodePort-based health probing. The review feedback highlights several critical improvements for the test harness to ensure reliability and safety in parallel CI environments. These include avoiding broad home directory globbing when resolving binary paths, using full UnixNano() values to prevent cluster and namespace name collisions, making the wait loop context-aware to avoid blocking on cancellation, and limiting the log reader size to prevent out-of-memory crashes. Additionally, a defensive nil check is recommended in the ClickHouse test to prevent potential panics when resolving component options.
Address automated code review feedback on PR #1105: - Replace home directory globbing in `resolveBinaryPath` with `os.UserHomeDir()` (`~/.local/bin`) and `/usr/local/bin`. - Remove modulo truncation on `time.Now().UnixNano()` when generating unique KinD cluster and test namespace names. - Make `waitUntil` immediately responsive to context cancellation using `time.NewTicker` and `select` on `ctx.Done()`. - Cap pod log reads in `fetchPodLogs` using `io.LimitReader` (1 MiB). - Add defensive `opts == nil` check in `assertClickHouseComponentResolver`. Comment link: #1105 (comment) Comment link: #1105 (comment) Comment link: #1105 (comment) Comment link: #1105 (comment) Comment link: #1105 (comment) Comment link: #1105 (comment) TAG=agy CONV=6bc68b62-40f7-4652-8de6-f5a036d676b2
VLukiantsev
marked this pull request as ready for review
October 7, 2026 08:54
janzak-dev
reviewed
Oct 7, 2026
…tTimeout in KinD 3P harness Address actionable review feedback on PR #1105: - Render `deploy/helm/templates/components/crd.yaml` via `helm template` instead of manual `{{-` line stripping (`installComponentCRD`). - Clear finalizers and poll until `apierrors.IsNotFound` when resetting the `Component` CR between subtests (`applyThirdPartyComponentCR`). - Remove redundant asynchronous Pod `Delete` before `Create` in `runMockMLWorkloadPod` (each subtest uses a distinct pod name). - Delegate `waitUntil` to `k8s.io/apimachinery/pkg/util/wait.PollUntilContextTimeout`. - Derive user `~/.local/bin` fallback in `resolveBinaryPath` when Bazel strips `$HOME`. Comment link: #1105 (comment) Comment link: #1105 (comment) Comment link: #1105 (comment) Comment link: #1105 (comment) TAG=agy CONV=6bc68b62-40f7-4652-8de6-f5a036d676b2
janzak-dev
approved these changes
Oct 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the OSS 3P ClickHouse and Redpanda end-to-end integration test suite on ephemeral KinD clusters (
test/e2e/clickhouse_3p_test.go,test/e2e/redpanda_3p_test.go, andtest/e2e/kind_3p_harness_test.go). The suite verifiesxmc-bridgestatus discovery, active health probing,.status.access.envVarsprojection, typed Go component resolvers, and live data-plane connectivity from mock ML workload Pods.Motivation & Context
As part of the XMC Components architecture, third-party infrastructure services (ClickHouse and Redpanda) can be integrated via either an Operator-Managed
wrapperRefCR (ClickHouseInstallation/ RedpandaCluster) or thestable-2026aOperator-less profile (spec.parameterswith active endpoint probing). This E2E suite validates both reconciliation paths end-to-end against real ClickHouse and Redpanda instances inside an isolated KinD cluster and confirms that downstream ML workload Pods can execute SQL queries and publish/consume Kafka messages using the injected connection parameters.Key Changes
test/e2e/kind_3p_harness_test.go):t.Cleanupteardown (or reusesKIND_CLUSTER_NAMEwhen created viatools/dev_env/dev_env.py), preloads container images, and installs thecomponents.xmc.devCRD.xmc-bridge(bridge.NewReconciler) wired to an in-processcms.NewServer(store.NewMemoryStore()),bridge.NewK8sStatusDetector, and akindNodePortProber(bridge.HealthProber) that translates<svc>.<ns>.svc.cluster.local:<port>to<kindNodeInternalIP>:<nodePort>for live TCP/HTTP readiness probing.runMockMLWorkloadPodto spawn mock ML workload Pods injected with.status.access.envVarsandSecretKeySelectorcredentials and capture container logs uponPodSucceeded.TestKind3PHarnessUnit) covering GVR mapping,NodePortendpoint translation, and CRD/Component readiness predicates.test/e2e/clickhouse_3p_test.go):clickhouseinstallations.clickhouse.altinity.comCRD, credentials Secret, and a single-nodeclickhouse/clickhouse-server:24.3-alpineinstance in KinD.OperatorManagedWrapperRefandOperatorlessStable2026aProfilemodes, assertingReady=True,.status.access.envVars,componentconfig.ResolveOptionsresolution, and liveCREATE TABLE/INSERT/SELECTexecution from a mock ML workload Pod.test/e2e/redpanda_3p_test.go):clusters.cluster.redpanda.comCRD, credentials Secret, and a single-nodedocker.redpanda.com/redpandadata/redpanda:v24.1.1broker in KinD.OperatorManagedWrapperRefandOperatorlessStable2026aProfilemodes, assertingReady=True,.status.access.envVars,ingestiongateway.ResolveRedpandaComponentConfigresolution, and live Kafka topic creation, message production, and consumption viarpkfrom a mock ML workload Pod.test/e2e/BUILD.bazel,test/e2e/README.md,cmd/ingestion_gateway/ingestiongateway/BUILD.bazel):kind_3p_harness_unit_test,clickhouse_3p_test, andredpanda_3p_testBazel test targets and documented CLI execution commands.Assumptions
enable_componentsDeployment Flag Scope:enable_components=true: Thecomponents.xmc.devCRD andxmc-bridgecontroller are active;ComponentCRs (clickhouse,redpanda) are reconciled, probed for endpoint readiness, and projected with.status.access.envVarsfor workload pod injection and typed component config resolution.enable_components=false: Thecomponents.xmc.devCRD andxmc-bridgecontroller are not deployed by the umbrella chart, and component resolvers fall back to legacy CLI flags/environment variables.enable_components=trueequivalent (livecomponents.xmc.devCRD +xmc-bridgereconciler + ClickHouse and RedpandaComponentCRs verified end-to-end in an ephemeral KinD cluster).cert-managerfrom external URLs during ephemeral KinD runs, the E2E suite installs theclickhouseinstallations.clickhouse.altinity.comandclusters.cluster.redpanda.comCRDs alongside real single-node ClickHouse (clickhouse/clickhouse-server:24.3-alpine) and Redpanda (docker.redpanda.com/redpandadata/redpanda:v24.1.1) Pods/Services — assumed to prevent network/webhook flakiness while exercisingxmc-bridgeK8sStatusDetectorJSONPath discovery, active TCP/HTTP health probing, and real SQL/Kafka data-plane execution — if full upstream operator controller reconciliation is additionally needed, a separate nightly operator-lifecycle test would be required.Verification & Testing
bazel test //test/e2e:kind_3p_harness_unit_test //pkg/components/... //pkg/clickhouse/... //cmd/ingestion_gateway/...bazel test //test/e2e:clickhouse_3p_test //test/e2e:redpanda_3p_test --test_output=streamed --test_arg=-test.v./.githooks/pre-commit./tools/analyze_diff_size.py --base-ref origin/main//test/e2e:kind_3p_harness_unit_test,//pkg/components/...,//pkg/clickhouse/...,//cmd/ingestion_gateway/...) passed.//test/e2e:clickhouse_3p_testand//test/e2e:redpanda_3p_test) passed end-to-end in both isolated ephemeral KinD cluster mode andKIND_CLUSTER_NAMEcluster-reuse mode.prod_code_lines: 1,test_code_lines: 984,total_code_lines: 985,doc_lines: 26).Related Issues / Bugs
Fixes bug 548436596
TAG=agy
CONV=6bc68b62-40f7-4652-8de6-f5a036d676b2