8000 fix: MaxNumberPerNode and CircularReplication podDistribution by Slach · Pull Request #2127 · Altinity/clickhouse-operator · GitHub
[go: up one dir, main page]

Skip to content

fix: MaxNumberPerNode and CircularReplication podDistribution - #2127

Open
Slach wants to merge 6 commits into
0.28.0from
fix/1252-max-number-per-node
Open

Slach wants to merge 6 commits into
0.28.0from
fix/1252-max-number-per-node

Conversation

@Slach
@Slach Slach commented Oct 31, 2026 •
Copy link
Copy Markdown
Collaborator

Fixes #1252
Fixes #1352

podDistribution: [{type: MaxNumberPerNode}] never worked, even with label.appendScope: "yes"

getMaxNumberOfPodsPerNode() walks pod templates via PodTemplatesIndex, but index is built in normalizeTemplates(), which runs after Fill() in normalizeConfiguration()
so maxNumberOfPodsPerNode was always 0, clusterScopeCycleSize was 0 and all hosts got clusterScopeCycleIndex=0

as result

  • with label.appendScope: "no" (default) anti-affinity matched nothing
  • with label.appendScope: "yes" all pods have the same clusterScopeCycleIndex label, so anti-affinity allowed only 1 pod per node and other pods stuck in Pending

fix: WalkPodTemplates() for CHI and CHK walks spec.templates.podTemplates list instead of the index

added TestMaxNumberPerNodeScopeCycles, 3 shards x 2 replicas with number: 2, pod template from CHI and from CHIT

also fixed type: CircularReplication (#1352)

  • shortcut expanded entries appended to PodDistribution during range and never normalized, so they got empty topologyKey and pod was rejected topologyKey: Required value: can not be empty, now appended entries normalized too
  • expanded entries inherit topologyKey from CircularReplication, so topologyKey: topology.kubernetes.io/zone is not lost
  • getMaxNumberOfPodsPerNode() counts CircularReplication as MaxNumberPerNode with replicas count, cause shortcut expanded after Fill()

added TestCircularReplication, 3 shards x 3 replicas, default and explicit topologyKey, lower case type

keep in mind, MaxNumberPerNode, PreviousTailAffinity and CircularReplication select pods by scope labels (clusterScopeIndex, clusterScopeCycleIndex), which added to pods only with operator config label.appendScope: "yes", default is "no"
without it MaxNumberPerNode anti-affinity matches nothing and PreviousTailAffinity required affinity can't be satisfied, pods stuck in Pending

apiVersion: clickhouse.altinity.com/v1
kind: ClickHouseOperatorConfiguration
metadata:
  name: append-scope-labels
spec:
  label:
    appendScope: "yes"

added TestPodDistributionScopeLabels in pkg/model/chi/creator, 3 shards x 3 replicas, creates StatefulSet for each host like controller does

  • without appendScope, scope labels absent and MaxNumberPerNode selector matches nothing
  • with appendScope, MaxNumberPerNode anti-affinity selects only pods from the same cycle
  • CircularReplication all affinity terms have topologyKey and PreviousTailAffinity required affinity matches exactly one existing pod (previous cycle tail or itself)

🤖 Generated with Claude Code

…cope cycles

WalkPodTemplates walked PodTemplatesIndex, which is built in normalizeTemplates(),
after Fill() calculates scope cycles, so maxNumberOfPodsPerNode was always 0
and every host got clusterScopeCycleIndex=0. Walk the pod templates list instead.

Fixes #1252

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y topologyKey

Expanded shortcut entries were appended to the slice being ranged, so they were
never normalized and got empty topologyKey, pods were rejected by API server.
Normalize appended entries, inherit topologyKey from CircularReplication and
count CircularReplication as MaxNumberPerNode of replicas count when host scope
cycles are calculated.

Fixes #1352

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Slach Slach changed the title fix: MaxNumberPerNode podDistribution ignored when calculating host scope cycles fix: MaxNumberPerNode and CircularReplication podDistribution Oct 31, 2026
Check that with operator config label.appendScope pods carry scope labels,
MaxNumberPerNode anti-affinity selects hosts of the same cycle only and
CircularReplication affinity terms are valid and satisfiable, and that without
appendScope scope labels are absent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Slach and others added 3 commits November 3, 2026 09:03
testflows 2.0.231215 references lzma._MODE_CLOSED, removed in python 3.13,
so live test output and tfs transform/report steps fail and only raw.log
is uploaded. See https://github.com/Altinity/clickhouse-operator/actions/runs/42177835548/job/127760322225

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Empty topology key for CircularReplication MaxNumberPerNode requires operator config label.appendScope: true

1 participant

0