8000 feat(helm): aggregate operator CRDs into builtin view, edit and admin roles by cwrau · Pull Request #2095 · Altinity/clickhouse-operator · GitHub
[go: up one dir, main page]

Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

* **Volume downscale tool now supports StorageClass migration.** Set `NEW_STORAGE_CLASS` in `tools/volume_downscale.sh` to move a volume to another StorageClass at the same or a smaller size; the script also detects repeated volume attachment and mount failures. See [runbook](tools/volume_downscale.md). by [@lesandie](https://github.com/lesandie) in [#2125](https://github.com/Altinity/clickhouse-operator/pull/2125)

* **The Helm chart now aggregates the operator's CRDs into the builtin `view`, `edit` and `admin` ClusterRoles.** `ClickHouseInstallationTemplate`, `ClickHouseOperatorConfiguration` and `ClickHouseKeeperInstallation` are readable by `view`; `edit` and `admin` can read and write all four operator kinds. `ClickHouseInstallation` is deliberately not readable by `view`, because the operator writes the normalized CR, including derived `password_sha256_hex` user hashes, into its status. Enabled by default, disable with `rbac.aggregate.enabled=false` (requires `rbac.create`).

### Changed

* **BACKWARD INCOMPATIBLE: Revised template precedence.** `ClickHouseInstallation` settings now override templates and operator defaults; later templates override earlier ones. Kubernetes resource lists merge by name or appropriate key. Existing installations may restart or recreate StatefulSets when effective configuration changes. See [upgrade notes](docs/operator_configuration.md#upgrading-from-027). by [@sunsingerus](https://github.com/sunsingerus)
Expand Down
1 change: 1 addition & 0 deletions deploy/helm/clickhouse-operator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ crdHook:
| podAnnotations | object | check the `values.yaml` file | annotations to add to the clickhouse-operator pod, check `kubectl explain pod.spec.annotations` for details |
| podLabels | object | `{}` | labels to add to the clickhouse-operator pod |
| podSecurityContext | object | `{}` | |
| rbac.aggregate.enabled | bool | `true` | create ClusterRoles aggregated into the builtin `view`, `edit` and `admin` roles for the operator CRDs (requires rbac.create). ClickHouseInstallation is only aggregated into `edit` and `admin`: the operator writes password hashes into its status |
| rbac.create | bool | `true` | specifies whether rbac resources should be created |
| rbac.namespaceScoped | bool | `false` | specifies whether to create roles and rolebindings at the cluster level or namespace level |
| secret.create | bool | `true` | create a secret with operator credentials |
Expand Down
102 changes: 102 additions & 0 deletions deploy/helm/clickhouse-operator/templates/aggregated-clusterroles.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
{{- if and .Values.rbac.create .Values.rbac.aggregate.enabled }}
# ClusterRoles aggregated into the builtin view, edit and admin roles via rbac.authorization.k8s.io/aggregate-to-* labels.
# The edit role carries the edit and admin labels: write verbs for the safe kinds and for ClickHouseInstallation.
# The view-sensitive role carries the edit and admin labels only (deliberately NOT aggregate-to-view): reads of ClickHouseInstallation.
# ClickHouseInstallation is NOT aggregated into view: the operator writes the normalized CR into
# status.normalized/normalizedCompleted, including password_sha256_hex hashes it derives for users
# (default password, operator access password). Every other kind has a Secret-based way to supply
# credentials (valueFrom.secretKeyRef, access.secret) and the operator writes no credentials into it.
# edit and admin can read all Secrets in a namespace anyway, so they may read and write ClickHouseInstallation.
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "altinity-clickhouse-operator.fullname" . }}-aggregate-view
labels:
{{- include "altinity-clickhouse-operator.labels" . | nindent 4 }}
rbac.authorization.k8s.io/aggregate-to-view: "true"
rbac.authorization.k8s.io/aggregate-to-edit: "true"
rbac.authorization.k8s.io/aggregate-to-admin: "true"
rules:
- apiGroups:
- clickhouse.altinity.com
resources:
- clickhouseinstallationtemplates
- clickhouseinstallationtemplates/status
- clickhouseoperatorconfigurations
verbs:
- get
- list
- watch
- apiGroups:
- clickhouse-keeper.altinity.com
resources:
- clickhousekeeperinstallations
- clickhousekeeperinstallations/status
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "altinity-clickhouse-operator.fullname" . }}-aggregate-edit
labels:
{{- include "altinity-clickhouse-operator.labels" . | nindent 4 }}
rbac.authorization.k8s.io/aggregate-to-edit: "true"
rbac.authorization.k8s.io/aggregate-to-admin: "true"
rules:
- apiGroups:
- clickhouse.altinity.com
resources:
- clickhouseinstallationtemplates
- clickhouseoperatorconfigurations
verbs:
- create
- delete
- deletecollection
- patch
- update
- apiGroups:
- clickhouse-keeper.altinity.com
resources:
- clickhousekeeperinstallations
verbs:
- create
- delete
- deletecollection
- patch
- update
- apiGroups:
- clickhouse.altinity.com
resources:
- clickhouseinstallations
verbs:
- create
- delete
- deletecollection
- patch
- update
---
# Read role for the unsafe kind. Deliberately NOT labelled aggregate-to-view: ClickHouseInstallation
# status holds derived password hashes, so only edit and admin may read it.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "altinity-clickhouse-operator.fullname" . }}-aggregate-view-sensitive
labels:
{{- include "altinity-clickhouse-operator.labels" . | nindent 4 }}
rbac.authorization.k8s.io/aggregate-to-edit: "true"
rbac.authorization.k8s.io/aggregate-to-admin: "true"
rules:
- apiGroups:
- clickhouse.altinity.com
resources:
- clickhouseinstallations
- clickhouseinstallations/status
verbs:
- get
- list
- watch
{{- end }}
9 changes: 9 additions & 0 deletions deploy/helm/clickhouse-operator/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -910,6 +910,15 @@
"rbac": {
"type": "object",
"properties": {
"aggregate": {
"type": "object",
"properties": {
"enabled": {
"description": "create ClusterRoles aggregated into the builtin `view`, `edit` and `admin` roles for the operator CRDs (requires rbac.create). ClickHouseInstallation is only aggregated into `edit` and `admin`: the operator writes password hashes into its status",
"type": "boolean"
}
}
},
"create": {
"description": "specifies whether rbac resources should be created",
"type": "boolean"
Expand Down
3 changes: 3 additions & 0 deletions deploy/helm/clickhouse-operator/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,9 @@ rbac:
create: true
# rbac.namespaceScoped -- specifies whether to create roles and rolebindings at the cluster level or namespace level
namespaceScoped: false
aggregate:
# rbac.aggregate.enabled -- create ClusterRoles aggregated into the builtin `view`, `edit` and `admin` roles for the operator CRDs (requires rbac.create). ClickHouseInstallation is only aggregated into `edit` and `admin`: the operator writes password hashes into its status
enabled: true
secret:
# secret.create -- create a secret with operator credentials
create: true
Expand Down
71 changes: 71 additions & 0 deletions dev/test_helm_chart.sh
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,77 @@ else
report "values.schema.json rejects a wrong type" yes
fi

echo
echo "aggregated RBAC"
# ClickHouseInstallation status holds operator-written password hashes, so CHI must stay out of view.
# Every other kind has a Secret-based credential path and is readable by view.
# The role documents are parsed with awk only: the yq on CI (mikefarah) and the one on developer
# machines (jq wrapper) share no expression syntax.
expect_count "three aggregate ClusterRoles on top of the two operator ones" ClusterRole 5
expect_contains "view aggregation label" 'rbac.authorization.k8s.io/aggregate-to-view: "true"'
expect_contains "edit aggregation label" 'rbac.authorization.k8s.io/aggregate-to-edit: "true"'
expect_contains "admin aggregation label" 'rbac.authorization.k8s.io/aggregate-to-admin: "true"'
expect_absent "rbac.aggregate.enabled=false drops the aggregate roles" 'aggregate-to-' --set rbac.aggregate.enabled=false
expect_absent "rbac.create=false drops the aggregate roles" 'aggregate-to-' --set rbac.create=false

# role_info <name-suffix|*> <label|*> <labels|verbs|resources>: for every aggregate ClusterRole whose
# name ends with <name-suffix> (or any) and which carries aggregate-to-<label> (or any), prints the
# sorted unique aggregate-to-* labels, verbs or resources as one space-separated line.
role_info() {
render --show-only templates/aggregated-clusterroles.yaml | awk -v suffix="$1" -v label="$2" -v what="$3" '
function flush( i, n, keys) {
if (doc_kind == "ClusterRole" && (suffix == "*" || name ~ (suffix "$")) && (label == "*" || (label in labels))) {
if (what == "labels") { for (k in labels) out[k] = 1 }
if (what == "verbs") { for (k in verbs) out[k] = 1 }
if (what == "resources") { for (k in resources) out[k] = 1 }
}
doc_kind = ""; name = ""; mode = ""; delete labels; delete verbs; delete resources
}
/^---$/ { flush(); next }
/^kind: / { doc_kind = $2 }
/^ name: / { name = $2 }
/^ rbac.authorization.k8s.io\/aggregate-to-[a-z]+: "true"$/ { split($1, p, "aggregate-to-"); sub(/:$/, "", p[2]); labels[p[2]] = 1 }
/^ (apiGroups|resources|verbs):$/ { mode = $1; sub(/:$/, "", mode); next }
/^ - / { if (mode == "verbs") verbs[$2] = 1; if (mode == "resources") resources[$2] = 1; next }
{ if ($0 !~ /^ - /) mode = "" }
END {
flush()
n = 0
for (k in out) keys[++n] = k
for (i = 1; i <= n; i++) for (j = i + 1; j <= n; j++) if (keys[j] < keys[i]) { t = keys[i]; keys[i] = keys[j]; keys[j] = t }
line = ""
for (i = 1; i <= n; i++) line = line (i > 1 ? " " : "") keys[i]
print line
}'
}

view_resources="$(role_info '*' view resources)"
edit_resources="$(role_info '*' edit resources)"
if grep -qE '(^| )clickhouseinstallations(/status)?( |$)' <<<"${view_resources}"; then
report "ClickHouseInstallation is not readable by view" no "view resources: ${view_resources}"
else
report "ClickHouseInstallation is not readable by view" yes
fi
expect_eq() {
local name="$1" want="$2" got="$3"
if [[ "${got}" == "${want}" ]]; then report "${name}" yes; else report "${name}" no "want: ${want} got: ${got}"; fi
}
# the view role carries all three labels and read verbs only; the edit role carries edit+admin
expect_eq "view role is aggregated into view, edit and admin" "admin edit view" "$(role_info '-aggregate-view' '*' labels)"
expect_eq "view role has read verbs only" "get list watch" "$(role_info '-aggregate-view' '*' verbs)"
expect_eq "write role is aggregated into edit and admin only" "admin edit" "$(role_info '-aggregate-edit' '*' labels)"
expect_eq "write role has write verbs only" "create delete deletecollection patch update" "$(role_info '-aggregate-edit' '*' verbs)"
# the sensitive read role is deliberately not aggregated into view
expect_eq "view-sensitive role has no aggregate-to-view label" "admin edit" "$(role_info '-aggregate-view-sensitive' '*' labels)"
expect_eq "view-sensitive role has read verbs only" "get list watch" "$(role_info '-aggregate-view-sensitive' '*' verbs)"
expect_eq "view-sensitive role covers ClickHouseInstallation and its status" "clickhouseinstallations clickhouseinstallations/status" "$(role_info '-aggregate-view-sensitive' '*' resources)"
for want in clickhouseinstallationtemplates clickhouseoperatorconfigurations clickhousekeeperinstallations; do
if grep -qw -- "${want}" <<<"${view_resources}"; then report "view role covers ${want}" yes; else report "view role covers ${want}" no "resources: ${view_resources}"; fi
done
for want in clickhouseinstallations clickhouseinstallationtemplates clickhouseoperatorconfigurations clickhousekeeperinstallations; do
if grep -qw -- "${want}" <<<"${edit_resources}"; then report "edit/admin role covers ${want}" yes; else report "edit/admin role covers ${want}" no "resources: ${edit_resources}"; fi
done

echo
echo "docs"
if [[ "${skip_docs}" == "yes" ]]; then
Expand Down
11 changes: 11 additions & 0 deletions docs/operator_installation_details.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,17 @@ helm upgrade clickhouse-operator clickhouse-operator/altinity-clickhouse-operato

Look https://github.com/Altinity/clickhouse-operator/tree/master/deploy/helm/clickhouse-operator/ for details

The chart also creates `<release>-aggregate-view`, `<release>-aggregate-edit` (write) and `<release>-aggregate-view-sensitive` (read of `ClickHouseInstallation`, aggregated into `edit` and `admin` only) ClusterRoles carrying the `rbac.authorization.k8s.io/aggregate-to-*` labels, so the builtin roles cover the operator's CRDs (disable with `rbac.aggregate.enabled=false`):

| Kind | view | edit, admin |
|---|---|---|
| `ClickHouseInstallation` | - | read, write |
| `ClickHouseInstallationTemplate` | read | read, write |
| `ClickHouseOperatorConfiguration` | read | read, write |
| `ClickHouseKeeperInstallation` | read | read, write |

`ClickHouseInstallation` is not readable by `view`: the operator writes the normalized CR, including derived `password_sha256_hex` user hashes, into its status. Pass credentials via `valueFrom.secretKeyRef` rather than inline, see [security hardening](security_hardening.md).

## Resources Description

Let's walk over all resources created along with ClickHouse operator, which are:
Expand Down
Loading
0