8000 Dashboard phase 2: read-only web UI, sign-in and API additions by abahmed · Pull Request #581 · abahmed/kwatch · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -323,6 +323,13 @@ jobs:
KWATCH_WATCH_SECRETS: ${{ matrix.secrets }}
run: ./scripts/test-kind-production.sh

# The dashboard as the chart installs it: password sign-in, the
# session cookie on /api/*, and shared mode refusing plain http.
# One matrix cell is enough; it does not depend on RBAC or Secrets.
- name: Run dashboard test
if: matrix.rbac == 'full' && matrix.secrets
run: ./scripts/test-kind-dashboard.sh

- name: Collect cluster diagnostics
if: always()
run: |
Expand Down
11 changes: 7 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,7 @@ enforces, are in `docs/contributor-architecture.md`.
| `internal/config/crd` | KwatchConfig resource watcher, including late install and restart |
| `internal/rbac` | Permission audit derived from `kube.SourceAccess`; reports capability gaps, never incidents |
| `internal/status` | The read-only "what is wrong now" report behind the opt-in `/status` endpoint; reads state kwatch already holds, runs no loop |
| `internal/dashboard` | The opt-in, Pro-only read-only dashboard on its own port: `/api/*` (bearer token or session cookie), password sign-in, and the embedded web UI (`web/`, plain HTML/CSS/JS, no build step, hashed asset names, strict CSP, no outside requests) |
| `cmd/kwatch-scorecard` | Offline tool that prints the noise scorecard from an audit log |
| `internal/audit`, `internal/scorecard` | Decision log and its offline noise scorecard |
| `internal/replay`, `internal/scenarios` | Test tooling only: observation record/replay on a simulated clock, and the labelled scenario library with the scorecard gate; nothing in production imports them |
Expand Down Expand Up @@ -976,10 +977,12 @@ Dynamic discovery dependencies must implement client-go's context-aware
discovery interface. Do not add a fallback to an unbounded, contextless
discovery call; discovery must stop with its watcher generation.

Kwatch exposes no user API. Liveness, readiness, availability, health, and
metrics are the only HTTP endpoints, plus the opt-in read-only `/status`
(`healthCheck.statusEndpoint`); do not add other diagnostics, debug, or
action endpoints until the core reaches its production goals
Kwatch exposes no write API. Liveness, readiness, availability, health, and
metrics are the only health-port endpoints, plus the opt-in read-only
`/status` (`healthCheck.statusEndpoint`). The opt-in dashboard
(`dashboard.enabled`, Pro) is read-only on its own port; its only POSTs are
sign-in and sign-out. Do not add other diagnostics, debug, or action
endpoints until the core reaches its production goals
(`docs/production-goals.md`).

Required components report progress even when their input is idle (the
Expand Down
28 changes: 28 additions & 0 deletions deploy/chart/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,34 @@ kubectl delete namespace kwatch
| `networkPolicy.extraEgressPorts` | Extra TCP egress ports open to any address, for providers not on 443: SMTP (`25`, `465` or `587`) for email, or a webhook's custom port | `[]` |
| `networkPolicy.kubeletCIDRs` | Node address ranges the kubelet egress rule is limited to; empty allows any address. Also admitted to the health port, for kubelet probes on a network plugin that enforces policy on node traffic | `[]` |
| `networkPolicy.healthIngressFrom` | Extra peers (`namespaceSelector`, `podSelector` or `ipBlock`) that may reach the health port (`/health`, `/metrics`, `/status`), such as your Prometheus namespace. Without it only pods in the release namespace (and `kubeletCIDRs`) can | `[]` |
| `dashboard.enabled` | Serve the read-only dashboard (web UI and `/api/*`) on its own port and ClusterIP Service `<release>-dashboard`. Pro only; Community never opens the port | `false` |
| `dashboard.port` | Dashboard port; must not equal the health port | `8061` |
| `dashboard.mode` | `local`: reached through `kubectl port-forward`; the cookie works on `http://localhost`. `shared`: https only; kwatch refuses sign-in over plain http and marks the cookie `Secure`. Needs `dashboard.trustProxy` or `dashboard.tls.secretName`, or the render fails | `local` |
| `dashboard.trustProxy` | Trust `X-Forwarded-Proto`/`X-Forwarded-For` from the Ingress or proxy that terminates TLS in front of kwatch. Without it the header is ignored | `false` |
| `dashboard.tls.secretName` | `kubernetes.io/tls` Secret kwatch serves https from itself, for a LoadBalancer or NodePort with nothing in front. Renewals are picked up without a restart | `""` |
| `dashboard.existingSecret` | Existing Secret with the keys `password` (sign-in) and `token` (API bearer token). Without it the chart creates `<release>-dashboard` with a generated password (`XXXX-XXXX-XXXX-XXXX`) and token, and keeps both across upgrades. Changing the password signs everyone out | `""` |
| `dashboard.networkPolicyFrom` | With `networkPolicy.enabled`: extra peers (your ingress controller, `ipBlock`s for load balancer clients) allowed to reach the dashboard port. The release namespace and `healthIngressFrom` always may | `[]` |

Sign-ins last 12 hours. A password change or a kwatch restart (upgrade,
rollout, rescheduling) signs everyone out. Behind an Ingress, set
`dashboard.trustProxy`: without it every client shares the Ingress's
address, and ten wrong passwords in a minute from anyone lock everyone
out of sign-in for that minute.

The generated `<release>-dashboard` Secret carries
`helm.sh/resource-policy: keep`: uninstalling or turning the dashboard off
leaves it, so a reinstall keeps the saved password. Delete it to forget the
password. The password and token are kept across upgrades through Helm
`lookup`, which offline renders (`helm template`, Argo CD, Flux with
post-rendering) cannot do: each sync would make a new password and sign
everyone out. With GitOps, create the Secret yourself (keys `password` and
`token`, for example with a sealed or external secret) and set
`dashboard.existingSecret`.

The chart does not expose the dashboard outside the cluster. To expose it,
run the kwatch.sh Dashboard menu or create your own Ingress or route
pointing at Service `<release>-dashboard` (port name `dashboard`).

| `image.repository` / `image.tag` / `image.pullPolicy` | Container image; an empty tag uses the chart's `appVersion` | `ghcr.io/abahmed/kwatch` / `""` / `IfNotPresent` |
| `crd.install` | Install the shared `KwatchConfig` CRD with this release; skipped automatically when another release owns it. `config.crd.enabled` is separate: it turns the live overlay on | `true` |
| `rbac.mode` | `least-privilege` grants an explicit read-only `list`/`watch` set, and kinds outside it (custom resources included) show as `permission_denied` in `/health` coverage (kwatch logs one warning about them and the install notes say so); `full` grants `list`/`watch` on every resource, except that `full` with `watch.secrets: false` renders the least-privilege list. Both grant `get` only on namespaces, nodes, `nodes/stats`, `nodes/metrics` and `pods/log`. Set `full` to watch custom resources | `least-privilege` |
Expand Down
16 changes: 16 additions & 0 deletions deploy/chart/templates/_dashboard.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{{/*
Dashboard settings. The chart never exposes the dashboard outside the
cluster: the kwatch.sh Dashboard menu, or you, point an Ingress, route
or Service at the ClusterIP Service <release>-dashboard.
mode local (port-forward) or shared (https only, Secure cookie)
trustProxy TLS ends at a proxy that sets X-Forwarded-Proto
tls kwatch serves https itself from a kubernetes.io/tls Secret
*/}}

{{- define "kwatch.dashboardCheck" -}}
{{- $d := default dict .Values.dashboard -}}
{{- $tls := default dict $d.tls -}}
{{- if and (eq (default "local" $d.mode) "shared") (not $d.trustProxy) (not $tls.secretName) }}
{{- fail "dashboard.mode=shared is https only: set dashboard.trustProxy=true (TLS ends at your Ingress or proxy) or dashboard.tls.secretName (kwatch serves https itself)" }}
{{- end }}
{{- end }}
3 changes: 2 additions & 1 deletion deploy/chart/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,8 @@ true
{{- end }}

{{/*
Secret that holds the dashboard bearer token (key token). An existing
Secret that holds the dashboard bearer token (key token) and sign-in
password (key password). An existing
Secret wins; otherwise it is the Secret this chart creates when
dashboard.enabled is true. The deployment reads it with optional: true,
so the pod starts without the dashboard when the Secret is missing.
Expand Down
34 changes: 30 additions & 4 deletions deploy/chart/templates/dashboard-secret.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,30 @@
{{- $dash := default dict .Values.dashboard -}}
{{- if and $dash.enabled (not $dash.existingSecret) }}
{{- $name := include "kwatch.dashboardSecretName" . }}
{{- /* Keep the token across upgrades: reuse the existing Secret's. */}}
{{- /* Keep the token and the password across upgrades: reuse the
existing Secret's values. A new password is 16 characters from an
alphabet without 0/O/1/I/L, in four groups of four (about 80 bits). */}}
{{- $existing := lookup "v1" "Secret" .Release.Namespace $name }}
{{- $token := randAlphaNum 32 | b64enc }}
{{- if and $existing $existing.data (index $existing.data "token") }}
{{- $token = index $existing.data "token" }}
{{- $data := dict }}
{{- if and $existing $existing.data }}
{{- $data = $existing.data }}
{{- end }}
{{- $token := index $data "token" | default (randAlphaNum 32 | b64enc) }}
{{- $password := index $data "password" }}
{{- if not $password }}
{{- /* crypto/rand bytes, each mapped onto the 32-letter alphabet: 256 is
a multiple of 32, so every letter is equally likely. */}}
{{- $alphabet := "ABCDEFGHJKLMNPQRSTUVWXYZ23456789" }}
{{- $raw := randBytes 16 | b64dec }}
{{- $plain := "" }}
{{- range $i := until 16 }}
{{- if and $i (eq (mod $i 4) 0) }}
{{- $plain = printf "%s-" $plain }}
{{- end }}
{{- $n := mod (int (index $raw $i)) 32 }}
{{- $plain = printf "%s%s" $plain (substr (int $n) (int (add1 $n)) $alphabet) }}
{{- end }}
{{- $password = $plain | b64enc }}
{{- end }}
apiVersion: v1
kind: Secret
Expand All @@ -14,7 +33,14 @@ metadata:
namespace: {{ .Release.Namespace }}
labels:
{{- include "kwatch.labels" . | nindent 4 }}
annotations:
# Kept when the release or the dashboard goes away, so a reinstall
# keeps the password people saved. Delete it to forget the password.
helm.sh/resource-policy: keep
type: Opaque
data:
# The bearer token for API clients such as the installer.
token: {{ $token }}
# The shared sign-in password of the web dashboard.
password: {{ $password }}
{{- end }}
6 changes: 4 additions & 2 deletions deploy/chart/templates/dashboard-service.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{{- $dash := default dict .Values.dashboard -}}
{{- if $dash.enabled }}
{{- include "kwatch.dashboardCheck" . }}
apiVersion: v1
kind: Service
metadata:
Expand All @@ -8,8 +9,9 @@ metadata:
labels:
{{- include "kwatch.labels" . | nindent 4 }}
annotations:
# Read-only dashboard API. No write endpoints. Access is bearer-token
# protected; do not expose without TLS.
# Read-only dashboard: web UI and API. No write endpoints. Sign-in
# with the password, or the bearer token for API clients. To expose
# it, point your own Ingress, route or Service at this one.
{{- if .Values.networkPolicy.enabled }}
networking.k8s.io/policy: "true"
{{- end }}
Expand Down
42 changes: 40 additions & 2 deletions deploy/chart/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,11 @@ spec:
mountPath: /config
- name: data
mountPath: /var/lib/kwatch
{{- if and (default dict .Values.dashboard).enabled (default dict (default dict .Values.dashboard).tls).secretName }}
- name: dashboard-tls
mountPath: /etc/kwatch/dashboard-tls
readOnly: true
{{- end }}
{{- with .Values.extraVolumeMounts }}
{{- toYaml . | nindent 10 }}
{{- end }}
Expand Down Expand Up @@ -141,14 +146,42 @@ spec:
name: {{ include "kwatch.licenseSecretName" . }}
key: license-key
optional: true
# The dashboard bearer token. optional: true lets the pod start
# without the dashboard (Community, or dashboard disabled).
# The dashboard settings and its Secret. optional: true lets
# the pod start without the dashboard (Community, or the
# dashboard disabled).
{{- $dashEnv := default dict .Values.dashboard }}
{{- if $dashEnv.enabled }}
- name: KWATCH_DASHBOARD_ENABLED
value: "true"
- name: KWATCH_DASHBOARD_PORT
value: {{ $dashEnv.port | default 8061 | quote }}
- name: KWATCH_DASHBOARD_MODE
value: {{ $dashEnv.mode | default "local" | quote }}
{{- if $dashEnv.trustProxy }}
- name: KWATCH_DASHBOARD_TRUST_PROXY
value: "true"
{{- end }}
{{- if (default dict $dashEnv.tls).secretName }}
- name: KWATCH_DASHBOARD_TLS_CERT
value: /etc/kwatch/dashboard-tls/tls.crt
- name: KWATCH_DASHBOARD_TLS_KEY
value: /etc/kwatch/dashboard-tls/tls.key
{{- end }}
- name: KWATCH_DEPLOYMENT_NAME
value: {{ include "kwatch.fullname" . | quote }}
{{- end }}
- name: KWATCH_DASHBOARD_TOKEN
valueFrom:
secretKeyRef:
name: {{ include "kwatch.dashboardSecretName" . }}
key: token
optional: true
- name: KWATCH_DASHBOARD_PASSWORD
valueFrom:
secretKeyRef:
name: {{ include "kwatch.dashboardSecretName" . }}
key: password
optional: true
{{- if eq (include "kwatch.watchSecrets" .) "false" }}
# watch.secrets=false: matches the RBAC, which grants no
# Secret access, even when config.yaml comes from a Secret.
Expand Down Expand Up @@ -196,6 +229,11 @@ spec:
{{- . | nindent 8 }}
{{- end }}
volumes:
{{- if and (default dict .Values.dashboard).enabled (default dict (default dict .Values.dashboard).tls).secretName }}
- name: dashboard-tls
secret:
secretName: {{ .Values.dashboard.tls.secretName }}
{{- end }}
- name: data
{{- if .Values.persistence.emptyDir }}
emptyDir:
Expand Down
14 changes: 12 additions & 2 deletions deploy/chart/templates/networkpolicy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,19 @@ spec:
ports:
- protocol: TCP
port: {{ $port }}
{{- if and $dash.enabled $dash.networkPolicyFrom }}
# Dashboard from outside the namespace: the peers in
# dashboard.networkPolicyFrom (your ingress controller, gateway or
# load balancer source ranges as ipBlocks).
- from:
{{- toYaml $dash.networkPolicyFrom | nindent 8 }}
ports:
- protocol: TCP
port: {{ $dashPort }}
{{- end }}
{{- if $dash.enabled }}
# Dashboard API (/api/*): bearer-token protected, ClusterIP only.
# Reachable from this namespace and healthIngressFrom sources.
# Dashboard (web UI and /api/*): password or bearer token. Reachable
# from this namespace (port-forward) and healthIngressFrom sources.
- from:
- namespaceSelector:
matchLabels:
Expand Down
78 changes: 78 additions & 0 deletions deploy/chart/test_helm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -572,4 +572,82 @@ if helm template test-old . --set bogusTopLevel=1 >/dev/null 2>&1; then
fi
echo "PASS: 0.10.5 values file"

echo "=== dashboard ==="
for kind in "kind: Ingress" "KWATCH_DASHBOARD_ENABLED" "port: 8061"; do
if grep -Fq "$kind" <<<"$OUT1"; then
echo "FAIL: the dashboard must be off by default ($kind)"; exit 1
fi
done
OUT_DASH=$(helm template kwatch . --set dashboard.enabled=true 2>&1)
for want in 'name: KWATCH_DASHBOARD_PASSWORD' 'key: password' \
'name: KWATCH_DASHBOARD_MODE' 'value: "local"' \
'name: KWATCH_DEPLOYMENT_NAME' ' password: ' ' token: '; do
grep -Fq -- "$want" <<<"$OUT_DASH" || {
echo "FAIL: dashboard render lacks $want"; exit 1;
}
done
if grep -Fq "kind: Ingress" <<<"$OUT_DASH"; then
echo "FAIL: local mode must not render an Ingress"; exit 1
fi
PW=$(grep -E '^ password: ' <<<"$OUT_DASH" | awk '{print $2}' |
base64 --decode)
grep -Eq '^([A-HJ-NP-Z2-9]{4}-){3}[A-HJ-NP-Z2-9]{4}$' <<<"$PW" || {
echo "FAIL: generated password has the wrong shape: $PW"; exit 1;
}
# expose_ok renders with dashboard.enabled and the given flags and checks
# every following string is present; expose_fail checks the render fails
# and names the given text.
expose_ok() {
local label=$1; shift
local args=()
while [ "$1" != "--" ]; do args+=("$1"); shift; done; shift
local out
out=$(helm template kwatch . --set dashboard.enabled=true "${args[@]}" \
2>&1) || { echo "FAIL: $label must render: $out"; exit 1; }
for want in "$@"; do
grep -Fq -- "$want" <<<"$out" || {
echo "FAIL: $label lacks $want"; exit 1;
}
done
}
expose_fail() {
local label=$1 want=$2; shift 2
local out
if out=$(helm template kwatch . --set dashboard.enabled=true "$@" 2>&1)
then
echo "FAIL: $label must not render"; exit 1
fi
grep -Fq -- "$want" <<<"$out" || {
echo "FAIL: $label: message lacks '$want': $out"; exit 1;
}
}
expose_fail "unknown mode" "must be one of" --set dashboard.mode=public
expose_fail "shared without https" "https only" --set dashboard.mode=shared
expose_ok "shared behind a proxy" --set dashboard.mode=shared \
--set dashboard.trustProxy=true -- \
'value: "shared"' 'name: KWATCH_DASHBOARD_TRUST_PROXY' "type: ClusterIP"
expose_ok "shared with own TLS" --set dashboard.mode=shared \
--set dashboard.tls.secretName=kwatch-tls -- \
"name: KWATCH_DASHBOARD_TLS_CERT" "secretName: kwatch-tls" \
"mountPath: /etc/kwatch/dashboard-tls"
expose_ok "networkPolicyFrom" --set networkPolicy.enabled=true \
--set 'dashboard.networkPolicyFrom[0].ipBlock.cidr=10.0.0.0/8' -- \
"cidr: 10.0.0.0/8" "port: 8061"
for kind in "kind: Ingress" "kind: HTTPRoute" "kind: Route" \
"kind: VirtualService" "type: LoadBalancer" "type: NodePort"; do
if grep -Fq "$kind" <<<"$(helm template kwatch . \
--set dashboard.enabled=true)"; then
echo "FAIL: the chart must not expose the dashboard ($kind)"; exit 1
fi
done
OUT_EXIST=$(helm template kwatch . --set dashboard.enabled=true \
--set dashboard.existingSecret=mine 2>&1)
if grep -Eq '^ password: ' <<<"$OUT_EXIST"; then
echo "FAIL: existingSecret must not render a password Secret"; exit 1
fi
grep -Fq "name: mine" <<<"$OUT_EXIST" || {
echo "FAIL: existingSecret must feed the dashboard env"; exit 1;
}
echo "PASS: dashboard"

echo "All helm template tests passed."
17 changes: 16 additions & 1 deletion deploy/chart/test_helm_lifecycle.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,17 @@ helm install "$release" deploy/chart \
--create-namespace \
--set image.tag=ci \
--set config.telemetry.enabled=false \
--set dashboard.enabled=true \
--wait=false

# The dashboard password and token must survive an upgrade (lookup).
dashboard_secret() {
kubectl get secret "$release-dashboard" --namespace "$namespace" \
-o jsonpath='{.data.password}{"/"}{.data.token}'
}
dashboard_before="$(dashboard_secret)"
test -n "$dashboard_before"

kubectl apply -f - <<'EOF'
apiVersion: kwatch.abahmed.dev/v1beta1
kind: KwatchConfig
Expand All @@ -53,7 +62,13 @@ awk '{ print } /helm.sh\/resource-policy: keep/ { print " lifecycle-test: upg
"$crd_template" >"$crd_template.new"
mv "$crd_template.new" "$crd_template"

helm upgrade "$release" "$tmp_chart/chart" --namespace "$namespace" --wait=false
helm upgrade "$release" "$tmp_chart/chart" --namespace "$namespace" \
--set image.tag=ci --set config.telemetry.enabled=false \
--set dashboard.enabled=true --wait=false
test "$(dashboard_secret)" = "$dashboard_before" || {
echo "FAIL: helm upgrade replaced the dashboard password or token"
exit 1
}

test "$(kubectl get crd kwatchconfigs.kwatch.abahmed.dev -o jsonpath='{.metadata.annotations.lifecycle-test}')" = upgraded
kubectl get kwatchconfig lifecycle-check --namespace "$namespace" >/dev/null
Expand Down
Loading
0