infra/docs/plans/2026-06-04-k8s-dashboard-sso-design.md

K8s Dashboard SSO via Authentik (oauth2-proxy) — Design

Date: 2026-06-04 Status: Approved (design) Author: Viktor + Claude Scope: Let namespace-owner users (e.g. gheorghe / vabbit81) open the Kubernetes Dashboard at https://k8s.viktorbarzin.me, authenticate once with their Authentik account, and manage their own namespace (inspect/edit Deployments, read logs) with read-only visibility elsewhere — no second login, no manually-pasted token.

---

1. Goal & Non-Goals

Goal

A user in an Authentik kubernetes-* group browses to the dashboard, completes the normal Authentik SSO redirect, and lands in a dashboard session whose permissions are their own K8s RBAC (scoped by the OIDC email/groups claims the apiserver already trusts). gheorghe gets full control of namespace vabbit81 and read-only cluster visibility.

Non-Goals

• No change to the kube-apiserver OIDC flags (already configured).
• No change to the RBAC module or the k8s_users schema (gheorghe is already onboarded; his RoleBindings already exist).
• No change to the existing CLI OIDC flow (kubelogin against the public kubernetes client) used by viktor/anca — it must keep working untouched.
• Not replacing the dashboard (Headlamp et al. were considered and rejected).
• Not hardening/removing the existing static cluster-admin SA (tracked as an optional future item in §9, deliberately out of scope here to keep blast radius minimal and break-glass intact).

---

2. Current State (verified)

Fact	Evidence
Dashboard deployed via Helm chart kubernetes-dashboard 7.12.0 in ns kubernetes-dashboard; fronted by kubernetes-dashboard-kong-proxy (HTTPS 443)	stacks/k8s-dashboard/main.tf
Ingress k8s.viktorbarzin.me currently auth = "required" (Authentik forward-auth) → kong-proxy. After auth, the dashboard only had a static cluster-admin SA token to talk to the apiserver — every authenticated user was effectively cluster-admin	stacks/k8s-dashboard/main.tf (ingress + admin-user CRB)
kube-apiserver already OIDC-configured: --oidc-issuer-url=https://authentik.viktorbarzin.me/application/o/kubernetes/, --oidc-client-id=kubernetes, --oidc-username-claim=email, --oidc-groups-claim=groups	stacks/rbac/modules/rbac/apiserver-oidc.tf
Per-user RBAC already created from k8s_users (Vault secret/platform): namespace-owners get a RoleBinding to ClusterRole admin in their namespace + a cluster-wide read-only ClusterRoleBinding, keyed on email	stacks/rbac/modules/rbac/main.tf
gheorghe = vabbit81, email vabbit81@gmail.com, namespace vabbit81, role namespace-owner. Bindings namespace-owner-vabbit81 + oidc-ns-owner-readonly-vabbit81 exist	Vault secret/platform → k8s_users
Authentik is Terraform-managed (provider adopted 2026-04-18, Wave 6a). Proxy providers/outposts/guest flow live in stacks/authentik/*.tf. The goauthentik/authentik provider is available to every stack via central terragrunt.hcl	stacks/authentik/authentik_provider.tf, guest.tf
The existing kubernetes OIDC application is still UI-managed (no client_id="kubernetes" in the repo) — must not be disturbed	repo grep (no match)
ingress_factory auth enum + comment-convention guard (scripts/check-ingress-auth-comments.py) require a # auth = "<tier>": … comment above any auth = "none"/"app" line	modules/kubernetes/ingress_factory/main.tf, infra/.claude/CLAUDE.md

The missing link is purely token injection: nothing today gives the dashboard the user's own OIDC id_token, so the apiserver can't apply the per-user RBAC that already exists.

---

3. Architecture

Browser
  → Cloudflare (proxied)
  → Traefik (ingress auth = "none"; oauth2-proxy is now the gate)
  → oauth2-proxy.kubernetes-dashboard.svc:4180
       ├─ no session → 302 → Authentik OIDC code-flow (+PKCE) → /oauth2/callback
       │     gated by a group policy: only kubernetes-{admins,power-users,namespace-owners}
       └─ session valid → proxies upstream + sets `Authorization: Bearer <id_token>`
  → kubernetes-dashboard-kong-proxy.svc:443  (UNCHANGED)
  → dashboard `api` → kube-apiserver (Bearer token)
       → OIDC validates: iss ✓, aud ⊇ {kubernetes} ✓, sig ✓
       → username = email, groups = groups claim
       → RBAC: namespace-owner-<user> (admin in their ns) + cluster read-only

The entire change is additive + one ingress repoint. New objects: oauth2-proxy Deployment/Service, an Authentik OIDC application/provider + scope mapping + group policy, and an ESO-synced secret. The ingress backend flips from kong-proxy → oauth2-proxy and auth flips required → none.

---

4. Components

4.1 Authentik (Terraform, in stacks/k8s-dashboard/)

Follows the stacks/authentik/guest.tf pattern (provider block reads secret/authentik → tf_api_token from Vault).

• authentik_provider_oauth2 "k8s_dashboard":
  • client_type = "confidential", client_id = "k8s-dashboard", client_secret from Vault.
  • allowed_redirect_uris = [{ matching_mode="strict", url="https://k8s.viktorbarzin.me/oauth2/callback" }].
  • authorization_flow = default implicit-consent (data source); invalidation_flow = default (data source).
  • access_token_validity = "hours=1", refresh_token_validity = "days=30".
  • include_claims_in_id_token = true (so the id_token carries email + groups; the apiserver reads the email claim for username regardless of sub_mode).
  • property_mappings = the default OIDC scope mappings (openid, email, profile) plus the goauthentik groups scope mapping plus the custom audience mapping below. (Resolved via data authentik_property_mapping_provider_scope lookups so we don't drop the standard claims.)
• authentik_property_mapping_provider_scope "k8s_dashboard_aud":
  • scope_name = "k8s-dashboard-audience", expression = return {"aud": ["kubernetes", "k8s-dashboard"]}.
  • Emits both audiences so the apiserver (kubernetes) and oauth2-proxy (k8s-dashboard) each find their own client id in aud.
• authentik_application "k8s_dashboard": slug k8s-dashboard, protocol_provider = the oauth2 provider, policy_engine_mode = "any".
• Group gate: authentik_policy_expression "k8s_dashboard_groups" → return ak_is_group_member(request.user, name="kubernetes-admins") or ak_is_group_member(request.user, name="kubernetes-power-users") or ak_is_group_member(request.user, name="kubernetes-namespace-owners"), bound to the application via authentik_policy_binding.

4.2 Vault + ESO

• Vault secret/k8s-dashboard (new): oauth2_proxy_client_id, oauth2_proxy_client_secret, oauth2_proxy_cookie_secret (32 random bytes, base64). The Authentik provider reads the same client id/secret so the two sides match.
• ExternalSecret in stacks/k8s-dashboard/main.tf → K8s Secret oauth2-proxy in ns kubernetes-dashboard. First-apply: target the ExternalSecret before the full apply (documented plan-time gotcha).

4.3 oauth2-proxy (Terraform, in stacks/k8s-dashboard/)

• Image quay.io/oauth2-proxy/oauth2-proxy:v7.x (pin a concrete tag; SHA-tag convention). linux/amd64.
• Deployment: 2 replicas (HA path), Recreate not needed (stateless), readiness /ping, requests 25m/64Mi, limit 128Mi. Standard dns_config ignore_changes (KYVERNO_LIFECYCLE_V1).
• Config (env OAUTH2_PROXY_* or args):
  • provider=oidc, oidc_issuer_url=https://authentik.viktorbarzin.me/application/o/k8s-dashboard/
  • client_id/client_secret/cookie_secret from the oauth2-proxy Secret
  • redirect_url=https://k8s.viktorbarzin.me/oauth2/callback
  • upstreams=https://kubernetes-dashboard-kong-proxy.kubernetes-dashboard.svc.cluster.local:443
  • ssl_upstream_insecure_skip_verify=true (kong self-signed cert)
  • pass_authorization_header=true (passes id_token to the dashboard)
  • set_authorization_header=true (belt-and-suspenders)
  • oidc_extra_audience=kubernetes (accept the apiserver audience too)
  • scope=openid email profile groups offline_access
  • email_domains=* (the Authentik group policy is the real gate)
  • cookie_secure=true, cookie_domains=k8s.viktorbarzin.me, whitelist_domains=k8s.viktorbarzin.me, cookie_refresh=30m, cookie_expire=168h
  • reverse_proxy=true, skip_provider_button=true
• Service oauth2-proxy (port 4180).

4.4 Ingress (edit existing in stacks/k8s-dashboard/main.tf)

• service_name = "oauth2-proxy", backend_protocol = "HTTP", port = 4180.
• auth = "none" with the mandatory comment: # auth = "none": oauth2-proxy is the gate — it runs the Authentik OIDC code-flow and injects the user's id_token as Bearer for dashboard→apiserver auth. Group policy on the Authentik app restricts to kubernetes-* groups.
• Keep dns_type = "proxied" and the homepage annotations.

---

5. The Audience Strategy (the crux) + Fallback

The apiserver is pinned to a single legacy --oidc-client-id=kubernetes; the CLI uses the public kubernetes client. oauth2-proxy must be its own confidential client (k8s-dashboard). A token with aud=k8s-dashboard alone would be rejected by the apiserver; a token with aud=kubernetes alone would be rejected by oauth2-proxy. Resolution: the Authentik scope mapping emits aud = ["kubernetes", "k8s-dashboard"]; both validators do a membership check and each finds its own id. oidc_extra_audience=kubernetes on oauth2-proxy is an extra safety net in case Authentik's scope mapping overrides aud to a single value rather than appending.

Apply-time verification (blocking): decode a freshly issued id_token and assert aud contains both kubernetes and k8s-dashboard, and that groups is present. If Authentik refuses to emit a multi-valued aud via scope mapping, fallback (documented, not preferred): repoint oauth2-proxy at the existing kubernetes client made confidential and add --oidc-client-secret to the kubelogin setup script — this unifies the audience at the cost of touching the CLI flow. We try the additive multi-aud path first precisely to avoid that.

---

6. Data Flow — gheorghe end-to-end

1. https://k8s.viktorbarzin.me → Cloudflare → Traefik (auth=none) → oauth2-proxy.
2. No session → 302 to Authentik (k8s-dashboard client, code+PKCE).
3. Authentik runs the group policy: vabbit81 ∈ kubernetes-namespace-owners ✓. Issues id_token: email=vabbit81@gmail.com, groups=[…,kubernetes-namespace-owners], aud=[kubernetes,k8s-dashboard].
4. oauth2-proxy validates token (k8s-dashboard ∈ aud), sets session cookie, proxies to kong-proxy with Authorization: Bearer <id_token>.
5. kong-proxy → dashboard api → kube-apiserver with the Bearer token.
6. apiserver OIDC: username=vabbit81@gmail.com, groups from claim.
7. RBAC: RoleBinding namespace-owner-vabbit81 (ClusterRole admin) in ns vabbit81 + ClusterRoleBinding oidc-ns-owner-readonly-vabbit81.
8. Dashboard shows full control of vabbit81, read-only elsewhere — the goal.

No RBAC changes required; bindings already key on his email.

---

7. Testing

1. Token shape (blocking): decode an issued id_token; assert aud ⊇ {kubernetes,k8s-dashboard} and groups present.
2. Admin: viktor logs in → sees/edits everything (cluster-admin group).
3. Namespace-owner: gheorghe logs in → can edit Deployments / read logs in vabbit81; gets Forbidden creating resources in other namespaces; can view (read-only) cluster resources.
4. No regression: viktor/anca CLI kubectl (kubelogin → public kubernetes client) still works.
5. Browser checks driven via Playwright MCP; screenshot on failure.

---

8. Rollback

Single-commit revert of the ingress edit restores service_name=kong-proxy, backend_protocol=HTTPS, port=443, auth=required → instant return to Authentik forward-auth gating. oauth2-proxy + Authentik objects are additive and inert once the ingress no longer points at them; they can be destroyed in a follow-up. No apiserver, RBAC, data, or CLI changes to unwind.

---

9. Security Notes & Out-of-Scope Hardening

• The id_token only ever lives server-side (Bearer header oauth2-proxy→kong); the browser holds an opaque oauth2-proxy session cookie (secure/httponly/samesite-lax, scoped to k8s.viktorbarzin.me).
• Two gates: Authentik group policy (only kubernetes-* groups complete the flow) and apiserver RBAC (per-user, by email). Defense in depth.
• The authentik_walloff blackbox guard is for auth=none carve-outs that must NOT redirect to Authentik. The dashboard intentionally does redirect (via oauth2-proxy), so it is not added to that guard.
• Out of scope (optional future hardening): the existing static admin-user cluster-admin ClusterRoleBinding + SA remain. They predate this change and provide break-glass. Removing them (admins would rely on SSO + Vault kubernetes/creds/dashboard-admin + kubelogin) is a separate, reversible security decision the user can request later. Not done here to keep blast radius minimal.

---

10. Monitoring & Docs

• Uptime-Kuma external monitor auto-created (dns_type=proxied → ingress_factory adds the external-monitor label). No manual monitor.
• oauth2-proxy readiness probe on /ping.
• Docs updated in the same commit: docs/architecture/authentication.md (new OIDC app + dashboard SSO flow), docs/architecture/multi-tenancy.md (dashboard access path for namespace-owners), .claude/reference/authentik-state.md (new app/provider/scope-mapping), .claude/reference/service-catalog.md (k8s-dashboard auth posture), and the companion 2026-06-04-k8s-dashboard-sso-plan.md.

---

11. Open Risks

Risk	Mitigation
Authentik scope mapping overrides rather than appends aud	oidc_extra_audience=kubernetes + blocking apply-time token decode; fallback in §5
Dashboard v7 ignores a pre-set Authorization header (known friction: kubernetes/dashboard #5105, #1213)	pass_authorization_header + set_authorization_header; validate in §7; kong forwards headers by default
ESO first-apply ordering	terragrunt apply -target the ExternalSecret first (documented plan-time pattern)
Single-master apiserver assumption (memory id=2484)	We don't touch apiserver flags; no new exposure

---

12. ADDENDUM (2026-06-04) — As-built pivoted to Option B (apiserver multi-issuer)

Sections 4–5 above describe the original plan: a separate k8s-dashboard confidential client whose token carries a dual aud so the apiserver (pinned to --oidc-client-id=kubernetes) would accept it without an apiserver change. That approach does not work, for a reason discovered during implementation:

1. The issuer is the binding constraint, not the audience. Every Authentik OAuth2 application has its own per-slug issuer. A token from the k8s-dashboard app has iss=…/o/k8s-dashboard/, but the apiserver does an exact issuer-string match against its single configured issuer (…/o/kubernetes/). The dual-aud scope mapping is irrelevant — the token is rejected on issuer before audience is even considered.

2. Apiserver OIDC was already silently broken. Inspecting the live kube-apiserver static-pod manifest showed no --oidc-* flags at all — the kubeadm v1.34 upgrade had regenerated the manifest and dropped the flags the rbac stack's null_resource had injected (its content-hash trigger never re-fired). So OIDC apiserver auth was off cluster-wide.

3. Reusing the kubernetes app (make it confidential) — rejected. It would force distributing the now-confidential client secret to every CLI user via the public k8s-portal /setup/script endpoint (a leak), plus re-onboarding existing CLI users. Too invasive.

As-built = Option B: structured AuthenticationConfiguration on the apiserver trusting BOTH issuers. stacks/rbac/modules/rbac/apiserver-oidc.tf now writes /etc/kubernetes/pki/auth-config.yaml (apiserver.config.k8s.io/v1) with two jwt issuers — kubernetes (audience kubernetes, for the kubelogin CLI) and k8s-dashboard (audience k8s-dashboard, for oauth2-proxy) — each mapping username<-email and groups<-groups with empty prefixes (to match existing RBAC subjects). The legacy --oidc-* flags are replaced by --authentication-config=…. The remote script health-gates /livez and auto-rolls-back the manifest if the single-master apiserver doesn't recover. The oauth2-proxy + k8s-dashboard Authentik app from §4 are reused unchanged (the dual-aud mapping is now harmless — issuer2 only requires k8s-dashboard ∈ aud).

This keeps the CLI flow 100% untouched (its own kubernetes issuer is one of the two trusted issuers) and restores the apiserver OIDC that the kubeadm upgrade had broken.

Known drift (carried forward): a future kubeadm upgrade will again regenerate the manifest and drop --authentication-config. The content-hash trigger won't auto-detect this. Operational mitigation: re-apply the rbac stack after every k8s control-plane upgrade (add to the upgrade runbook). The rbac provisioner needs TF_VAR_ssh_private_key (an SSH key authorized on the master) — it is not wired from Vault yet.