Repository navigation
fix(cozystack-basics): refuse a direct delete of a tenant namespace - #5011
europrinter (yankawai) wants to merge 1 commit into
Conversation
A tenant namespace is meant to go away with its Tenant. The tenant chart's pre-delete hook deletes the applications first and the tenant modules such as etcd after them, and helm-controller deletes the namespace at the end. A direct delete hands the namespace to the namespace controller instead. It deletes the pods first and then everything else together, so the tenant etcd and its etcd-ca-tls Secret can be gone before the TenantControlPlanes that use them. Kamaji then cannot reach the datastore to clean up, the TenantControlPlane finalizer never clears and the namespace stays Terminating. The policy refuses DELETE of a namespace labelled tenant.cozystack.io/tenant-root, which the tenant chart puts on every tenant namespace below tenant-root. tenant-root carries no such label and stays with the no-delete guardrail. Two callers are let through. helm-controller and its shards run as the flux ServiceAccount in cozy-fluxcd and delete the namespace during a Tenant uninstall. The garbage collector runs in kube-system and removes a child namespace through its ownerReference once the parent namespace is gone. The namespace controller only finalizes a namespace and never sends a DELETE for it, so it is not affected. Everyone else is refused, system:masters included, as with the no-delete guardrail. The message names the Tenant to delete and the label to remove when the Tenant is already gone. The policy test helpers take the template path now, so the new test compiles the shipped template with the apiserver CEL compiler like the external TLS policy test does. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
IvanHunters
left a comment
There was a problem hiding this comment.
Verified the invariant rather than only the guard: enumerated every namespace deleter (no Go code deletes a tenant namespace; helm-controller in cozy-fluxcd and the GC in kube-system are exempt, so legitimate teardown via kubectl delete tenant still works). The direct human DELETE is refused, confirmed with the in-process CEL decision test (11/11) and a mutation that drops the kube-system exemption (the GC case then fails). No blocking issues.
Non-blocking:
tenant-namespace-delete-policy.yaml:69-73: thesystem:serviceaccounts:kube-systemexemption is broad (any SA in kube-system, not just the GC). Acceptable as a footgun guardrail since RBAC still gates who can delete, but worth awareness.tenant-namespace-delete-policy.yaml:74-84: the denial message parses tenant/parent by splitting on the last-, which is correct only while tenant names stay single-word. Deny fires regardless, so this is message accuracy only.- Migration 49 labels every
^tenant-namespace, so a non-tenant namespace namedtenant-*becomes undeletable until the label is removed (the message gives the command). Protection also depends on thetenant-rootlabel being present, with a brief gap for a not-yet-reconciled deep tenant namespace.
What this PR does
Fixes #5002. A DELETE of a tenant namespace is refused unless it comes from the identities that remove it on the normal path, and the denial says to delete the Tenant instead.
The new ValidatingAdmissionPolicy in cozystack-basics matches DELETE of core namespaces, and its binding selects namespaces labelled
tenant.cozystack.io/tenant-root. The tenant chart puts that label on every tenant namespace below tenant-root (tenant.ancestorTenantLabels, and migration 49 backfilled older ones). tenant-root itself has no such label and stays withcozystack-no-delete-guardrail. Thetenant-*probe namespaces of the e2e suites carry onlynamespace.cozystack.io/hostand are not selected either.Two groups pass.
system:serviceaccounts:cozy-fluxcd, because helm-controller and its shards run asfluxthere, and a Tenant uninstall is helm-controller deleting the namespace after the pre-delete hook.system:serviceaccounts:kube-system, for the garbage collector following the ownerReference of a child namespace once its parent is gone. The namespace controller never sends a DELETE for a namespace, it updatesfinalize, so it is not affected (pkg/controller/namespace/deletionandpkg/registry/core/namespace/storageat v1.33.12 and v1.34.1).system:mastersis not exempt, the same as with the no-delete guardrail.This is the denial kube-apiserver 1.35 returns.
There are a few limits.
tenant-*namespace that existed when it ran, so a non-tenant namespace with such a name is refused too and needs the same label removal.--default-service-account, orspec.serviceAccountNameon Tenant HelmReleases) would need that account allowed. Cozystack sets neither.The helper rename in
external_tls_policy_test.go(loadPolicy(t, path)andcompilePolicy) is byte for byte the one in #2622, so the two merge in either order.Checked locally.
go test ./pkg/registry/apps/application/ -run 'TestTenantNamespaceDeletePolicy|TestDatabaseExternalTLSPolicy' -count=1 -vpasses 4 tests, the newTestTenantNamespaceDeletePolicy_Decisionswith 11 cases.go test ./pkg/registry/... ./pkg/config/... ./pkg/cmd/server/...passes in all 11 packages that have tests.helm unittestinpackages/system/cozystack-basicswith helm-unittest 1.1.2 in docker passes 12 suites and 70 tests, against 11 and 66 on main. The new tests on main without the template fail, both Go tests and 4 helm tests in 2 suites.system:masters, drop either exempt group, select on the no-delete label, add UPDATE, always name tenant-root as the parent, drop the capability gate, label tenant-root, switch to Warn, switch failurePolicy to Ignore, drop the bypass from the message and drop the objectSelector.tenant-*namespace without the label and a tenant namespace after its label is removed are accepted for deletion. With this binding removed, a dry-run delete by the cluster admin goes through. That harness is not part of the PR.hack/bats-no-negated-assert.bats,hack/bats-no-exit-trap.bats,hack/bats-runner-coverage.bats,hack/helmrelease-readiness.batsandhack/route-hostname-policy-version-coverage.batspass underhack/cozytest.sh.The new e2e case in
hack/e2e-install-cozystack.batscreates an empty Tenant, checks that a server dry-run delete of its namespace is refused with the message, deletes the Tenant and waits for the namespace to go. It was not run here, there is no Cozystack cluster at hand. Its kubectl steps were run against the envtest apiserver 1.35, the selector and field selector read, the refused dry-run with the exact strings it greps for, andkubectl wait --for=deleteon a namespace that is gone. The Chainsaw suites that delete Tenants, the gateway inheritance ones among them, go through the uninstall path as well.The release-1.6 backport is #5012.
Screenshots
Not a UI change.
Downstream repositories
The trigger map was walked against the diff. It adds a policy and tests to cozystack-basics and an e2e case, and no entry covers that. The website docs do not tell users to delete a tenant namespace directly, and terraform-provider-cozystack deletes Tenants through the apps API.
Release note