A battle-tested toolkit for dealing with stuck namespaces and backing up Kubernetes resources.
knsk.sh— diagnoses and kills namespaces stuck inTerminatingstate, and cleans up other cluster-level problemsbackup.sh— backs up any namespace to clean,kubectl apply-ready YAML files
Requires
kubectl≥ 1.20 andpython3.backup.shadditionally requiresyqv4.
Warning
knsk.sh permanently deletes Kubernetes resources and removes finalizers. Review the source before running it, run it without flags first (that is a read-only diagnosis), and preview with --dry-run before using any --delete-* or --force flag. See the Disclaimer.
Running without flags performs a read-only diagnosis and reports:
| Check | What it finds |
|---|---|
| Unavailable API services | Aggregated APIs that are down and blocking namespace cleanup |
| Broken admission webhooks | Validating/Mutating webhooks whose backend Service is missing — a leading cause of stuck namespaces in k8s 1.20+ |
| Stuck namespaces | Namespaces in Terminating state and the resources blocking them |
| Stuck cluster resources | Resources in Terminating state across all namespaces |
| Orphan resources | Resources that belong to a namespace that no longer exists |
| Lost PVCs | PersistentVolumeClaims in Lost phase whose backing PV no longer exists — blocks pod scheduling indefinitely |
# Clone and run (diagnosis only — no changes made)
git clone https://github.com/provadigital/knsk.git
cd knsk && chmod +x knsk.sh
./knsk.sh
# See what commands would run without executing them
./knsk.sh --dry-run --delete-all --force
# Fix everything automatically
./knsk.sh --delete-all --forceknsk.sh [options]
--dry-run Show what will be executed instead of executing it
--skip-tls Set --insecure-skip-tls-verify on all kubectl calls
--delete-api Delete broken API services
--delete-resource Delete stuck resources inside stuck namespaces
--delete-orphan Delete orphan resources found in the cluster
--delete-webhook Delete broken admission webhooks blocking namespace deletion
--delete-lost Delete PersistentVolumeClaims stuck in Lost phase
--delete-all All of the above combined
--force Force-finalize namespaces that survive --delete-all
--port {number} kubectl proxy port for legacy fallback (default: 8765)
--timeout {number} Max seconds to wait for kubectl responses (default: 15)
--no-color Plain output, no ANSI colors (useful in CI/scripts)
--kubeconfig {path} Path to a custom kubeconfig file
-h --help Show this help
--force clears the spec.finalizers on any namespace that is still stuck after all other steps. It requires --delete-all or --delete-resource to be set alongside it.
Modern method (k8s 1.20+): Uses kubectl replace --raw /api/v1/namespaces/<name>/finalize — no proxy, no token required.
Legacy fallback: If the above fails, falls back to kubectl proxy + curl. Token retrieval tries kubectl create token (k8s 1.24+) first, then falls back to SA secret lookup for older clusters.
Exports every resource in a namespace to individual YAML files, stripped of all Kubernetes-managed fields, so they can be re-applied to a fresh namespace with a single command.
chmod +x backup.sh
# Back up a single namespace
./backup.sh my-app
# Back up all non-system namespaces (asks for confirmation)
./backup.sh
# Use a custom output directory
./backup.sh my-app --outdir /mnt/backups
./backup.sh my-app -o /mnt/backups
# Restore — the Namespace object is always applied first, so this is all you need
kubectl apply -f knsk_backup/20260618_17h00/my-app/knsk_backup/
└── 20260618_17h00/ ← timestamp, never overwritten
└── my-app/
├── 00-Namespace-my-app.yaml ← always first (sorts before A–Z)
├── ConfigMap-app-config.yaml
├── Deployment-api.yaml
├── PersistentVolumeClaim-data.yaml
├── PersistentVolume-pvc-xxx.yaml ← cluster-scoped, included when bound to this namespace
├── Secret-app-secrets.yaml
├── Service-api.yaml
└── ...
The 00-Namespace-*.yaml prefix guarantees the namespace is created before any namespaced resources when running kubectl apply -f <dir>/.
The following fields are removed from every exported resource so that kubectl apply works cleanly on a fresh cluster:
status · metadata.uid · metadata.resourceVersion · metadata.generation · metadata.creationTimestamp · metadata.deletionTimestamp · metadata.managedFields · metadata.ownerReferences · metadata.selfLink · metadata.annotations[kubectl.kubernetes.io/last-applied-configuration]
Two additional fields are stripped to avoid binding conflicts on restore:
| Resource | Extra field stripped | Reason |
|---|---|---|
PersistentVolumeClaim |
spec.volumeName |
Allows dynamic provisioning of a fresh PV instead of pointing to a non-existent one |
PersistentVolume |
spec.claimRef |
Returns the PV to Available state so it can be re-bound |
Resources that are ephemeral or auto-reconstructed by Kubernetes are excluded:
| Category | Skipped |
|---|---|
| Always skipped kinds | Event, Endpoints, EndpointSlice, Lease, ControllerRevision, ReplicationController, PodMetrics, NodeMetrics |
| Controller-managed Pods | Pods with ownerReferences set (created by Deployments, StatefulSets, Jobs, etc.) — standalone Pods are backed up |
| Controller-managed ReplicaSets | ReplicaSets owned by a Deployment — standalone ReplicaSets are backed up |
| Auto-generated Secrets | Secrets of type kubernetes.io/service-account-token |
| Auto-generated ConfigMaps | kube-root-ca.crt |
| Default ServiceAccount | The default SA in every namespace |
| System namespaces | kube-system, kube-public, kube-node-lease (full-cluster mode only) |
backup.sh [namespace] [options]
namespace Namespace to back up. Omit to back up all non-system namespaces.
--outdir, -o <path> Write output to this directory instead of the default knsk_backup/<timestamp>/
A test_scripts.yaml file is included to spin up sample workloads across three namespaces (team-a, team-b, team-c) and verify the full backup/restore cycle:
# 1. Deploy test resources
kubectl apply -f test_scripts.yaml
# 2. Run a full-cluster backup
./backup.sh
# 3. Delete the test namespaces
kubectl delete ns team-a team-b team-c
# 4. Restore from backup (replace the timestamp with the one generated in step 2)
kubectl apply -f knsk_backup/<timestamp>/team-a/
kubectl apply -f knsk_backup/<timestamp>/team-b/
kubectl apply -f knsk_backup/<timestamp>/team-c/Expected result: all namespaces, Deployments, ConfigMaps, Secrets, Services, StatefulSets, PersistentVolumes, PersistentVolumeClaims, and Jobs are recreated cleanly with no errors.
Intentional edge case:
team-bcontains a PVC that will end up inLostphase after restore — the backing PV is recreated but the binding reference no longer matches. This is by design, to exerciseknsk.sh --delete-lost. Run the following to clean it up and allow the StatefulSet pods to schedule:./knsk.sh --delete-lostAfter this step, all scripts have been fully tested end-to-end.
| Tool | Required by | Install |
|---|---|---|
kubectl |
both scripts | https://kubernetes.io/docs/tasks/tools/ |
python3 |
knsk.sh --force |
pre-installed on most systems |
yq v4 |
backup.sh |
brew install yq · GitHub releases |
This software is provided "as is", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. See LICENSE for the full text.
Read the source before you run it. knsk.sh and backup.sh are deliberately plain, readable shell scripts. You are expected to review them yourself and satisfy yourself that they do what you intend before running them against any cluster you care about. Never pipe them from a URL into a shell without reading them first.
Use at your own risk. These scripts force-delete Kubernetes resources, strip finalizers, and deliberately bypass safeguards the API server would normally enforce. Used incorrectly — or used correctly against the wrong cluster — they can cause irreversible data loss, orphan resources, and permanently break running workloads. The --force option in particular removes namespace finalizers, which tells Kubernetes to forget cleanup work it had not finished.
The author and contributors accept no responsibility or liability for any damage, downtime, data loss, or malfunction arising from the use or misuse of this software, whether or not that behaviour results from a defect in the software. You are solely responsible for verifying the code, for testing it, and for the consequences of running it.
Recommended before any destructive run:
- Confirm the target cluster —
kubectl config current-context - Take a backup —
./backup.sh(or your own tooling) - Preview the commands —
./knsk.sh --dry-run --delete-all --force - Test on a non-production cluster before trusting it on a production one
- Only then run it for real
If knsk doesn't work for your cluster, open an issue — stuck namespace reproduction cases are always welcome.