New to Kubed? Please start here.

Kubernetes Recycle Bin

Kubed provides a recycle bin for deleted and/or updated Kubernetes objects. Once activated, any deleted and/or updated object is stored in YAML format in folder mounted inside Kubed pod. This tutorial will show you how to use Kubed to setup a recycle bin for Kubernetes cluster objects.

Before You Begin

At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube.

Deploy Kubed

To enable config syncer, you need a cluster config like below.

$ cat ./docs/examples/recycle-bin/config.yaml

clusterName: unicorn
recycleBin:
  handleUpdates: false
  path: /tmp/kubed/trash
  ttl: 168h

Now, create a Secret with the Kubed cluster config under config.yaml key.

$ kubectl create secret generic kubed-config -n kube-system \
    --from-file=./docs/examples/recycle-bin/config.yaml
secret "kubed-config" created

# apply app=kubed label to easily cleanup later
$ kubectl label secret kubed-config app=kubed -n kube-system
secret "kubed-config" labeled

$ kubectl get secret kubed-config -n kube-system -o yaml
apiVersion: v1
data:
  config.yaml: bm90aWZpZXJTZWNyZXROYW1lOiBrdWJlZC1ub3RpZmllcgpyZWN5Y2xlQmluOgogIGhhbmRsZV91cGRhdGU6IGZhbHNlCiAgcGF0aDogL3RtcC9rdWJlZAogIHJlY2VpdmVyOgogICAgbm90aWZpZXI6IG1haWxndW4KICAgIHRvOgogICAgLSBvcHNAZXhhbXBsZS5jb20KICB0dGw6IDE2OGgK
kind: Secret
metadata:
  creationTimestamp: 2017-07-26T18:55:54Z
  labels:
    app: kubed
  name: kubed-config
  namespace: kube-system
  resourceVersion: "32920"
  selfLink: /api/v1/namespaces/kube-system/secrets/kubed-config
  uid: 0d3aa21b-7234-11e7-af79-08002738e55e
type: Opaque

Now, deploy Kubed operator in your cluster following the steps here. Once the operator pod is running, go to the next section.

Using Recycle Bin

In this tutorial, a ConfigMap will be used to show how recycle bin feature can be used.

To keep things isolated, this tutorial uses a separate namespace called demo throughout this tutorial. Run the following command to prepare your cluster for this tutorial:

$ kubectl create namespace demo
namespace "demo" created

~ $ kubectl get namespaces
NAME          STATUS    AGE
default       Active    6h
kube-public   Active    6h
kube-system   Active    6h
demo          Active    4m

Create a ConfigMap called omni in the demo namespace.

$ kubectl create configmap omni -n demo --from-literal=hello=world
configmap "omni" created

$ kubectl get configmaps omni -n demo -o yaml
apiVersion: v1
data:
  hello: world
kind: ConfigMap
metadata:
  creationTimestamp: 2017-07-26T19:18:40Z
  name: omni
  namespace: demo
  resourceVersion: "34414"
  selfLink: /api/v1/namespaces/demo/configmaps/omni
  uid: 3b77f592-7237-11e7-af79-08002738e55e

Now, delete the ConfigMap omni. Kubed operator pod will notice this and stored the deleted object in YAML format in a file matching the selfLink for that object inside the recycleBin.path folder.

# Exec into kubed operator pod
$ kubectl exec -it $(kubectl get pods --all-namespaces -l app=kubed -o jsonpath={.items[0].metadata.name}) -n kube-system sh

# running inside kubed operator pod
/ # find /tmp/kubed/trash/
/tmp/kubed/trash/
/tmp/kubed/trash/api
/tmp/kubed/trash/api/v1
/tmp/kubed/trash/api/v1/namespaces
/tmp/kubed/trash/api/v1/namespaces/demo
/tmp/kubed/trash/api/v1/namespaces/demo/configmaps
/tmp/kubed/trash/api/v1/namespaces/demo/configmaps/omni.20170726T193302.yaml

/ # cat /tmp/kubed/trash/api/v1/namespaces/demo/configmaps/omni.20170726T193302.yaml
apiVersion: v1
data:
  hello: world
kind: ConfigMap
metadata:
  creationTimestamp: 2017-07-26T19:33:02Z
  name: omni
  namespace: demo
  resourceVersion: "35481"
  selfLink: /api/v1/namespaces/demo/configmaps/omni
  uid: 3d50fba0-7239-11e7-af79-08002738e55e

Supported Kubernetes Objects

Following Kubernetes objects are supported by recycle bin:

• v1:
  • ComponentStatus
  • ConfigMap
  • Endpoints
  • Event
  • LimitRange
  • Namespace
  • Node
  • PersistentVolume
  • PersistentVolumeClaim
  • Pod
  • ReplicationController
  • Secret
  • Service
  • ServiceAccount
• apps/v1beta1:
  • Deployment
  • StatefulSet
• batch/v1:
  • Job
• batch/v2alpha1:
  • CronJob
• extensions/v1beta1:
  • DaemonSet
  • Deployment
  • Ingress
  • ReplicaSet
  • ThirdPartyResource
• kubedb/v1alpha1:
  • DormantDatabase
  • Elasticsearch
  • Postgres
  • Snapshot
• monitoring.coreos.com:
  • Prometheus
  • ServiceMonitor
• rbac/v1alpha1:
  • ClusterRole
  • ClusterRoleBinding
  • Role
  • RoleBinding
• rbac/v1:
  • ClusterRole
  • ClusterRoleBinding
  • Role
  • RoleBinding
• monitoring.appscode.com/v1alpha1:
  • ClusterAlert
  • NodeAlert
  • PodAlert
• kubed.appscode.com/v1alpha1:
  • Restic
• storage/v1:
  • StorageClass
• storage/v1beta1:
  • StorageClass
• voyager.appscode.com/v1beta1:
  • Certificate
  • Ingress

To add support for additional object types, please file an issue. We are exploring ways to watch for any object deletion here.

Using Persistent Storage

The installation scripts for Kubed mounts an emptyDir under /tmp path. This tutorial used /tmp/kubed/trash to store objects in recycle bin. If you want to use a PersistentVolume to recycle bin data, mount a PV and updated the recycleBin.path accordingly.

Disable Recycle Bin

If you would like to disable this feature, remove the recyclebin portion of your Kubed cluster config. Then update the kubed-config Secret and restart Kubed operator pod(s).

Cleaning up

To cleanup the Kubernetes resources created by this tutorial, run the following commands:

$ kubectl delete ns demo
namespace "demo" deleted

To uninstall Kubed operator, please follow the steps here.

Next Steps

• Learn how to use Kubed to take periodic snapshots of a Kubernetes cluster here.
• Backup your persistent volumes with Stash.
• Need to keep configmaps/secrets synchronized across namespaces or clusters? Try Kubed config syncer.
• Want to keep an eye on your cluster with automated notifications? Setup Kubed event forwarder.
• Out of disk space because of too much logs in Elasticsearch or metrics in InfluxDB? Configure janitors to delete old data.
• Wondering what features are coming next? Please visit here.
• Want to hack on Kubed? Check our contribution guidelines.