Updating a Kubernetes Opaque Secret

introduction

Yes, Kubernetes, what is it? Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications.¹ The Software Development Life Cycle (SDLC), isn’t only about developing software - as the name may imply - but also all operations involved in smoothly running and using the software or service. This involves, of course, deploying your software for external use (unless you intend it to remain internal) and scaling it to support the growing load as time goes on. These developer operations, among many others, is where Kubernetes comes in handy.

This writing focuses on the use of Kubernetes to manage secrets², more specifically, to update them. I hadn’t interacted with Kubernetes until recently, so I am by no means an expert and hope to improve this post and write more on it in the future. In this piece, I’ll be sharing my approach to updating Kubernetes secrets.

types of k8s secret

Kubernetes supports different types of secrets³, each with a specific use case. I’ll focus on Opaque secrets, which are arbitrary user-defined data. Opaque secrets are the default type for Kubernetes secrets, making them ideal for general-purpose use to protect various sensitive data across your cluster.⁴ Application code typically relies on this type of secret. Information such as database name and credentials, API keys, tokens, encryption keys — basically, any sensitive data that shouldn’t be hard-coded in application code — can be stored in an Opaque secret in Kubernetes.

updating an opaque secret - my approach

As mentioned earlier, I will be sharing my current approach to upating an opaque Kubernetes secret. I basically follow 5 steps, yes 5, but it works fine for me and you might find it useful. Let’s start by taking a look at a sample opaque secret config file; sample below:

apiVersion: v1
data:
  secret.yaml: c2VydmVyOiBteXNlcnZlci5jb20KdXNlcjogYWRtaW4K...
kind: Secret
metadata:
  creationTimestamp: "2020-01-01T13:02:15Z"
  name: my-service-secret
  namespace: my-namespace
  resourceVersion: "123456
  uuid: some-uuid
type: Opaque

This sample is a standard definition of a Kubernetes Opaque secret. The components include:

1. apiVersion: Specifies the Kubernetes API version used for the secret.
2. data: Stores key-value pairs; base64-encoded to protect sensitive information.
3. metadata: Contains information such as the secret’s name, namespace, and timestamps.
4. type: Defines the secret as Opaque, allowing it to hold generic data.

To interact with Kubernetes clusters, a user typically needs a versatile tool. Examples of these tools include kubectl, k9s, among others. Kubectl⁵ is the command-line tool I am most familiar with, so that will be the primary tool used in the process.

steps

Right, let’s dive into the actual action. Outlined below are the steps I follow in updating an Opaque Kuberbetes secret. These steps will assume we are updating the secret above. It also assumes basic knowledge of concepts in the Kubernetes ecosystem such as namespaces, pods, etc.

1. Get and decode the secret & pipe the result to a yaml file

   kubectl get secret my-secet -n my-namespace -o jsonpath=' {.data.secret\.yaml}' | base64 --decode > s.yaml
   

2. Open the yaml file (with the decoded secret data) and make the necessary changes/updates

   vim s.yaml
   

3. Re-encode the updated secret data and pipe to a new file. The -i flag isn’t required on linux systems.

   base64 -i s.yaml | tr -d '\n' > s_encoded.yaml
   

4. Create a JSON file which contains the request to update the secret; sample content below

   vim s_req.json
   

   [
     {
         "op": "replace",
         "path": "/data/secret.yaml",
         "value": "<base64-encoded value in s_encoded.yaml>"
     }
   ]
   

5. Make a patch request using kubectl to update the secret

   kubectl patch secret my-service-secret -n my-namespace --type='json' --patch-file=s_req.json
   

In most cases, you would need to restart the Kubernetes deployment⁶ for the pods to pick up the updated secrets.

conclusion

This post shared one of the many approaches used for updating an Opaque Kubernetes secret. There are simpler or faster ways to do the same thing, but this is my brain dump of what works for me at the moment.

There is a handy kubectl plugin that I sometimes use for the same purpose, called kubectl-modify-secret. However, it tends not to parse some multi-line secret files too well, which is what pushed me to find an alternative approach in case this tool doesn’t work for me as I expect. The issue is documented here. Since I have experienced this issue first-hand, I am hoping to investigate further and most likely contribute a workaround for this, and you can too! You should give the tool a try; kudos to its creator. I hope you find this post useful!