Part 2 - Managing Configuration

Daniel LopesDaniel Lopes
8 min read

As in current days a new feature or a correction moves through several environments (Development, Staging, UAT) until is finally available to final users (Production environment). While the application moves throughout those environments, it will need to assume different configurations - connect to different databases, and get data from different resources. To allow this kind of flexibility K8s has specific resources like ConfigMap and Secrets to manage all that and this article will show how to use them in several ways.

ConfigMap vs Secrets

ConfigMap is the recommended place to put non-sensitive configurations like a link to a database but not things like the corresponding username and password, for that we have Secrets K8s object.

Setting ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  # this configmap name
  name: myConfigmap
  namespace: develop
data:
  # property-like keys; each key maps to a simple value
  mysql_connection: "jdbc:mysql://fake-mysql-db:3306/mydb"
  oracle_connection: "jdbc:oracle:thin:@fake-oracle-db:4443:SID"

  # file-like keys
  kafka.yaml: |
    spring:
      kafka:
        bootstrap-servers: 127.0.0.1:19091
        consumer:
          group-id: car-group

  user-interface.properties: |
    color.good=green
    color.bad=red
    allow.textmode=true

Fig. 1 - Example of a ConfigMap file

In the above example, it can be seen how to create a Configmap file. Section "metadata" provides the "name" by which its keys can be found. Section data gives the keys with values in two possible formats:

  • key-value pair: mysql_connection: "jdbc:mysql://fake-mysql-db:3306/mydb"

  • key-file pair: kafka.yaml: | ....

In Fig. 2 we can see how those keys are used:

apiVersion: v1
kind: Pod
metadata:
  name: mydemo-pod
spec:
  containers:
    - name: demo
      image: alpine
      command: ["sleep", "3600"]
      env:
        # Define the environment variable
        - name: MYSQL_CONNECTION # Notice that the case is different here
                                 # from the key name in the ConfigMap.
          valueFrom:
            configMapKeyRef:
              name: myConfigmap  # The ConfigMap this value comes from.
              key: mysql_connection # The key to fetch.

#...unneeded code...

Fig. 2 - Example of usage of Configmap in Pod creation

It can be seen the values for the Mysql and Oracle databases are given in the valueFrom.configMapKeyRef section: we must provide the name given to the configmap (myConfigmap) and the key where it is located.

To apply/create a config map on K8s cluster using a file, open a console and perform
kubectl apply -f configmap_file_name.yaml

Setting Secrets

Secrets are the K8s object where sensitive data like passwords and tokens can be stored.

Secrets can be of several types [2], but here we will explore the Opaque type because is the one used for credentials like username and password for a Pod to use to connect to resources like a database.

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
  namespace: develop
type: Opaque
data:
  mysql_username: bXlzcWxfdXNlcm5hbWU=
  mysql_password: bXlzcWxfcGFzc3dvcmQ=

Fig. 3 - Example of a Secret file with Mysql and Oracle credentials

In Fig. 3 shows how to create a Secret file. The username and passwords must be previously 64 bit encoded and the output pasted on the correspondent place:

echo -n 'mysql_username' | base64
#output bXlzcWxfdXNlcm5hbWU=
echo -n 'mysql_password' | base64
#output bXlzcWxfcGFzc3dvcmQ=

After the secret's file is created execute kubectl apply -f secret_file_name.yaml

Next, it's needed to setup a Pod configuration to make use of those secret values.

The next example shows how to setup a Pod to make use of the secrets we just created.

apiVersion: v1
kind: Pod
metadata:
  name: mydemo-pod
spec:
  containers:
    - name: demo
      image: alpine
      command: ["sleep", "3600"]
      env:
        - name: MYSQL_USERNAME   # Define the environment variable
          valueFrom:
              secretKeyRef:
                  name: mysecret # The Secret this value come from
                  key: mysql_username # The key to fetch.
        - name: MYSQL_PASSWORD
          valueFrom:
            secretKeyRef:
                  name: mysecret 
                  key: mysql_password

Fig. 4 - Secrets applied to a Pod

Here we can see that it is created an environment variable in CAPS that will be used by the application and subsection valueFrom where we indicate the secret object name where to get the value and the key that has the value.

Setting configuration by mounting volumes

K8s allows making the configuration objects (both ConfigMap and Secrets) accessible by volume mounts. The configurations are created normally, then in the Pod configurations file they are defined as volume mount points and the keys are available inside those same mount points.

Here how it's done:

Create the Configmap

apiVersion: v1
kind: ConfigMap
metadata:
  # this configmap name
  name: my-configmap
data:
  # property-like keys; each key maps to a simple value
  oracle_connection: "jdbc:oracle:thin:@fake-oracle-db:4443:SID"

  # file-like keys
  kafka.yaml: |
    spring:
      kafka:
        bootstrap-servers: 127.0.0.1:19091
        consumer:
          group-id: car-group

  user-settings.properties: |
    colors.color1=yellow
    colors.color2=red
    fruits.fruit1=banana
    fruits.fruit2=strawberry

Fig. 5: ConfigMap to be used as a Volume

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
type: Opaque
data:
  oracle_username: b3JhY2xlX3VzZXJuYW1l
  oracle_password: b3JhY2xlX3Bhc3N3b3Jk

Fig. 6: Secret to be used as a Volume

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
    - name: demo
      image: alpine
      resources:
          limits:
            memory: 600Mi
            cpu: 1
      command: ["sleep", "3600"]
      volumeMounts:
      - name: configmap-vol
        mountPath: "/etc/config/configMap"
      - name: secret-vol
        mountPath: "/etc/config/secret"
  volumes:
  # You set volumes at the Pod level, then mount them into containers
  # inside that Pod
  - name: configmap-vol
    configMap:
      # Provide the name of the ConfigMap you want to mount.
      name: my-configmap
      # An array of keys from the ConfigMap to create as files
      items:
      - key: "kafka.yaml" # the key on configmap
        path: "kafka_file.yaml" # the name to be given on the volume
      - key: "user-settings.properties"
        path: "user-settings_file.properties"
  - name: secret-vol
    secret:
      secretName: my-secret

Fig. 7: Pod definition with the ConfigMap and Secret to be mounted as Volumes

After "applying" the K8s definition files it is possible to access the pod console and see how the artifacts were created.

Fig. 8 - Accessing the configuration as volumes inside a pod

Setting configuration as a Pod's environment variable

Another way to have configuration variables available is by setting them as environment variables. The way to set this up is simple. Create the ConfigMap with the variable names in upper case format:

apiVersion: v1
kind: ConfigMap
metadata:
  # this configmap's name
  name: my-configmap-posix
data:
  # property-like keys; each key maps to a simple value
  # the key must be UPPER CASE
  ORACLE_CONNECTION: "jdbc:oracle:thin:@fake-oracle-db:4443:SID"
  MYSQL_CONNECTION: "jdbc:mysql://localhost:3306/mydb"

Fig. 9 - ConfigMap defining 2 variables to be available as environment variables in a Pod

In the Pod definition file, set the ConfigMap's name to be used under spec.containers.envFrom.configMapRef.name:

apiVersion: v1
kind: Pod
metadata:
  name: my-pod-posix
spec:
  containers:
    - name: demo
      image: alpine
      resources:
          limits:
            memory: 300Mi
            cpu: 1
      command: ["sleep", "3600"]
      envFrom:
        - configMapRef:
            name: my-configmap-posix # the ConfigMap's name to be used

Fig. 10 - Pod definition file with the ConfigMap as environment variables placeholder

Apply both definition files and enter in the pod's console by issuing kubectl exec --stdin --tty my-pod-posix -- /bin/sh .
By issuing printenv it can be seen all the environment variables including the ones declared in the ConfigMap.

Fig. 11 - printenv command showing Pod's console showing environment variables

Using ConfigMap and Secrets as a file in a Pod

If you need a configuration file to setup a MySQL database and have those configurations available as a file in the proper place so the MySql engine can use it.

To create a ConfigMap:

Create the required file, for our example it will be mysql.conf.
To generate a ConfigMap from the file issue: kubectl create configmap mysql-conf-file --from-file mysql.conf . If you list the ConfigMaps in K8s cluster

> kubectl get configmap
NAME                 DATA   AGE
kube-root-ca.crt     1      38d
my-configmap         3      4d20h
my-configmap-posix   2      4h7m
mysql-conf-file      1      07s   #the configmap just created

To create a Secret is the same process:

Create the data in a file and issue the follwoing comand :

kubectl create secret <secret_name> --from-file <file_name>

In this example two certificates where created: my-certificate-1.crt and my-certificate-2.crt. To create the secrets input:

kubectl create secret generic certificate-1 --from-file my-certificate-1.crt
kubectl create secret generic certificate-2 --from-file my-certificate-2.crt

If you list the Secrets in K8s:

> kubectl get secret
NAME            TYPE     DATA   AGE
certificate-1   Opaque   1      4m56s
certificate-2   Opaque   1      5m5s

Now we have to define in the Pod definition file how to associate the mysql-conf-file and my-certificate to the proper files.

As a note the secrets will be made available as a file but only inside of different folders.

apiVersion: v1
kind: Pod
metadata:
  name: my-pod-conf-as-file
spec:
  containers:
    - name: demo
      image: alpine
      resources:
          limits:
            memory: 300Mi
            cpu: 1
      command: ["sleep", "3600"]
      volumeMounts:
        - name: mysql-config-volume # name of the volume that has the mysql.conf data
          mountPath: /etc/mysql/my.cnf # the file name to mount the configmap
        - name: my-certificate-1-volume # name of the volume that has the certificate
          mountPath: /etc/certificate-folder-1 # folder where to put the secret
        - name: my-certificate-2-volume
          mountPath: /etc/certificate-folder-2 
  volumes:
    - name: mysql-config-volume      # the name to give to this volume
      configMap:
        name: mysql-conf-file        # the name of the configmap

    - name: my-certificate-1-volume  # the name to give to this volume
      secret: 
        secretName: certificate-1    # the name of the Secret

    - name: my-certificate-2-volume  # the name to give to this volume
      secret: 
        secretName: certificate-2    # the name of the Secret

If we access the pod by issuing kubectl exec --stdin --tty my-pod-conf-as-file -- /bin/sh we can verify if the items were created:

> ls /etc/cert*              # list all content starting with my
/etc/certificate-folder-1:   # the folder where the 1st certificate was mounted
my-certificate-1.crt         # the certificate inside the folder

/etc/certificate-folder-2:   # the folder where the 2nd certificate was mounted
my-certificate-2.crt         #  the certificate inside the folder

References

[1] “Configuration,” Kubernetes. https://kubernetes.io/docs/concepts/configuration/ (accessed Nov. 19, 2023).

[2] “Secrets,” Kubernetes. https://kubernetes.io/docs/concepts/configuration/secret/
(accessed Nov. 19, 2023).

‌

0
Subscribe to my newsletter

Read articles from Daniel Lopes directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Kuberneteskubernetes-configmapkubernetes-secrets

Written by

Daniel Lopes
Daniel Lopes