Google Cloud: Working with Google Kubernetes Engine Secrets and ConfigMaps

In this lab, you set up configuration information, both encrypted and unencrypted. Encrypted configuration information is stored as secrets. Unencrypted configuration information is stored as ConfigMaps. This approach avoids hard coding such information into code bases. Credentials (like API keys) that belong in secrets should never travel inside code repositories like GitHub (unless they are encrypted before going in, but even then it is better to keep them separate).

In this lab, you learn how to perform the following tasks:

  • Create secrets by using the command and manifest files
  • Create ConfigMaps by using the command and manifest files
  • Consume secrets in containers by using environment variables or mounted volumes
  • Consume ConfigMaps in containers by using environment variables or mounted volumes

Task 1. Work with Secrets

In this task, you authenticate containers with Google Cloud in order to access Google Cloud services. You set up a Cloud Pub/Sub topic and subscription, try to access the Cloud Pub/Sub topic from a container running in GKE, and see that the access request fails. To properly access the pub/sub topic, you create a service account with credentials, and pass those credentials through Kubernetes Secrets.

Prepare a service account with no permissions

  1. In the Google Cloud Console, on the Navigation menu > click IAM & admin > Service accounts.
  2. Click + Create Service Account.
  3. In the Service Account Name text box, enter and then click Create.
  4. Click Continue and then click Done.

5. Find the no-permissions service account in the list, and copy the email address associated with it to a text file for later use.

no-permissions@qwiklabs-gcp-00–8092901054e2.iam.gserviceaccount.com

Create a GKE cluster

When you create this cluster you will specify the service account you created earlier. To illustrate the need for service accounts with proper permissions, that service account has no permissions to any other Google Cloud services, and therefore will be unable to connect to the Cloud Pub/Sub test application you will later deploy. You will fix this later in the lab.

  1. In Cloud Shell, type the following command to create environment variables for the Google Cloud zone and cluster name that will be used to create the cluster for this lab.
export my_zone=us-central1-a
export my_cluster=standard-cluster-1

2. In Cloud Shell, type the following command to set the environment variable for the service account name, where is the email address of the service account that you created earlier.

export my_service_account=[MY-SERVICE-ACCOUNT-EMAIL]

3. In Cloud Shell, type the following command to create a Kubernetes cluster.

gcloud container clusters create $my_cluster \
--num-nodes 2 --zone $my_zone \
--service-account=$my_service_account

Note: You need to wait a few minutes for the cluster deployment to complete.

4. Configure access to your cluster for kubectl:

gcloud container clusters get-credentials $my_cluster --zone $my_zone

Set up Cloud Pub/Sub and deploy an application to read from the topic

  1. In Cloud Shell, type the following command to set the environment variables for the pub/sub components.
export my_pubsub_topic=echo
export my_pubsub_subscription=echo-read

2. To create a Cloud Pub/Sub topic named echo and a subscription named echo-read that is associated with that topic, execute the following commands:

gcloud pubsub topics create $my_pubsub_topic
gcloud pubsub subscriptions create $my_pubsub_subscription \
--topic=$my_pubsub_topic

Deploy an application to read from Cloud Pub/Sub topics

You create a deployment with a container that can read from Cloud Pub/Sub topics. Since specific permissions are required to subscribe to, and read from, Cloud Pub/Sub topics this container needs to be provided with credentials in order to successfully connect to Cloud Pub/Sub.

The Deployment manifest file called is provided for you.

apiVersion: apps/v1
kind: Deployment
metadata:
name: pubsub
spec:
selector:
matchLabels:
app: pubsub
template:
metadata:
labels:
app: pubsub
spec:
containers:
- name: subscriber
image: gcr.io/google-samples/pubsub-sample:v1
  1. In Cloud Shell enter the following command to clone the repository to the lab Cloud Shell.
git clone https://github.com/GoogleCloudPlatform/training-data-analyst

2. Create a soft link as a shortcut to the working directory.

ln -s ~/training-data-analyst/courses/ak8s/v1.1 ~/ak8s

3. Change to the directory that contains the sample files for this lab.

cd ~/ak8s/Secrets/

4. Deploy the application.

kubectl apply -f pubsub.yaml

5. After the application is deployed, query the Pods by executing the following command:

kubectl get pods -l app=pubsub

This command filters the list of Pods, only including those that have a matching label of .

Note: You need to wait for about a minute. The application is starting, but it will not succeed.

6. List the Pods again:

kubectl get pods -l app=pubsub

Notice the status of the Pod. It has an error and has restarted several times.

7. To inspect the logs from the Pod, execute the following command:

kubectl logs -l app=pubsub

The error message displayed at the end of the log indicates that the application doesn’t have permissions to query the Cloud Pub/Sub service.

StatusCode.PERMISSION_DENIED, User not authorized to perform this action.

Create service account credentials

You will now create a new service account and grant it access to the pub/sub subscription that the test application is attempting to use. Instead of changing the service account of the GKE cluster nodes, you will generate a JSON key for the service account, and then securely pass the JSON key to the Pod via Kubernetes Secrets.

  1. In the Google Cloud Console, on the Navigation menu, click IAM & admin > Service Accounts.
  2. Click + Create Service Account.
  3. In the Service Account Name text box, enter and then click Create.
  4. In the Role drop-down list, select Pub/Sub > Pub/Sub Subscriber.
  5. Confirm the role is listed, and then click Continue. and then click Done.

6. In the Service Account overview screen, click the three dots on the right hand side of the service account, then select + Create Key.

7. Select JSON as the key type, and then click Create.

8. A JSON key file containing the credentials of the service account will download to your computer. You can see the file in the download bar at the bottom of your screen. You will use this key file to configure the sample application to authenticate to Cloud Pub/Sub API.

9. Click Close and then click Done.

10. On your hard drive, locate the JSON key that you just downloaded and rename the file to .

Import credentials as a secret

  1. In Cloud Shell, click the three dots icon in the Cloud Shell toolbar to display further options.

2. Click Upload file and upload the file from your local machine to the Cloud Shell VM.

3. In Cloud Shell, enter the following command to confirm that the file was uploaded.

ls ~/

You should see the credentials file that has been uploaded along with the lab files directory you cloned earlier.

4. To save the key file to a Kubernetes Secret named execute the following command.

kubectl create secret generic pubsub-key \
--from-file=key.json=$HOME/credentials.json

This command creates a secret named that has a value containing the contents of the private key that you downloaded from the Google Cloud Console.

5. Remove the file from your computer.

rm -rf ~/credentials.json

Configure the application with the secret

You now update the deployment to include the following changes:

  • Add a volume to the Pod specification. This volume contains the secret.
  • The secrets volume is mounted in the application container.
  • The environment variable is set to point to the key file in the secret volume mount.

The environment variable is automatically recognized by Cloud Client Libraries, in this case, the Cloud Pub/Sub client for Python.

The updated Deployment file called has been provided for you.

apiVersion: apps/v1
kind: Deployment
metadata:
name: pubsub
spec:
selector:
matchLabels:
app: pubsub
template:
metadata:
labels:
app: pubsub
spec:
volumes:
- name: google-cloud-key
secret:
secretName: pubsub-key
containers:
- name: subscriber
image: gcr.io/google-samples/pubsub-sample:v1
volumeMounts:
- name: google-cloud-key
mountPath: /var/secrets/google
env:
- name: GOOGLE_APPLICATION_CREDENTIALS
value: /var/secrets/google/key.json
  1. Deploy the configuration file.
kubectl apply -f pubsub-secret.yaml

2. After the configuration is deployed, execute the following command to display the Pod status.

The Pod status should be Running. The status change might take several seconds to appear.

kubectl get pods -l app=pubsub

Test receiving Cloud Pub/Sub messages

  1. Now that you configured the application, publish a message to the Cloud Pub/Sub topic you created earlier in the lab.
gcloud pubsub topics publish $my_pubsub_topic --message="Hello, world!"

Output:

Within a few seconds, the message should be picked up by the application and printed to the output stream.

2. To inspect the logs from the deployed Pod, execute the following command:

kubectl logs -l app=pubsub

The output should look like the example.

Task 2. Work with ConfigMaps

ConfigMaps bind configuration files, command-line arguments, environment variables, port numbers, and other configuration artifacts to your Pods’ containers and system components at runtime. ConfigMaps enable you to separate your configurations from your Pods and components. But ConfigMaps aren’t encrypted, making them inappropriate for credentials. This is the difference between secrets and ConfigMaps: secrets are encrypted and are therefore better suited for confidential or sensitive information such as credentials. ConfigMaps are better suited for general configuration information such as port numbers.

Use the kubectl command to create ConfigMaps

You use to create ConfigMaps by following the pattern and adding a flag for file () or literal ().

  1. Start with a simple literal in the following command:
kubectl create configmap sample --from-literal=message=hello

2. To see how Kubernetes ingested the ConfigMap, execute the following command:

kubectl describe configmaps sample

The output should look like below.

Next you will create a ConfigMap from a file. The file has been provided for you and contains some sample data:

message2=world
foo=bar
meaningOfLife=42

3. To create the ConfigMap from , execute the following command:

kubectl create configmap sample2 --from-file=sample2.properties
  1. To see how Kubernetes ingested the ConfigMap, execute the following command:
kubectl describe configmaps sample2

The output should look like below.

Use manifest files to create ConfigMaps

You can also use a YAML configuration file to create a ConfigMap. The file contains a ConfigMap definition called . You will use this ConfigMap later to demonstrate two different ways to expose the data inside a container.

apiVersion: v1
data:
airspeed: africanOrEuropean
meme: testAllTheThings
kind: ConfigMap
metadata:
name: sample3
namespace: default
selfLink: /api/v1/namespaces/default/configmaps/sample3
  1. Create the ConfigMap from the YAML file:
kubectl apply -f config-map-3.yaml

2. To see how Kubernetes ingested the ConfigMap, execute the following command:

kubectl describe configmaps sample3

The output should look like below.

Now you have some non secret, unencrypted, configuration information properly separated from your application and available to your cluster. You’ve performed this task using ConfigMaps in three different ways to demonstrate the various options, but in practice you typically pick one method, most likely the YAML configuration file approach. Configuration files provide a record of the values that you’ve stored so that you can easily repeat the process in the future.

Next, you’ll learn how to access this information from within your application.

Use environment variables to consume ConfigMaps in containers

In order to access ConfigMaps from inside Containers using environment variables the Pod definition must be updated to include one or more . The file is an updated version of the Cloud Pub/Sub demo Deployment that includes the following additional setting at the end of the file to import environmental variables from the ConfigMap into the container.

- name: INSIGHTS
valueFrom:
configMapKeyRef:
name: sample3
key: meme
  1. To reapply the updated configuration file, execute the following command:
kubectl apply -f pubsub-configmap.yaml

Now your application has access to an environment variable called , which has a value of .

2. To verify that the environment variable has the correct value, you must gain shell access to your Pod, which means that you need the name of your Pod. To get the name of the Pod, execute the following the command:

kubectl get pods

The output should look below.

3. To start the shell session, execute the following command, substituting the name of your Pod for :

kubectl exec -it [MY-POD-NAME] -- sh

4. To print a list of environment variables, execute the following command:

printenv

should appear in the list.

To exit the container’s shell session, execute the following command:

exit

Use mounted volumes to consume ConfigMaps in containers

You can populate a volume with the ConfigMap data instead of (or in addition to) storing it in an environment variable.

In this Deployment the ConfigMap named that you created earlier in this task is also added as a volume called in the Pod spec. The volume is then mounted inside the container on the path . The original method using Environment variables to import ConfigMaps is also configured.

The updated Deployment file called has been provided for you.

apiVersion: apps/v1
kind: Deployment
metadata:
name: pubsub
spec:
selector:
matchLabels:
app: pubsub
template:
metadata:
labels:
app: pubsub
spec:
volumes:
- name: google-cloud-key
secret:
secretName: pubsub-key
- name: config-3
configMap:
name: sample3
containers:
- name: subscriber
image: gcr.io/google-samples/pubsub-sample:v1
volumeMounts:
- name: google-cloud-key
mountPath: /var/secrets/google
- name: config-3
mountPath: /etc/config
env:
- name: GOOGLE_APPLICATION_CREDENTIALS
value: /var/secrets/google/key.json
- name: INSIGHTS
valueFrom:
configMapKeyRef:
name: sample3
key: meme
  1. Reapply the updated configuration file.
kubectl apply -f pubsub-configmap2.yaml

2. Reconnect to the container’s shell session to see if the value in the ConfigMap is accessible. The Pod names will have changed. To get the name of the Pod, execute the following the command:

kubectl get pods

The output should look like below.

3. To start the shell session, execute the following command, substituting the name of your Pod for :

kubectl exec -it [MY-POD-NAME] -- sh

4. Navigate to the appropriate folder.

cd /etc/config

5. List the files in the folder.

The filenames as keys from should be listed.

ls

Output:

airspeed  meme

6. To view the contents of one of the files, execute the following command:

cat airspeed

Note that the value of airspeed did not include a carriage return, therefore the command prompt (the # sign) is at the end of the returned value.

7. To exit the container’s shell, execute the following command:

exit

This concludes our lab for — Working with Google Kubernetes Engine Secrets and ConfigMaps

Happy Learning !!!

Artificial Intelligence | Machine Learning | DevOps enthusiast

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store