September 28, 2021
How to deploy a simple Kubernetes app with a K8s operator
Written by: Matt Conway
One of the interesting aspects of Kubernetes is that everything is declarative. This means that everything is driven by configuration and thus driving config and deploying a Kubernetes app with an automated K8s operator is a natural fit.
The CloudTruth integration for Kubernetes (a.k.a kubetruth) is a Kubernetes operator that can be installed via helm. By default, it is set up to push the parameters from CloudTruth projects into ConfigMap and Secret resources that are named after the project. This makes it just as easy to manage the Kubernetes configuration for many services as it is for one.
Whilst managing ConfigMaps and Secrets is a great first step to getting familiar with using CloudTruth to configure your services, there is much more that can be accomplished. Kubetruth allows one to manage any Kubernetes resource by defining a template that represents that resource, and parameterizing it to use data from CloudTruth parameters and templates. Since re-deploying pods is frequently tied to configuration changes, a natural progression is to use CloudTruth to manage a Kubernetes deployment.
Setup a CloudTruth Account
To start with a simple example, we’ll use CloudTruth to manage the lifecycle of an nginx deployment by creating a CloudTruth project to hold the configuration for our application. You can follow along by signing up for a free CloudTruth account, and using the CloudTruth CLI (or Web UI) to input data. The commands below reference files available in a github repository, so your first step should be to clone the repository and switch to the directory containing the files:
git clone https://github.com/cloudtruth/blog-examples cd blog-examples/kubernetes-deploy/part1
The commands to create the cloudtruth project are also present in the setup.sh script present in that directory.
Install the CLI:
curl -sL https://github.com/cloudtruth/cloudtruth-cli/releases/latest/download/install.sh | sudo sh
Login using the CLI to create an api key, then assign it to an environment variable to save it for installing the operator:
cloudtruth login export CLOUDTRUTH_API_KEY='<your_api_key>'
Configure a Kubernetes project
Now lets create a project in Cloudtruth called k8s-myapp-nginx:
cloudtruth projects set k8s-myapp-nginx
Then we’ll add some parameters to that project which will be used to control the basic functionality:
cloudtruth --project k8s-myapp-nginx parameter set --value myapp app_name cloudtruth --project k8s-myapp-nginx parameter set --value 80 app_port cloudtruth --project k8s-myapp-nginx parameter set --value 80 service_port cloudtruth --project k8s-myapp-nginx parameter set --value nginx image_name cloudtruth --project k8s-myapp-nginx parameter set --value 1.20 image_version
At this point, the project in the cloudtruth web ui will look like:
Finally, we’ll create templates for the deployment and service resources that we’d like CloudTruth to manage:
cloudtruth --project k8s-myapp-nginx template set --body deployment.tmpl.yaml deployment cloudtruth --project k8s-myapp-nginx template set --body service.tmpl.yaml service
Where deployment.tmpl.yaml contains:
apiVersion: apps/v1 kind: Deployment metadata: name: namespace: ctdeploytest labels: app: spec: replicas: 1 selector: matchLabels: app: template: metadata: labels: app: spec: containers: - name: image: : ports: - containerPort:
and service.tmpl.yaml contains:
apiVersion: v1 kind: Service metadata: name: namespace: ctdeploytest spec: selector: app: type: NodePort ports: - protocol: TCP port: targetPort:
By taking a look at the web UI, we can verify that these templates were created and can drill into to each of them to verify the contents match expectations:
Provision the Kubernetes Operator
On the Kubernetes side, if you don’t already have a local Kubernetes cluster to test against, then we recommend setting up minikube. You’ll need to install the operator and configure it to use the templates we defined above for our project. We’ll install kubetruth in a namespace to make cleanup easier at the end of this tutorial:
helm repo add cloudtruth https://packages.cloudtruth.com/charts/ helm install --create-namespace --namespace ctdeploytest --values ctdeploytest-values.yaml --set appSettings.apiKey=$CLOUDTRUTH_API_KEY kubetruth cloudtruth/kubetruth
where ctdeploytest-values.yaml contains the operator configuration:
# Applying deployment resources requires additional permissions rbac: additionalRoleRules: - apiGroups: ["", "extensions", "apps"] resources: ["deployments", "replicasets", "pods", "services"] verbs: ["*"] # Kubetruth is configured using a CustomResourceDefinition called # ProjectMapping, which allows one to specify how the operator # transforms the data from your CloudTruth projects into kuberenetes # resources projectMappings: # Define the root project mapping, skipping all projects except for the # example we care about root: scope: "root" environment: default skip: true # Define an override ProjectMapping to enable deployment of CloudTruth projects named k8s* k8s-deploy: scope: "override" skip: false project_selector: "^k8s-" resource_templates: deployment: '' service: ''
Verify the Kubernetes app deploy
That's it! All the configuration and connections are complete, and changes to the CloudTruth project will propagate into Kubernetes automatically. To check that everything ran correctly and that our nginx service is running, you can try the following commands:
kubectl describe -n ctdeploytest deployment kubetruth kubectl logs -n ctdeploytest deployment/kubetruth kubectl describe -n ctdeploytest deployment myapp kubectl logs -n ctdeploytest deployment/myapp # Use minikube service to open a web browser connected to the deployed service # force a 404 to see the nginx version string by adding /xxx to the end of the url minikube -n ctdeploytest service nginx
If you’d rather investigate in a more visual fashion, you can run
minikube dashboard to get a web console into your minikube cluster.
One useful aspect of having CloudTruth deploy your components is that a Kubernetes app deployment can now be triggered with a configuration change. For example, one can bump the Nginx version to the newest version, and the deployment will happen automatically. This could easily be done in CI for your own components after building and pushing a new image.
kubectl describe -n ctdeploytest deployment myapp | grep -i image # shows 1.20 cloudtruth --project k8s-myapp-nginx parameter set --value 1.21 image_version kubectl -n ctdeploytest exec deployment/kubetruth -- wakeup # trigger an update instead of waiting for polling cycle to elapse kubectl describe -n ctdeploytest deployment myapp | grep -i image # shows 1.21
To uninstall the operator and clear out any resources it created, run the following series of commands (or run cleanup.sh).
helm delete --namespace ctdeploytest kubetruth kubectl delete customresourcedefinition projectmappings.kubetruth.cloudtruth.com kubectl delete namespace ctdeploytest
While this is a fairly simple application of CloudTruth for driving Kubernetes application configuration, it does illustrate the basic methodology of transforming simple configuration into a domain specific representation. This allows the different cohorts of your teams to focus on what’s important to them, with CloudTruth connecting the dots. Your Operations teams (Kubernetes experts) can define common and compliant patterns across all your application components. Thereby simplifying the life of Developers who now only have to set a handful of simple parameters to get their components running in Kubernetes.
Moving on from a toy example, upcoming parts to this blog series will share some tips on how to use CloudTruth to manage a multi-service, multi-team application at scale, including topics like:
- Using checksums to link Deployments to ConfigMaps and Secrets
- Keeping configuration interfaces up to date with a GitOps lifecycle
- Compose-able templates
- Cross-project sharing
CloudTruth has open source in its DNA. Make sure you check out the kubetruth and CLI repositories if you are interested in seeing how we engineer the code that you’ll be running in your environment. You can find some other usage examples for CloudTruth in Kubernetes there as well.