Learn how to – Automate App Deployment updates on Kubernetes using Keel.
Kubernetes usually as k8s or Kube is a perfect container orchestration system used for automation and management. It was initially developed by Google but is currently a community project. Recently, the popularity of Kubernetes and its ecosystem has grown immensely due to its ability to its workload types, design patterns, and behavior.
The basic resource in Kubernetes is called a pod. This is the smallest deployable unit which is a wrapper around containers. A pod may be made up of one or more containers each bearing its own configurations.
There are 3 different resources provided when deploying pods. These are:
- Deployments: the easiest and most used resource. They are usually used for stateless applications. However, the application can be made stateful by attaching a persistent volume to it.
- StatefulSets: This resource is used to manage the deployment and scaling of a set of Pods. It provides the guarantee about ordering and uniqueness of these Pods.
- DaemonSets: This controller ensures all the pod runs on all the nodes of the cluster. In case a node is added/removed from the cluster, DaemonSet automatically adds or removes the pod.
Updating deployments can be done manually by:
- Update image tag in
deployment.yaml - Apply the manifest
kubectl apply -f deployment.yaml
This method is time-consuming and does not give ample time for the user to focus on other important tasks like writing code, tests e.t.c. Today, there are many tools that can be used to build container images, these include Kaniko, Img image builder e.t.c. But there has been a missing gap on how to automatically apply these updates when new images are available.
Keel aims to provide a simple, robust, background service that automatically updates deployments, allowing users to focus on some other tasks. This is important, especially in situations where your application is re-build and needs to be deployed each time the code changes.
The below image will help you understand how Keel works.
In this guide, we will discuss how to automate app deployment updates on Kubernetes using Keel.
Getting Started.
Since this guide demonstrates how to automatically update deployments on Kubernetes, you need to be familiar with how to build and push images to your container registry such as Docker Hub.
Therefore you will require:
- Github repository with a Dockerfile: we will use the repo URL as the path of the Dockerfile
- Docker hub account: to be able to authenticate and push the Docker image.
- Access to Kubernetes cluster: to be able to deploy the Kaniko pod and create the docker registry secret.
- Webhook Relay – will relay public webhooks to our internal Kubernetes environment so we don’t have to expose Keel to the public internet
You also need a Kubernetes cluster set up. This page provides several options on how to set a Kubernetes cluster:
- Install Kubernetes Cluster on Rocky Linux 8 with Kubeadm & CRI-O
- Install Kubernetes Cluster on Ubuntu using K3s
- Deploy Kubernetes Cluster on Linux With k0s
- Run Kubernetes on Debian with Minikube
You need kubectl as well.
curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/binEnable access to the cluster:
# For k0s
export KUBECONFIG=/var/lib/k0s/pki/admin.conf
# For Vanilla Kubernetes
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/configConfirm kubectl works by listing available namespaces:
$ kubectl get ns
NAME STATUS AGE
default Active 18d
demo Active 2m3s
kube-flannel Active 18d
kube-node-lease Active 18d
kube-public Active 18d
kube-system Active 18d
metallb-system Active 18d
web Active 18d
Step 1 – Set up GitHub repository
For this guide, we will have a simple Go application that will be built and updated automatically. The private git repository will be created with the 2 files below:
- Dockerfile
FROM golang:1.17-alpine as build-env
# Set environment variable
ENV APP_NAME sample-dockerize-app
ENV CMD_PATH main.go
# Copy application data into image
COPY . $GOPATH/src/$APP_NAME
WORKDIR $GOPATH/src/$APP_NAME
# Budild application
RUN CGO_ENABLED=0 go build -v -o /$APP_NAME $GOPATH/src/$APP_NAME/$CMD_PATH
# Run Stage
FROM alpine:3.14
# Set environment variable
ENV APP_NAME sample-dockerize-app
# Copy only required data into this image
COPY --from=build-env /$APP_NAME .
# Expose application port
EXPOSE 8081
# Start app
CMD ./$APP_NAME
- main.go
package main
import (
"fmt"
"log"
"net/http"
)
var version = "version1"
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Welcome to my website! Version %s", version)
})
fmt.Printf("App is starting, version: %s n", version)
log.Fatal(http.ListenAndServe(":8081", nil))
}This application will be used to print the version to help us identify the automatic image update. The files should be created on Github.
An example showing how the files should appear:
Step 2 – Enable Webhook Relay forwarding
Triggers are entry points into the Keel. They collect information regarding updated images and send events to providers.
Creating a Webhook Relay operator allows us to create a public point and destination where webhooks are forwarded.
To keep things isolated we will run them in a separate namespace. Create the namespace with the command:
kubectl create namespace push-workflowNow set the created namespace as the current context.
kubectl config set-context $(kubectl config current-context) --namespace=push-workflowInstall Helm charts using the aid in the below guide:
Add the repository:
helm repo add webhookrelay https://charts.webhookrelay.com
helm repo updateObtain access tokens from the webhook Relay page. While on this page, navigate to your profile and click ‘Create Token‘. You will have the token generated.
Export the token as variables
export RELAY_KEY=0***9cc-1765-4***-b968-397***1c6
export RELAY_SECRET=Qsr1R****U3Install the tokens using Helm:
helm upgrade --install webhookrelay-operator --namespace=push-workflow webhookrelay/webhookrelay-operator
--set credentials.key=$RELAY_KEY --set credentials.secret=$RELAY_SECRETCheck if the application has been installed.
$ helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
webhookrelay-operator push-workflow 1 2022-06-17 11:10:18.377937954 +0000 UTC deployed webhookrelay-operator-0.3.10.5.1 Now create a manifest that will allow you receive and forward hooks.
vim webhookrelay_cr.yamlAdd the below lines to the file.
apiVersion: forward.webhookrelay.com/v1
kind: WebhookRelayForward
metadata:
name: keel-forward
spec:
buckets:
- name: dockerhub-to-keel
inputs:
- name: dockerhub-endpoint
description: "Public endpoint"
responseBody: "OK"
responseStatusCode: 200
outputs:
- name: keel
destination: http://keel:9300/v1/webhooks/dockerhub
internal: trueApply the manifest
kubectl apply -f webhookrelay_cr.yamlYou should have the 2 pods running;
# kubectl get pods
NAME READY STATUS RESTARTS AGE
keel-forward-whr-deployment-5d8db5b4b7-lwkbr 1/1 Running 0 21s
webhookrelay-operator-c694f6c8b-msv86 1/1 Running 0 112sObtain the endpoint by describing the pod, say:
$ kubectl describe webhookrelayforwards.forward.webhookrelay.com keel-forward
...
...
Status:
Agent Status: Running
Public Endpoints:
https://n5jmxkbviuucl395nwjxr7.hooks.webhookrelay.com
Ready: true
Routing Status: ConfiguredAlternatively, obtain the public endpoint from the buckets page.
Step 3 – Configure DockerHub
Now create a new repository on DockerHub. This can be public or private depending on your preference.
Now configure the repository Webhooks by creating a new one (dockerhub-to-keel) using the public endpoint.
Step 4 – Build and Push Container Image
There are several guides to help you build and push docker images to your container registry. In this guide, we will use Kaniko deployed using the aid of the guide below.
You can use the guide to deploy the DockerHub secrets and proceed as below.
Create the manifest:
vim pod.ymlThe file will contain the lines:
apiVersion: v1
kind: Pod
metadata:
name: kaniko
spec:
containers:
- name: kaniko
image: gcr.io/kaniko-project/executor:latest
args:
- "--context=git://<Git API token>@github.com/learnhowto/kubernetes-kaniko.git#refs/heads/master"
- "--destination=<dockerhub-username>/keel-demo-image:latest"
volumeMounts:
- name: kaniko-secret
mountPath: /kaniko/.docker
restartPolicy: Never
volumes:
- name: kaniko-secret
secret:
secretName: dockercred
items:
- key: .dockerconfigjson
path: config.jsonThis manifest will help us create and push the image to Dockerhub with the tag keel-demo-image:latest. Apply the manifest.
kubectl apply -f pod.ymlFollow the build process.
$ kubectl logs kaniko --follow
Enumerating objects: 28, done.
Counting objects: 100% (28/28), done.
Compressing objects: 100% (25/25), done.
Total 28 (delta 4), reused 0 (delta 0), pack-reused 0
INFO[0001] Resolved base name golang:1.17-alpine to build-env
INFO[0001] Retrieving image manifest golang:1.17-alpine
INFO[0001] Retrieving image golang:1.17-alpine from registry index.docker.io
INFO[0002] Retrieving image manifest alpine:3.14
INFO[0002] Retrieving image alpine:3.14 from registry index.docker.io
INFO[0004] Built cross stage deps: map[0:[/sample-dockerize-app]]
INFO[0004] Retrieving image manifest golang:1.17-alpine
INFO[0004] Returning cached image manifest
INFO[0004] Executing 0 build triggers
.......
INFO[0019] ENV APP_NAME sample-dockerize-app
INFO[0019] COPY --from=build-env /$APP_NAME .
INFO[0019] Taking snapshot of files...
INFO[0019] EXPOSE 8081
INFO[0019] cmd: EXPOSE
INFO[0019] Adding exposed port: 8081/tcp
INFO[0019] CMD ./$APP_NAME
INFO[0019] Pushing image to klinsmann1/keel-demo-image:latest
INFO[0023] Pushed index.docker.io/klinsmann1/keel-demo-image@sha256:51d44171d79919df45e27c61ef929bf48fb2cabcade724c5a76cda501ad4303b Now you should have the image pushed to Dockerhub and tagged as latest
Step 5 – Deploy Keel on Kubernetes.
Using Helm charts, we can easily deploy keel with the below steps:
Add the chart repo:
helm repo add keel https://charts.keel.sh
helm repo updateWhen working regularly with Kubernetes manifests, you need to install Keel without Helm provider support as shown:
helm upgrade --install keel --namespace=push-workflow keel/keel --set helmProvider.enabled="false" --set service.enabled="true" --set service.type="ClusterIP"Step 6 – Using Keel to Automate App Deployment updates.
When using Keel, policies are used to define how and when you want the application update to occur. Although providers may have different methods of getting the configuration for your applications, policies are similar across them all. The available Keel policies are;
- all: this is used to initiate updates whenever a new version bump or a new prerelease is created. For example 1.0.0 -> 1.0.1-rc1
- major: this updates major, minor, and patch versions
- minor: updates minor and patch versions but ignores major versions.
- patch: updates patch versions only and ignore minor and major versions.
- force: this forces the update even when the tag is not semver. For example latest. You can also enforce a tag say keel.sh/match-tag=true
- glob: use wildcards to match versions
Policies are specified with a special annotation. For example:
...........
annotations:
keel.sh/policy: minor # update policy (available: patch, minor, major, all, force)
keel.sh/trigger: poll # enable active repository checking (webhooks and GCR would still work)
keel.sh/approvals: "1" # required approvals to update
keel.sh/match-tag: "true" # only makes a difference when used with 'force' policy, will only update if tag matches :dev->:dev, :prod->:prod
keel.sh/pollSchedule: "@every 1m"
keel.sh/notify: chan1,chan2 # chat channels to sent notification toDeploy Your Application
Create a manifest to use the docker image pushed to DockerHub:
vim deployment.yamlAdd the below lines to the file specifying the keel policies as required.
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo
labels:
name: "demo"
annotations:
# force policy will ensure that deployment is updated
# even when tag is unchanged (latest remains)
keel.sh/policy: force
spec:
replicas: 1
revisionHistoryLimit: 5
selector:
matchLabels:
app: demo
template:
metadata:
name: demo
labels:
app: demo
spec:
containers:
- image: klinsmann1/keel-demo-image:latest
imagePullPolicy: Always # this is required to force pull image
name: demo
ports:
- containerPort: 8081
livenessProbe:
httpGet:
path: /
port: 8081
initialDelaySeconds: 10
timeoutSeconds: 5Apply the manifest.
kubectl create -f deployment.yamlVerify if everything is okay.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
demo-66db46dfd-7x6bw 1/1 Running 0 9s
kaniko 0/1 Completed 0 103s
keel-65fcb79c6d-n7vz4 1/1 Running 0 6m23s
keel-forward-whr-deployment-5d8db5b4b7-kzdzn 1/1 Running 0 11m
webhookrelay-operator-c694f6c8b-4vrpc 1/1 Running 0 13m
$ kubectl logs demo-66db46dfd-7x6bw
App is starting, version: version1 Step 7 – Update and Push Container Image
Now we will edit our main.go code on Github, and build and push a newer image with the program’s version string to version2
package main
import (
"fmt"
"log"
"net/http"
)
var version = "version2"
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Welcome to my website! Version %s", version)
})
fmt.Printf("App is starting, version: %s n", version)
log.Fatal(http.ListenAndServe(":8080", nil))
}Build and push the image by running the same kaniko manifest in step 4. The commands used here are:
kubectl delete pod kaniko
kubectl apply -f pod.ymlThe new image will be pushed to Dockerhub. You can fill the process with the command:
kubectl logs kaniko --followStep 8 – Verify Updates on your Kubernetes App
Once a new version is built and pushed, Keel automatically updates the application. To verify this, view the deployment history of the application.
kubectl rollout history deployment/demoSample Output:
Check if the pod is running.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
demo-86df5f4b76-p76th 1/1 Running 0 103s
kaniko 0/1 Completed 0 2m9s
keel-65fcb79c6d-n7vz4 1/1 Running 0 11m
keel-forward-whr-deployment-5d8db5b4b7-kzdzn 1/1 Running 0 16m
webhookrelay-operator-c694f6c8b-4vrpc 1/1 Running 0 18mNow verify if the update was automated.
kubectl logs demo-86df5f4b76-p76th
Conclusion.
It is that simple!
Now you can easily automate app deployment updates on Kubernetes using Keel. I hope this was of value.
Related posts:
Automate Penetration Testing Operations with Infection Monkey
Automate Icinga2 Configurations with Icinga Director on CentOS 8|RHEL 8
Automate Windows Server & Windows Desktop Administration with Ansible
Leave a Reply
You must be logged in to post a comment.