Understanding KEDA
Srujan Reddy
What is KEDA?
KEDA is Kubernetes-based Event Driven Autoscaler. It can be used to scale any container based resource like Deployments, StatefulSets etc whenever an event occurs and then scale back to zero when it is not necessary or it can be used to create Kubernetes Jobs during events. It can be used to scale both Horizontally and Vertically.
Need for KEDA
KEDA is an important player to maximise your usage efficiency. You simply cannot keep your deployments running in the hope of traffic/event occurs. You must be able to create resources at the minute of need and must be able to terminate them as soon as their purpose is served. This is especially useful for Jobs. KEDA can ingest metrics from any sources(currently 68 scalers available). Here are some of the examples.
Vertical scaling of pods based on avg resource usage metrics from Prometheus or Datadog.
Scaling based on ALB metrics.
Scaling jobs based on Kafka or ActiveMQ queues.
Scaling based on query results from databases like MongoDb, DynamoDb and MySQL etc.
Scaling Github runners based number of jobs pending in GHA.
These are some of the examples but you can use any event to scale your resources.
KEDA Architecture
Lets first see all the CRDs that KEDA offers
➜ keda k get crds |grep keda
cloudeventsources.eventing.keda.sh 2024-07-12T02:20:03Z
clustertriggerauthentications.keda.sh 2024-07-12T02:20:03Z
scaledjobs.keda.sh 2024-07-12T02:20:03Z
scaledobjects.keda.sh 2024-07-12T02:20:03Z
triggerauthentications.keda.sh 2024-07-12T02:20:03Z
ScaledObjects
ScaledObjects defines what resource you want to scale and based on what triggers. The resource would be typically Deployment or StatefulSet.
spec:
scaleTargetRef:
apiVersion: {api-version-of-target-resource} # Optional. Default: apps/v1
kind: {kind-of-target-resource} # Optional. Default: Deployment
name: {name-of-target-resource} # Mandatory. Must be in the same namespace as the ScaledObject
Triggers is the external source based on which you want to scale the said object. ScaledJobs are similar to ScaledObjects. The only difference is you create/scale Jobs based on the external source.
TriggerAuthentication
As you define external triggers, you also need a way to authenticate to that source. For this purpose you need TriggerAuthentication. ClusterTriggerAuthentication is a similar one with cluster-scope.
CloudEventSource
CloudEventSource resource can be used in KEDA for subscribing to events that are emitted to the user’s defined CloudEvent sink.
Here's how all works together. ScaledObject first checks if TriggerAuthentication works or not and if could able to get required metrics from the external source. Also checks if the object that it needs to be scaled is defined or not. Then it will create a Horizontal Pod Autoscaler(HPA) for the object and starts passing metrics to HPA based on the data from external source.
Demo: Using MongoDB to scale Deployment
Here I am trying to scale deployment based on MongoDb query results. I have created a deployment for nginx with one replica, pretty standard
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx-dep
name: nginx-dep
spec:
replicas: 1
selector:
matchLabels:
app: nginx-dep
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx-dep
spec:
containers:
- image: nginx
name: nginx
resources: {}
status: {}
My target is to scaledown this deployment to zero when my mongodb document has "enabled" value is set as "no" and scales it whenever the value is "yes". So my scaled object definition looks something like
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: nginx-so
namespace: default
spec:
scaleTargetRef:
name: nginx-dep
cooldownPeriod: 30
minReplicaCount: 0
maxReplicaCount: 5
triggers:
- type: mongodb
metadata:
dbName: "drinks"
collection: "softdrinks"
query: '{ "enabled": "yes" }'
queryValue: "1"
authenticationRef:
name: mongodb-trigger
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: mongodb-trigger
spec:
secretTargetRef:
- parameter: connectionString
name: mongodb-secret
key: connect
You need to create mongodb secret with the key connect and value should be mongodb connection string mongodb+srv://:@******.*****.mongodb.net/<dbname>
apiVersion: v1
kind: Secret
metadata:
name: mongodb-secret
type: Opaque
data:
connect: bW9uZ29kYitzcnY6Ly86QCoqKioqKi4qKioqKi5tb25nb2RiLm5ldC8K==
Now we have everything in place. Once you create ScaledObject and triggerauthentications, SO will create a HPA and converts the query metric to HPA understandable metric.
Here I have changed value {"enabled": "no"}. Now when you describe ScaledObject you will see
➜ keda k get so nginx-so
NAME SCALETARGETKIND SCALETARGETNAME MIN MAX TRIGGERS AUTHENTICATION READY ACTIVE FALLBACK PAUSED AGE
nginx-so apps/v1.Deployment nginx-dep 0 5 mongodb mongodb-trigger True False False Unknown 28m
Name: nginx-so
Namespace: default
Labels: scaledobject.keda.sh/name=nginx-so
Annotations: <none>
API Version: keda.sh/v1alpha1
Kind: ScaledObject
Metadata:
Creation Timestamp: 2024-07-13T06:44:57Z
Finalizers:
finalizer.keda.sh
Generation: 1
Resource Version: 144637
UID: 9b385df8-88a4-4d0a-bc68-4e7ffea7f8f7
Spec:
Cooldown Period: 30
Max Replica Count: 5
Min Replica Count: 0
Scale Target Ref:
Name: nginx-dep
Triggers:
Authentication Ref:
Name: mongodb-trigger
Metadata:
Collection: softdrinks
Db Name: drinks
Query: { "enabled": "yes" }
Query Value: 1
Type: mongodb
Status:
Conditions:
Message: ScaledObject is defined correctly and is ready for scaling
Reason: ScaledObjectReady
Status: True
Type: Ready
Message: Scaling is not performed because triggers are not active
Reason: ScalerNotActive
Status: False
Type: Active
Message: No fallbacks are active on this scaled object
Reason: NoFallbackFound
Status: False
Type: Fallback
Status: Unknown
Type: Paused
External Metric Names:
s0-mongodb-softdrinks
Health:
s0-mongodb-softdrinks:
Number Of Failures: 0
Status: Happy
Hpa Name: keda-hpa-nginx-so
Last Active Time: 2024-07-13T07:01:28Z
Original Replica Count: 0
Scale Target GVKR:
Group: apps
Kind: Deployment
Resource: deployments
Version: v1
Scale Target Kind: apps/v1.Deployment
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning KEDAScalerFailed 29m keda-operator failed to parsing mongoDB metadata, because of missing required field in scaler config: no host given
Warning ScaledObjectCheckFailed 29m keda-operator failed to ensure HPA is correctly created for ScaledObject
Normal KEDAScalersStarted 29m keda-operator Scaler mongodb is built.
Normal KEDAScalersStarted 29m keda-operator Started scalers watch
Normal ScaledObjectReady 29m keda-operator ScaledObject is ready for scaling
Normal KEDAScaleTargetActivated 28m keda-operator Scaled apps/v1.Deployment default/nginx-dep from 0 to 1, triggered by mongoDBScaler
Normal KEDAScaleTargetDeactivated 12m keda-operator Deactivated apps/v1.Deployment default/nginx-dep from 1 to 0
The statementsMessage: Scaling is not performed because triggers are not active. and Normal KEDAScaleTargetDeactivated 12m keda-operator Deactivated apps/v1.Deployment default/nginx-dep from 1 to 0 shows that the trigger is not active so scaled to zero.
HPA reflects the same
➜ keda k get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
keda-hpa-nginx-so Deployment/nginx-dep <unknown>/1 (avg) 1 5 0 32m
➜ keda k describe hpa keda-hpa-nginx-so
Name: keda-hpa-nginx-so
Namespace: default
Labels: app.kubernetes.io/managed-by=keda-operator
app.kubernetes.io/name=keda-hpa-nginx-so
app.kubernetes.io/part-of=nginx-so
app.kubernetes.io/version=2.14.0
scaledobject.keda.sh/name=nginx-so
Annotations: <none>
CreationTimestamp: Sat, 13 Jul 2024 01:44:58 -0500
Reference: Deployment/nginx-dep
Metrics: ( current / target )
"s0-mongodb-softdrinks" (target average value): <unknown> / 1
Min replicas: 1
Max replicas: 5
Deployment pods: 0 current / 0 desired
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True SucceededGetScale the HPA controller was able to get the target's current scale
ScalingActive False ScalingDisabled scaling is disabled since the replica count of the target is zero
ScalingLimited False DesiredWithinRange the desired count is within the acceptable range
Now if I change enabled to yes
Name: nginx-so
Namespace: default
Labels: scaledobject.keda.sh/name=nginx-so
Annotations: <none>
API Version: keda.sh/v1alpha1
Kind: ScaledObject
Metadata:
Creation Timestamp: 2024-07-13T06:44:57Z
Finalizers:
finalizer.keda.sh
Generation: 1
Resource Version: 146171
UID: 9b385df8-88a4-4d0a-bc68-4e7ffea7f8f7
Spec:
Cooldown Period: 30
Max Replica Count: 5
Min Replica Count: 0
Scale Target Ref:
Name: nginx-dep
Triggers:
Authentication Ref:
Name: mongodb-trigger
Metadata:
Collection: softdrinks
Db Name: drinks
Query: { "enabled": "yes" }
Query Value: 1
Type: mongodb
Status:
Conditions:
Message: ScaledObject is defined correctly and is ready for scaling
Reason: ScaledObjectReady
Status: True
Type: Ready
Message: Scaling is performed because triggers are active
Reason: ScalerActive
Status: True
Type: Active
Message: No fallbacks are active on this scaled object
Reason: NoFallbackFound
Status: False
Type: Fallback
Status: Unknown
Type: Paused
External Metric Names:
s0-mongodb-softdrinks
Health:
s0-mongodb-softdrinks:
Number Of Failures: 0
Status: Happy
Hpa Name: keda-hpa-nginx-so
Last Active Time: 2024-07-13T07:19:58Z
Original Replica Count: 0
Scale Target GVKR:
Group: apps
Kind: Deployment
Resource: deployments
Version: v1
Scale Target Kind: apps/v1.Deployment
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning KEDAScalerFailed 35m keda-operator failed to parsing mongoDB metadata, because of missing required field in scaler config: no host given
Warning ScaledObjectCheckFailed 35m keda-operator failed to ensure HPA is correctly created for ScaledObject
Normal KEDAScalersStarted 35m keda-operator Scaler mongodb is built.
Normal KEDAScalersStarted 35m keda-operator Started scalers watch
Normal ScaledObjectReady 35m keda-operator ScaledObject is ready for scaling
Normal KEDAScaleTargetDeactivated 18m keda-operator Deactivated apps/v1.Deployment default/nginx-dep from 1 to 0
Normal KEDAScaleTargetActivated 34s (x2 over 33m) keda-operator Scaled apps/v1.Deployment default/nginx-dep from 0 to 1, triggered by mongoDBScaler
and HPA looks like
Name: keda-hpa-nginx-so
Namespace: default
Labels: app.kubernetes.io/managed-by=keda-operator
app.kubernetes.io/name=keda-hpa-nginx-so
app.kubernetes.io/part-of=nginx-so
app.kubernetes.io/version=2.14.0
scaledobject.keda.sh/name=nginx-so
Annotations: <none>
CreationTimestamp: Sat, 13 Jul 2024 01:44:58 -0500
Reference: Deployment/nginx-dep
Metrics: ( current / target )
"s0-mongodb-softdrinks" (target average value): 1 / 1
Min replicas: 1
Max replicas: 5
Deployment pods: 1 current / 1 desired
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True ReadyForNewScale recommended size matches current size
ScalingActive True ValidMetricFound the HPA was able to successfully calculate a replica count from external metric s0-mongodb-softdrinks(&LabelSelector{MatchLabels:map[string]string{scaledobject.keda.sh/name: nginx-so,},MatchExpressions:[]LabelSelectorRequirement{},})
ScalingLimited False DesiredWithinRange the desired count is within the acceptable range
Events: <none>
Source:
Subscribe to my newsletter
Read articles from Srujan Reddy directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
Srujan Reddy
Srujan Reddy
I am a Kubernetes Engineer passionate about leveraging Cloud Native ecosystem to run secure, efficient and effective workloads.