Traffic doubles at 9:00. Within about 30 seconds the HorizontalPodAutoscaler asks for six more API replicas. Four start; two stay Pending, because no node has room for their CPU requests. The node autoscaler asks the cloud for a machine, and the last two start once it has booted and joined the cluster.
That morning ran on several control loops, each with its own input and its own clock. Tuned well, nobody notices them. With wrong requests, or two autoscalers that disagree, you get pods evicted in a loop or nodes that never scale down.
This post covers each loop with its current defaults (Kubernetes 1.37), where the loops fight, and one diagram that shows them all.
The autoscaling loops at a glance
| Loop | Changes | Watches | Default period |
|---|---|---|---|
| HorizontalPodAutoscaler (HPA) | Replica count | Resource, custom and external metrics | 15 s |
| VerticalPodAutoscaler (VPA) | CPU and memory requests | Usage history | 1 min |
| KEDA | Replica count, down to zero | Event sources such as queue length | 30 s polling |
| Cluster Autoscaler | Node group size | Unschedulable pods | 10 s scan |
| Karpenter | Individual nodes | Unschedulable pods, node cost | 1 to 10 s batches |
Pods connect the layers, and requests are the currency: the scheduler and both node autoscalers decide on requested CPU and memory, not real usage, as the Kubernetes node autoscaling docs stress.
HPA: replicas from metrics
The HPA controller runs in kube-controller-manager and recalculates every 15 seconds. CPU and memory come from metrics.k8s.io, served by metrics-server, which scrapes every kubelet every 15 seconds. That API became stable as v1 in Kubernetes 1.37, but the HPA still reads v1beta1. Other metrics arrive through adapters for your metrics pipeline: custom.metrics.k8s.io for values such as requests per second, external.metrics.k8s.io for values outside the cluster such as a cloud queue. The HPA reference documents the formula:
desiredReplicas = ceil(currentReplicas × currentMetricValue / desiredMetricValue)
With 4 replicas at 90% CPU and a 60% target, the HPA asks for ceil(4 × 90 / 60) = 6. At 64% it does nothing: the ratio of 1.07 is within the default tolerance of 10%. Utilization is relative to requests, so a container without a CPU request leaves the HPA unable to act on CPU. With several metrics, the largest result wins.
| Setting | Default |
|---|---|
| Sync period | 15 s |
| Tolerance | 10%; a per-HPA field is stable since 1.37 |
| Scale-down stabilization | 300 s, using the highest recommendation in that window |
| Scale-up limit | 100% or 4 pods per 15 s, whichever is more |
| CPU initialization period | 5 min; CPU from pods not yet ready is ignored |
Scale to zero is beta and enabled by default in 1.37: with an object or external metric you can set minReplicas: 0, as the release announcement explains. CPU and memory cannot wake a workload from zero, because they are measured on running pods.
spec:
minReplicas: 3
maxReplicas: 30
metrics:
- type: Resource
resource:
name: cpu
target: {type: Utilization, averageUtilization: 60}
behavior:
scaleUp:
tolerance: 0.05
scaleDown:
stabilizationWindowSeconds: 300
VPA: requests from usage history
VPA is an add-on from the Kubernetes autoscaler project, where 1.8 is the current release line. It has three components:
- The recommender computes a target, a lower bound and an upper bound per container every minute. Memory follows peaks over 8 days; CPU history loses half its weight every 24 hours.
- The updater checks pods every minute and evicts or resizes those outside the bounds, respecting PodDisruptionBudgets.
- The admission controller, a mutating webhook, writes the recommended requests into new pods.
| Update mode | Effect |
|---|---|
Off |
Recommendations only |
Initial |
Requests set at pod creation only |
Recreate |
Pods evicted and recreated |
InPlaceOrRecreate |
Resized in place, evicted if that fails |
InPlace |
Resized in place only; alpha in VPA 1.7 |
Auto |
Deprecated alias for Recreate |
In-place pod resize is stable since Kubernetes 1.35, and InPlaceOrRecreate is generally available since VPA 1.6. The updater still evicts when a resize is infeasible or stays deferred for over five minutes. Run Off for a week and compare recommendations with your requests before you enable updates. VPA does not yet support workloads with pod-level resources; the VPA reference covers the rest.
Cluster Autoscaler vs Karpenter
Both react to pods the scheduler marked unschedulable, and both simulate scheduling with requests, not node usage.
Cluster Autoscaler grows node groups you define in advance, such as AWS Auto Scaling groups. Every 10 seconds it tries unschedulable pods on a template node of each group, picks a group with an expander (least-waste by default) and asks the cloud to grow it; its FAQ targets under 30 seconds for that on clusters below 100 nodes. A node whose pods request less than 50% of its allocatable CPU and memory, and could move elsewhere, is removed after 10 minutes. Restrictive PodDisruptionBudgets, local storage and pods annotated safe-to-evict: "false" block that.
Karpenter has no node groups. A NodePool sets constraints such as instance families, zones and limits; a NodeClass holds cloud settings. Karpenter batches pending pods for 1 to 10 seconds, bin-packs them and launches instances directly, on AWS by offering EC2 Fleet up to 60 instance types. Its disruption docs cover removal: consolidation deletes or replaces underused nodes (WhenEmptyOrUnderutilized by default), drifted nodes are replaced, and budgets cap both at 10% of a NodePool's nodes by default. Nodes also expire after 720 hours.
| Cluster Autoscaler | Karpenter | |
|---|---|---|
| Unit of scaling | Predefined node groups | Individual nodes, chosen per batch |
| Instance types | Fixed per group | Any type the NodePool allows |
| Removing nodes | Under 50% requested for 10 minutes | Consolidation and drift within budgets, plus expiry |
| Cloud providers | Many | Fewer, AWS and Azure among them |
At the time of writing (November 2026), EKS Auto Mode builds on Karpenter, AKS node auto-provisioning runs Karpenter for you, and GKE manages its own cluster autoscaler. Run one node autoscaler per set of nodes; Karpenter's migration guide scales Cluster Autoscaler to zero replicas.
KEDA: queues and scale to zero
KEDA, a CNCF graduated project, adds event-driven scaling with more than 70 scalers: consumer lag on Kafka partitions, RabbitMQ or SQS queue length, Prometheus queries, cron. A ScaledObject points at a Deployment. The KEDA operator handles 0 to 1 and back itself, polling every 30 seconds and scaling to zero after a 300-second cooldown. From 1 to N it manages an HPA and feeds it through the external metrics API, as the ScaledObject reference describes.
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: worker
spec:
scaleTargetRef:
name: worker
minReplicaCount: 0
maxReplicaCount: 50
triggers:
- type: rabbitmq
metadata:
queueName: orders
mode: QueueLength
value: "20"
hostFromEnv: RABBITMQ_HOST
Scale to zero costs a cold start, which suits a durable queue. HTTP needs a buffering layer, because Services do not hold requests while no pod is ready. With HPA scale to zero in beta, an HPA on an external metric can now reach zero alone, but KEDA still brings the scalers, trigger authentication and ScaledJobs for batch work. The event-driven architecture diagram shows the queues around such workers.
How the loops interact, and where they fight
- HPA and VPA on one metric. VPA raises CPU requests, utilization drops, the HPA removes replicas, the remaining pods run hotter, and VPA raises requests again. Pair an HPA on CPU with a VPA limited to memory (
controlledResources: ["memory"]), or scale on custom metrics. - KEDA plus your own HPA. Two controllers would write one replica count. Hand an existing HPA over with KEDA's
transfer-hpa-ownershipannotation. - VPA on DaemonSets. It spoils the node autoscalers' prediction of what a new node will run; the Kubernetes docs advise against it.
- Wrong requests. Too low, and new nodes do not help; too high, and nodes never look underused.
Timing matters too. A CPU spike reaches a decision after up to 15 seconds for the next scrape plus 15 for the next HPA sync. Pods that do not fit wait up to 10 more seconds for a Cluster Autoscaler scan, or 1 to 10 for Karpenter's batch. Then the VM boots: AWS says Karpenter launches capacity in under a minute; the Cluster Autoscaler FAQ quotes three to four minutes on Google Compute Engine. For faster headroom, run low-priority placeholder pods that real pods preempt, as that FAQ describes.
The Kubernetes autoscaling diagram
Diagram contents
- orders queue
- KEDA
- worker
- new pod
- metrics-server
- HPA: api
- api
- VPA: api memory
- Cluster Autoscaler
- Karpenter
- Cloud provider API
- New node
- queue length
- replicas 0–N
- CPU usage
- replicas
- memory usage
- memory requests
- new replica
- Unschedulable
- resize node group
- launch instances
- VM joins
How to read it:
- Top row, teal: KEDA reads the orders queue and scales the worker Deployment between zero and N replicas.
- Middle row, blue: the HPA scales the api Deployment from CPU usage every 15 seconds.
- Bottom, purple: the VPA sets only memory requests on api, so it never competes with the HPA.
- Center: a replica that fits on no node stays Pending. The node layer reacts to that pod, not to CPU.
- Right, amber: Cluster Autoscaler resizes a node group, or Karpenter launches instances, through the cloud API, and the new VM joins. The Markdown block lists the defaults.
Drawing autoscaling diagrams clearly
- One loop, one line. The signal goes into the controller; what it changes comes out.
- Write the period next to every controller. It answers most "why didn't it scale" questions.
- Name the resource each autoscaler owns, so an HPA and a VPA on one workload visibly do not overlap.
- Draw the pending pod. It is the hand-off between the pod layer and the node layer.
- Leave out individual replicas, kubelets and extra scalers.
Common mistakes are an HPA arrow straight to nodes, both node autoscalers drawn as if they run together, and no mention of requests. For the cluster around the loops, see how to draw a Kubernetes architecture diagram.
Build it in Archboard
- Sign in with Google or GitHub and create a diagram.
- From the Kubernetes pack, add two Deployment nodes, worker and api, plus an HPA and a VPA for api. Set the VPA update mode to InPlaceOrRecreate.
- Add a Metrics node from the DevOps pack for metrics-server and a Queue from the Data pack.
- Add a Custom Resource from the Kubernetes pack with kind ScaledObject for KEDA, and a Pod with phase Pending between the Deployments.
- For the node layer, add a Deployment for Cluster Autoscaler, a Custom Resource with kind NodePool for Karpenter, a Cloud resource from the Cloud pack for the cloud API, and a Node.
- Connect them with labeled edges (CPU usage, replicas, Unschedulable and so on), give each loop a color, and add a Core Text label with its period.
- Add a Markdown block from the Core pack with the defaults.
The Kubernetes YAML to diagram tool turns existing manifests into a first draft of the workloads; a repeatable method for turning YAML into a readable diagram covers the clean-up. More on the Kubernetes use case page.
The example uses 13 diagram nodes, within the Free plan.
FAQ
What is the difference between HPA and VPA? The HPA changes how many pods run, based on metrics such as CPU utilization. The VPA changes each pod's CPU and memory requests, based on usage history.
Can I use HPA and VPA together? Yes, if they act on different metrics: HPA on CPU with VPA limited to memory, or HPA on custom or external metrics with VPA on both.
Should I use Cluster Autoscaler or Karpenter? Karpenter, if your cloud supports it and you want it to choose instance types and replace nodes with cheaper ones. Cluster Autoscaler for other providers and fixed node groups. Never both on the same nodes.
Can Kubernetes scale a deployment to zero? Yes, with KEDA, or with HPA scale to zero, beta and enabled by default since Kubernetes 1.37. Both need a metric that exists without running pods, such as queue length.
Why does my HPA show unknown CPU targets?
Usually metrics-server is missing, or a container has no CPU request, so utilization cannot be computed. Run kubectl describe hpa to see the error.