Kubernetes Autoscaling Needs Coordinated Decisions on Pods and Nodes

Oct 24, 2026 803 views
Digital abstract neon wave interference pattern on black, symbolizing algorithmic control loops and infrastructure scaling metrics.

A Kubernetes workload can be simultaneously described as too large, too small and short on capacity. Consider checkout-api: its Helm configuration declares three replicas, each requesting 1,000 millicores of CPU and 2 GiB of memory. Under load, HPA raises the count to 11. VPA recommends reducing CPU to 350 millicores while increasing memory to 3.1 GiB. A Kafka lag spike prompts KEDA to ask for 24 replicas. Meanwhile, Karpenter provisions nodes based on the resource requests Kubernetes sees, not the lower CPU consumption the containers may actually have.

Each component can have a reasonable explanation for its action. The problem is that none is making the whole capacity decision. Kubernetes operators have separate controllers adjusting variables that affect one another, and the resulting workload can become more expensive or less stable than any individual controller intended.

Independent controllers share one workload

Kubernetes autoscaling follows a modular design. HPA adjusts replica counts using resource or custom metrics. VPA recommends or applies container resource sizes based on observed usage. KEDA connects external events, such as queue depth, to scaling. Karpenter and Cluster Autoscaler add node capacity when pods cannot be scheduled. GitOps tools such as Argo CD and Flux reconcile cluster resources against version-controlled configuration.

That separation works well when components have distinct responsibilities. Capacity decisions, however, are coupled. Replica count affects per-pod utilization; resource requests affect HPA’s utilization calculation and pod placement; replica growth can trigger node provisioning; and GitOps reconciliation can restore a declared baseline after a controller has changed the live deployment. These controllers run at the same time, each observing only part of the system.

The Unix model of small tools composing through a sequence of inputs and outputs doesn’t map neatly to concurrent control loops. Autoscalers aren’t passing a finished result down a pipeline. They’re writing to shared workload state and responding to the effects of one another’s changes. As the source article puts it, “Determining workload capacity is a single, coupled mathematical equation.” Treating its variables as unrelated settings can produce oscillation instead of a stable configuration.

Four ways scaling decisions collide

Resource requests can distort HPA’s signal

HPA commonly evaluates CPU utilization as a percentage of declared CPU requests. In simplified terms, it compares actual usage with the total requested capacity. That makes the request value part of the scaling signal, not merely a scheduling reservation.

If VPA reduces CPU requests to match lower observed consumption, the denominator in HPA’s calculation shrinks. The same traffic can then register as higher utilization and cause HPA to add replicas. With more replicas sharing the work, CPU use per pod may fall; VPA can respond by trimming requests again. Repeated adjustments risk a cycle in which each controller’s sensible local response amplifies the other’s.

The operational issue is broader than finding the right CPU number. Teams need to decide replica count and per-pod capacity together, while accounting for how those choices change the metrics used to make the next decision.

Queue demand and CPU demand can disagree

KEDA and HPA can also compete when both influence the scale subresource for the same deployment. KEDA may increase replicas to clear a growing event queue. If the added workers spend much of their time waiting on I/O, CPU utilization can remain low. An HPA policy driven by CPU may interpret that low reading as excess capacity and scale the deployment down.

The cluster then receives conflicting instructions: one controller sees work accumulating, while another sees lightly loaded processors. Alternating scale events don’t necessarily improve throughput. The relevant signal depends on the workload, so a queue-driven worker’s capacity plan needs to consider queue depth and processing behavior alongside CPU.

GitOps can restore an outdated replica count

GitOps brings another control loop into the picture. Argo CD or Flux may treat repository configuration as the desired state. When HPA changes a deployment’s replica count, the live object can differ from the version in Git. With automatic synchronization enabled, a GitOps controller may reset replicas to the committed baseline during a demand spike.

Teams can configure GitOps to ignore replica differences, but that weakens the connection between the repository and runtime state. The source article highlights the tradeoff: production changes may no longer be reflected in Git, while later rollouts can reapply a baseline that no longer matches workload needs. Coordinating autoscaling with GitOps therefore requires an intentional way to represent or review dynamic capacity changes.

Requests can turn utilization gaps into node costs

Node autoscalers make decisions from scheduling requirements. If a container requests 1,000 millicores but uses about 150, Kubernetes still reserves the requested core for placement. Karpenter may add physical capacity to fit pods according to those declared requests, even when actual CPU use is much lower.

This can magnify poor pod sizing across a cluster. The article reports average CPU utilization of around 8% across production fleets despite substantial node provisioning. That figure points to a mismatch between requested and consumed resources; it doesn’t, by itself, establish that every cluster can safely reduce requests. The practical challenge is to balance headroom, scheduling guarantees, latency targets and node cost using signals that span both workloads and infrastructure.

A capacity plan needs a shared view

A more capable autoscaling system would evaluate the variables together: CPU and memory consumption, queue depth, transaction rates, latency objectives, node prices and the configuration stored in application repositories. It would decide on replica counts and resource requests as a related set of actions, rather than allowing separate controllers to optimize each value without accounting for the others.

Such coordination also has to include the path back to Git. The source article proposes submitting reasoned changes to application repositories, so automated capacity decisions can be reviewed and recorded without asking GitOps to disregard runtime differences. That would make the relationship between declared configuration and live behavior explicit.

Timing matters too. Reactive scaling starts after a metric crosses a threshold, while new pods and nodes take time to become useful. Historical usage can reveal recurring daily or weekly demand patterns. A system that forecasts demand could prepare capacity before an expected surge, reducing the interval in which traffic rises faster than available compute. The article describes this as a shift from reflexive response toward planned adjustment.

For operators to trust that process, every adjustment needs an understandable rationale: what demand signal changed, what capacity was added or removed, and how the decision weighs cost against latency requirements. Explainability won’t eliminate uncertainty in forecasts, but it gives SREs a basis for assessing and auditing decisions.

Automation needs coordination

HPA, VPA, KEDA and Karpenter remain useful as mechanisms for carrying out specific Kubernetes actions. The gap lies in deciding which actions should happen together and what outcome they should serve. A controller that reacts quickly to one metric is automated; autonomy, as framed by the source article, means observing the wider operating environment and taking responsibility for the combined result.

That distinction gives platform teams a better starting point than choosing HPA or VPA in isolation. The relevant question is who sets the workload’s capacity, whether that decision-maker can see demand, performance, resource sizing, node cost and GitOps state together, and how it explains its choices. Until those signals inform one coordinated plan, adding another controller can add another competing opinion. The next step for Kubernetes autoscaling is to make the capacity decision shared, explicit and tied to workload outcomes.

The post Why Kubernetes autoscalers keep fighting each other (and why the solution must be autonomous) appeared first on The New Stack.

Source: Nikhil Kurup · thenewstack.io

Comments

Sign in to comment.
No comments yet. Be the first to comment.

Related Articles

Why Kubernetes autoscalers keep fighting each other (and ...