Traffic Spikes Without Pending Pods: How to Make Kubernetes Scale Itself
A two-minute self-check and a four-step plan so your cluster adds capacity by itself when traffic jumps: right-size requests, add the Cluster Autoscaler, mix node pools and watch pending pods.
A calm cluster during a traffic spike is a design choice, not luck. The pattern is simple: pods scale on demand, and the cluster adds the nodes they need within minutes. This guide gives you a two-minute check and the four steps we use. It is based on a real engagement with a real-time data platform whose traffic jumps by millions of requests in minutes.
Two-minute self-check
Answer these honestly:
- When traffic doubles, do new pods find a node, or do they wait in Pending?
- Are pod requests based on measured usage, for example the 95th percentile, or on a guess?
- Does the cluster add nodes by itself, and how long until new pods are running?
- Do you get an alert when pods stay Pending?
- Which part of your capacity can run on spot instances if an interruption happens?
If you can answer four or more, you are in good shape. Each answer you cannot give is a spot where a spike can still surprise you.
The pattern that works
Scaling pods is only half of autoscaling. If the nodes are full, new pods wait and traffic queues. The cluster has to react to what is actually pending, not to a forecast.
The plan, in four steps
- Right-size requests. Set CPU and memory requests from observed usage, using the 95th percentile. In the engagement this removed over-provisioning that had been taking about 60% of capacity, and it gives the autoscaler an honest picture.
- Add the Cluster Autoscaler. It watches for pending pods, adds nodes when they cannot be scheduled and removes idle nodes. Set pod disruption budgets so scale-down stays graceful.
- Mix node pools. Keep reserved capacity for the steady baseline and run burst workloads on spot instances, spread across several instance types to lower interruption risk.
- Watch and rehearse. Alert on pending pods, track scale-up and scale-down timing and test what happens when a spot node is interrupted.
What it looked like in practice
After the change, pods were running within 2 to 3 minutes of a spike starting, and the cluster scaled back down about 10 minutes after the spike ended. Requests no longer timed out for lack of capacity, nobody scales nodes by hand and the configuration is plain YAML. Details: the case study.
Next step
Run the self-check today. If it shows gaps, book a call and we will look at your cluster together.
Ready to simplify your platform?
We work with a small number of clients at a time. If you’re dealing with reliability problems, complexity, or a platform that’s hard to operate, let’s talk.