Are You Getting the Most From Kubernetes Autoscaling? | Play the ‘Black Friday’ Game

Table of Contents

One of the main benefits of Kubernetes is its ability to dynamically change the environment to meet traffic needs. Containers, pods and even nodes can be created and removed as needed based on real-time traffic – which is referred to as autoscaling. 

Autoscaling makes Kubernetes an extremely resilient and attractive container management environment. 

But this capability comes with many challenges and complexities, with numerous parameters that have financial AND operational implications. Take this: Let’s say you will scale your database when it reaches 50% CPU utilization. That guarantees you’re paying for a system that will never be more than half utilized. Is that a problem? Maybe.

It’s extremely common for different components to scale different rates. You might be able to bring up a new copy of your front-end application within two or three minutes, but, your backend database may take several minutes or even hours to replicate itself.  

So given the above constraints maybe it’s not such a bad idea to start replicating your database at 50% CPU utilization.

The game: Black Friday

We took this situation, and made it into a game called ‘Black Friday’. It’s no secret that Black Friday, the day after Thanksgiving in the US, is one of the busiest days for online retail merchants.

From 2019 to 2020, online Black Friday sales rose about 22% to $9 billion.

Source: Adobe via CNBC, 2020

Written by Appvia Solutions Architect Chris Nesbitt-Smith, the idea of the game is that you are in a large retailer whose Kubernetes environment needs to be tuned for scaling on Black Friday.

There are four tunable entities: Nodes, Frontend, Backend and Database. Each one can be changed individually to affect how they scale. Nodes and Database take the longest to scale, while Frontend and Backend take the least amount of time. 

Since Nodes and Database scale more slowly, the temptation is to set their threshold to a low CPU value (say 20%). But that guarantees you will be wasting 80% capacity (and therefore money). On the other hand, you don’t want to scale too slowly since you will get failed requests which will affect your bottom line (as well as your SLAs).  

So the goal is to save money by effectively utilizing the current capacity, scale when you need additional capacity, and cause as few failed requests as possible. 

For example, here the Frontend/Backend CPU scaling values were set to 90% and the Database to 30%. These are the results:

black friday savings

The result was that I didn’t save any money, but I did cost myself $4 in SLA penalties and this means I actually lost profits because of it. 

Can you do better? We challenge you to try!

About Appvia

Appvia enables businesses to solve complex cloud challenges with products and services that make Kubernetes secure, cost-effective and scalable.

Our founders have worked with Kubernetes in highly regulated, highly secure environments since 2016, contributing heavily to innovative projects such as Kops and fully utilizing Kubernetes ahead of the curve. We’ve mastered Kubernetes, and experienced its complexities, so our customers don’t have to. 

Share this article
profile-112x112-crop-1 (6)
Tennis Smith
Tennis has spent over 40 years in the business, starting from a stint in the US Air Force he’s worked in various capacities from equipment installation, software QA, app development and DevOps. During his 30 years in Silicon Valley, he worked for the likes of Apple, Cisco and Visa International. On the personal front, he’s been married for 25 years, is an enthusiastic martial artist and spends too much money on his cats.

The podcast that takes a lighthearted look at the who, what, when, where, why, how and OMGs of cloud computing

Related insights

Managing Kubernetes Secrets with HashiCorp Vault vs. Azure Key Vault Keeping secrets secure...
Namespaces are a vital feature of Kubernetes. They allow you to separate uniquely named...
DevOps teams have rapidly adopted Kubernetes as the standard way to deploy and...
Once you start working with Kubernetes, it’s natural to think about how you...
Self-service of cloud resources Kubernetes has been brilliant at delivering an ecosystem for...
Pods, deployments, and services are just some of the concepts that you need to understand in...
Last week I published a blog, “How to spot gaps in your Public Cloud...
Breaking down the core areas that you should be aware of when considering...
5 tips to help you manage more with less Not every manager of...
Public cloud has provided huge benefits in getting infrastructure and services to people...
This is the story of how three Appvia Engineers contributed so much to...
Overview The UK Home Office is a large government organisation, whose projects and...