WHITE PAPER

The Guide to Kubernetes

Cost Optimization
How to embrace a FinOps model in Kubernetes
 THE GUIDE TO KUBERNETES COST OPTIMIZATION // 2

TABLE OF CONTENTS

Introduction.............................................................................. 3

Why Kubernetes Cost Estimation Is Hard .................................. 4

Application Right-Sizing........................................................... 7

How to Right-Size Application CPU and Memory....................... 8

Small Workloads.................................................................. 8

Multiple Workloads.............................................................. 9

Enable FinOps in Kubernetes....................................................10

About Fairwinds...................................................................... 11
 THE GUIDE TO KUBERNETES COST OPTIMIZATION // 3

INTRODUCTION
As cloud adoption has skyrocketed, the need to manage cloud cost has
increased. According to the 2024 State of FinOps Survey, organizations are
focused on reducing waste and managing commitments, but while there are
many companies that help organizations manage cloud spend, the
management layer remains a challenge. For those organizations that adopted
containers and Kubernetes, cost allocation and optimization quickly
becomes a black hole for platform engineering leaders and finance teams.

The biggest questions around spend:

Harmony between
financial and operations • Cost allocation: How much am I spending? How should that cost be divided
between teams, applications, and business units?
teams is achievable
through efficient • Service ownership: Are my teams enabled to monitor cost for their apps
resource usage and
and make changes to lower costs?
collaboration.
• Right-sizing resources: Are my apps over-provisioned and using too much
compute or memory?

Allocation, optimization, and management of Kubernetes is understanding

and having visibility into associated costs. It’s the process of providing cost
transparency to financial teams to compare spend against allocated budget and
enable show backs and charge backs to build a true FinOps culture.

This FinOps culture is essential to departments that often appear to have

opposing requirements. Engineering and operations teams are concerned
about application availability and service reliability, while the financial teams
are focused on application and infrastructure efficiency. While these might
seem like opposite seats on a seesaw, by not aligning, there can be extreme
consequences. The reality is that harmony between financial and ops teams is
achievable. By focusing on whether teams are using resources efficiently,
you create a culture of collaboration.

The key to effectively managing an environment is the ability to measure it.

Cost management provides that measurement capability by gathering and
refining usage metrics and adding a cost perspective. With cost visibility across
clusters and applications, teams can properly align business priorities with
resources.

This guide helps both platform engineers and DevOps leaders, as well as
finance teams, understand the challenges around Kubernetes cost optimization.
It provides a technical overview of why right-sizing applications in Kubernetes
is hard and then offers suggestions on how to do it better, including digging into
both open source and commercial solutions. Finally, it discusses how
organizations can apply the benefits of the FinOps discipline to ensure costs are
aligned with application performance requirements.
 THE GUIDE TO KUBERNETES COST OPTIMIZATION // 4

WHY KUBERNETES COST ESTIMATION IS HARD

Traditional FinOps cost allocation (for example, mapping resource costs back to
teams or projects one-to-one) doesn’t easily work for containerized workloads.
It's hard to allocate billed resources (such as nodes), because in Kubernetes,
those resources are shared between applications and teams. Platform
engineering and DevOps leaders need to figure out how to divide up the cost of
a node between several different teams based on resource usage. And, even
then, understanding how to estimate cost is challenging.

There are five problems that impact cost estimation: bin packing, resource
asymmetry, cost attribution, resource ranges, and noisy neighbors.

1. Bin Packing: Kubernetes cannot split up one pod across multiple nodes. So if a
whole number of pods does not fit inside a single node, a certain amount of
capacity will be wasted. In the example below, a node has 5 GB of memory and 5
CPUs, each workload requires 2GB of memory and 2 CPUs. A single node can fit
two workloads, but wastes 1GB of memory and 1 CPU. If a third workload is
added, it requires a new node, meaning all nodes are underutilized.

CPU Mem. CPU Mem. CPU Mem.

A. Two workloads require B. 3 workloads require 2

1 node, wastes 1 GB/1CPU nodes, severely underutilized
 THE GUIDE TO KUBERNETES COST OPTIMIZATION // 5

2. Resource Asymmetry: If workloads use much more memory than CPU, or

vice versa, there can be a lot of unused capacity. For example, if a CPU-intensive
workload requires 3 CPUs, but only 1 GB, a single node can only accommodate
one workload.

CPU Mem. CPU Mem.

CPU intensive workload: Single node can only have

Requires 3 CPUs, 1GB one pod worth of workload,
with lots of wasted memory
 THE GUIDE TO KUBERNETES COST OPTIMIZATION // 6

3. Cost Attribution: Deciding what each CPU and each GB of memory costs
is pretty straightforward - we can simply attribute half the cost of a node to its
memory, and half to its CPUs. For example, if our nodes cost $16/month, with
4 CPUs and 4GB of memory, we can say that’s $2 per CPU and $2 per GB of
memory. We can then apply those unit costs to our workloads, so a workload
that needs 2 CPUs and 2GB would cost $8. This makes sense because the
workload takes up exactly half a node. But things get trickier when you have
resource asymmetry (see above). Should we charge the workload for the space
that it might waste? An optimistic strategy would say no - Kubernetes will bin-
pack workloads efficiently. In that case a workload with 3 CPUs and 1 GB of
memory would also cost $8. But a conservative strategy would charge you for 2
GB of wasted memory, leading to a total cost of $12.

CPU Mem. CPU Mem. CPU Mem.

Node: $16/month Workload: 2 CPU | 2 GB Workload: 3 CPUs | 1 GB

4 CPUs - $2/CPU Cost: $2/CPU | $2/GB Optimistic total cost: $8
4 GBs - $2/GB Total cost = $8 Conservative total cost: $12

Kubernetes can help pack workloads together in a cost-efficient way, however,

engineers need to decide how to plan for this using a conservative or optimistic
method. The conservative method prepares for the worst-case scenario and
assumes there will be waste. The optimistic method assumes optimal bin-packing.

4. Resource Ranges: A Kubernetes workload doesn’t just use a fixed amount of

CPU and memory. Resource usage can vary from second to second, and pods may
get killed or moved between nodes in response to these fluctuations.
 THE GUIDE TO KUBERNETES COST OPTIMIZATION // 7

To help with this issue, Kubernetes configuration gives engineers two fields for
managing resources:

• Requests set the minimum amount of resources reserved for the workload.
When a workload gets scheduled on a node, users can be sure that at least
this much resource is available.
• Limits set the maximum resource usage of the workload. When a pod for the
workload tries to use more than this amount, the pod will either get throttled
(CPU) or killed (memory) and a new one will start.

The actual resource usage of a given workload will end up somewhere between
these two numbers, and will fluctuate heavily from second to second. The
challenge is guessing what resource range is needed for all applications.

5. Noisy Neighbors: Kubernetes can only do its best with the information it's
given. This is why it’s so important to specify requests and limits for every single
one of your workloads. For example, if one of your workloads duly requests
1GB of memory, but another workload on the same node hasn’t set requests or
limits, the second workload could easily begin to gobble up all the resources
available on that node. This could prevent the original workload from getting all
the memory it needs, hurting its performance. It can be tricky to accurately tune
these settings.

Understanding applications’ financial impact is important, but it can be difficult

when multiple workloads are running in Kubernetes.

APPLICATION RIGHT-SIZING
To make cloud spend as efficient as possible, engineers need to right-size
instances and workloads. However, it can be a real challenge to dig into
the details of Kubernetes application resources and historical usage to
make decisions.

Missing Resource Limits and Requests

Most teams lack visibility or inconsistently configure CPU and memory. This
lack of visibility makes it challenging to optimize compute and workloads in a
dynamic environment like Kubernetes. Multiple teams and clusters, combined
with management complexity, means there’s a lot of information that needs to
be reviewed and evaluated to make the best decision possible for CPU and
memory settings.
 THE GUIDE TO KUBERNETES COST OPTIMIZATION // 8

A key feature of Kubernetes that enables scaling is the capability to set specific
resource requests and limits for each workload. By setting sensible and correct
limits and requests on how much CPU and memory each pod uses, you can
maximize resource utilization. Setting limits too low on an application will cause
problems. For example, if memory limits are too low, Kubernetes is bound to kill
an application for violating its limits. Meanwhile, if set too high, resources will be
overallocated, resulting in higher bills.

While Kubernetes best practices dictate that resource limits and requests on
workloads should consistently be set, it is not always easy to know what values
to use for each application. As a result, some teams never set requests or limits
at all, while others set them too high during initial testing and then never course
correct. The key to ensuring scaling actions work properly is tuning your
resource limits and requests on each pod so workloads run efficiently.

HOW TO RIGHT-SIZE APPLICATION CPU AND MEMORY

Small Workloads
Right-sizing application CPU and memory is really hard without a tool to help
out. Lots of times if CPU and memory is set, those settings are just guesses.
Right-sizing workloads is the first step to optimizing costs.

Goldilocks is an open source tool that helps right-size workloads. It's ideal for
users new to Kubernetes or those teams managing only a few workloads in
Kubernetes.

It provides a dashboard utility to identify a baseline for setting Kubernetes

resource requests and limits. Goldilocks gets those challenging settings “just
right” — so engineers are bin-packing the right way. Goldilocks includes a
Vertical Pod Autoscaler (VPA) sub-chart that is used to install both the VPA
controller and the resources for it, which improves its ability to handle large
clusters with hundreds of namespaces and VPA objects.
 THE GUIDE TO KUBERNETES COST OPTIMIZATION // 9

Multiple Workloads
When teams start to manage multiple workloads across multiple teams, it's easy
to forget to rightsize those applications or be unsure how to configure them
properly. This is where a platform like Fairwinds Insights can help. It uses
findings from Goldilocks and Prometheus to help teams rightsize each workload.
Insights also provides additional benefits, including:

Consistency: Insights ensures all workloads have CPU and memory

settings prior to deploying to production.

Monitoring: Insights monitors clusters over time and make

recommendations on changes to configurations that will save cloud
spend and help applications perform better.

Integration: Insights uses actual cloud spend from AWS, Google,

Azure, and makes cost recommendations based on actual usage.

Documentation: Insights includes documentation to demonstrate

to financial teams and business leaders how much each
Kubernetes workload costs, where savings have been made, and
where spend should be increased.
 THE GUIDE TO KUBERNETES COST OPTIMIZATION // 10

ENABLE FINOPS IN KUBERNETES

The FinOps Foundation says that “At its core, FinOps is a cultural practice. It’s
the way for teams to manage their cloud costs, where everyone takes ownership
of their cloud usage supported by a central best-practices group.” This holds
true for Kubernetes as well.

We talk a lot about Kubernetes service ownership, the idea that DevOps teams
provide developers the tools and guardrails they need to build, deploy, and own
an application from development through deployment and maintenance. When
thinking about how Kubernetes service ownership affects overall cost
management, proper configuration plays a major role. Service owners (aka
developers) need to understand how much an application ultimately costs—and
whether this amount aligns with their pricing metrics. In the past, businesses
“Fairwinds is helping us could rely on cloud cost tools to provide visibility into the underlying cloud
with our shift in how we do infrastructure. But now, Kubernetes introduces a new layer of obfuscation in
business. We can say, cloud cost management, a black hole for traditional cloud cost monitoring tools.
‘this application team,'
‘this service,' ‘this product,' By employing a FinOps / service ownership model of Kubernetes, teams can
and ‘this feature’ has a understand the “true cost” of a workload and perform proper cost allocation
specific cost and put from development through deployment and maintenance of applications, across
alerting around it. It helps products and teams. This level of clarity into cloud resources, typically found
us to know what we’re
through a Kubernetes governance platform, allows teams to make better
doing as a business, where
decisions around the finances of Kubernetes. Without it, organizations have
we need to focus and
where we need to change trouble optimizing compute and workloads in a dynamic environment like
our investments.” Kubernetes. Multiple teams, multiple clusters, and a lot of complexity translate
into copious amounts of information to review and evaluate when trying to
Rishi Malik, VP of Platform make data-driven business decisions.
Engineering at Clover
Network A FinOps approach with Kubernetes can be viewed as just another model for
proper cost management and service ownership. Whatever you may call it, if
using Kubernetes, you need to empower development teams to own and run
their applications in production, removing Ops as a bottleneck to delivery, while
also providing finance management the detail they need to make better
decisions.

CONCLUSION
Don’t let Kubernetes be a black hole of cloud spend. Adopt a FinOps approach
that empowers platform engineering leaders to set Kubernetes guardrails for
developers to follow when configuring Kubernetes. Doing so will help teams
demonstrate to finance teams how the budget is being allocated and spent,
including where savings have been identified.

The open source tool, Goldilocks, and Fairwinds Insights can help Kubernetes
organizations right-size applications, monitor spend over time, and demonstrate
where more or fewer resources are needed. The result is consistent Kubernetes
deployments with cost optimization top of mind.
WHY FAIRWINDS
Fairwinds is your trusted partner for Kubernetes security, policy, and governance. With Fairwinds,
customers ship cloud-native applications faster, more cost effectively, and with less risk. We provide a
unified view between dev, sec, and ops, removing friction between those teams with software that
simplifies complexity. Fairwinds Insights is built on Kubernetes expertise and integrates our leading open
source tools to help you save time, reduce risk, and deploy with confidence.

WWW.FAIRWINDS.COM