BABEŞ-BOLYAI UNIVERSITY CLUJ-NAPOCA

FACULTY OF MATHEMATICS AND COMPUTER

SCIENCE
SPECIALIZATION COMPUTER SCIENCE

DIPLOMA THESIS

Infrastructure design for student

collaboration projects using
Kubernetes

Supervisor
Lect. Dr. Pop Emilia

Author
Sidorac Constantin-Radu

2024
 ABSTRACT

One of the banes of a software developer’s existence is getting their code to prop-
erly work on a different machine. This is more so the case for students, who don’t
have the necessary experience to efficiently troubleshoot deployment issues, or to
properly document the process of running their application.
Students need to collaborate on projects as part of their education, but working
in a team can become a pain point for them, often for the reason stated above. Even
outside of the school context, students with different knowledge sets and prefer-
ences may come together to build something.
The following pages lay out the choices made when designing the infrastructure
for a complex collaborative project between students.
At first, the core tools will be introduced. Kubernetes will be used as the base of
the infrastructure, with applications running in Docker containers.
Following, both cost and performance optimizations will be tackled. To be able
to quantify the possible improvements, multiple results will be presented.
The design of the infrastructure will be explored in-depth, tackling topics such
as observability, security, and stability. Multiple tools or methods will be explored
in this part.
A chapter will deal with the processes and tools involved in developing an ap-
plication, and how these are used in the presented project.
Finally, there will be some case studies of applications developed by students
using this infrastructure: a volunteer management solution, and some websites.
Unless otherwise noted, this paper only contains my own work based on my
own research. I have neither given nor received unauthorized assistance on this
thesis.

Sidorac Constantin-Radu

i
Contents

1 Introduction 1

2 Cloud Computing and Kubernetes 3

2.1 Core Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1.1 Cloud Computing . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1.2 Containers & Docker . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1.3 Microservices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1.4 Infrastructure as Code . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Kubernetes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.2 Hardware Resources . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.3 Workloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.4 Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.5 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.6 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3 Kubernetes Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3.1 Helm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3.2 Kustomize . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.3 Custom Resources & the Operator pattern . . . . . . . . . . . 12

3 Cost and performance optimizations 13

3.1 Free resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1.1 Cloud providers . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1.2 Free tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.1.3 Open source community . . . . . . . . . . . . . . . . . . . . . . 15
3.1.4 The Cloud Native Landscape . . . . . . . . . . . . . . . . . . . 15
3.2 Cost optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.1 VM Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.2 Spot VMs & Commitments . . . . . . . . . . . . . . . . . . . . 17
3.2.3 Network Configuration . . . . . . . . . . . . . . . . . . . . . . 18
3.3 Resource optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

ii
 CONTENTS

3.3.1 Kubernetes Distribution . . . . . . . . . . . . . . . . . . . . . . 18

3.3.2 Resource allocation . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3.3 Docker image size . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.4 Performance optimization . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.4.1 Improving the read path - Caching . . . . . . . . . . . . . . . . 20
3.4.2 Distributing workloads . . . . . . . . . . . . . . . . . . . . . . 21

4 Infrastructure design 22
4.1 Kubernetes Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.1.1 VM configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.1.2 Network configuration . . . . . . . . . . . . . . . . . . . . . . . 24
4.1.3 Software Configuration . . . . . . . . . . . . . . . . . . . . . . 25
4.2 Managing the Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2.1 ArgoCD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2.2 Repository file structure & namespaces . . . . . . . . . . . . . 26
4.3 Observing the Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.3.1 Metrics & Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.3.2 Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.3.3 Traces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.4 Securing the Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.4.1 Encrypted traffic . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.4.2 Secret management . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.4.3 Network policies . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.5 Repairing the Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.5.1 Backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.5.2 Restoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.5.3 Dynamic DNS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.6 Downtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.7 Centralized services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.7.1 Authentication service - OAuth2 proxy . . . . . . . . . . . . . 33
4.7.2 Central database - PostgreSQL . . . . . . . . . . . . . . . . . . 34

5 Application development 35
5.1 Source Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.1.1 Git & GitHub . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.1.2 Contributing strategy . . . . . . . . . . . . . . . . . . . . . . . 35
5.2 Password Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.2.1 Need . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.2.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.3 Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

iii
 CONTENTS

5.4 Task management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5.5 CI/CD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.6 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

6 Case studies 39
6.1 Volunteer Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.1.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.1.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.2 Project Websites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.2.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

7 Conclusions & Future Work 42

Bibliography 43

iv
Chapter 1

Introduction

The main goal of this paper is establishing a set of tools and practices as the
foundation on which multiple collaborative projects between students will be build.
Most of the decisions presented throughout the thesis are made with a concrete
purpose in mind, but alternatives and their use cases will always be presented as
well.
The real-life use case that this thesis is tackling boils down to the building of the
infrastructure that is going to be used by a number of Computer Science students.
These students are part of a student’s association, which means that their work will,
for all intents and purposes, be regarded as volunteering.
The background of these students varies, they can be first years or pursuing
a Master’s degree. They may already work in the field, or have no professional
experience. The languages, frameworks, and interests of any student can also vary
widly. All this variation will be felt to a higher degree as time passes and new
volunteers try to deploy changes to the applications created by the older ones.
To make all of this variation manageable we’ll be using a microservice-based
architecture, enforcing Docker as the main tool of choice. Kubernetes is used to
orchestrate the containers, and Helm is used to bundle up the applications that will
be deployed on Kubernetes. These concepts will be tackled in Chapter 2 - Cloud
Computing and Kubernetes.
An important part of any project is budgeting, and the budget can be rather
sparse in the case of nonprofit organizations (like a student’s association is). For
this reason we want to maximize what we can get out of the available resources
by entailing multiple optimization strategies. Chapter 3 - Cost and performance
optimizations will present some of these strategies.
Afterwards, in Chapter 4 - Infrastructure design, the actual design of the Kuber-
netes cluster will be tackled. A big part of the chapter will consist of the presentation
of different tools and the purposes for which they’re used. By the end of this chapter,
a production-ready cluster will be in place.

1
 CHAPTER 1. INTRODUCTION

The cluster needs to be used and applications need to be developed, tested, and
deployed on it for any of the prior work to have any use. Chapter 5 - Application
development delves into this topics such as Source Control, documentation, and
CI/CD.
Finally, 2 projects will be studied in Chapter 6 - Case studies. These were devel-
oped by student volunteers, each of them having a different amount of experience.

2
Chapter 2

Cloud Computing and Kubernetes

2.1 Core Concepts

2.1.1 Cloud Computing

Cloud computing can be defined as the delivery of computing services over the
internet. These come in the form of servers (usually through VMs), networking,
storage, and so on.[1]
There are multiple benefits to hosting your solutions in the cloud, even if it may
come at a higher cost per unit. The initial investment is way smaller, because you
don’t need to acquire expensive hardware, set it up, and only then deploy your
application: provisioning 50 machines only takes a few clicks.
Most billing models are ‘pay as you go’. This enables flexibility, as you can
rapidly scale in case of a burst in traffic, or tone down resources if you realize you
don’t need as many. Costs can also fluctuate throughout the day based on load, com-
pared to having physical infrastructure which has the initial investment overhead
and fixed costs after that (unless, of course, you unplug some of the servers).
Hosting an application in the cloud also enables higher availability, because
there’s no longer a single point of failure (your server room). Sometimes compo-
nents of a software solution are hosted in different availability zones, meaning that
your application won’t experience downtime even if an entire data center went up
in flames one day.

Nonprofit organizations may struggle to have dedicated physical hardware for

their computing needs (besides it being overkill), which makes the cloud a handy
alternative. This paper focuses on cloud native technologies for the reasons stated
above.

3
 CHAPTER 2. CLOUD COMPUTING AND KUBERNETES

2.1.2 Containers & Docker

Containers are a type of virtualization which takes advantage of the common
parts of an operating system. Thanks to the kernel being shared, multiple instances
can run on the same underlying operating system, most commonly Linux. This
enables running applications in isolated and secure environments, but without the
overhead of an entire Virtual Machine, which employs hardware virtualization [2],
[3].
Docker[4] is a platform which takes advantage of this technology, providing fast
and consistent deployments of applications.
One of its main advantages is its portability, which enables users to store the in-
structions needed for configuring and running the application in a file called ’Dock-
erfile’. This Dockerfile, along with the source code, is then used to build an image
that can then be run on any machine that uses the same kernel. In practice, this
means that Linux-based containers can run on any Linux distribution (and even on
Windows), which covers the vast majority of web servers.
The resulting image is then saved in some kind of registry. The images are tagged
to provide useful information about the version and contents of the image.

For the purposes of this paper, Docker is the main building block, and it lends
itself well to student projects and the problems that may arise in them:

• “It worked on my machine” - even veterans run into this issue, and it’s the
norm for students, but with the help of containerization the problem becomes
trivial.

• Isolation - changes to the environment in which a program is running can

have an effect on the other programs there. For example, upgrading a depen-
dency used by multiple apps can have a catastrophic effect if the new version
contains breaking changes. Moreover, any general configuration change of the
OS could have the same catastrophic effect.

• Automation - building and deploying are done as easily as running one com-
mand. Switching to a new version is nothing more than using a different im-
age.

2.1.3 Microservices
Microservices can be thought of as a set of small services that are developed and
deployed independently, but work together, communicating through APIs,

4
 CHAPTER 2. CLOUD COMPUTING AND KUBERNETES

The main advantage of a microservice-based architecture is the ability to break

software down into multiple independent components. This means that each ser-
vice can use a different technology stack, be developed by a different team, or fail
without taking the whole application down with it.[5]
Docker is a perfect tool for this type of architecture, wrapping the different ser-
vices and transforming them in self-contained workloads.
Figure 2.1 shows the design for a mock web application using a microservice-
based architecture. The application would enable a user to subscribe to articles, and
receive emails whenever a new one is published.

Figure 2.1: Example of a microservice-based application

Students have highly-varied knowledge sets and passions, so finding a group of

people that align on these can be a challenge. Moreover, volunteering work is, by
definition, voluntary, meaning that contributors to the project can come and go as
time passes.
Microservices enable small teams to develop small parts of the whole, with fea-
tures being deployed progressively. The loss isn’t as significant when a project is
abandoned, which is helpful in the volatile world of volunteering.
For the purposes of this paper, students have to develop the backend and fron-
tend of a solution separately for the reasons mentioned above.

2.1.4 Infrastructure as Code

Infrastructure as Code[6] is a way to automate infrastructure by taking inspira-
tion from software development practices. The configuration is kept in source con-

5
 CHAPTER 2. CLOUD COMPUTING AND KUBERNETES

trol, providing a single source of truth that is used to automatically apply changes to
systems. Things are built the same way every time and the usual development prac-
tices can be applied (modularization, versioning). Looking at the code also provides
a real overview of the infrastructure that’s in place.
Some of the tools presented in the following chapters are based on this concept,
mainly Kubernetes (which uses yaml files called ’manifests’ for its resource defini-
tions), and Helm (which templates and bundles up Kubernetes manifests).
Terraform is another IaC tool used to configure resources from different cloud
providers.

All the infrastructure presented in this paper will be IaC, including every single
VM and all of the preparation that is needed for them. All of the cloud-provider
resources are deployed using Terraform, and everything Kubernetes (besides some
initial steps) is deployed using ArgoCD. Besides the advantages presented above,
IaC is used to future-proof the infrastructure: if everything was lost, or we had to
switch cloud providers, there would be an extremely fast and simple way to get
everything up and running again.

2.2 Kubernetes
This is the main tool that’s going to be used, and everything else is going to be
build on top of it. All of the applications, tools, and services are going to be running
in Kubernetes, as well as all of the configuration and routing.

2.2.1 Overview
Kubernetes[7] is an open source platform used to manage containerized work-
loads. It can be considered a container orchestrator that facilitates automation and
declarative configuration. Kubernetes is highly customizable, and focuses on offer-
ing flexibility to its users by providing building blocks which can then be imple-
mented, extended, and configured in multiple ways.
Kubernetes runs on a cluster, which is a set of machines called ‘nodes’. These
nodes host the basic deployment unit of Kubernetes, the Pod, which contains the
containerized applications.
The state of the cluster is dictated by Kubernetes Resources, which are objects
expressed in the ‘yaml’ format. These objects have unique names per resource, but
can be isolated further through the use of namespaces, which provide a scope for
names and enable grouping of resources.

6
 CHAPTER 2. CLOUD COMPUTING AND KUBERNETES

2.2.2 Hardware Resources

An application needs resources to be able to run. The ones all applications need
are CPU and memory, but some also need storage, or sometimes even GPU. All
applications also use network resources, but those are harder to quantify and not
declared in advance.
For CPU and Memory, requests and limits are set. Requests specify the amount
of resources a container needs to be able to run, and that container is guaranteed to
have that amount. A container can go over its request, and that’s where the limits
come in: they stop the container from going above the specified limit.
Storage is different and more complicated, as there are multiple types of storage
available. Files can be stored on the underlying node’s disk, on a separate disk, in a
storage bucket, and so on.

2.2.3 Workloads
A pod contains one or more containers, all of which share the same context,
as well as storage and network resources. Pods are meant to run one instance of
an application, meaning that you can easily horizontally scale your application by
adding more pods.
Pods are scheduled on nodes mostly based on the resources they request, but
there are other mechanisms that ensure certain pods run only on certain nodes.
Most of the time pods are managed by other workload resources, which are
shortly presented below:

Deployments are generally used for stateless applications that need a certain num-
ber of pods to be running. They handle pod failures, and do rolling updates in
case something about the pod definition changes.

Jobs are used for one-time batch jobs. There is also a CronJob resource that creates
Jobs based on a predefined schedule.

StatefulSets are used for stateful applications, like databases, which store their
state using persistent storage.

DaemonSets ensure that all nodes have an instance of the pod running on them.
They’re used, for example, for getting node metrics.

2.2.4 Networking
All pods receive their own unique IP address per cluster, meaning that all pods
are normally able to communicate with each other. There are a few resources used

7
 CHAPTER 2. CLOUD COMPUTING AND KUBERNETES

for networking which take advantage of this feature.

The Service resource is used to expose an application that’s running as one or more
pods. Because pods can be killed and created at any point, the backend is
dynamic, and the service takes care of knowing which pods are part of the
backend at any point.

The Ingress resource is used to route HTTP requests to services based on rules. For
the ingress to work, an ingress controller needs to be running. There are many
ingress controllers, some of which are native to a certain cloud provider, and
some which are vendor-independent.

Moreover, there are DNS records made for every Service and Pod, enabling the
use of DNS names instead of IPs.

2.2.5 Configuration
Kubernetes has a few resources that are used to configure applications: Con-
figMaps and Secrets. ConfigMaps are used for storing non-confidential application
configuration in a place separate from the application code. Secrets are used to store
confidential data, like passwords.

2.2.6 Example
This subsection presents an example of an application running in Kubernetes.

Figure 2.2: Distribution of Pods across Nodes

8
 CHAPTER 2. CLOUD COMPUTING AND KUBERNETES

The cluster contains 4 nodes, each having 2CPU, 4GB of memory, and 30GB of
storage. The same mock web application described in 2.1.3 is deployed, but an
observability tool is also added for collecting node metrics. Figure 2.2 shows how
pods could be distributed across the nodes based on the requests they set.

Namespace Resource Name Pods

apps Deployment Article article-pod-1
article-pod-2
Notification notification-pod-1
Frontend frontend-pod-1
frontend-pod-2
frontend-pod-3
frontend-pod-4
UserManagement usermanagement-pod-1
StatefulSet PostgreSQL postgresql-pod-1
monitoring DaemonSet node-exporter node-exporter-pod-1
node-exporter-pod-2
node-exporter-pod-3
node-exporter-pod-4
StatefulSet Prometheus prometheus-pod-1

Figure 2.3: Workload distribution by namespace

Namespace Service Pods

apps Article article-pod-1
article-pod-2
Notification notification-pod-1
Frontend frontend-pod-1
frontend-pod-2
frontend-pod-3
frontend-pod-4
UserManagement usermanagement-pod-1
PostgreSQL postgresql-pod-1
monitoring node-exporter node-exporter-pod-1
node-exporter-pod-2
node-exporter-pod-3
node-exporter-pod-4
Prometheus prometheus-pod-1

Figure 2.4: Service distribution by namespace

The Kubernetes Resources were separated in namespaces for convenience. This

way, the actual application is logically separated from the observability tools, even
though the underlying pods may exist on the same node. This separation is laid out
in Figure 2.3 and Figure 2.4.

9
 CHAPTER 2. CLOUD COMPUTING AND KUBERNETES

Figure 2.5: Services and the backend Pods they point to.

A Service resource is created for each microservice. The grouping of pods can be
seen in Figure 2.5.

Figure 2.6: Routing of public Services through Ingresses. Non-public services can
communicate with each other using the Cluster’s DNS names.

10
 CHAPTER 2. CLOUD COMPUTING AND KUBERNETES

Two ingresses are set up, one for the application itself, and one for Prometheus
(which will be described in more detail in section 4.3). All client requests are routed
based on the rules set on the ingress. This means that requests made to the arti-
cle.example.com domain are routed to one of three services, depending on the most
specific prefix they match against, and all of the requests that were made to the
prometheus.example.com domain go directly to the Prometheus service.
It is not desirable to have all services exposed to the internet. As such, in those
cases communication between services should be done internally to the cluster.
Pods can be referenced by IP, by their DNS name, or by the DNS name of a Ser-
vice that selects them: to make a request to the PostgreSQL database you could use
‘10.0.2.4’, ‘postgresql-pod-1.apps’ or ‘PostgreSQL.apps‘. In practice, the DNS name
of a service is used for internal communication.
Figure 2.6 shows the communication paths between services, including public-
facing ones.

2.3 Kubernetes Tools

2.3.1 Helm
Kubernetes uses declarative configuration, which means that the desired state
of the cluster is held in manifest files. Manifest files are yaml files in which the
specification of a Kubernetes resource is declared.
A Kubernetes application usually consists of multiple manifests, and the sheer
amount of files can become unwieldy if the application has any complexity. Helm[8]
is a tool built to manage these manifests, fulfilling the role of a package manager.
Helm packages are called ‘Charts’ and
an example of the file structure can be seen
in Figure 2.7, where the same web applica-
tion presented in the previous sections was
used.
The ‘templates’ folder contains yaml
files templated using the GO template lan-
guage, which should be rendered to valid
manifests after being processed by the
templating engine using the values found
in the ‘values.yaml‘ file. The resulting
manifests are then applied to the clus-
ter.
Figure 2.7: Example of the elements
of a helmchart

11
 CHAPTER 2. CLOUD COMPUTING AND KUBERNETES

This paper makes heavy use of helm, packaging the entire state of the cluster in
charts. Helm is also used for automation, being used internally by ArgoCD (which
will be tackled in a future section).

2.3.2 Kustomize
Kustomize[9] is a tool used to apply modifications to manifest files without mod-
ifying the original files. It works by patching over the original resources, making it
a great tool for when you want slightly different configuration based on the envi-
ronment.
There are cases where Kustomize is the better tool for the job than Helm, and
that’s the case for the infrastructure presented in this paper as well (ArgoCD is de-
ployed using Kustomize).

2.3.3 Custom Resources & the Operator pattern

Custom Resources are an extension of the Kubernetes API. A resource is an end-
point in the Kubernetes API that stores a collection of objects. This means that once
a Custom Resource is installed, objects of that kind can be created and manipulated.
At the same time, a custom controller is deployed, which reads the spec of an
object, tries to ensure that the state of the cluster is in sync with that specification,
and then updates the status of that object.
A combination of a Custom Resource and a controller managing that resource is
called an Operator.

The infrastructure described in this paper uses operators made by the Kuber-
netes community to easily deploy and configure some tools. These tools will be
covered in chapter 4.

12
Chapter 3

Cost and performance optimizations

3.1 Free resources

One of the biggest cost saves comes from using free available resources, and this
section focuses on where to find such resources. Students in particular have access
to plenty of free tools, and certain groups have the opportunity to apply for grants.
This paper takes advantage of a combination of those.

3.1.1 Cloud providers

Most Cloud providers offer a free number of credits that can be used for a fixed
period of time to test the services they provide. Besides those, there are certain
grants that can be applied for. For example, there are grants for non governmental
organizations or nonprofits, there are academic or research grants, and sometimes
even startups may be offered free resources for a period of time. Figure 3.1 contains
a table presenting some of these.
Offering these kinds of ‘free trials’ is useful for the Cloud Provider in the long-
term by allowing possible future customers to try out their services, which some-
times results in a ‘vendor lock-in’.

Figure 3.1: Opportunities for free resources from different Cloud providers

Cloud Provider Credits Grants Free resources

Academic Research,
GCP [10]
Teaching credits,
[11] [12] $300 for 90 days Free tier products
Google Workspace for
[13]
Nonprofits

13
 CHAPTER 3. COST AND PERFORMANCE OPTIMIZATIONS

Azure [14] Microsoft Cloud for

$200 for 30 days Free services
[15] [16] Nonprofit
aws startups, Grant
AWS [17] $300 for Connected
for Research and
[18] [19] Community AWS Free tier
Education, AWS for
[20] members
Nonprofits
Credits for FOSS,
DigitalOcean DigitalOcean for
[21] [22] $200 for 60 days Startups, Free resources
[23] [24] DigitalOcean for
Nonprofits
Always-free
IBM Cloud
$200 for 30 days products, Free
[25]
trial

The infrastructure presented in this paper started as a proof-of-concept built us-

ing GCP credits, but was later moved to a more permanent solution using cloud
resources that are paid for using the nonprofit Azure grant.
Google Workspaces for Nonprofits is also used to manage volunteer emails, ac-
counts, and offers 100TB of Google Drive storage (that will be used for backups, as
will be seen later).
Lastly, the free tier computing power offered by Google is used to host a pass-
word manager.

3.1.2 Free tools

Students have access to a plethora of free tools through their university. Usually
all that’s needed is a valid university email address, but sometimes you may need to
provide additional proof, such as a picture of your student ID. If a certain university
isn’t part of the list of partners that have access to a certain tool, a request can always
be made to add that university to the list.
In any case, getting access to these tools is a relatively straightforward process,
but most students may not be aware of them. Below is a list of a few corporations,
and some of the free tools they provide to students.

Microsoft - offers licenses for Office365, Windows, Visual Studio, and other Mi-
crosoft products.

14
 CHAPTER 3. COST AND PERFORMANCE OPTIMIZATIONS

GitHub - offers the GitHub Student Developer Pack, which contains benefits pro-
vided directly by GitHub (like GitHub Copilot and GitHub Pro), as well as
benefits provided by partners.

JetBrains - offers licenses to a lot of its IDEs.

3.1.3 Open source community

Open source software is software that makes its source code available to the pub-
lic. Depending on the license chosen, you can view the code, use it as-is, or modify
it. Open source is incredibly important in today’s ecosystem, as around 96% of the
total codebases contain open source[26], and it’s estimated that the total value of
open source software reaches $8.8 trillion[27].
As such, the value added from using open source software becomes obvious,
and subsection 3.1.4 will cover some important projects that are part of the Cloud
Native Computing Foundation.
Developing an open source project comes with its own perks, such as possibly
receiving contributions from other developers down the line, or having other people
continue a project that would otherwise be left unmaintained. To encourage open
source, GitHub offers some of its features for free to public repositories.

The infrastructure developed in this paper is open source, as are all of the projects
developed using it. Thanks to that, GitHub Actions as well as GitHub Packages are
used as part of the deployment pipeline.

3.1.4 The Cloud Native Landscape

The Cloud Native Landscape[28] is a compilation of all cloud native projects of-
ficially acknowledged by the Cloud Native Computing Foundation[29]. This project
was started by the Linux Foundation to help advance container technology, meaning
that the landscape is full of open source projects that revolve around containeriza-
tion.
The landscape can provide an end-user using Kubernetes with a comprehensive
list of useful open source tools. The user can then pick the best ones for their use-
case. The incredible size of the landscape is represented by Figure 3.2

The infrastructure presented in this paper makes heavy use of the tools found in
the Cloud Native Landscape. Helm, kube-router, k3s, Redis, PostgreSQL, ArgoCD,
Nginx, flannel, cert-manager, CoreDNS, Oauth2-proxy, Prometheus and Grafana are

15
 CHAPTER 3. COST AND PERFORMANCE OPTIMIZATIONS

Figure 3.2: The Cloud Native Landscape as of May 2024

just some of the tools used, with many more potential tools added as the needs of
the infrastructure increase.

3.2 Cost optimization

3.2.1 VM Configuration
Most cloud providers offer a high number of VM types. The total number of
VM types comes from a combination of parameters: operating system, number of
virtual CPUs, amount of RAM, processor type, temporary storage, hosting region,
and disks. For the rest of the section Azure VM pricing[30] will be used.

The most straightforward cost-to-performance relation comes from the amount

of CPU and Memory the VMs have assigned, and this holds true for things like disk
size as well. Something that may not be obvious is that multiple smaller VMs are
cheaper than fewer bigger VMs. For example, a 8GB 2vCPU VM is twice as expen-
sive as a 4GB 2vCPU VM. So, if you were to count the total amount of resources
available, multiple smaller VMs are better, and this was done in this paper.
A nice performance increase + cost decrease can be achieved thanks to the ARM
architecture: ARM processors are generally more efficient[31], as well as cheaper(half
as cheap on Azure), but there may be less software options available for them, in-
cluding operating systems. Something that needs to be kept in mind is that, when
building Docker images, the architecture on which they’ll be deployed matters. All
VMs created for this thesis use ARM processors.
The cheapest operating system is the free one, but also the one that works on the

16
 CHAPTER 3. COST AND PERFORMANCE OPTIMIZATIONS

cheapest processor, and the one that’s the most lightweight, and Debian was chosen
from the list provided by Azure, as it supports the ARM processor.

Hosting region is an important aspect for production workloads, as the farther

away a VM is hosted from the people that make use of its computing power, the
higher the latency and the worse the service. That being said, for workloads which
aren’t as vulnerable to a bit of extra delay, the hosting region doesn’t matter as much.
Resources hosted in different regions cost different amounts, and that can be taken
advantage of after a bit of looking around. A VM hosted in Central India is 36%
cheaper than a VM hosted in Central Sweden, for example.

Disks can get a bit complicated because of the options available and the limita-
tions set on them. Research is needed to be able to come to the most cost-efficient
result. For example, in Azure a Premium SSD v2 has way better IOPS and it’s way
cheaper than a normal Premium SSD, but you are unable to use a Premium SSD v2
as a boot disk, meaning that adding a second disk to the VM would be better over-
all. At the same time, at lower amounts of storage, a Standard SSD may be more
performant, as well a cheaper, than a Premium SSD.

3.2.2 Spot VMs & Commitments

Many cloud providers allow committing resources: paying in advance for a cer-
tain amount and type of resources at a cheaper price. The advantage is the lower
cost, but you can lose the flexibility of scaling down in case you’re not using all of
them. That being said, if you are sure you are going to use at least a certain amount,
or if you expect growth, commitments are a great way to save money (for example,
committing a 2vCPU 4GB RAM VM in Central India for 1 year results in a 32% cost
decrease, which goes up to 55% when committing for 3 years).
Because the organization that holds this infrastructure gets a fixed budget per
year through the Azure Grant for Nonprofits, 1 year commitments are employed.

Spot VMs[32] are VMs which make use of the extra capacity the Cloud provider
has on hand. They have a significant cost reduction( 75%), but no Service-Level
Agreement exists for them, meaning that they can be taken down at any time with
just 30 seconds of notice. They are also not guaranteed, because the cloud provider
may not have that many spare resources.
Spot VMs are great for workloads that can handle interruptions, so they would
be a great addition to a Kubernetes cluster. A possible strategy would be scheduling
pods on Spot VMs and falling back on normal VMs when the extra capacity is gone.

17
 CHAPTER 3. COST AND PERFORMANCE OPTIMIZATIONS

Because of a quota limit, only 1 Spot VM can be used by this infrastructure, but the
terraform scripts provided can be configured to use more.

3.2.3 Network Configuration

Most of the time, when adopting a cloud provider, clients would consume mul-
tiple of their solutions because they are useful and convenient. Exaggerated de-
pendence on vendor-specific solutions could lead to a hard time in the case of a
migration. Moreover, these solutions and services come at an extra cost, and, as this
chapter has shown, the main priority is minimizing cost.
Networking costs are usually unexpected and they vary with load, meaning that
they can be insanely high. In the case of clusters, traffic between zones can turn out
to be the biggest money eater. A best practice used to ensure high availability is
deploying resources in multiple zones and regions, lowering the chance of a natural
disaster bringing all of your infrastructure down. Ignoring this practice and dealing
with the consequences of potential downtime is sometimes preferred.
Load balancers are a service provided by cloud providers, enabling traffic to go
to multiple backends based on certain rules. They are handy because they allow
the backends to be dynamic, and provide customization and security features, but
they can become expensive. The alternative to a vendor-provided load balancer is
a self-managed one. As such, one could assign a public IP address to a single VM
which houses a load balancer and connect it to the other nodes through a private
subnetwork.
Extra measures can be taken to reduce ingested traffic, but their cost-efficiency
depends on the context. CDNs could be used to cache highly requested data, but
those usually cost extra. In today’s ecosystem, where websites are often used for
AI training, scrapers can become a huge problem. Therefore, solutions like rate-
limiting and IP blocking are needed more and more.

3.3 Resource optimization

3.3.1 Kubernetes Distribution

Kubernetes provides extensive customization capabilities, which goes as far as
enabling users to create alternative solutions to core features by implementing inter-
faces. Thanks to this, some more lightweight Kubernetes distributions were created.
Besides the standard K8s distribution, the most well-known ones are K3s, K0s,
and MicroK8s. All of these mostly reduce the memory footprint of Kubernetes, with
K3s having the smallest binary.[33]

18
 CHAPTER 3. COST AND PERFORMANCE OPTIMIZATIONS

K3s was chosen for this paper, as it had the smallest binary, it was easy to setup
quickly, and had just enough features for the desired use-cases. Those features could
also be turned off and replaced with the desired ones.

3.3.2 Resource allocation

In Kubernetes, pods are running on nodes, which are physical machines that can
handle a certain amount of work, which is defined by the amount of cores and the
amount of memory each machine has. Each pod uses some amount of CPU and
memory of the node it’s running on.
To ensure your pod has enough resources, they are scheduled based on the
amount of resources they request. These amounts are assigned by the user, meaning
that good utilization of the available resources depends on how the user allocates
them.
To ease this endeavor, some tools exist which make recommendations based on
historic resource usage, like the VerticalPodAutoscaler[34].

3.3.3 Docker image size

All docker images are build upon other docker images. The only exception are
a handful of “base” images which are build from scratch. As a consequence, the
resulting image of an application will depend on the base image that’s used. Because
you don’t always need all of the features an image can provide, widely-used images
usually come in different forms. You might see image tags containing the words
“slim” or “alpine”, which sometimes indicate the base image that was used.
There are a couple of popular base images, like debian, ubuntu, alpine, busybox,
or distroless. Depending on which base image you use, your resulting image size
can decrease significantly, as can be seen in Figure 3.3 and Figure 3.4.

debian:12.5 117MB
debian:12.5-slim 74.8MB
ubuntu:24.04 76.2MB
distroless:base-debian12 20.7MB
distroless:base-nossl-debian12 14.8MB
distroless:static-debian12 1.99MB
busybox:3.16.1 4.26MB

Figure 3.3: Size comparison of popular base images

Ideally, you would like to have the smallest image possible, but sometimes that
comes at the cost of time. Getting your application to work is harder, or downright

19
 CHAPTER 3. COST AND PERFORMANCE OPTIMIZATIONS

python:3.11 1.01 GB
python:3.11-slim 131 MB
python:3.11-alpine 52.7 MB
python3 distroless 52.8 MB

Figure 3.4: Size comparison of different python3.11 images

impossible, when it depends on a lot of things you’d usually find in a normal linux
distribution (some images don’t even have a shell available).
For example, for alpine-based images, you would usually add dependencies us-
ing apk (alpine’s packet manager). For distroless images, you would need to com-
pile your application to a binary in a different build step, and copy that binary, as
well as any required libraries, on top of the distroless base.
Figure 3.4 shows how big of a size decrease can be achieved by using the right
base image for a python-based application. In most use-cases, you’d probably use
the python:3.11-slim base image, but that still provides a 87% decrease in image size
compared to the default image.

3.4 Performance optimization

In this section, we’ll consider performance from the viewpoint of an end-user:
how much time is needed to get the desired result. There are multiple possible
bottlenecks in a software solution, and possible improvements for some of them
will be presented in the following pages.

3.4.1 Improving the read path - Caching

In computer science, everything reads or writes data. Oftentimes, generating the
data takes the longest amount of time, not reading or writing it. If some data is
going to be used multiple times in a short amount of time, it would be pointless to
make the computations necessary to generate that data every single time. It would
be faster to store that data somewhere, and just read it.
That is what a cache is, a memory from which faster retrieval is possible. Caches
are used everywhere, starting from the processor level[35] to all across the internet[36].
In the case of the internet, reading data sometimes becomes a problem of avail-
ability, as what you may want to access is on the other side of the world. For faster
retrieval, proxy caches are employed between the user and the server. Sometimes
ISPs may cache data for multiple users at once.
A cache may be set right in front of the application, to cut out unnecessary com-
putations from being made. The application itself may cache data it uses from other

20
 CHAPTER 3. COST AND PERFORMANCE OPTIMIZATIONS

sources (like databases).

Caching usually means a trade-off between resources: less CPU usage/ lower
latency in exchange for higher memory usage. This trade-off may not be worth it,
and caching could not even make sense in some cases[37].
Even so, generally, caching can speed things up significantly. For images, it’s
normal to see content being served from the cache 50%+ of the time when using a
proxy cache such as Varnish [38], and caching database queries with Redis could
result in a 10x speedup of the query[39].

3.4.2 Distributing workloads

Doing things in parallel can be a way to reduce the time it would take for a
request to be fulfilled, even though the amount of work done in total would be
the same or slightly elevated. This has been known for a long time, and parallel
workloads can be implemented whenever there’s more than one core available.
The speed of a single CPU can only get up to a certain point, but the number
of CPUs can grow to crazy amounts. Workloads can then be distributed between
multiple cores on the same machine, or between multiple machines.
One way in which an application can distribute work is through the use of multi-
threading, especially for applications that serve multiple clients. A thread that runs
parallel to the others (within limitations caused by the number of cores) is created
for each client, and then disposed of when the request is done. Threads are not free,
and they come with certain amounts of overhead.
A higher-scale strategy is having multiple applications of the same type running
at the same time on different machines, the traffic being distributed equally between
them through the use of load balancing. Just as with threads, there’s the overhead
of an entire application to take into account.

21
Chapter 4

Infrastructure design

This chapter will be presenting the actual infrastructure that was created using
some of the techniques, practices and tools presented in this paper. The context
and reasoning for each choice will be presented, along with possible alternatives.
Because of the sheer breadth of tools and information available, some choices were
based on popularity and good reputation.
Once again, this infrastructure is made to be used by a group of Computer Sci-
ence students that are part of a nonprofit organization. Fast, small, cheap solutions
are most of the time preferred to resilient ones.

4.1 Kubernetes Cluster

4.1.1 VM configuration
All of the infrastructure is based on having a couple of VMs organized in a cluster
structure and running Kubernetes on them.
The Azure Grant for Nonprofits provides $2000 worth of credits to spend in a
year. Going above that amount would entail paying it out of pocket, and consider-
ing how expensive the cloud can get, and how little money a student’s association
would have, minimizing costs is a must.
For this reason, the cheapest VMs were searched for. It turns out that on Azure,
a B2pls VM hosted in Central India with 2vCPU and 4GB of memory is as cheap
as they come. Memory is always a concern, which is why the smaller VM variants
weren’t considered. As mentioned, a high number of smaller VMs provides a higher
total amount of resources per dollar. This VM has an ARM processor, and Debian
installed on it.
Disk-wise, the default for the OS was used, which is a 30GB Standard SSD, but
most VMs would also have a secondary disk, a 32GB Premium v2 SSD. The OS disk
would only be used as ephemereal storage by Kubernetes, as well as to cache docker

22
 CHAPTER 4. INFRASTRUCTURE DESIGN

images, regarding its performance as not that important. The secondary disk would
be used to store persistent data, especially for databases, meaning that the best was
desired(and, funnily enough, this disk type is also the cheapest for what it provides).

Figure 4.1: The VMs of the Kubernetes cluster

In total, 7 VMs were provisioned and were paid for upfront for a 41% cost re-
duction that came from reserving the instances for 1 year. Because the amount of
money the organization has is fixed per year, it makes sense to get as much comput-
ing power as possible, even though not all of it would be consistently used. All of
those 7 VMs have their mandatory boot disk, as well as a secondary disk. The cost
for all of this, in total for a year, is $1300, leaving $700 for traffic costs, as well as
unexpected costs.
Using Spot VMs was desired and planned for, but unfortunately Azure has a
quota for them, only allowing the use of one B2pls v2 on the subscription that was
used. In any case, the terraform module supports Spot VMs, and their configuration
would have been just slightly different: because of their ephemereal nature, nothing
would be persisted on them, meaning they wouldn’t have that secondary disk. Extra
measures would be necessary to use Spot VMs, such as running a scheduled job that
regularly turns the VM back on if it was deallocated.
If more capacity is required, Azure support will probably be contacted to in-

23
 CHAPTER 4. INFRASTRUCTURE DESIGN

crease the quota for Spot VMs.

4.1.2 Network configuration

All of these VMs are part of the same virtual network and, by default, they aren’t
able to communicate with each other. Firewall rules were set up in such a way that
all of the nodes of the cluster can communicate with each other (internally to the
subnet) on all available ports.
One VM, the ‘master’ VM, has a public IP. Ports 80 and 443 are open for traffic
from all sources, to be able to receive and handle http traffic. Port 22 is also open on
all VMs for debugging purposes, but it should only be used sparringly, as needed.
K3s comes with some default networking options by default, a few of which
are used here as well. Flannel is used to take care of the networking that happens
in Kubernetes: networking between pods, and ensuring pods from different nodes
can communicate with each other. ServiceLB is used to provide load balancing to
the cluster without the need for an external cloud provider, and CoreDNS is used to
provide DNS to the cluster.
For ingresses, the nginx-ingress controller is used to be able to use many of the
features of nginx. It is also one of the most popular ingress controller choices, and
because of that it’s well documented.

Figure 4.2: Network configuration of the Cluster

24
 CHAPTER 4. INFRASTRUCTURE DESIGN

Everything passes through the master VM, as all nodes are connected directly to
it, and all traffic goes to it first. This makes the master VM the single point of failure
of this infrastructure. For all intents and purposes, this infrastructure isn’t crucial,
and the possiblity of downtime is worth the cost optimizations.
For DNS, Clouflare is used, without proxying. CNAME records are used for
everything besides the root domain, because all subdomains would point to the
same public IP.

4.1.3 Software Configuration

K3s is installed on each of the nodes, but with slightly different configuration.
Through terraform, a startup script is passed to the VM, along with a few environ-
ment variables containing useful data.
K3s has 2 types of nodes: a server node, and an agent node. The server node is
set up to be the master VM, and all of the other nodes are agent nodes. Terraform
generates a random token which is then passed to all VMs to be able to connect to
one another.
The terraform module contains 3 types of nodes, which are labelled in Kuber-
netes accordingly:

• the master node, which is where the k3s server resides and which holds the
public IP

• normal nodes, which have the same VM configuration as the master nodes,
but the k3s agent is installed on it

• spot nodes, which are just like normal nodes, but they don’t have a secondary
disk and don’t allow persistent workloads on them

The local-path provisioner that comes with k3s is used for storing persistent data.
It makes use of the existing storage of the nodes, and stores data in a folder on them.
It’s configured to use only the second disk for persistent storage, which is mounted
on /data/ssd.

4.2 Managing the Cluster

4.2.1 ArgoCD
ArgoCD is, as its name implies, a Kubernetes tool for continuous delivery. It uses
Git repositories as sources of truth, watches them, and modifies the cluster in such
a way that the cluster state matches what is declared in a respository.

25
 CHAPTER 4. INFRASTRUCTURE DESIGN

After the initial startup script that installs ArgoCD, it is used to deploy every-
thing else in the cluster, including itself. Application resources are used to define
what and where is deployed, and both helm and kustomize are supported for pack-
aging Kubernetes applications.
ArgoCD makes deploying a new version of an application as simple as changing
a version number in a file. At least every 3 minutes ArgoCD checks the repository,
and then makes the necessary changes.
A nice dashboard is provided, which also allows the users to manage resources.
ArgoCD is configured to only allow users of the organization to be able to access
the UI, and users can access certain parts of the UI based on the Google Group
they’re part of (for example, developers only get access to the production-apps and
development-apps applications).

Figure 4.3: ArgoCD UI showing all Application resources

4.2.2 Repository file structure & namespaces

The repository that is used as the source of truth contains a ‘helmcharts’ folder,
in which most of the used helmcharts are organized. The most important helmchart
is ‘apps’, which is used to deploy everything else, and it only contains Application
resource definitions. These Applications can reference any accessible repository.
Besides the ‘apps’ one, there are 2 kinds of helmcharts: infra, and hermes. In-
frastructure helmcharts contain the tools that are used by the entire cluster, while
the hermes ones contain the actual applications developed by students.
The infrastructure resources are split into namespaces in an arbitrary way for
ease of use, but they could all be deployed in the same namespace as well. One

26
 CHAPTER 4. INFRASTRUCTURE DESIGN

Figure 4.4: ArgoCD managing the cluster through repos

important feature that is used here are helm subcharts: you can define a dependency
to an external chart, and have that deployed. Many charts are already made by other
people, and this provides a cleaner way to use them.
The hermes resources are split into 2: production, and development. Production
resources are the ones that contain the state that can be seen by the outside world,
like the main website, while development resources can only be accessed by devel-
opers. The helmcharts are templated using the values.yaml file for the development
environment and the values.prd.yaml file for the production environment.
The repository could be differently organized, and may be modified with time.

4.3 Observing the Cluster

Observability reffers to the ability to observe the internal state of a system. In
this case, the system is made out of the Kubernetes cluster, the applications that run
on it, and the underlying nodes the cluster is made out of.
Observability allows developers to gain insights into the behaviour and perfor-
mance of their systems. This is done through 3 major components: metrics, logs,

27
 CHAPTER 4. INFRASTRUCTURE DESIGN

and traces.

4.3.1 Metrics & Alerts

Metrics can be defined as data points that measure certain aspects of a system’s
performance or behaviour. They usually consist of a timestamp, a number, and
some metadata about what that number means. Metrics can be used to make visu-
alizations, which are very handy when investigating something, or alerts, which let
the developers know that something wrong happened to the cluster.

Figure 4.5: Observability tools and how they’re linked

In Kubernetes, there are a few popular tools used for collecting and visualizing
metrics, as well as sending alerts. These are Prometheus, Grafana, and Alertman-
ager.
Prometheus is complex tool: it scrapes metrics from endpoints, stores them in a
time-series database, and allows querying of said metrics with the possibility of also
creating alerts.
Grafana is tool used for visualization. It can be used to visualize all the observ-
ability components: logs, metrics, and traces alike. Grafana supports multiple data

28
 CHAPTER 4. INFRASTRUCTURE DESIGN

sources, meaning that you may only need one visualization tool for the entire ob-
servability stack.
Alertmanager is used to send alerts to different receivers. It receives alerts from
Prometheus, and then forwards them. For this paper, Alertmanager was configured
to send alerts to a Discord channel, as that is where the project communication is
done.

Kube-prometheus-stack is a helmchart containing these tools, along with a large

amount of Grafana Dashboards and alerting rules. It also comes with some extra
exporters, which generate metrics based on cluster state (for example, node-exporter
generates metrics about the cluster nodes). For these reasons, kube-prometheus-
stack was used to deploy these tools.
All 3 of these tools have UIs available. To not have them publicly available, an
Oauth2 proxy is used to secure them: users can only access the web UI if they log in
with their Google account, if their account is part of the organization, and if they’re
part of the Google groups that are allowed access.

4.3.2 Logs
Logs are part of any application, and they’re crucial in figuring out what went
wrong where.
Clusters containing many applications have a massive amount of logs. Thank-
fully, this isn’t the case just yet, so for now the ArgoCD UI is enough to be able to
debug applications.
Later, when logs would start to matter more a solution consisting of fluent-bit
for picking up all the logs from the cluster, Loki for storing them, and Grafana for
visualization would be used.

4.3.3 Traces
Traces are records that capture the flow and timing of requests as they move
through the system. They are important when it comes to understand the behaviour
of microservices.
Ideally, a Kubernetes cluster would have a tracing solution. As things stand, this
wasn’t a priority, but might become one when more and more services are added.

29
 CHAPTER 4. INFRASTRUCTURE DESIGN

4.4 Securing the Cluster

4.4.1 Encrypted traffic

Because of the distributed nature of the internet, traffic can be intercepted. This
means that anyone could look at the requests you’re making. If those requests con-
tain sensitive information, such as passwords or card details, bad actors can steal
them.
A solution devised for securing traffic while in transit is TLS, which is using
asymmetric encryption (which means that a public key and a private key are used).
Certificates are documents that prove the ownership of a public key.

For traffic to and from the cluster to be considered secure, certificates need to be
generated. Doing it manually is a time-consuming process, but may be desired for
some production workloads where paid certificates are used because it’s important
to verify business legitimacy (for shops for example).
For the purposes of this infrastructure, automation and cost-efficiency are a pri-
ority, and free certificates encrypt traffic in the same way as paid certificates.

Cert-manager is a Kubernetes tool that automates the generation, issuance, and

renewal of TLS certificates. Generating a new certificate is as simple as creating a
Certificate resource, which is why cert-manager was deployed as part of the infras-
tructure.

4.4.2 Secret management

Kubernetes secrets are used to store information that shouldn’t be available to
everyone, and because of that it’s not obvious how to automatically create them, or
where to store them.
There are multiple external secret managers that let you store secrets and provide
APIs to consume those secrets, but very few that are actually free. Most of them
impose certain restrictions(such as only being able to store 6 secrets until payment
is required). As such, external secret managers aren’t a great option in this context.
One solution would be self-hosting a secret manager, but holding it in the Kuber-
netes cluster would make us lose everything if we lost the cluster. Holding it outside
the cluster is also a solution, but finding enough free computing power outside of
our current provider would be a challenge.
There’s the concept of sealed secrets, which let you encrypt secrets using a certifi-
cate that’s only available in the cluster, and storing them in source control. They’d
then be decrypted to normal secrets in the cluster. This option adds some manage-

30
 CHAPTER 4. INFRASTRUCTURE DESIGN

ment overhead that may result in undesired behaviours: creating these secrets also
requires some form of access to the cluster, so people might start creating normal
secrets and not have them in source control anymore.

A rather unconventional approach was chosen, but based on a few advantages

that come with the current use case. People that have access to the Kubernetes
cluster can already see all of the secrets because only already highly privileged users
can use the cluster. This means that we could have a private repository in which to
store secrets, and give access to users that could already see them.
Secrets are stored in a private GitHub repository. There’s an SSH key that needs
to be given as a parameter when running the terraform module for creating the
cluster to be able to connect to that repository. The SSH key will be stored in a
password manager, so bootstrapping the cluster is as easy as copy and pasting.

4.4.3 Network policies

Network policies are Kubernetes resources used to specify rules regarding how
traffic can travel through the cluster. This is especially useful in multi-tenant clus-
ters, where separation between different clients is required. For example, consider-
ing 2 different companies using the same Kubernetes cluster, it would be undesired
for the Pods of one company to be able to access the database of the other company.
K3s comes with some basic NetworkPolicy implementation based on kube-router.
For now, NetworkPolicies aren’t greatly used in this cluster, but they will be
implemented to a great degree later on.

4.5 Repairing the Cluster

4.5.1 Backups
Thanks to Google Workspaces for Nonprofits, the organization has 100TB of free
Google Drive storage. Other forms of cloud storage would cost money, and this is an
insane amount of free storage that’s just sitting there, waiting to be taken advantage
of.
As most backup solutions focus on using disks or bucket stores, some custom
backup solutions were created with the help of the rclone tool.

For the postgresql database, a CronJob runs daily, dumps the database to a file,
archives and compresses it, and then uses rclone to copy that file to Google Drive.

31
 CHAPTER 4. INFRASTRUCTURE DESIGN

A similar approach was taken for the secret manager: rclone saves the necessary
files daily to Google Drive.
There are currently no backup solutions in place for persistent volumes, but a
similar approach would probably be taken.

4.5.2 Restoring
In case of an incident in which everything becomes lost, there comes the need to
restore everything. This process should be as simple and fast as possible, to prevent
insane amounts of downtime, and to ensure that actual restoration is possible.
There are currently 4 things which would need to be restored: the secret man-
ager, the secret manager database, the postgresql database, and the Kubernetes clus-
ter.
The secret manager is deployed using terraform, and only requires setting in a
few variables that are needed for proper functioning. The backups only need to be
copied from the bucket, and dropped on the VM.
The postgresql database has a restoration job which takes the specified backup
from the drive, decompresses it, and restores the database.
The Kubernetes cluster would be restored in the same way the secret manager
would be restored, but may take longer and it should be observed just in case some-
thing goes wrong. As of now, besides the database, persistent data wouldn’t be able
to be restored.

4.5.3 Dynamic DNS

This tool improves the quality-of-life of creating the Kubernetes cluster from
scratch by taking the public IP of the cluster and assigning it to the right domain
in Cloudflare. This is also useful if, for some reason, the IP changes while the cluster
is running. This might not be the case when everything is hosted in the cloud, but
for deployments on physical servers, where a static IP might not be a possibility, this
would come in very handy.
DDclient is used in the password manager deployment, because the IP is ephemeral
in that case.

4.6 Downtime
Availability in general is going to suffer in favor of more computing power and
lower costs in the current implementation of the infrastructure.

32
 CHAPTER 4. INFRASTRUCTURE DESIGN

The reason for why the trade-off is worth it is that traffic is generally low, and
downtime has close to 0 impact, meaning that, as things currently stand, highly
available solutions are not a priority.
At some point, as the apps become more and more used, availability will become
important, and the focus will be shifted to accommodate that as well.

4.7 Centralized services

To make things simple and consistent, some services are used in a centralized
manner. This allows the developers to not worry about implementing certain fea-
tures, as they are provided through the cluster.

4.7.1 Authentication service - OAuth2 proxy

This cluster would be used to host applications and tools for a NGO. These ap-
plications may be hosted publicly, but it wouldn’t be desired for just anyone to have
access to them, meaning that some type of authentication should be in place. More-
over, access to certain features may be provided based on what kind of user tries to
access the application.
Self-made authentication solutions can be incredibly insecure, especially if they’re
developed by students. This means that, ideally, a trustworthy authentication pro-
cess should be in place, like using Oauth2 to login with a Google account. This
would be a great solution, especially because all of the user accounts of the organi-
zation are Google accounts created thanks to Google Workspaces for Nonprofits.
The overhead of implementing Oauth2 authentication for every single applica-
tion could drive students away from spending time on the actual important features
of their application.

For these purposes, the Oauth2-proxy is used to disallow unauthenticated users

from accessing some applications. A single Oauth2-proxy is deployed, and opting
into authentication is done through Ingress configuration. This configuration can
go as far as only allowing users of certain Google groups to be able to reach certain
tools.
Moreover, headers containing user information are passed to the application, al-
lowing the developers to use that information in whatever way they see fit.

Currently, the Oauth2-proxy is used for all the infrastructure dashboards (Ar-
goCD, Grafana, Alertmanager, Prometheus), as well as the student management
application.

33
 CHAPTER 4. INFRASTRUCTURE DESIGN

Figure 4.6: Basic flow of accessing apps protected by the Oauth2 proxy

4.7.2 Central database - PostgreSQL

A central database implementation is used. This is done to centralize things
such as backups and permissions, and to relieve the developers of having to set up
a database themselves. PostgreSQL was used, due to its popularity and robustness.
There is currently no way to authenticate to the database from outside of the
cluster. Having the database not be publicly available is great for keeping it safe,
but not necessarily great for doing manual operations on it.
A postgres proxy requiring authentication could be put in front of the database,
or a VPN solution could be created to allow acess to cluster resources from outside
of it.

34
Chapter 5

Application development

Another important part of this infrastructure pertains to actual development of

applications, and this chapter tackles some of those aspects.

5.1 Source Control

The source code of any software solution is stored somewhere. Some issues arise
as development is done on the application, and things break or change significantly.
Sometimes it may be desired to return to an older version of an application because
of some breaking change. Sometimes we want to see when and how a certain change
was implemented. An even bigger issue is the way in which multiple people can
make changes to that application and keeping all of those changes in sync. All of
these problems are why a way to control and version the source code is needed.

5.1.1 Git & GitHub

Git[40] is a free distributed revision control system that enables multiple devel-
opers to work on the same project and easily see all of the changes that happened
throughout time.
GitHub is a git repository hosting provider that promotes cooperation in open
source projects (sometimes by offering access to features for free), and thanks to this
it was chosen to host all of the source code presented in this paper.

5.1.2 Contributing strategy

Git allows something called a ‘branch’ to be created. Branches are parallel ver-
sion of the files in a repository. Usually, there’s a ‘main’ branch that’s considered the
absolute source of truth.

35
 CHAPTER 5. APPLICATION DEVELOPMENT

When one wants to add new functionality of an application, they should first
create a branch that’s based on the main branch. This kind of branch is called a
‘feature branch’. When development is done and the feature is considered complete,
the feature branch should be merged into main branch. This process is done through
something called a ‘Pull Request’, in which a user requests that their feature branch
is merged.
In GitHub, pull requests have a lot of options available, but the most impor-
tant one is approvals. Another person (or more, depending on the strategy that’s in
place) needs to look at the code that’s about to be merged and approve the changes.
If something is wrong or needs to be modified, the reviewer can leave some com-
ments, after which the initial developer resolves them. This process continues until
approval is given, and the feature branch is merged through a squash commit (this
means that all of the commits of the feature branch become just one commit on the
main branch).
This branching strategy is uncomplicated, and allows the main repository to only
contain commits that add entire features.

5.2 Password Manager

5.2.1 Need
A software solution can depend on sensitive information that comes in the form
of passwords, tokens, keys, and generally secrets. Teams should be able to share
this information in a safe manner, which is why a password manager should be
employed.
Passwords managers aren’t only useful for software development though. The
activity of volunteers includes quite a few of these secrets that need to be shared:
Instagram or Facebook credentials, shared email accounts, and so on.
In the past, all of these credentials would be shared through channels like pri-
vate messaging, some text files, and even public Google Docs (which were only safe
thanks to obfuscation).

5.2.2 Implementation
Most password managers that allow access to groups of people need to be paid.
There are self-hosted solutions, but they should be hosted somewhere separate from
the Kubernetes cluster. As mentioned in subsection 3.1.3, Google has some free tier
products, and one of those is a month worth computing power of a small VM, along
with a public IP, which is enough for hosting a password manager.

36
 CHAPTER 5. APPLICATION DEVELOPMENT

As such, Vaultwarden[41] is used, which is a lightweight alternative implemen-

tation of Bitwarden. An open source repository[42] is taken advantage of to config-
ure the VM and to deploy the password manager.

Terraform is used for VM creation, with variables for required configuration

(such as as a Cloudflare API key). The VM is made in such a way to conform with
the requirements of the Google free tier. This means that the public IP is epheme-
real and could go away without notice, so Dynamic DNS is used. Moreover, traffic
from certain countries isn’t included in the free tier, so there’s a solution in place for
geo-blocking. Rclone is used to backup the password manager to a Google Drive
folder.
Most of these features come from the mentioned repository, but the configura-
tion as well as the deployment automation through terraform was made for this
infrastructure.

5.3 Communication
Developer communication is important, to help one another, discuss implemen-
tation details, and make decisions. As such, the communication channels that are
used become important.
Discord was chosen for communication. It has built-in share screening capabili-
ties, voice and video calls, as well as text channels. Another very important feature
is the possiblity of extending Discord functionality through the use of bots and web-
hooks (which is how Kubernetes alerts are received in a discord channel). The actual
organization of the server is not definitive, with changes being made as the needs of
the projects are more and more understood.
Entire communities exist on Discord, with complex servers managing thousands
of users and a plethora of topics, so it should lend itself quite well to handling a few
students working on computer science projects.

5.4 Task management

GitHub Issues is used to manage the features that need to be implemented, what
task each user has, and when everything should be done. Specifically how this is
done isn’t set in stone yet, because there wasn’t much development going on. But, as
projects are created and students start working on them, GitHub Issues will become
a core tool of the development process.

37
 CHAPTER 5. APPLICATION DEVELOPMENT

5.5 CI/CD
CI/CD[43] is a set of practices and tools designed to improve the software de-
velopment lifecycle.

Continuous integration means that code changes are merged into the main branch
multiple times per day. The integration of the code can have a lot of automation
behind it: validating, building and testing, and only then allowing the merge to
happen.
Continuous deployment is concerned with the automation of the deployment
process.

For this infrastructure, CD is offered by ArgoCD, while CI is done through the

use of GitHub Actions. Currently, the projects are only set up to automatically build
Docker images after a merge to main happened, but with time more steps would be
added to the pipeline.

5.6 Documentation
Documentation written using Markdown, with at least a README.md file con-
taining basic information about what the application does, how to run it, and how
it can be configured. Developers may fail to write proper documentation even in
professional environments, but, considering the ephemeral nature of computer sci-
ence students, it is crucial to have adequate documentation for all of the developed
software solutions. As applications grow, API documentation will also be required.

38
Chapter 6

Case studies

This chapter will tackle 2 ‘projects’, their design, and how the infrastructure pre-
sented in this paper was used to make them come to life.

6.1 Volunteer Management

The context is as follows: every semester, computer science students can choose
to sign up to the student’s association, becoming volunteers. The process has 3
steps: students sign up, moment in which they become candidates, student are in-
terviewed to see if they’re a good fit for the organization, and then they’re accepted
or rejected based on how their interviews went.
Naturally, existing volunteers also need to be managed. In the last few years,
around 300 students show up as being part of the organization, so their data should
be safely stored in a database.
For both of these purposes, the volunteer management application was devel-
oped.

6.1.1 Design
The volunteer management application is made up of 2 distinct projects: the
volunteer-api[44], and the volunteer-management-frontend[45]. The idea was to
decouple the management itself from the interface, to allow for alternative imple-
mentations in the future.
The API has a few endpoints which allow CRUD operations on the students, as
well recruitment campaigns and candidates. The frontend consumes this API and
provides the actual management operations.
A volunteer has built both of these components, using GO for the API, and React
+ Typescript for the frontend, with minimal adjustments made by me to allow them
to properly run in Kubernetes.

39
 CHAPTER 6. CASE STUDIES

The components are configurable through the use of environment variables.

Figure 6.1: Initial design of the application

Figure 6.2: The application in the context of the infrastructure

40
 CHAPTER 6. CASE STUDIES

6.1.2 Results
As can be seen in figure Figure 6.2, the application makes use of the oauth2 proxy
to only allow a certain group of people to manage volunteers. The ingress is config-
ured in such a way that all requests made to endpoints that start with ‘/api/v1’ go
to the API service, while all other requests go directly to the frontend. Cert-manager
is used to generate a certificate for the desired subdomain. The application connects
to the centralised PostgreSQL database using secrets deployed from a private repos-
itory with the use of ArgoCD. To reach the database pod, the cluster DNS was used.
The Docker image of the application is built every time a new change is pushed
to the main branch thanks to GitHub actions, and the resulting image is stored in
GitHub Packages.
Grafana was used to check some metrics of the application, specifically how
much CPU and memory was used when the application is idle. During the initial
deployment phases, alerts from alertmanager showed that the application wasn’t
deployed properly, and the ArgoCD UI was used to check the logs of the containers.

6.2 Project Websites

Each semester, the organization has a number of projects. These are projects
made especially for computer science students, and sometimes they’re made in col-
laboration with IT organizations. Each one of these projects had a website made a
couple of years ago in PHP, using Laravel.
Because at some point all of the infrastructure was lost, the knowledge of how to
run those PHP applications was lost as well.

6.2.1 Design
The old Laravel applications were containerized and updated to a version that
supported ARM processors. Minimal adjustments were made to the websites to be
up to date, but they are subpar by all standards, and they need to be migrated to
some more lightweight and simpler technologies.

6.2.2 Results
For each of these websites, the design was simple. There would be 1 pod running
the containerized application, with an Ingress pointing to it, and a certificate created
for encryption.
Grafana and ArgoCD were used in the process of making the websites work.
GitHub actions and GitHub packages were made use of here as well.

41
Chapter 7

Conclusions & Future Work

This paper has shown how Kubernetes could be used to host computer science
projects developed by students. A focus was put on minimizing costs, with quite
a few strategies being presented. The usage of community-created tools was a big
part of the infrastructure, providing efficient and cost-effective solutions to many
problems.

Future work would expand more on the processes involved during the devel-
opment of applications, particularly on the improvement of the CI pipeline. Focus
would be put on the actual management of the projects, and how best to handle
ephemeral contributors, taking inspiration from the open source movement.
The tool belt offered by the cluster could also by improved, mainly by adding a
logging solution to the observability stack. Some first steps that could be taken to
make the cluster more secure would be implementing a VPN solution and having
concrete NetworkPolicies.

42
Bibliography

[1] Benneth Uzoma and Bonaventure Okhuoya. A research on cloud computing,

12 2022.

[2] Sachchidanand Singh and Nirmala Singh. Containers docker: Emerging roles
future of cloud technology. In 2016 2nd International Conference on Applied and
Theoretical Computing and Communication Technology (iCATccT), pages 804–807,
2016.

[3] Tamanna Siddiqui, Shadab Alam Siddiqui, and Najeeb Ahmad Khan. Compre-
hensive analysis of container technology. In 2019 4th International Conference on
Information Systems and Computer Networks (ISCON), pages 218–223, 2019.

[4] Docker Development Community. Docker documentation. https://docs.

docker.com/get-started/overview/. Online; accessed May 2024.

[5] Omar Al-Debagy and Peter Martinek. A comparative review of microservices

and monolithic architectures. In 2018 IEEE 18th International Symposium on Com-
putational Intelligence and Informatics (CINTI), pages 000149–000154, 2018.

[6] Kief Morris. Infrastructure as Code: Dynamic Systems for the Cloud Age. O’Reilly
Media, Inc., 2nd edition, 2020.

[7] Kubernetes Development Community. Kubernetes documentation. https:

//kubernetes.io/docs/home/. Online; accessed May 2024.

[8] Helm Development Community. Helm documentation. https://helm.sh/

docs/. Online; accessed May 2024.

[9] The Kubernetes Authors. Kustomize documentation. https://kubectl.

docs.kubernetes.io/references/kustomize/. Online; accessed May
2024.

[10] Google. Google Research Credits. https://edu.google.com/programs/

credits/research/. Online; accessed May 2024.

43
 BIBLIOGRAPHY

[11] Google. Google Free Trial and Free Tier Services. https://cloud.google.
com/free/docs/free-cloud-features#free-tier. Online; accessed
May 2024.

[12] Google. Google Teaching Credits. https://edu.google.com/programs/

credits/teaching/. Online; accessed May 2024.

[13] Google. Google Workspace for Nonprofits. https://www.google.com/

nonprofits/offerings/workspace/. Online; accessed May 2024.

[14] Microsoft. Azure for nonprofits. https://www.microsoft.com/en-us/

nonprofits/azure. Online; accessed May 2024.

[15] Microsoft. Azure free services. https://azure.microsoft.com/en-us/

pricing/free-services. Online; accessed May 2024.

[16] Microsoft. Azure Free Trial. https://azure.microsoft.com/en-us/

pricing/offers/ms-azr-0044p. Online; accessed May 2024.

[17] Amazon Web Services. AWS Free Tier. https://aws.amazon.com/free/.

Online; accessed May 2024.

[18] Amazon Web Services. AWS for Nonprofits. https://aws.amazon.com/

government-education/nonprofits/. Online; accessed May 2024.

[19] Amazon Web Services. AWS Programs for Research and Education. https:
//aws.amazon.com/grants/. Online; accessed May 2024.

[20] Amazon Web Services. aws startups. https://aws.amazon.com/

startups/credits. Online; accessed May 2024.

[21] DigitalOcean. DigitalOcean Open Source Credits. https://www.

digitalocean.com/open-source/credits-for-projects. Online;
accessed May 2024.

[22] DigitalOcean. DigitalOcean for Nonprofits. https://www.digitalocean.

com/impact/nonprofits. Online; accessed May 2024.

[23] DigitalOcean. DigitalOcean for Startups. https://www.digitalocean.

com/hatch. Online; accessed May 2024.

[24] DigitalOcean. DigitalOcean trial. https://try.digitalocean.com/

freetrialoffer/. Online; accessed May 2024.

[25] IBM. IBM Cloud Free Tier. https://www.ibm.com/cloud/free. Online;

accessed May 2024.

44
 BIBLIOGRAPHY

[26] Synopsys. 2024 open source security and risk analysis report.
https://www.synopsys.com/software-integrity/resources/
analyst-reports/open-source-security-risk-analysis.html.
Online; accessed May 2024.

[27] Manuel Hoffmann, Frank Nagle, and Zhou Yanuo. The value of
open source software. Harvard Business School Strategy Unit Working
Paper No. 24-038, Available at SSRN: https://ssrn.com/abstract=4693148 or
http://dx.doi.org/10.2139/ssrn.4693148, (January 1, 2024).

[28] The Linux Foundation. The Cloud Native Landscape. https://landscape.

cncf.io/. Online; accessed May 2024.

[29] The Linux Foundation. Cloud Native Computing Foundation. https://www.

cncf.io/. Online; accessed May 2024.

[30] Microsoft. Azure Virtual Machine Pricing. https://azure.microsoft.

com/en-us/pricing/details/virtual-machines/. Online; accessed
May 2024.

[31] Khushi Gupta and Tushar Sharma. Changing trends in computer architecture
: A comprehensive analysis of arm and x86 processors. International Journal
of Scientific Research in Computer Science, Engineering and Information Technology,
pages 619–631, 06 2021.

[32] Microsoft. Spot Virtual Machines. https://azure.microsoft.com/

en-us/products/virtual-machines/spot. Online; accessed May 2024.

[33] Georgios Koukis, Sotiris Skaperas, Ioanna Angeliki Kapetanidou, Lefteris Ma-
matas, and Vassilis Tsaoussidis. Performance evaluation of kubernetes net-
working approaches across constraint edge environments, 2024.

[34] The Kubernetes Authors. Vertical Pod Autoscaler. https:

//github.com/kubernetes/autoscaler/blob/master/
vertical-pod-autoscaler/README.md. Online; accessed May 2024.

[35] Ameer Khan. Brief overview of cache memory. Technical report, 04 2020.

[36] Sarina Sulaiman, Siti Mariyam Shamsuddin, Ajith Abraham, and Shahida Su-
laiman. Web caching and prefetching: What, why, and how? In 2008 Interna-
tional Symposium on Information Technology, volume 4, pages 1–8, 2008.

[37] Jhonny Mertz and Ingrid Nunes. A qualitative study of application-level

caching. IEEE Transactions on Software Engineering, 43(9):798–816, 2017.

45
 BIBLIOGRAPHY

[38] Shahab Bakhtiyari. Performance evaluation of the apache traffic server and
varnish reverse proxies. 2012.

[39] Maksim Privalov and Mariya Stupina. Improving web-oriented information

systems efficiency using redis caching mechanisms. Indonesian Journal of Elec-
trical Engineering and Computer Science, 33:1667, 03 2024.

[40] John D. Blischak, Emily R. Davenport, and Greg Wilson. A quick introduction
to version control with git and github. PLOS Computational Biology, 12(1):1–18,
01 2016.

[41] Daniel Garcı́a. Vaultwarden. https://github.com/dani-garcia/

vaultwarden. Online; accessed May 2024.

[42] Brad Ford. Bitwarden Gcloud. https://github.com/dadatuputi/

bitwarden_gcloud. Online; accessed May 2024.

[43] Mojtaba Shahin, Muhammad Ali Babar, and Liming Zhu. Continuous inte-
gration, delivery and deployment: A systematic review on approaches, tools,
challenges and practices. IEEE Access, PP, 03 2017.

[44] Ardelean George Daniel. Volunteer API. https://github.com/

Giuco-senpai45/volunteer-api. Online; accessed May 2024.

[45] Ardelean Geroge Daniel. Volunteer Management Frontend. https://

github.com/Giuco-senpai45/volunteer-management. Online; ac-
cessed May 2024.

46