Running Kubernetes in

Azure
VAIBHAV GUJRAL
CLOUD ARCHITECT | MICROSOFT AZURE MVP

https://vaibhavgujral.com/ 1
About me
Director, Global Microsoft Cloud CoE at Capgemini
Born and brought up in India and based out of Omaha, NE since 2016
Microsoft Azure MVP since 2020
Leader, Omaha Azure User Group(https://omahaazure.org)
15+ cloud certifications and counting…

https://vaibhavgujral.com/ 2
What is Kubernetes?
Kubernetes is a portable, extensible, open-source
platform for automating the deployment, scaling,
and management of containerized workloads.

Kubernetes (k(j)uːbərˈnɛtɪs)
Greek for “helmsman of a ship”

https://vaibhavgujral.com/ 3
Kubernetes History
First announced by Google in 2014
Heavily influenced by Google’s Borg system.
Original codename for Kubernetes project was Project 7 (a reference to the Star Trek ex-
Borg character Seven of Nine)
V1.0 was released on July 21, 2015
Current version is 1.26.1 (released on 01/18/2023) (Release History)
Version 1.27 will be available 04/11/2023 (Schedule)
Originally written in C++, the current system is written in Go language.

https://vaibhavgujral.com/ 4
Kubernetes Architecture – Big Picture
Control Plane Workers Plane

Master Nodes Worker Nodes

Kubernetes Cluster

https://vaibhavgujral.com/ 5
Kubernetes Architecture

API Server Scheduler

kubelet Container Runtime

etcd Controller Manager kube-proxy Container

Control Plane Worker Nodes

https://vaibhavgujral.com/ 6
Kubernetes Components
1. Control Plane: manages the agent nodes and the
pods in the cluster
• api-server: front end of the Kubernetes control plane;
exposes Kubernetes API
• controller-manager: runs the controller processes
• scheduler: tracks newly created pods and selects node to run
them on
• etcd: stores the state of the cluster (config, running workloads
status, etc.)

2. Worker nodes: run your application workloads

• Pods: a collection of containers co-located on a single
machine
• kube-proxy: a network proxy that runs on each node in a
cluster
• kubelet: agent that runs on each node in a cluster; ensures
containers are running in a pod
• Containers: software responsible for running containers

https://vaibhavgujral.com/ 7
Kubernetes Core Concepts
Pods are smallest unit in Kubernetes providing an abstraction over containers. Pods are
ephemeral and get their own IP Address.
Services provide a persistent IP Address for a set of pods running an application and acts like a
load balancer. The lifecycle of a service is not linked to the lifecycle of a pod.
Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster.
ConfigMaps are text-based key-value stores to store the external configuration for your
application.
Secrets are base-64 encoded store for confidential data like passwords and secrets.
Volumes offer data storage for persistent data that needs to exist beyond the lifecycle of a pod.

https://vaibhavgujral.com/ 8
Kubernetes Core Concepts
A ReplicaSet's purpose is to maintain a stable set of replica Pods running at any given time.
A Deployment provides declarative updates for Pods and ReplicaSets.
A StatefulSet is similar to deployment, but it maintains a sticky identity for each of their Pods.
A DaemonSet is used for deploying ongoing background tasks that you need to run on all or
certain nodes, and which do not require user intervention.
A Job creates one or more Pods and continues to retry execution of the Pods until a specified
number of them successfully terminate.
A CronJob creates Jobs on a repeating schedule.

https://vaibhavgujral.com/ 9
 Managing Kubernetes Cluster

API Server Scheduler

kubelet Container Runtime

kubectl
etcd Controller Manager kube-proxy Container

Control Plane Worker Nodes

GitHub - kelseyhightower/kubernetes-the-hard-way: Bootstrap Kubernetes the hard way on Google Cloud Platform. No scripts.

https://vaibhavgujral.com/ 10
 =
Azure Kubernetes Azure Kubernetes
Service

https://vaibhavgujral.com/ 11
Azure Kubernetes service
Hosted Kubernetes service in Azure
Reduces the complexity and operational overhead of managing Kubernetes
The control plane is provided as a managed Azure resource abstracted from the user
The control plane includes the core Kubernetes components like kube-apiserver, etcd, kube-
scheduler and kube-controller-manager
An AKS cluster has one or more worker nodes, which is an Azure virtual machine (VM) that runs
the Kubernetes node components and container runtime
You only pay for worker nodes and control plane comes at no charge to you

https://docs.microsoft.com/en-us/azure/aks/

https://vaibhavgujral.com/ 12
Azure Kubernetes Service

Managed by Azure Managed by Customer

API Server Scheduler

kubelet Container Runtime

etcd Controller Manager kube-proxy Container

Control Plane Worker Nodes

https://vaibhavgujral.com/ 13
 Shared Responsibility
Azure managed control plane
Managed
DIY with
Responsibilities Kubernetes
Kubernetes
on Azure App/
Self-managed master node(s)
workload Kubernetes etcd
Containerization User definition API endpoint API server Store

Application iteration,
debugging
Controller Cloud
Scheduler Manager Controller
CI/CD

Provisioning, upgrades,
patches

Reliability availability
Schedule pods over
private tunnel
Scaling Customer VMs

Docker Docker Docker Docker Docker

Monitoring and logging Pods Pods Pods Pods Pods

Customer Microsoft

https://vaibhavgujral.com/ 14
Benefits of AKS
Azure managed control plane

• Automated upgrades, patches App/

Self-managed master node(s)
workload Kubernetes etcd
User definition API endpoint API server Store

• High reliability, availability

• Easy, secure cluster scaling Controller Cloud

Scheduler Manager Controller

• Self-healing

• API server monitoring

Schedule pods over
private tunnel
• At no charge Customer VMs

Docker Docker Docker Docker Docker

Pods Pods Pods Pods Pods

https://vaibhavgujral.com/ 15
Create/Configure AKS Cluster
1. Using Azure Command Line Interface (CLI)
az aks create --resource-group myResourceGroup --name myAKSCluster --node-count 1 --generate-ssh-keys

2. Using Azure PowerShell

New-AzAksCluster -ResourceGroupName myResourceGroup -Name myAKSCluster -NodeCount 1

3. Using Azure Portal

4. Using ARM Templates and Bicep
5. Using Azure REST API
PUT
https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/
providers/Microsoft.ContainerService/managedClusters/{resourceName}?api-version=2021-05-01

https://vaibhavgujral.com/ 16
Manage Azure Kubernetes Service
Task The Old Way With Azure

Create a cluster Provision network and VMs az aks create

Install dozens of system components including etcd
Create and install certificates
Register agent nodes with control plane

Upgrade a cluster Upgrade your master nodes az aks upgrade

Cordon/drain and upgrade Agent nodes individually

Scale a cluster Provision new VMs az aks scale

Install system components
Register nodes with API server

https://vaibhavgujral.com/ 17
https://vaibhavgujral.com/ 18
AKS Networking
Two different options -
1. Kubenet networking - The network resources are typically created and configured as the AKS
cluster is deployed.

Pod Pod

Container IP ……... Container IP

Subnet A
DB 10.10.0.0 App 10.10.0.n
192.168.0.0/20

Pods CIDR Range

IP 192.168.0.0
10.10.0.1/16 Node

https://vaibhavgujral.com/ 19
 Pod Pod
10.10.0.1/16 -> 192.168.0.0
10.11.0.1/16 -> 192.168.0.1
Container IP ……... Container IP
DB 10.11.0.0 App 10.11.0.n Route Table

Pods CIDR Range

10.11.0.1/16 Node IP 192.168.0.1

Pod Pod

Container IP ……... Container IP

DB Subnet A
10.10.0.0 App 10.10.0.n
192.168.0.0/20

Pods CIDR Range

IP 192.168.0.0
10.10.0.1/16 Node

https://vaibhavgujral.com/ 20
AKS Networking
Two different options -
2. Azure Container Networking Interface (CNI) networking - The AKS cluster is connected to
existing virtual network resources and configurations.

Pod Pod

Container IP ……... Container IP

DB App 192.168.0.n
192.168.0.1

Node IP 192.168.0.0

Subnet A
192.168.0.0/20

https://vaibhavgujral.com/ 21
 AKS Security
Enforce compliance rules with Azure Policy
Identity and access control using Azure
Active Directory

Encrypt using your own keys, stored in Azure

Key Vault
Gain unmatched security management with
Azure Security Center integration
Interact securely with Kubernetes API server
using Azure Private Link
Use application gateway (and WAF) with
Ingress Controller

https://vaibhavgujral.com/ 22
 AKS Governance with Azure Policies
Cloud Azure
Architect Policy

1. Cloud architect assigns a

deployment policy across cluster(s)
Compliance reports
2. Developer uses standard
Assigns a policy across
Kubernetes API to deploy to the clusters
Cluster-1 Cluster-2 Cluster-3
cluster

3. Real-time deployment
enforcement (acceptance/denial)
Compliance reports for the entire
environment, with pod-level
granularity O
provided to developer based on
policy
Real-time enforcement of
AKS
Developer
4. Cloud architect obtains compliance policy and feedback
report for the entire environment
and can drill down to individual
Cluster-1 Cluster-2 Cluster-3
O
pod level

https://vaibhavgujral.com/ 23
AKS Identity and Management
Storage

AKS with RBAC

Active VNet
Directory Active
SQL
Directory
Node Node Database

Pod Pod

AAD Pod Identity

Cosmos
DB

Key Vault

https://vaibhavgujral.com/ 24
AKS Identity and Management

https://docs.microsoft.com/en-us/azure/aks/concepts-identity#azure-ad-integration

https://vaibhavgujral.com/ 25
Azure AD Workload Identities
Used by a software workload (such as an application, service, script, or container) to authenticate
and access other services and resources.
In Azure Active Directory (Azure AD), workload identities are applications, service principals, and
managed identities.

https://vaibhavgujral.com/ 26
AKS Auto-scaling

The cluster autoscaler watches for pods that

can't be scheduled on nodes because of
resource constraints. The cluster then
automatically increases the number of nodes.
The horizontal pod autoscaler uses the Metrics
Server in a Kubernetes cluster to monitor the
resource demand of pods. If an application
needs more resources, the number of pods is
automatically increased to meet the demand.

https://vaibhavgujral.com/ 27
 Kubernetes-based event-driven auto-
scaling (KEDA)
Open-source component jointly built by Microsoft and
RedHat Kubernetes cluster

Event-driven container creation & scaling

Allows containers to “scale to zero” until an event comes Scaler AKS cluster
in, which will then create the container and process the External
event, resulting in more efficient utilization and reduced trigger source
costs Controller

Native triggers support

Containers can consume events directly from the event
Metrics adapter
source, instead of routing events through HTTP

Can be used in any Kubernetes service

This includes in the cloud (e.g., AKS, EKS, GKE, etc.) or on-
premises with OpenShift—any Kubernetes workload that KEDA
requires scaling by events instead of traditional CPU or
memory scaling can leverage this component.
https://docs.microsoft.com/en-us/azure/azure-functions/functions-kubernetes-keda
https://vaibhavgujral.com/ 28
Azure Monitor for Containers
Visualization
Visualize overall health and performance
from clusters to containers with drill
downs and filters
Observability
Insights
Azure Kubernetes Azure Monitor Provide insights with multi-cluster health
Pipelines control for containers roll up view
Monitor & Analyze
Monitor and analyze Kubernetes and
container deployment performance,
events, health, and logs
ACI burst
Response
from AKS Native alerting with integration to issue
managements and ITSM tools
Observability
Observe live container logs on container
deployment status

https://vaibhavgujral.com/ 29
AKS Diagnostics
Faster resolution of common issues with AKS diagnostics
Sample diagnostics web
an intelligent, self-diagnostic experience portal

right in the portal Zero configuration

<\> and zero cost
Cluster-specific observations Intelligent detectors
based on AKS-specific
telemetry
Recommended actions for Cluster-specific
observations
troubleshooting Recommended
actions
for troubleshooting
Azure
portal

AKS
Azure
production cluster
backend Node
User telemetry Node 1 2

https://vaibhavgujral.com/ 30
AKS Storage Options

https://vaibhavgujral.com/ 31
Azure Container registry
First-class Azure resource
Managed, private Docker registry service based on the open-source Docker Registry 2.0.
Can be used with existing container development and deployment pipelines
Use Azure Container Registry Tasks to build container images in Azure
Three pricing tiers:
1. Basic
2. Standard
3. Premium
https://azure.microsoft.com/en-us/pricing/details/container-registry/
https://docs.microsoft.com/en-us/azure/container-registry/container-registry-skus

https://vaibhavgujral.com/ 32
Azure container instances
Fastest and simplest way to run a container in Azure
https://docs.microsoft.com/en-us/azure/container-instances/

https://vaibhavgujral.com/ 33
Serverless Kubernetes with Virtual Nodes
Node Node

Elastically provision compute capacity Pods Pods

in seconds

No infrastructure to manage
Azure Container
Instances (ACI)
Built on open sourced Virtual Kubelet Kubernetes
technology, donated to the Cloud Native control plane Pods

Computing Foundation (CNCF)

Virtual node

https://vaibhavgujral.com/ 34
Azure Container Apps
Azure Container Apps enables you to run microservices and containerized applications
on a serverless platform.

https://vaibhavgujral.com/ 35
https://vaibhavgujral.com/ 36
CI/CD using Azure DevOps
Azure Container Registry

5
Engineer Azure Kubernetes Service Azure SQL Database
4
11 2 3
6

Visual Studio Azure Repos Azure Pipelines

Code

8 Azure DevOps 7

Kiabana Azure Monitor

https://vaibhavgujral.com/ 37
CI/CD using GitHub and Jenkins
Azure Container Registry

5
Engineer Azure Kubernetes Service Azure SQL Database
4
11 2 3
6

Visual Studio GitHub Jenkins

Code

8 7

Grafana Azure Monitor

https://vaibhavgujral.com/ 38
Helm Package Manager
Helm is a package manager for Kubernetes that combines all your application's resources and
deployment information into a single deployment package.

https://vaibhavgujral.com/ 39
AKS – Best Practices

https://docs.microsoft.com/en-us/azure/aks/best-practices
https://vaibhavgujral.com/ 40
Further Reading
Kubernetes on Azure - https://aka.ms/K8sonAzure
Microsoft Learn - https://aka.ms/LearnKubernetes
What is Kubernetes - https://aka.ms/k8sLearning
AKS Case Studies - https://aka.ms/AKS/casestudy
AKS Roadmap - https://aka.ms/k8sroadmap
Getting started for free - https://aka.ms/AKS/trial

https://vaibhavgujral.com/ 41
https://vaibhavgujral.com/ 42
Contact Information
https://vaibhavgujral.com
@vaibhavgujral_
https://www.linkedin.com/in/vaibhavgujral/
https://www.youtube.com/c/VaibhavGujral
[email protected]

LinkedIn Twitter Email

https://vaibhavgujral.com/ 43
Slides

https://vaibhavgujral.com/ 44
https://vaibhavgujral.com/ 45