Load Balancing

What is load balancing?

• LB is the method of distributing network traffic
equally across a pool of resources that support an
application.
• Modern applications must process millions of users
simultaneously and return the correct text, videos, images,
and other data to each user in a fast and reliable manner.
• A load balancer
• is a device that sits between the user and the server group
• acts as an invisible facilitator, ensuring that all resource
servers are used equally.
WHAT IS CLOUD LOAD BALANCING?
▪ CLB is the process of distributing workloads across
computing resources in a cloud computing
environment and carefully balancing the network
traffic accessing those resources.
▪ To meet workload demands by routing incoming
traffic to multiple servers, networks or other resources
▪ improving performance and protecting against
disruptions in services.

▪ To distribute workloads across two or more

geographic regions.
 CONT…
▪ CLB helps enterprises achieve high-performance
levels for potentially lower costs than traditional on-
premises load balancing technology.
▪ Cloud load balancing takes advantage of the cloud's
scalability and agility to meet the demands of
distributed workloads with high numbers of client
connections.
▪ It also improves overall availability, increases
throughput and reduces latency.
▪ In addition to workload and traffic distribution, cloud
load balancing services typically offer other features,
such as application health checks, automatic scaling
and failover and integrated certificate management.
 HOW DOES IT WORK?

▪ In cloud computing, load-balancing

technology resides between back- end
servers and client devices.

▪ As requests are received, the load balancer

distributes them to available servers using
an algorithm that may take into account a
variety of criteria, including geographical
distance and server load.
Types of load balancing Technology?

 Hardware load balancers

• A hardware-based load balancer is a hardware
appliance that can securely process and redirect
gigabytes of traffic to hundreds of different
servers. You can store it in your data centers and
use virtualization to create multiple digital or
virtual load balancers that you can centrally
manage.
 Software load balancers
• Software-based load balancers are applications
that perform all load balancing functions. You can
install them on any server or access them as a
fully managed third-party service.
Comparison of hardware balancers to software
load balancers

• Hardware load balancers require an initial investment,

configuration, and ongoing maintenance. You might
also not use them to full capacity, especially if you
purchase one only to handle peak-time traffic spikes.
• If traffic volume increases suddenly beyond its
current capacity, this will affect users until you can
purchase and set up another load balancer.

• In contrast, software-based load balancers are much

more flexible. They can scale up or down easily and
are more compatible with modern cloud computing
environments. They also cost less to set up, manage,
and use over time.
WHAT IS CLOUD LOAD BALANCING VS.
TRADITIONAL LOAD BALANCING?

▪ Traditional load balancing technology is

hardware-based, requiring IT teams to install,
manage, and maintain proprietary hardware
within a data center.
▪ In contrast, cloud load balancing tends to be a
software-based technology, as most cloud
vendors will not allow customer hardware to run
within their environment.
▪ Software-based load balancers can run in any
location or environment, and they are more
affordable for smaller businesses.
CONT.

• Reduced latency. Load balancing minimizes response

time for application users by spreading cloud
workloads evenly across available resources.
• Easier automation. Cloud load balancing improves
automation by enabling organizations to deliver
insight into applications in near-real time and use
predictive analytics to identify potential bottlenecks
in advance.
• Faster recovery. During network emergencies or
natural disasters, providers offering cloud load
balancing can redirect traffic to other regions to
ensure continuity and availability.
WHAT IS CLOUD LOAD BALANCING AS
A SERVICE (LBAAS)?
▪ Load balancing as a service provides cloud load
balancing on an as- needed basis, replacing on-
premises, dedicated appliances.
▪ LBaaS provides greater scalability, as load balancing in
the cloud can accommodate traffic spikes without
needing to reconfigure physical infrastructure.
▪ Greater availability is ensured by connecting to the
closest servers, and LBaaS can also reduce the cost of
investment and maintenance when compared to
hardware-based appliances.
What are the benefits of cloud load balancing?
Cloud Load balancing directs and controls internet traffic between the application
servers and their visitors or clients. As a result, it improves an application’s availability,
scalability, security, and performance.
Application availability
Server failure or maintenance can increase application downtime, making your
application unavailable to visitors. Load balancers increase the fault tolerance
of your systems by automatically detecting server problems and redirecting
client traffic to available servers. You can use load balancing to make these
tasks easier:
 Run application server maintenance or upgrades without application downtime
 Provide automatic disaster recovery to backup sites
 Perform health checks and prevent issues that can cause downtime
Application scalability
You can use load balancers to direct network traffic intelligently among
multiple servers. Your applications can handle thousands of client
requests because load balancing does the following:
 Prevents traffic bottlenecks at any one server
 Predicts application traffic so that you can add or remove different servers, if
needed
 Adds redundancy to your system so that you can scale with confidence
 Contd.
Application security
Load balancers come with built-in security features to add another layer
of security to your internet applications. They are a useful tool to deal
with distributed denial of service attacks, in which attackers flood an
application server with millions of concurrent requests that cause server
failure. Load balancers can also do the following:
 Monitor traffic and block malicious content
 Automatically redirect attack traffic to multiple backend servers to minimize
impact
 Route traffic through a group of network firewalls for additional security
Application performance
Load balancers improve application performance by increasing
response time and reducing network latency. They perform several
critical tasks such as the following:
 Distribute the load evenly between servers to improve application
performance
 Redirect client requests to a geographically closer server to reduce latency
 Ensure the reliability and performance of physical and virtual computing
resources
What a r e l o a d balancing algorithms
A load balancing algorithm is the set of rules that a load balancer follows to
determine the best server for each of the different client requests. Load
balancing algorithms fall into two main categories.
Static load balancing
Static load balancing algorithms follow fixed rules and are independent of
the current server state. The following are examples of static load balancing.
 Round-robin method
• Servers have IP addresses that tell the client where to send requests. The IP address
is a long number that is difficult to remember. To make it easy, a Domain Name
System maps website names to servers. When you enter aws.amazon.com into your
browser, the request first goes to our name server, which returns our IP address to
your browser.
• In the round-robin method, an authoritative name server does the load balancing
instead of specialized hardware or software. The name server returns the IP
addresses of different servers in the server farm turn by turn or in a round-robin
fashion.
 Weighted round-robin method
• In weighted round-robin load balancing, it can assign different weights to each
server based on their priority or capacity. Servers with higher weights will receive
more incoming application traffic from the name server.
 IP hash method
• In the IP hash method, the load balancer performs a mathematical computation,
called hashing, on the client IP address. It converts the client IP address to a
number, which is then mapped to individual servers.
Dynamic load balancing
Dynamic load balancing algorithms examine the current state of the servers before
distributing traffic. The following are some examples of dynamic load balancing algorithms.
 Least connection method
• A connection is an open communication channel between a client and a server.
When the client sends the first request to the server, they authenticate and establish
an active connection between each other. In the least connection method, the load
balancer checks which servers have the fewest active connections and sends traffic
to those servers. This method assumes that all connections require equal processing
power for all servers.
 Weighted least connection method
• Weighted least connection algorithms assume that some servers can handle more
active connections than others. Therefore, you can assign different weights or
capacities to each server, and the load balancer sends the new client requests to the
server with the least connections by capacity.
 Least response time method
• The response time is the total time that the server takes to process the incoming
requests and send a response. The least response time method combines the server
response time and the active connections to determine the best server. Load balancers
use this algorithm to ensure faster service for all users.
 Resource-based method
• In the resource-based method, load balancers distribute traffic by analyzing the
current server load. Specialized software called an agent runs on each server and
calculates usage of server resources, such as its computing capacity and memory.
Then, the load balancer checks the agent for sufficient free resources before
distributing traffic to that server.
Types of load balancing?
 Application load balancing
• Complex modern applications have several server farms with multiple
servers dedicated to a single application function. Application load
balancers look at the request content, such as HTTP headers or SSL session
IDs, to redirect traffic.
• For example, an ecommerce application has a product directory, shopping
cart, and checkout functions. The application load balancer sends requests
for browsing products to servers that contain images and videos but do not
need to maintain open connections. By comparison, it sends shopping cart
requests to servers that can maintain many client connections and save cart
data for a long time.

 Network load balancing

• Network load balancers examine IP addresses and other network
information to redirect traffic optimally. They track the source of the
application traffic and can assign a static IP address to several servers.
Network load balancers use the static and dynamic load balancing
algorithms described earlier to balance server load.
Types of load balancing? Cont..
 Global server load balancing
• Global server load balancing occurs across several geographically
distributed servers. For example, companies can have servers in multiple
data centers, in different countries, and in third-party cloud providers
around the globe. In this case, local load balancers manage the application
load within a region or zone. They attempt to redirect traffic to a server
destination that is geographically closer to the client. They might redirect
traffic to servers outside the client’s geographic zone only in case of server
failure.
 DNS load balancing
• In DNS load balancing, you configure your domain to route network
requests across a pool of resources on your domain. A domain can
correspond to a website, a mail system, a print server, or another service
that is made accessible through the internet. DNS load balancing is helpful
for maintaining application availability and balancing network traffic across
a globally distributed pool of resources.
How does load balancing work?
• Companies usually have their application running on
multiple servers. Such a server arrangement is called a
server farm.
• User requests to the application first go to the load
balancer. The load balancer then routes each request to a
single server in the server farm best suited to handle the
request.
• Load balancing is like the work done by a manager in a
restaurant. Consider a restaurant with five waiters. If
customers were allowed to choose their waiters, one or
two waiters could be overloaded with work while the
others are idle. To avoid this scenario, the restaurant
manager assigns customers to the specific waiters who are
best suited to serve them.
TITLE AND CONTENT LAYOUT
WITH CHART
 LB in AMAZON WEB SERVICES (AWS)
▪ Amazon Web Services (AWS) Elastic Load Balancing distributes
incoming client traffic and routes it to registered targets such as EC2
instances, containers, or IP addresses, in one or more Availability
Zones.
▪ AWS provides managed services to implement load balancing
effectively, ensuring high availability, fault tolerance, and better
application performance..
▪ Elastic Load balancing supports four types of load balancers:
▪ Application
▪ Network
▪ Gateway
▪ Classic.
▪ The load balancers differ in the features offered,
▪ the network layers at which they operate
▪ and supported communication protocols.
 AWS Elastic Load Balancer (ELB)
1. Application Load Balancer (ALB)
• Operates at the Application Layer (Layer 7) of the OSI model.
• Best suited for HTTP and HTTPS traffic.
• Supports features like:
 Content-based routing (e.g., path-based, host-based).
 WebSocket support.
 SSL/TLS termination.
 Enhanced integration with AWS services like ECS, EKS, and Lambda.
 Ideal for microservices and containerized applications.
2. Network Load Balancer (NLB)
• Operates at the Transport Layer (Layer 4).
• Designed for ultra-low latency and high throughput.
• Handles TCP, UDP, and TLS traffic.
• Scales automatically to millions of requests per second.
• Best suited for applications requiring extreme performance, such as gaming or
real-time data streaming.
 Elastic Load Balancer (ELB)
3. Gateway Load Balancer (GWLB)
• Operates at Layer 3 (Network Layer).
• Used primarily for deploying and scaling third-party virtual appliances
(e.g., firewalls, intrusion detection/prevention systems).
• Simplifies the deployment of security and network appliances.

4.Classic Load Balancer (CLB)

• Operates at both Layer 4 and Layer 7.
• A legacy service, recommended only for existing setups.
• Limited features compared to ALB and NLB.
 Use Cases
ALB: Micro-services, REST APIs, web applications,
content-based routing.
NLB: High-performance applications, financial services,
gaming platforms.
GWLB: Security appliances, network appliances at
scale.
CLB: Legacy applications that require minimal features.
 LB in GOOGLE CLOUD PLATFORM
▪ In Google Cloud Platform (GCP), load balancing is a fully managed
service that distributes incoming traffic across multiple instances,
regions, or services to ensure high availability, reliability, and
scalability.
• These distribute traffic across multiple regions and are designed for
applications that need low latency, high availability worldwide and
based on usecases.
▪ The Cloud Load Balancing service available on Google Cloud Platform
is built on the same front-end server infrastructure that powers
Google.
▪ GCP provides global and regional load balancing options, categorized based on
traffic type (HTTP(S), TCP/UDP, or internal).
▪ The service offers a range of load balancers that vary depending on whether the
customer needs
▪ external or internal load balancing,
▪ global or regional load balancing,
▪ Premium or Standard network service tiers,
▪ proxy or pass-through services, among other factors.
 Types of LB in GOOGLE CLOUD
1. Global Load Balancers HTTP(S) Load Balancer
Operates at Layer 7 (Application Layer).
Ideal for web applications, APIs, and content delivery.
Features:
Content-based routing (path and host-based).
Integration with Cloud CDN for caching and faster content delivery.
SSL/TLS termination.
WebSocket support.
Types:
External HTTP(S) Load Balancer: Handles traffic from the internet to your application.
Internal HTTP(S) Load Balancer: Handles traffic within your VPC.
SSL( Secure Socket Layer) Proxy Load Balancer
Operates at Layer 4 (Transport Layer) but terminates SSL/TLS( Transport Layer Security).
Designed for encrypted web traffic.
Distributes HTTPS or SSL traffic globally.
TCP Proxy Load Balancer
Operates at Layer 4.
Best for non-HTTP traffic requiring global load balancing, such as gaming applications or
VPNs (Virtual Private Networks).
 Load Balancing in Google Cloud
2. Regional Load Balancers
These operate within a single region and are suitable for applications confined to specific
geographic areas.
TCP/UDP Load Balancer
Operates at Layer 4.
Ideal for high-performance applications that rely on TCP/UDP protocols, such as gaming,
media streaming, and IoT.
3. Internal Load Balancers
Used for traffic distribution within a Virtual Private Cloud (VPC), typically for backend services.
Internal TCP/UDP Load Balancer
Operates at Layer 4.
Balances traffic within a private network for backend systems.
Internal HTTP(S) Load Balancer
Operates at Layer 7.
Used for distributing HTTP(S) traffic across services within a VPC.
4. Network Load Balancer
Operates at Layer 4.
Handles large volumes of TCP/UDP traffic at low latency.
Ideal for performance-critical and latency-sensitive applications
Features of GCP Load Balancing
Fully Managed: No need to provision or manage instances
manually.
Global or Regional Options: Depending on application
requirements.
Autoscaling: Automatically scales up or down based on traffic
demand.
Integrated with GCP Services:
 Cloud CDN for caching and performance.
 Identity-Aware Proxy (IAP) for secure access.
 Cloud Armor for DDoS protection.
Cross-Region Failover: Automatically redirects traffic in case of
regional outages (global load balancers).
 Common Use Cases
External HTTP(S) Load Balancer: Global web applications and APIs.

Internal HTTP(S) Load Balancer: Microservices communication

within a VPC (Virtual Private Cloud).

TCP/UDP Load Balancer: Gaming applications, IoT, and streaming.

SSL Proxy Load Balancer: Secure web applications needing SSL/TLS

termination.

Network Load Balancer: High-performance, latency-sensitive

backend services.
Load balancing in Microsoft Azure
Load balancing in Microsoft Azure involves distributing incoming
network traffic across multiple backend resources, such as virtual
machines (VMs), containers, or application instances, to ensure
scalability, high availability, and reliability.

Azure offers a variety of load balancing solutions, catering to both

internet-facing and internal applications.

Azure provides several types of load balancers based on traffic type

(HTTP, TCP/UDP, or internal), application layer, and deployment
scenario.
1. Azure Load Balancer
2. Azure Application Gateway
3. Azure Traffic Manager
4. Azure Front Door
5. Azure API Management Gateway
 MICROSOFT AZURE
▪ Microsoft Azure offers four load balancing
services. Azure Traffic Manager is a (OSI model)
layer 7 DNS-based traffic load balancer for
delivering services across global Azure regions.
▪ Azure Load Balancer is a layer 4 network load
balancer for routing traffic between VMs.
▪ Azure Application Gateway is a layer 7 delivery
controller for regional applications.
▪ Azure Front Door is a highly secure, layer 7 global
load balancer for microservice
 Comparison of Azure Load Balancing Options

Service OSI Layer Traffic Type Scope Key Features

Azure Load Balancer Layer 4 TCP/UDP Regional High throughput, health probes,
Zone Redundancy
Azure Application Layer 7 HTTP/HTTPS Regional WAF, URL-based routing,
Gateway SSL Termination

Azure Traffic Manager DNS Level All Global Geo-routing, performance-based routing.

Azure Front Door Layer 7 HTTP/HTTPS Global CDN integration, WAF, global load
balancing.

API Management Layer 7 API Traffic Global API rate limiting, authentication,
Gateway /Regional versioning
Use Cases
High-performance backend services: Use Azure Load
Balancer.

Secure web applications: Use Azure Application

Gateway with WAF.

Global web applications: Use Azure Front Door.

Multi-region disaster recovery: Use Azure Traffic

Manager.
Cloud Scaling
 Cloud Scaling
A Cloud-computing model’s Scalability is defined as the ability to
seamlessly and instantly increase or decrease an IT solution’s size
or capacity.
There are two types of scalabilities in the Cloud architecture,
 Vertical Scaling
 Horizontal Scaling.
A server’s ability to enhance its capacity to handle the increasing
workload is termed Vertical Scaling.

A system’s ability to add new instances to manage the

distributed workload is Horizontal Scaling.

The Cloud service providers like ESDS – The Digital

Transformation catalyst offer auto Scaling Cloud architecture to
optimize resources’ performance.
 Benefits of Cloud Scaling
Fast and Easy
Cloud Scaling architecture enables instructing additional VMs to
manage the increasing workloads with just a few clicks. It
eliminates the delay caused due to rising workloads.
Cost efficiency
Scaling horizontally or vertically is highly cost-efficient
compared to an infrastructure where resources are idle most of
the time.
Optimized Performance
A scalable Cloud architecture efficiently manages a drastic rise
and fall in traffic, thus optimizing the performance. It ensures
the effective utilization of resources, thus eliminating idle
resources or insufficient resource circumstances.
Capacity
The scalable architecture of the Cloud expands its capacity to
manage the growing business requirements.
 Horizontal Scaling
It is also called ‘scaling out’.

Horizontal cloud scaling improves the cloud throughput by

adding new computing infrastructure.

At a basic level, scaling out can mean adding new computing

nodes or machines to enhance the data processing and storage
capabilities.
Horizontal scaling is primarily useful for organizations that need
high availability and near-zero downtime for their online
services.

Compared to vertical scaling, horizontal scaling is quicker and

easier to accomplish.

Generally, it takes place with minimal downtime or other

 Benefits of Horizontal Scaling

Flexible Scaling tools

Easy fault-tolerance
Enhanced resilience due to discrete, multiple systems
Easy to upgrade
Limitless Scaling with endless additions of server instances
Cost-effective implementation
 Vertical Scaling
Vertical cloud scaling enhances the technical specifications of existing
infrastructure by adding or replacing CPU, HDD, or other components.
Decommissioning existing systems and replacing them with higher
capability infrastructure would also qualify as vertical scaling or ‘scaling
up’.

The old infrastructure is either discarded for scrap, resold, or

repurposed for less intensive business processes.
Compared to horizontal scaling, vertical scaling can take longer and may
entail a period of downtime. However, scaling up is usually cheaper than
scaling out.
Cont..
Using the services of a third-party cloud vendor simplifies vertical scaling for
businesses.

This is because ensuring sufficient cloud computing resources for fulfilling scalability
needs becomes the service provider’s duty.

For the end user, scaling up can then be as simple as modifying the instance size on a
web interface.
 Benefits of Vertical Scaling
Flexible Scaling of resources

Consumes less power

Lower cooling cost

Software cost-effective

Reduced administrative efforts to manage a single system