ELL 887 - CLOUD COMPUTING

Cloud Native –
Communication & Data
 2

Outline
• Introduction
• Pillars of Cloud Native
• Cloud Native Applications
• Cloud-native Communication Patterns
• Cloud-native Data Patterns
• Cloud-native Resiliency
• Monitoring & Health
• Devops
 3

Cloud-native Communication Patterns

 In a monolithic application, communication is straightforward.
• The code modules execute together in the same executable space (process) on a
server.
• This approach can have performance advantages as everything runs together in
shared memory
• However, it results in tightly coupled code that becomes difficult to maintain, evolve,
and scale.
 Cloud-native systems implement a microservice-based architecture with
many small, independent microservices.
 While they provide many benefits, microservices have some limitations.
• Local in-process method calls between components are now replaced with network
calls.
• Each microservice must communicate over a network protocol, which adds
complexity to your system:
− Network congestion, latency, and transient faults are a constant concern.
− Resiliency (that is, retrying failed requests) is essential.
− Some calls must be idempotent as to keep consistent state.
− Each microservice must authenticate and authorize calls.
− Each message must be serialized and then deserialized - which can be expensive.
− Message encryption/decryption becomes important.
 4

Cloud-native Communication Patterns

 When constructing a cloud-native system, communication becomes a significant
design decision.
• How does a front-end client application communicate with a back-end microservice?
• How do back-end microservices communicate with each other?
• What are the principles, patterns, and best practices to consider when implementing
communication in cloud-native applications?
 Each microservice executes in a separate process and typically runs inside a
container that is deployed to a cluster.
 A cluster groups a pool of virtual machines together to form a highly available
environment.
 They're managed with an orchestration tool, which is responsible for deploying
and managing the containerized microservices.
 Across the cluster, microservices communicate with each other through APIs
and messaging technologies.
 5

Front-end Client Communication

• In a cloud-native system, front-end clients (mobile, web, and desktop applications)
require a communication channel to interact with independent back-end
microservices
• To keep things simple, a front-end client could directly communicate with the
back-end microservices
 6

Direct Client to Service Communication

 With this approach, each microservice has a public endpoint that is accessible by
front-end clients.
 In a production environment, you'd place a load balancer in front of the
microservices, routing traffic proportionately.
 While simple to implement, direct client communication would be acceptable only
for simple microservice applications.
 This pattern tightly couples front-end clients to core back-end services, opening the
door for many problems, including:
• Client susceptibility to back-end service refactoring.
• A wider attack surface as core back-end services are directly exposed.
• Duplication of cross-cutting concerns across each microservice.
• Overly complex client code - clients must keep track of multiple endpoints and handle
failures in a resilient way.
 7

API Gateway Service

• Instead, a widely accepted cloud design pattern is to implement an API Gateway
Service between the front-end applications and back-end services.
• The API Gateway service abstracts the back-end core microservices.
• Implemented as a web API, it acts as a reverse proxy, routing incoming traffic to the
internal microservices.
• The gateway insulates the client from internal service partitioning and refactoring.
• If you change a back-end service, you accommodate for it in the gateway without
breaking the client.
• It's also your first line of defense for cross-cutting concerns, such as identity, caching,
resiliency, metering, and throttling.
• Many of these cross-cutting concerns can be off-loaded from the back-end core
services to the gateway, simplifying the back-end services.
• Care must be taken to keep the API Gateway simple and fast.
• Typically, business logic is kept out of the gateway.
• A complex gateway risks becoming a bottleneck and eventually a monolith itself.
 8

API Gateway Service

API gateway pattern

 9

API Gateway Service

• Larger systems often expose multiple API Gateways segmented by client type (mobile,
web, desktop) or back-end functionality.
• Incoming traffic is sent to a specific API gateway - based upon client type: web, mobile, or
desktop app.
• This approach makes sense as the capabilities of each device differ significantly across
form factor, performance, and display limitations.
• Typically mobile applications expose less functionality than a browser or desktop
applications.
• Each gateway can be optimized to match the capabilities and functionality of the
corresponding device.

Backend for frontend pattern

 10

Service to Service Communication

 Moving from the front-end client, we now address back-end microservices communicate
with each other.
 When constructing a cloud-native application, you'll want to be sensitive to how back-end
services communicate with each other.
 Ideally, the less inter-service communication, the better.
 However, avoidance isn't always possible as back-end services often rely on one another to
complete an operation.
 There are several widely accepted approaches to implementing cross-service communication.
 The type of communication interaction will often determine the best approach.
 Consider the following interaction types:
• Query
− when a calling microservice requires a response from a called microservice, such as, "Hey, give me the buyer
information for a given customer Id."
• Command
− when the calling microservice needs another microservice to execute an action but doesn't require
a response, such as, "Hey, just ship this order."
• Event
− when a microservice, called the publisher, raises an event that state has changed or an action has occurred.
− Other microservices, called subscribers, who are interested, can react to the event appropriately.
− The publisher and the subscribers aren't aware of each other.
 Microservice systems typically use a combination of these interaction types when executing operations
that require cross-service interaction.
 11

Query
• Many times, one microservice might need to query another, requiring
an immediate response to complete an operation.
• A shopping basket microservice may need product information and a
price to add an item to its basket.
• There are many approaches for implementing query operations.
 12

Request-Response Messaging
• One option for implementing this scenario is for the calling back-end microservice to
make direct HTTP requests to the microservices it needs to query.
• While direct HTTP calls between microservices are relatively simple to implement, care
should be taken to minimize this practice.
• To start, these calls are always synchronous and will block the operation until a result is
returned or the request times outs.
• What were once self-contained, independent services, able to evolve independently and
deploy frequently, now become coupled to each other.
• As coupling among microservices increase, their architectural benefits diminish.

Direct HTTP communication

 13

Request-Response Messaging - Limitations

• Executing an infrequent request that makes a single direct HTTP call to another
microservice might be acceptable for some systems.
• However, high-volume calls that invoke direct HTTP calls to multiple
microservices aren't advisable.
• They can increase latency and negatively impact the performance, scalability, and
availability of your system.
• Even worse, a long series of direct HTTP communication can lead to deep and
complex chains of synchronous microservices calls
 14

Request-Response Messaging - Limitations

• Chaining HTTP queries can lead to many problems
• What happens if Step #3 fails?
• How do you recover if Step #8 fails?
• What if Step #6 is slow because the underlying service is busy?
• Even if all works correctly, think of the latency this call would incur, which is the sum of
the latency of each step.
 The large degree of coupling suggests the services weren't optimally modeled.
 15

Materialized View Pattern

• A popular option for removing microservice coupling is
the Materialized View pattern.
• With this pattern, a microservice stores its own local, denormalized
copy of data that's owned by other services.
• Instead of the Shopping Basket microservice querying the Product
Catalog and Pricing microservices, it maintains its own local copy of
that data.
• This pattern eliminates unnecessary coupling and improves reliability
and response time.
• The entire operation executes inside a single process.
 16

Service Aggregator Pattern

 Another option for eliminating microservice-to-microservice coupling is an Aggregator
microservice, shown in purple in the figure below.
 The pattern isolates an operation that makes calls to multiple back-end microservices,
centralizing its logic into a specialized microservice.
 The purple checkout aggregator microservice in the figure orchestrates the workflow for the
Checkout operation.
• It includes calls to several back-end microservices in a sequenced order.
• Data from the workflow is aggregated and returned to the caller.
• While it still implements direct HTTP calls, the aggregator microservice reduces direct dependencies
among back-end microservices.
 17

Request/Reply Pattern
• Another approach for decoupling synchronous HTTP messages is a Request-Reply Pattern, which uses
queuing communication.
• Communication using a queue is always a one-way channel, with a producer sending the message and
consumer receiving it.
• With this pattern, both a request queue and response queue are implemented, shown in figure below.
• Here, the message producer creates a query-based message that contains a unique correlation ID and
places it into a request queue.
• The consuming service dequeues the messages, processes it and places the response into the response
queue with the same correlation ID.
• The producer service dequeues the message, matches it with the correlation ID and continues
processing.
 18

Commands
• Another type of communication interaction is a command.
• A microservice may need another microservice to perform an action.
• The Ordering microservice may need the Shipping microservice to create a shipment for an
approved order.
• In figure below, one microservice, called a Producer, sends a message to another microservice, the
Consumer, commanding it to do something.

Command interaction with a queue

 19

Commands
 Most often, the Producer doesn't require a response and can fire-and-forget the
message.
 If a reply is needed, the Consumer sends a separate message back to Producer on
another channel.
 A command message is best sent asynchronously with a message queue. supported by
a lightweight message broker.
 In the previous diagram, note how a queue separates and decouples both services.
 A message queue is an intermediary construct through which a producer and consumer
pass a message.
• Queues implement an asynchronous, point-to-point messaging pattern.
• The Producer knows where a command needs to be sent and routes appropriately.
• The queue guarantees that a message is processed by exactly one of the consumer instances
that are reading from the channel.
• In this scenario, either the producer or consumer service can scale out without affecting the
other.
• Moreover, technologies can be disparate on each side, meaning that we might have a Java
microservice calling a Golang microservice.
 Message queues are backing services.
 20

Events
 Message queuing is an effective way to implement communication where a
producer can asynchronously send a consumer a message.
 However, what happens when many different consumers are interested in
the same message?
• A dedicated message queue for each consumer wouldn't scale well and would
become difficult to manage.
 To address this scenario, we move to the third type of message interaction,
the event.
• One microservice announces that an action had occurred.
• Other microservices, if interested, react to the action, or event.
• This is also known as the event-driven architectural style.
 Eventing is a two-step process.
• For a given state change, a microservice publishes an event to a message
broker, making it available to any other interested microservice.
• The interested microservice is notified by subscribing to the event in the
message broker.
• The Publish/Subscribe pattern is used to implement event-based
communication.
 21

Events

Event-Driven messaging - a shopping basket microservice publishing an event with two other
microservices subscribing to it.

• Note the event bus component that sits in the middle of the communication
channel.
• It's a custom class that encapsulates the message broker and decouples it
from the underlying application.
• The ordering and inventory microservices independently operate the event
with no knowledge of each other, nor the shopping basket microservice.
• When the registered event is published to the event bus, they act upon it.
 22

Topic

• With eventing, we move from queuing technology to topics.

• A topic is similar to a queue, but supports a one-to-many messaging pattern.
• One microservice publishes a message.
• Multiple subscribing microservices can choose to receive and act upon that message.
• In the previous figure, publishers send messages to the topic.
• At the end, subscribers receive messages from subscriptions.
• In the middle, the topic forwards messages to subscriptions based on a set of rules, shown in
dark blue boxes.
• Rules act as a filter that forward specific messages to a subscription.
• Here, a "GetPrice" event would be sent to the price and logging subscriptions as the logging
subscription has chosen to receive all messages.
• A "GetInformation" event would be sent to the information and logging subscriptions.
 23

gRPC
 REST-based communication is widely implemented, .
• REST is a flexible architectural style that defines CRUD-based operations against entity
resources.
• Clients interact with resources across HTTP with a request/response communication model.
• A newer communication technology, gRPC, has gained tremendous momentum across
the cloud-native community.
• gRPC is a modern, high-performance framework that evolves the age-old remote
procedure call (RPC) protocol.
• At the application level, gRPC streamlines messaging between clients and back-end
services.
• Originating from Google, gRPC is open source and part of the Cloud Native Computing
Foundation (CNCF) ecosystem of cloud-native offerings.
 24

gRPC
• A typical gRPC client app will expose a local, in-process function that implements a
business operation.
• Under the covers, that local function invokes another function on a remote machine.
• What appears to be a local call essentially becomes a transparent out-of-process call
to a remote service.
• The RPC plumbing abstracts the point-to-point networking communication,
serialization, and execution between computers.
• In cloud-native applications, developers often work across programming languages,
frameworks, and technologies.
• This interoperability complicates message contracts and the plumbing required for
cross-platform communication.
• gRPC provides a "uniform horizontal layer" that abstracts these concerns.
• Developers code in their native platform focused on business functionality, while gRPC
handles communication plumbing.
• gRPC offers comprehensive support across most popular development stacks,
including Java, JavaScript, C#, Go, Swift, and NodeJS.
 25

gRPC Benefits
 gRPC uses HTTP/2 for its transport protocol.
 While compatible with HTTP 1.1, HTTP/2 features many advanced
capabilities:
• A binary framing protocol for data transport - unlike HTTP 1.1, which is text based.
• Multiplexing support for sending multiple parallel requests over the same connection
- HTTP 1.1 limits processing to one request/response message at a time.
• Bidirectional full-duplex communication for sending both client requests and server
responses simultaneously.
• Built-in streaming enabling requests and responses to asynchronously stream large
data sets.
• Header compression that reduces network usage.
 gRPC is lightweight and offers high performance
• It can be up to 8x faster than JSON serialization with messages 60-80% smaller.
 26

gRPC Usage
• Synchronous backend microservice-to-microservice communication where an
immediate response is required to continue processing.
• Polyglot environments that need to support mixed programming platforms.
• Low latency and high throughput communication where performance is
critical.
• Point-to-point real-time communication - gRPC can push messages in real
time without polling and has excellent support for bi-directional streaming.
• Network constrained environments – binary gRPC messages are always
smaller than an equivalent text-based JSON message.
 27

Service Mesh Communication Infrastructure

 We have explored different approaches for implementing synchronous HTTP
communication and asynchronous messaging.
 In each of the cases, the developer is burdened with implementing communication
code.
• Communication code is complex and time intensive.
• Incorrect decisions can lead to significant performance issues.
 A more modern approach to microservice communication centers around a new and
rapidly evolving technology entitled Service Mesh.
 A service mesh is a configurable infrastructure layer with built-in capabilities to handle
service-to-service communication, resiliency, and many cross-cutting concerns.
 It moves the responsibility for these concerns out of the microservices and into service
mesh layer.
 Communication is abstracted away from the microservices.
 A key component of a service mesh is a proxy.
 In a cloud-native application, an instance of a proxy is typically co-located with each
microservice.
 While they execute in separate processes, the two are closely linked and share the
same lifecycle.
 This pattern, known as the Sidecar pattern
 28

Service Mesh with a side car

• Note in the figure how messages are intercepted by a proxy that runs alongside
each microservice.
• Each proxy can be configured with traffic rules specific to the microservice.
• It understands messages and can route them across your services and the
outside world.
 29

Service Mesh
 Along with managing service-to-service communication, the Service Mesh
provides support for service discovery and load balancing.
 Once configured, a service mesh is highly functional.
• The mesh retrieves a corresponding pool of instances from a service discovery
endpoint.
• It sends a request to a specific service instance, recording the latency and response
type of the result.
• It chooses the instance most likely to return a fast response based on different
factors, including the observed latency for recent requests.
 A service mesh manages traffic, communication, and networking concerns at
the application level.
• It understands messages and requests.
• A service mesh typically integrates with a container orchestrator.
• Kubernetes supports an extensible architecture in which a service mesh can be
added.
 30

Istio
 While a few service mesh options currently exist, Istio is the most popular at
the time of this writing.
 Istio is a joint venture from IBM, Google, and Lyft.
 It's an open-source offering that can be integrated into a new or existing
distributed application.
 The technology provides a consistent and complete solution to secure,
connect, and monitor microservices.
 Its features include:
• Secure service-to-service communication in a cluster with strong identity-based
authentication and authorization.
• Automatic load balancing for HTTP, gRPC, WebSocket, and TCP traffic.
• Fine-grained control of traffic behavior with rich routing rules, retries, failovers, and
fault injection.
• A pluggable policy layer and configuration API supporting access controls, rate
limits, and quotas.
• Automatic metrics, logs, and traces for all traffic within a cluster, including cluster
ingress and egress.
 31

Envoy
 A key component for an Istio implementation is a proxy service entitled the
Envoy proxy.
 It runs alongside each service and provides a platform-agnostic foundation for
the following features:
• Dynamic service discovery.
• Load balancing.
• TLS termination.
• HTTP and gRPC proxies.
• Circuit breaker resiliency.
• Health checks.
• Rolling updates with canary deployments
 Envoy is deployed as a sidecar to each microservice in the cluster.
 32

Outline
• Introduction
• Pillars of Cloud Native
• Cloud Native Applications
• Cloud-native Communication Patterns
• Cloud-native Data Patterns
• Cloud-native Resiliency
• Monitoring & Health
• Devops
 33

Data Management in Monolithic Applications

 In the monolithic application, business service components
collocate together in a shared services tier, sharing data from a
single relational database.
 In many ways, a single database keeps data management simple.
• Querying data across multiple tables is straightforward.
• Changes to data update together or they all rollback.
• ACID transactions guarantee strong and immediate consistency.
 34

ACID
 ACID is an acronym that refers to the set of 4 key properties that define a
transaction: Atomicity, Consistency, Isolation, and Durability.
 ACID transactions guarantee that each read, write, or modification of a table
has the following properties:
• Atomicity
− Each statement in a transaction (to read, write, update or delete data) is treated as a single unit.
− Either the entire statement is executed, or none of it is executed.
− This property prevents data loss and corruption from occurring if, for example, if your streaming data source
fails mid-stream.
• Consistency
− Ensures that transactions only make changes to tables in predefined, predictable ways.
− Transactional consistency ensures that corruption or errors in your data do not create unintended
consequences for the integrity of your table.
• Isolation
− When multiple users are reading and writing from the same table all at once, isolation of their transactions
ensures that the concurrent transactions don't interfere with or affect one another.
− Each request can occur as though they were occurring one by one, even though they're actually occurring
simultaneously.
• Durability
− ensures that changes to your data made by successfully executed transactions will be saved, even in the
event of system failure.
 35

Data Management in Cloud Native Applications

 Designing for cloud-native, we take a different approach.
 Business functionality segregates into small, independent
microservices.
 Each microservice encapsulates a specific business
capability and its own data.
 The monolithic database decomposes into a distributed
data model with many smaller databases, each aligning
with a microservice.
 Thus we have a design that exposes a database per
microservice.
 This database per microservice provides many benefits,
especially for systems that must evolve rapidly and
support massive scale.
 With this model...
• Domain data is encapsulated within the service
• Data schema can evolve without directly impacting other
services
• Each data store can independently scale
• A data store failure in one service won't directly impact other
services
 While encapsulating data into separate microservices can
increase agility, performance, and scalability, it also
presents many challenges.
 36

Cross-service Queries
• While microservices are independent and focus on specific functional capabilities, like
inventory, shipping, or ordering, they frequently require integration with other microservices.
• Often the integration involves one microservice querying another for data.

 A shopping basket microservice that adds an item to a user's shopping basket.

 While the data store for this microservice contains basket and line item data, it doesn't
maintain product or pricing data.
 Instead, those data items are owned by the catalog and pricing microservices.

 This aspect presents a problem.

• How can the shopping basket microservice add a product to the user's shopping basket when it
doesn't have product nor pricing data in its database?
 37

Cross-service Queries
 One option is a direct HTTP call from the shopping basket to the catalog and
pricing microservices.
• However synchronous HTTP calls couple microservices together, reducing their
autonomy and diminishing their architectural benefits.
 We could also implement a request-reply pattern with separate inbound and outbound
queues for each service.
• However, this pattern is complicated and requires plumbing to correlate request and
response messages.
• While it does decouple the backend microservice calls, the calling service must still
synchronously wait for the call to complete.
• Network congestion, transient faults, or an overloaded microservice and can result
in long-running and even failed operations.
 Instead, a widely accepted pattern for removing cross-service dependencies is the
Materialized View Pattern
 38

Materialized View Pattern

• With Materialized View pattern, you place a local data table (known as a read model) in the shopping
basket service.
• This table contains a denormalized copy of the data needed from the product and pricing microservices.
• Copying the data directly into the shopping basket microservice eliminates the need for expensive cross-
service calls.
• With the data local to the service, you improve the service's response time and reliability.
• Additionally, having its own copy of the data makes the shopping basket service more resilient.
• If the catalog service should become unavailable, it wouldn't directly impact the shopping basket service.
• The shopping basket can continue operating with the data from its own store.
• The catch with this approach is that you now have duplicate
data in your system.
• However, strategically duplicating data in cloud-native
systems is an established practice and not considered an
anti-pattern, or bad practice.
• Keep in mind that one and only one service can own a data
set and have authority over it.
• You'll need to synchronize the read models when the system
of record is updated.
• Synchronization is typically implemented via asynchronous
messaging with a publish/subscribe pattern
 39

Distributed Transactions
• While querying data across microservices is difficult, implementing a transaction
across several microservices is even more complex.
• The inherent challenge of maintaining data consistency across independent data
sources in different microservices can't be understated.
• The lack of distributed transactions in cloud-native applications means that you
must manage distributed transactions programmatically.
• You move from a world of immediate consistency to that of eventual consistency.
 40

Distributed Transactions - Problem

• Five independent microservices participate in a distributed transaction that creates an

order.
• Each microservice maintains its own data store and implements a local transaction for its
store.
• To create the order, the local transaction for each individual microservice must succeed,
or all must abort and roll back the operation.
• While built-in transactional support is available inside each of the microservices, there's
no support for a distributed transaction that would span across all five services to keep
data consistent.
• Instead, this distributed transaction must be constructed programmatically.
 41

Rolling back a Transaction

• A popular pattern for adding distributed transactional support is the Saga pattern.
• It's implemented by grouping local transactions together programmatically and sequentially invoking each
one.
• If any of the local transactions fail, the Saga aborts the operation and invokes a set of compensating
transactions.
• The compensating transactions undo the changes made by the preceding local transactions and restore
data consistency.

• Rolling back a transaction:

• The Update Inventory operation has
failed in the Inventory microservice.
• The Saga invokes a set of
compensating transactions (in red) to
adjust the inventory counts, cancel the
payment and the order, and return the
data for each microservice back to a
consistent state.
 42

Polyglot Data Persistence in Cloud Native Systems

• Segregating data also enables each microservice to implement the data store type that is
best optimized for its workload, storage needs, and read/write patterns.
• Choices include relational, document, key-value, and even graph-based data stores.

 Each microservice supports a different type of data store.

• The product catalog microservice consumes a relational database to accommodate the rich relational
structure of its underlying data.
• The shopping cart microservice consumes a distributed cache that supports its simple, key-value data
store.
• The ordering microservice consumes both a NoSql document database for write operations along
with a highly denormalized key/value store to accommodate high-volumes of read operations.
 43

Relational vs noSQL data

 Relational and NoSQL are two types of database systems commonly implemented in cloud-native apps.
• They're built differently, store data differently, and accessed differently.
 Relational databases have been a prevalent technology for decades.
• They remain relevant for microservices with complex data
• They're mature, proven, and widely implemented.
• Competing database products, tooling, and expertise abound.
• Relational databases provide a store of related data tables.
• These tables have a fixed schema, use SQL (Structured Query Language) to manage data, and support ACID guarantees.
 No-SQL databases refer to high-performance, non-relational data stores.
• They have gained considerable popularity.
• They excel in their ease-of-use, scalability, resilience, and availability characteristics.
• Their schemaless nature allows developers to move away from an architecture of typed data classes and ORMs that make
change expensive and time-consuming.
• Instead of joining tables of normalized data, NoSQL stores unstructured or semi-structured data, often in key-value pairs or
JSON documents.
• No-SQL databases typically don't provide ACID guarantees beyond the scope of a single database partition.
• High volume services that require sub second response time favor NoSQL datastores.
• The impact of NoSQL technologies for distributed cloud-native systems can't be overstated.
• The proliferation of new data technologies in this space has disrupted solutions that once exclusively relied on relational
databases.
 44

Data Models for noSQL databases

 45

CAP Theorem
 As a way to understand the differences between these types of
databases, consider the CAP theorem, a set of principles applied
to distributed systems that store state.
 The theorem states that distributed data systems will offer a
trade-off between consistency, availability, and partition tolerance
and any database can only guarantee two of the three properties:
• Consistency:
− Every node in the cluster responds with the most recent data, even if the system
must block the request until all replicas update.
− If you query a "consistent system" for an item that is currently updating, you'll
wait for that response until all replicas successfully update.
− However, you'll receive the most current data.
• Availability:
− Every node returns an immediate response, even if that response isn't the most
recent data.
− If you query an "available system" for an item that is updating, you'll get the
best possible answer the service can provide at that moment.
• Partition Tolerance.
− Guarantees the system continues to operate even if a replicated data node fails
or loses connectivity with other replicated data nodes.
 CAP theorem explains the tradeoffs associated with managing
consistency and availability during a network partition
 However tradeoffs with respect to consistency and performance
also exist with the absence of a network partition
 46

CAP Theorem – Relational Database

 Relational databases typically provide consistency and availability, but not partition tolerance.
 They're typically provisioned to a single server and scale vertically by adding more resources to
the machine.
 Many relational database systems support built-in replication features where copies of the
primary database can be made to other secondary server instances.
 Write operations are made to the primary instance and replicated to each of the secondaries.
 Upon a failure, the primary instance can fail over to a secondary to provide high availability.
 Secondaries can also be used to distribute read operations.
 While writes operations always go against the primary replica, read operations can be routed to any of the
secondaries to reduce system load.
 Data can also be horizontally partitioned across multiple nodes, such as with sharding.
 But, sharding dramatically increases operational overhead by spitting data across many pieces that cannot
easily communicate.
 It can be costly and time consuming to manage.
 Relational features that include table joins, transactions, and referential integrity require steep
performance penalties in sharded deployments.
 Replication consistency and recovery point objectives can be tuned by configuring whether
replication occurs synchronously or asynchronously.
 If data replicas were to lose network connectivity in a "highly consistent" or synchronous relational
database cluster, you wouldn't be able to write to the database.
 The system would reject the write operation as it can't replicate that change to the other data replica.
 Every data replica has to update before the transaction can complete.
 47

CAP Theorem – noSQL Database

 NoSQL databases typically support high availability and partition tolerance.
 They scale out horizontally, often across commodity servers.
• This approach provides tremendous availability, both within and across geographical regions at a reduced cost.
• You partition and replicate data across these machines, or nodes, providing redundancy and fault tolerance.
• Consistency is typically tuned through consensus protocols or quorum mechanisms.
• They provide more control when navigating tradeoffs between tuning synchronous versus asynchronous replication
in relational systems.
 If data replicas were to lose connectivity in a "highly available" NoSQL database cluster, you could still
complete a write operation to the database can still be completed.
• The database cluster would allow the write operation and update each data replica as it becomes available.
• NoSQL databases that support multiple writable replicas can further strengthen high availability by avoiding the
need for failover when optimizing recovery time objective.
 Modern NoSQL databases typically implement partitioning capabilities as a feature of their system
design.
• Partition management is often built-in to the database, and routing is achieved through placement hints - often
called partition keys.
• A flexible data models enables the NoSQL databases to lower the burden of schema management and improve
availability when deploying application updates that require data model changes.
 High availability and massive scalability are often more critical to the business than relational table
joins and referential integrity.
• Developers can implement techniques and patterns such as Sagas, CQRS, and asynchronous messaging to embrace
eventual consistency.
 48

Relational vs noSQL data

Referential integrity states that all references in data are valid, and that if one attribute's value references
another attribute's value, then the referenced value must exist.
 49

Database as a Service
• Cloud-native applications favor data services exposed as a Database
as a Service (DBaaS).
• Fully managed by a cloud vendor, these services provide built-in
security, scalability, and monitoring.
• Instead of owning the service, users simply consume it as a backing
service.
• The provider operates the resource at scale and bears the
responsibility for performance and maintenance.
• They can be configured across cloud availability zones and regions to
achieve high availability.
• They all support just-in-time capacity and a pay-as-you-go model.
 50

NewSQL Databases
• NewSQL is an emerging database technology that combines the distributed scalability
of NoSQL with the ACID guarantees of a relational database.
• NewSQL databases are important for business systems that must process high-volumes
of data, across distributed environments, with full transactional support and ACID
compliance.
• While a NoSQL database can provide massive scalability, it does not guarantee data
consistency.
• Intermittent problems from inconsistent data can place a burden on the development
team.
• Developers must construct safeguards into their microservice code to manage
problems caused by inconsistent data.
• A key design goal for NewSQL databases is to work natively in Kubernetes, taking
advantage of the platform's resiliency and scalability.
• NewSQL databases are designed to thrive in ephemeral cloud environments where
underlying virtual machines can be restarted or rescheduled at a moment's notice.
• The databases are designed to survive node failures without data loss nor downtime.
 51

NewSQL Databases
The Cloud Native Computing Foundation (CNCF) features several NewSQL
database projects.
 52

High Volume Data

 Large cloud-native applications often support high-volume data
requirements.
 In these scenarios, traditional data storage techniques can cause
bottlenecks.
 For complex systems that deploy on a large scale, two techniques can
improve application performance:
• Command and Query Responsibility Segregation (CQRS)
• Event Sourcing
 53

CQRS
• CQRS, is an architectural pattern that can help maximize performance, scalability, and
security.
• The pattern separates operations that read data from those operations that write data.
• For normal scenarios, the same entity model and data repository object are used for both
read and write operations.
• However, a high volume data scenario can benefit from separate models and data tables
for reads and writes.
• To improve performance, the read operation could query against a highly denormalized
representation of the data to avoid expensive repetitive table joins and table locks.
• The write operation, known as a command, would update against a fully normalized
representation of the data that would guarantee consistency.
• A mechanism need to be implemented to keep both representations in sync.
• Typically, whenever the write table is modified, it publishes an event that replicates the
modification to the read table.
• Implementing CQRS can improve application performance for cloud-native services.
• However, it does result in a more complex design.
• This principle should be carefully and strategically applied to those sections of a cloud-
native application that will benefit from it.
 54

CQRS

• As shown in figure separate command and query models are implemented.

• Each data write operation is saved to the write store and then propagated to the read store.
• The data propagation process operates on the principle of eventual consistency.
• The read model eventually synchronizes with the write model, but there may be some lag in
the process.
• This separation enables reads and writes to scale independently.
• Read operations use a schema optimized for queries, while the writes use a schema
optimized for updates.
• Read queries go against denormalized data, while complex business logic can be applied to
the write model.
• Tighter security may be imposed on write operations than those exposing reads.
 55

Event Sourcing
 A system typically stores the current state of a data entity.
• If a user changes their phone number, for example, the customer record is updated with
the new number.
• We always know the current state of a data entity, but each update overwrites the previous
state.
• In most cases, this model works fine.
 In high volume systems, however, overhead from transactional locking and
frequent update operations can impact database performance, responsiveness, and
limit scalability.
 Event Sourcing takes a different approach to capturing data.
 Each operation that affects data is persisted to an event store.
 Instead of updating the state of a data record, we append each change to a sequential list
of past events.
 The Event Store becomes the system of record for the data.
 It's used to propagate various materialized views within the bounded context of a
microservice.
 56

Event Sourcing

• In the figure, note how each entry (in blue) for a user's shopping cart is appended to an
underlying event store.
• In the adjoining materialized view, the system projects the current state by replaying all the
events associated with each shopping cart.
• This view, or read model, is then exposed back to the UI.
• Events can also be integrated with external systems and applications or queried to determine
the current state of an entity.
• With this approach, history is maintained.
• You know not only the current state of an entity, but also how you reached this state.
 57

Event Sourcing
 Mechanically speaking, event sourcing simplifies the write model.
• There are no updates or deletes.
• Appending each data entry as an immutable event minimizes contention, locking, and
concurrency conflicts associated with relational databases.
 Building read models with the materialized view pattern enables the decoupling of
the view from the write model and choose the best data store to optimize the
needs of your application UI.
 While event sourcing can provide increased performance and scalability, it comes
at the expense of complexity and a learning curve.
 58

Caching in Cloud Native Apps

 The benefits of caching are well understood.
 The technique works by temporarily copying frequently accessed data from a backend data
store to fast storage that's located closer to the application.
 Caching is often implemented where...
• Data remains relatively static.
• Data access is slow, especially compared to the speed of the cache.
• Data is subject to high levels of contention.
 Benefits
• Caching can increase performance, scalability, and availability for individual microservices and the
system as a whole.
• It reduces the latency and contention of handling large volumes of concurrent requests to a data store.
• As data volume and the number of users increase, the greater the benefits of caching become.
 Caching is most effective when a client repeatedly reads data that is immutable or that
changes infrequently.
• Examples include reference information such as product and pricing information, or shared static
resources that are costly to construct.
 While microservices should be stateless, a distributed cache can support concurrent access to
session state data when absolutely required.
 Caching can also be used to avoid repetitive computations.
• If an operation transforms data or performs a complicated calculation, the result can be cached for
subsequent requests.
 59

Caching Architecture
 Cloud native applications typically implement a distributed caching architecture.
 The cache is hosted as a cloud-based backing service, separate from the microservices.
 In the figure, note how the cache is independent of and shared by the microservices.
• The cache is invoked by the API Gateway.
• The gateway serves as a front end for all incoming requests.
• The distributed cache increases system responsiveness by returning cached data whenever possible.
• Additionally, separating the cache from the services allows the cache to scale up or out independently to meet increased
traffic demands.
 The figure presents a common caching pattern known as the cache-aside pattern.
• For an incoming request, you first query the cache (step #1) for a response.
• If found, the data is returned immediately.
• If the data doesn't exist in the cache (known as a cache miss), it's retrieved from a local database in a downstream service
(step #2).
• It's then written to the cache for future requests (step #3), and returned to the caller.
• Care must be taken to periodically evict cached data so that the system remains timely and consistent.
 60

Caching Architecture
• As a shared cache grows, it might prove beneficial to partition its data across multiple nodes.
• Doing so can help minimize contention and improve scalability.
• Many Caching services support the ability to dynamically add and remove nodes and
rebalance data across partitions.
• This approach typically involves clustering.
• Clustering exposes a collection of federated nodes as a seamless, single cache.
• Internally, however, the data is dispersed across the nodes following a predefined distribution
strategy that balances the load evenly.
 61

Readings
• Architecting Cloud Native .NET Applications for Azure.
https://dotnet.microsoft.com/en-us/download/e-book/cloud-native-
azure/pdf