eBook

Modern
Distributed
Database
Fundamentals
What Organizations Need to Know to Increase Scalability,
Meet Ever-Increasing Data Requirements, and Streamline
Tech Stacks
Table of
Contents
Introduction 3

The Evolution of Transactional Databases 4

What is a Distributed SQL Database? 5
What to Expect in this eBook 7

1. Why Distributed SQL Databases Matter 8

Handling Ever-Increasing Data Requirements 9

Increasing Modern Application Scalability and Availability 11
Streamlining the Tech Stack Jungle 14

2. Fundamentals of Distributed SQL Databases 17

Scalable by Design 17
Versatile by Nature 20
Reliable by Default 22

3. Distributed SQL Use Cases 26

Database Modernization 26
Tech Stack Unification 28
Operational Data Management 29

4. Choosing a Distributed SQL Database 31

Key Factors 31
Evaluation Criteria 32
Best Practices 33

5. Evolving Transactional Data with TiDB 35

Introducing TiDB 35
Origins of TiDB 36
Inside TiDB’s Distributed SQL Architecture 36
The Advantages of TiDB 38

Conclusion 40

© 2023 PingCAP. All rights reserved. 2

Introduction
Transactional databases have been a fundamental component
of modern computing for decades. They are designed to store
and manage large volumes of data, while ensuring that the data
remains accurate, consistent, and available. The evolution of
transactional databases can be traced back to the early days of
computing, with the emergence of the first computer systems.

© 2023 PingCAP. All rights reserved. 3

The Evolution of Transactional Databases
During the 1950s and 1960s, the first By the early 2000s, the emergence of
transactional databases were developed for web-based applications and the increasing
mainframe computers. These early popularity of e-commerce led to the
databases were typically built on top of development of new types of transactional
hierarchical and network-based data databases, such as NoSQL databases. These
models, which were common at the time. databases were designed to handle large
These databases were designed to handle volumes of unstructured data, and were used
simple transaction processing tasks, such as in applications such as social media and web
banking transactions. analytics.

In the 1970s, the first relational database As organizations generate and process
management systems (RDBMS) were ever-increasing amounts of data, the need
developed. These databases were built on for scalable and efficient databases is
the relational data model, which allowed becoming more important. Traditional SQL
data to be stored in tables with relationships databases have long been used for
between them. This made it easier to managing and processing data with strong
manage data, and allowed for more complex consistency, but they have scalability and
queries to be executed. performance limitations. NoSQL databases, on
the other hand, are masters at data scalability
During the 1980s and 1990s, relational and performance, but they tend to fall short
databases became increasingly popular, when data consistency is a requirement.
with the emergence of commercial database
management systems, such as Oracle, IBM
DB2, and Microsoft SQL Server. These
databases were designed to handle large
volumes of data and complex queries, and
were used in a wide range of applications,
including banking, retail, and healthcare.

© 2023 PingCAP. All rights reserved. 4

What if there was a solution that combined database alternative. They offer the benefits
the best of traditional SQL and NoSQL of traditional SQL and NoSQL databases while
databases for modern applications? also allowing for more efficient data
processing of mixed workloads and storage
In recent years, distributed SQL databases across multiple nodes.
have emerged as a popular relational

What is a Distributed SQL Database?

Distributed SQL is a type of database data, distributed SQL databases distribute
architecture that distributes data across data across multiple servers, also known as
multiple nodes, allowing for elastic scalability, nodes. Each node operates independently,
relentless reliability, and faster query but also communicates with other nodes to
processing of mixed workloads. Unlike ensure that data is consistent and available
traditional SQL databases that rely on a for processing.
single-node server to store and process

Figure 1. A typical distributed SQL data architecture.

© 2023 PingCAP. All rights reserved. 5

Distributed SQL databases work by partitioning data into smaller, more manageable subsets,
known as shards. Each shard is stored on a separate node, and queries that involve data from
multiple shards are executed across multiple nodes simultaneously. This allows for faster query
processing and better performance, as each node can process queries in parallel.

Additionally, distributed SQL databases with mixed workload processing capabilities can
combine row and column storage in a single database. This provides a single endpoint for mixed
workloads while guaranteeing strong data consistency. Data can also be collected from
multiple applications and aggregated instantly, allowing real-time queries to be performed on
online operational data.

Benefits of Distributed SQL

Distributed SQL databases offer several benefits over traditional SQL databases, including:

1. Scalability: 3. High availability:

As data volumes grow, these databases These databases can provide high
can easily scale up or down to handle the availability, ensuring that data is always
load, by adding or removing nodes as accessible, even in the event of a node
needed. This makes it possible to handle failure. This is achieved through data
large-scale data processing and storage replication, which ensures that data is
without sacrificing performance. always available on multiple nodes.

2. Fault tolerance: 4. Mixed workload processing:

These databases are designed to be These databases can provide efficient
fault-tolerant, which means they can complex query processing of mixed
continue to operate even if one or more workloads, enabling greater developer
nodes fail. This is achieved by replicating productivity, a simplified architecture, and
data across multiple nodes, so that if one real-time data aggregations.
node fails, data can still be retrieved from
other nodes.

Distributed SQL databases are becoming increasingly popular as organizations look for ways to
manage and process large volumes of data efficiently. As data continues to grow at an
exponential rate, distributed SQL databases will become even more important for modern
application development.

© 2023 PingCAP. All rights reserved. 6

What to Expect in this eBook
In this eBook, designed for application development, data architecture, and infrastructure
leaders, we’ll take a tour through the fundamentals of a modern distributed database. With
distributed SQL databases as our lens, we’ll uncover why these databases matter, how they’re
architected, as well as how they’re used in real-world environments. We’ll also explore what you
need to know when choosing a distributed SQL database, as the proliferation of choices can be
daunting to navigate.

By the end of this eBook, you’ll have the knowledge and confidence to take the next step in your
cloud-native journey. You’ll also be able to pinpoint precisely what makes distributed SQL a
unique modern distributed database solution for transactional data.

© 2023 PingCAP. All rights reserved. 7

 1
Why Distributed
SQL Databases
Matter

Choosing the right database to power separate databases for transactional and
modern applications can be challenging. For analytical workloads, adding even more
starters, as data volumes grow when using a technical complexity while opening up major
traditional relational database, challenges in data reliability and consistency.
performance and scalability radically
degrade. These problems can only be With the explosive growth of data and the need
remedied with additional data processing, for scalable and efficient systems, traditional
aggregation, and integration tools. However, relational and NoSQL databases have faced
such solutions create greater technical limitations. This has led to the emergence of
complexity for developers, poor real-time distributed SQL databases, revolutionizing how
performance, and higher data storage costs organizations handle their data.

Additionally, modern applications need to

meet an increasingly mixed and complex
set of requirements. When using a traditional
relational database, teams end up adopting

© 2023 PingCAP. All rights reserved. 8

Handling Ever-Increasing Database
Requirements
In today’s data-driven world, organizations are generating and collecting vast amounts of data
at an unprecedented rate. From user interactions to IoT devices, the volume, velocity, and variety
of data are continually expanding.

Figure 2. The acceleration of new customer experiences into digital channels is driving the creation of modern
software applications as digital services.

As a result, businesses face the significant challenge of effectively managing and processing this
ever-increasing data. Distributed SQL databases have emerged as a robust solution to address
these escalating data requirements.

© 2023 PingCAP. All rights reserved. 9

Scalable Data Storage Data Partitioning and Sharding
Traditional relational databases often To efficiently handle ever-increasing data
struggle to accommodate the rapidly requirements, distributed SQL databases
growing data volumes. Distributed SQL employ data partitioning and sharding
databases address this challenge by techniques. Data partitioning involves
providing scalable data storage capabilities. dividing the data into smaller, more
Instead of relying on a single server, these manageable subsets called partitions. Each
databases distribute data across multiple partition is then distributed across different
nodes in a cluster. As data grows, nodes in the cluster.
organizations can seamlessly add new nodes
to the cluster, allowing for horizontal scaling. This approach allows organizations to
This distributed nature of storage enables distribute the data processing workload
organizations to handle massive data growth evenly and achieve better query
while maintaining optimal performance and performance. Furthermore, distributed SQL
ensuring data availability. databases support sharding, which involves
horizontally splitting the data across multiple
nodes based on specific criteria such as a
Elastic Computing Power
range of values or a hash function. Sharding
In addition to scaling storage, distributed ensures that data is evenly distributed across
SQL databases also offer elastic computing nodes, preventing hotspots and enabling
power. As data requirements increase, the efficient data retrieval.
demand for processing power grows.
Traditional relational databases may
Data Compression and
experience performance bottlenecks when
Optimization
attempting to handle large volumes of con-
current queries and complex analytical work- With the exponential growth of data, storage
loads.Distributed SQL databases, on the other costs and network bandwidth become
hand, leverage the distributed nature of their critical considerations. Distributed SQL
architecture to distribute query execution databases incorporate advanced data
across multiple nodes. This parallel compression and optimization techniques to
processing capability enables organizations minimize storage requirements and improve
to leverage the combined computing power data transfer efficiency.
of the cluster, resulting in faster query
response times and improved overall system By compressing data, these databases
performance. reduce the storage footprint, allowing

© 2023 PingCAP. All rights reserved. 10

organizations to store more data within the same infrastructure. Additionally, optimized data
transfer protocols and algorithms ensure efficient movement of data across the distributed
cluster, reducing network latency and bandwidth consumption.

Real-Time Data Processing

Ever-increasing data requirements often demand real-time or near-real-time data processing

capabilities. Distributed SQL databases excel in this aspect by offering the ability to store data
in real time when paired with distributed processing frameworks like Apache Kafka or Apache
Flink. With this combined capability, organizations can handle continuous data streams, perform
real-time analytics, and make data-driven decisions promptly.

As data requirements continue to grow, distributed SQL databases have proven their
effectiveness in handling the challenges posed by this rapid data expansion. Through scalable
data storage, elastic computing power, data partitioning and sharding, data compression and
optimization, and real-time data processing capabilities, these databases empower
organizations to efficiently scale and manage ever-increasing data volumes. By leveraging the
distributed nature of their architecture, distributed SQL databases provide the scalability,
availability, and flexibility required to meet the demands of modern data-driven applications.

Increasing Modern Application

Scalability and Availability
Organizations strive to deliver highly-scalable and always-on applications that provide an
exceptional user experience. However, traditional database systems often struggle to keep up
with the scalability and availability demands of modern applications, which require real-time
responsiveness.

© 2023 PingCAP. All rights reserved. 11

 Figure 3. An example of a traditional database system that implements sharding, adding technical complexity.

Distributed SQL databases have emerged as a powerful solution to address these challenges and
significantly improve application scalability and availability.

Distributed Query Execution

Distributed SQL databases leverage their distributed architecture to execute queries in parallel
across multiple nodes. This parallel processing capability allows for faster query execution times,
resulting in improved application performance. By dividing the query workload across the cluster,
distributed SQL databases can harness the collective computational power of the nodes,
effectively reducing the response times for complex queries. This distributed query execution
ensures that modern applications can deliver real-time results to users, enabling them to
interact seamlessly with the application.

© 2023 PingCAP. All rights reserved. 12

Intelligent Data Placement

Efficient data placement is crucial for maximizing application availability. Distributed SQL
databases can intelligently distribute and replicate data across data nodes in multiple
availability zones (AZs), offering high availability and fault tolerance. This means if a single node
or less than half of the nodes fail, the system can continue to function, a characteristic
traditional monolithic databases can never achieve. This intelligent data placement ensures that
data is located closer to the nodes that require it, optimizing application availability.

Disaggregated Storage and Compute Architecture

To further enhance application scalability and availability, distributed SQL databases utilize a
disaggregated storage and compute architecture. This architecture separates computing from
storage, so each layer can be deployed separately and scaled independently.

In a disaggregated storage and compute architecture, different functionalities are divided and
allocated to two types of nodes: the Write Node and the Compute Node. This means you can
decide the number of Write Nodes and Compute Nodes to be deployed as needed. Additionally,
you can scale out or scale in the computing or storage capacity online as needed. The scaling
process is transparent to application operations and maintenance staff.

Integration with Modern Application Frameworks

Distributed SQL databases seamlessly integrate with modern application frameworks, enabling
developers to leverage their performance-enhancing features. These databases support popular
frameworks and libraries for application development, such as Spring Boot, Django, or Ruby on
Rails.

By integrating with these frameworks, distributed SQL databases provide a familiar development
environment and enable developers to take advantage of performance optimizations specific to
the database. This integration ensures that modern applications can harness the full potential of
distributed SQL databases and deliver exceptional performance to end users.

© 2023 PingCAP. All rights reserved. 13

Streamlining the Tech Stack Jungle
In a rapidly-evolving technological landscape, companies often find themselves navigating
through a complex jungle of technologies, frameworks, and tools. Managing multiple components
and integrating them seamlessly can be a daunting task. Distributed SQL databases offer a
valuable solution by streamlining the tech stack jungle, simplifying the architecture, and reducing
the complexity associated with data management.

Figure 4. An example of a distributed SQL architecture with scalability and reliability for modern transactional apps
coupled with real-time analytics on transactional data.

© 2023 PingCAP. All rights reserved. 14

Consolidated Data Management

A significant challenge in the tech stack jungle is dealing with multiple data management sys-
tems. Traditional architectures often involve separate databases for different purposes, such as
relational databases, NoSQL databases, caching systems, and message brokers. This fragmenta-
tion introduces complexities in data modeling, data synchronization, and maintaining consisten-
cy across systems.

Distributed SQL databases consolidate these different data management needs into a single,
unified system. By consolidating data management, organizations can simplify their tech stack,
reduce integration challenges, and streamline their operations.

Integration with Ecosystem Tools Simplified Data Operations

and Frameworks
Effective data operations are essential for
Navigating the tech stack jungle often managing the tech stack jungle efficiently.
involves integrating various tools and Traditional databases often require
frameworks to build end-to-end solutions. specialized knowledge and expertise for
Distributed SQL databases are designed to administration, monitoring, and scaling.
seamlessly integrate with popular ecosystem These monolithic systems may also require
tools and frameworks. They provide system downtime, as you’re upgrading a
connectors and APIs for integration with single node. Distributed SQL databases
programming languages, frameworks, and address these challenges by providing
data processing platforms. built-in automation and management tools.
Even better, these databases utilize
Whether it’s integrating with data streaming automatic rolling upgrades since they
frameworks like Apache Kafka or connecting upgrade nodes one-by-one, minimizing
with analytics tools like Apache Spark, impact to the running cluster. These tools
distributed SQL databases—when designed and built-in processes simplify tasks such as
as an open-source product—offer robust database deployment, configuration,
integrations that simplify the process of monitoring, and scaling, allowing
building comprehensive data-driven organizations to streamline their data
solutions, which require less effort for operations.
adoption. This integration capability
reduces complexity in the tech stack and
ensures a smoother development and
deployment experience.

© 2023 PingCAP. All rights reserved. 15

Additionally, distributed SQL databases often offer intuitive web-based interfaces or
command-line tools that provide a unified view and control over the entire distributed database
cluster. This simplification of data operations minimizes the complexity associated with
managing the tech stack jungle.

© 2023 PingCAP. All rights reserved. 16

 2
Fundamentals
of Distributed
SQL Databases

In today’s data-driven world, organizations face the daunting challenge of managing ever-growing
volumes of data, ensuring reliable access, and accommodating dynamic workloads. To overcome
these challenges, distributed SQL databases have emerged as a powerful solution, offering
scalability, reliability, and versatility. This chapter will explore the fundamental principles that
underpin distributed SQL databases and their significance in modern data management.

Scalable by Design
Scalability is a key advantage of distributed SQL
databases, enabling organizations to efficiently
handle growing data volumes, user demands,
and transactional workloads. In this section,
we’ll explore how distributed SQL databases
are designed to be inherently scalable. We will
delve into their horizontal scalability, automatic
sharding capabilities, distributed transactions,
and concurrency control mechanisms.

© 2023 PingCAP. All rights reserved. 17

Horizontal Scalability

Distributed SQL databases excel at horizontal scalability, allowing organizations to seamlessly

expand their data management capabilities as their needs evolve. Horizontal scalability refers to
the ability to add more nodes to a distributed system, thereby increasing its capacity and
performance. Key features contributing to horizontal scalability include:

1. Node Addition and Removal: 2. Load Balancing:

Distributed SQL databases allow for easy Distributed SQL databases incorporate
addition and removal of nodes in the load balancing mechanisms that
cluster. When the workload increases, evenly distribute data and query
organizations can add more nodes to processing across the available nodes.
distribute the data and processing load, Load balancing ensures that the workload
effectively scaling the system. Conversely, is efficiently distributed, preventing any
if the workload decreases, nodes can be single node from becoming a
removed to optimize resource utilization. performance bottleneck. By evenly
This elasticity in node management distributing the data and queries,
ensures that distributed SQL databases distributed SQL databases optimize
can scale up or down based on demand. resource utilization and maximize system
throughput.

Automatic Sharding

Automatic sharding is a vital capability of distributed SQL databases that allows them to partition
data across multiple nodes transparently. Sharding ensures that data is distributed evenly and
managed efficiently in a distributed environment. Key features of automatic sharding include:

1. Data Partitioning: 2. Transparent Data Distribution:

Distributed SQL databases automatically Automatic sharding ensures that data
partition data into smaller, manageable distribution is transparent to applications.
chunks known as shards. Each shard Applications can interact with the
contains a subset of the data and resides distributed SQL database as if it were a
on different nodes within the cluster. Data single logical database, without needing
partitioning allows for parallel processing to handle the complexity of data
of queries across multiple shards, distribution.
enabling high-performance query
execution.

© 2023 PingCAP. All rights reserved. 18

This transparent data distribution simplifies application development and maintenance while
allowing the database to scale seamlessly.

Distributed Transactions

Distributed SQL databases provide support for distributed transactions, allowing organizations
to maintain transactional integrity across multiple nodes. Distributed transactions ensure that a
group of database operations is treated as a single unit, guaranteeing consistency and
durability. Key features of distributed transactions include:

1. Atomicity and Consistency: 2. Two-Phase Commit Protocol:

Distributed transactions in distributed SQL Distributed SQL databases employ the
databases adhere to ACID (Atomicity, two-phase commit protocol to ensure the
Consistency, Isolation, Durability) atomicity of distributed transactions. The
properties. ACID properties ensure that a two-phase commit protocol coordinates
distributed transaction either commits the commit decision across all
entirely or rolls back entirely, preserving participating nodes, ensuring that all
data consistency across multiple nodes. nodes either commit or rollback the
Distributed SQL databases employ transaction consistently. This protocol
consensus protocols to coordinate the enables distributed SQL databases to
commit or rollback of distributed maintain data consistency and
transactions, maintaining transactional transactional integrity in a distributed
integrity. environment.

Distributed SQL databases are designed to be inherently scalable, allowing organizations to

handle increasing data volumes, user demands, and transactional workloads effectively. Through
horizontal scalability, automatic sharding, distributed transactions, and concurrency control
mechanisms, distributed SQL databases empower organizations to scale their data
management systems seamlessly while ensuring consistent performance, data integrity, and
transactional reliability. By leveraging these scalability features, organizations can confidently
meet the challenges of modern data-intensive applications and achieve optimal resource
utilization in distributed environments.

© 2023 PingCAP. All rights reserved. 19

Versatile by Nature
Distributed SQL databases form the backbone
of modern data management systems,
offering versatility that enables organizations
to handle a wide range of data-intensive
tasks. In this section, we’ll explore the inherent
versatility of distributed SQL databases,
focusing on their high-performance
architecture and their ability to handle mixed
workloads efficiently.

High-Performance Architecture

The high-performance architecture of distributed SQL databases is one of their key strengths,
enabling them to deliver exceptional performance for a variety of data processing tasks. The
following aspects contribute to their high-performance capabilities:

1. Distributed Query Execution: 2. Data Partitioning and Replication:

Distributed SQL databases leverage their Distributed SQL databases employ
distributed architecture to execute intelligent data partitioning and replication
queries in parallel across multiple nodes. techniques to ensure efficient data
This parallel processing capability allows distribution and availability. Data is
for faster query execution times, enabling divided into smaller, manageable
organizations to obtain real-time insights partitions that are distributed across the
from large datasets. By dividing the query nodes in the cluster. This distribution
workload across the cluster, distributed enables the database to process
SQL databases harness the collective queries in parallel, as each node operates
computational power of the nodes, on a subset of the data. Additionally, data
resulting in improved application replication ensures fault tolerance and
performance and responsiveness. high availability by storing copies of data
across multiple nodes. In case of node
failures, the database can seamlessly
retrieve data from other replicas,
minimizing downtime and ensuring
continuous operations.

© 2023 PingCAP. All rights reserved. 20

Mixed Workload Processing

Another aspect of the versatility of distributed SQL databases is their ability to handle mixed
workloads efficiently. Whether it involves processing analytical queries, transactional operations,
or a combination of both, distributed SQL databases excel in accommodating diverse workloads.
Here’s how they achieve this:

1. Hybrid Transactional/Analytical optimizing queries, distributed SQL

Processing (HTAP): databases ensure that analytical and
Distributed SQL databases support HTAP, transactional workloads coexist
allowing organizations to perform both harmoniously without impacting each
transactional and analytical operations other’s performance.
within a single database system. This
eliminates the need for separate systems 3. Workload Isolation and Prioritization:
for different workloads, simplifying the Distributed SQL databases allow for
architecture and reducing operational workload isolation and prioritization,
complexities. Organizations can run ensuring that critical transactional
complex analytical queries while operations are not affected by
simultaneously serving transactional resource-intensive analytical queries.
requests, enabling real-time Workload management features enable
decision-making and faster organizations to allocate resources based
time-to-insights. on predefined priorities and dynamically
adjust resource allocation as per

2. Automatic Query Optimization: changing workload demands. This

Distributed SQL databases employ ensures that mission-critical

advanced query optimization techniques transactions receive the necessary

to efficiently handle mixed workloads. resources and performance, even in the

These techniques analyze query patterns, presence of heavy analytical workloads.

data distribution, and available resources

within the cluster to generate optimized
execution plans. By automatically

The versatility of distributed SQL databases makes them a fundamental component of modern
data management systems. Their high-performance architecture, characterized by distributed
query execution and intelligent data partitioning, enables organizations to achieve optimal

© 2023 PingCAP. All rights reserved. 21

performance for a wide range of data processing tasks. Additionally, their ability to efficiently
handle mixed workloads, including HTAP scenarios, ensures flexibility and agility in meeting
diverse data processing requirements. By harnessing the power of distributed SQL databases,
organizations can unlock the potential of their data and drive innovation in today’s data-driven
world.

Reliable by Default
Reliability is a foundational characteristic of
distributed SQL databases, ensuring
consistent and dependable data
management in distributed environments. In
this section, we will explore how distributed
SQL databases are designed to be reliable by
default. We will delve into their strong
consistency guarantees, high availability
features, fault tolerance mechanisms, and
disaster recovery capabilities.

Strong Consistency

Distributed SQL databases provide strong consistency guarantees, ensuring that data remains
consistent across all nodes in the distributed system, even in the presence of concurrent
operations. Strong consistency is essential for applications requiring accurate and reliable data
access. Key features contributing to strong consistency include:

1. Distributed ACID Transactions:

Distributed SQL databases support ACID (Atomicity, Consistency, Isolation, Durability)
properties across distributed transactions. ACID transactions ensure that multiple operations
within a transaction either complete successfully or fail entirely, maintaining data
consistency across distributed nodes. These transactions guarantee that the database is
always in a consistent state, regardless of concurrent modifications.

© 2023 PingCAP. All rights reserved. 22

2. Consensus Protocols:
Distributed SQL databases employ consensus protocols, such Raft, to achieve strong
consistency. These protocols enable nodes to agree on the order of operations and reach
consensus on the state of the distributed database. Through consensus, distributed SQL
databases ensure that data modifications are applied uniformly across all nodes,
guaranteeing strong consistency across the system.

High Availability

High availability is a crucial aspect of distributed SQL databases, ensuring that applications
remain accessible and responsive even in the face of node failures or network interruptions.
Distributed SQL databases achieve high availability through various features, including:

1. Automatic Data Replication: 2. Failover Mechanisms:

Distributed SQL databases automatically In the event of a node failure, distributed
replicate data across multiple nodes in SQL databases employ failover
the cluster. This replication ensures that mechanisms to ensure uninterrupted
copies of data are readily available, even service. Failover mechanisms detect failed
if a node fails. By replicating data, nodes and automatically promote replica
distributed SQL databases maintain data nodes to take over the responsibilities of
availability and enable seamless failover the failed nodes. This automatic failover
in case of node outages. process ensures high availability by
quickly restoring operations and
minimizing downtime.

Fault Tolerance

Fault tolerance is a critical capability of distributed SQL databases, enabling them to withstand
hardware failures, network issues, or other system failures. Distributed SQL databases implement
fault tolerance through the following mechanisms:

1. Data Replication:
Distributed SQL databases replicate data across multiple nodes, providing redundancy and
safeguarding against data loss. If a node fails, data can be retrieved from replicas, ensuring
that data remains accessible and preserving system functionality.

© 2023 PingCAP. All rights reserved. 23

2. Self-Healing Architecture:
Distributed SQL databases employ self-healing architectures that automatically detect and
recover from failures. When a node fails, the system identifies the failure and redistributes its
responsibilities to other available nodes, ensuring fault tolerance and maintaining system
integrity.

Disaster Recovery

Disaster recovery (DR) is a critical aspect of distributed SQL databases, ensuring data integrity
and business continuity in the face of catastrophic events. Distributed SQL databases provide
robust DR capabilities through the following mechanisms:

1. Backup and Restore:

Distributed SQL databases can realize high availability of clusters based on the Raft
consensus protocol and a reasonable deployment topology. That means when a few nodes
in a cluster fail, the cluster can still be available. On this basis, to further ensure data safety, a
distributed SQL database provides backup and restore as the last resort to recover data from
natural disasters and operational failures.

Backup and restore in a distributed SQL database satisfies the following requirements:

a. Backs up cluster data to a DR system with a Recovery Point Objective (RPO) as short
as 5 minutes, reducing data loss in disaster scenarios.
b. Handles operational failures from applications by rolling back data to a time before
the error event.
c. Performs history data auditing to meet the requirements of judicial supervision.
d. Clones the production environment, which is convenient for troubleshooting,
performance tuning, and simulation testing.

2. Replication Across Geographical Regions:

Distributed SQL databases offer the ability to replicate data across multiple geographical
regions or data centers. By replicating data in different locations, organizations can achieve
geographical redundancy, ensuring that data remains available even if an entire region or
data center is affected by a disaster.

© 2023 PingCAP. All rights reserved. 24

Distributed SQL databases prioritize reliability by default, enabling organizations to have
confidence in their data management systems. Through strong consistency guarantees, high
availability features, fault tolerance mechanisms, and disaster recovery capabilities, distributed
SQL databases ensure consistent and dependable operations in distributed environments. By
leveraging these inherent reliability features, organizations can build robust and resilient
applications, meeting the demands of modern data-intensive scenarios while maintaining data
integrity and availability.

© 2023 PingCAP. All rights reserved. 25

 3 Distributed
SQL Use Cases

As we’ve demonstrated so far throughout this guide, distributed SQL databases have emerged as
a transformative technology. They’re revolutionizing the way organizations manage their data
infrastructure while unlocking new levels of scalability, reliability, and versatility. In this chapter,
we’ll explore several use cases that demonstrate the practical applications and benefits of
distributed SQL databases across different domains and industries. We’ll also provide real-world
examples to support these use cases.

Database Modernization
Organizations are constantly seeking ways to
modernize their data infrastructure to meet the
evolving needs of their applications and users.
Database modernization has become a critical
initiative, as legacy systems often struggle to
cope with the demands of scalability,
performance, and agility required by modern
use cases. Distributed SQL databases have
emerged as a powerful solution to drive this
database modernization journey.

© 2023 PingCAP. All rights reserved. 26

Legacy Database Challenges Industry Use Case:

Legacy databases often present several Financial Services

challenges and limitations that hinder
organizations from achieving optimal
performance and scalability. Some Distributed SQL databases offer a compelling
common challenges include: solution for financial services organizations seeking
to modernize their data infrastructure.
• Scalability:
Legacy databases may struggle to A global financial services company needed a
scale horizontally, limiting their ability solution with the combination of scale, simplicity,
to handle increasing data volumes and strong consistency. Its architecture was made
and high concurrency demands. up of sharded MySQL on Vitess. This solution
required major application refactoring, complicated
• Performance Bottlenecks: sharding orchestration, and lacked strong
As data sizes and query complexity transactional consistency.
grow, legacy databases can
experience performance bottlenecks, After extensive research and testing, the company
resulting in slow response times and chose a distributed SQL database with MySQL
degraded user experiences. compatibility. Being MySQL compatible made the
transition simple, while the distributed SQL
• Limited Agility: architecture eliminated manual sharding,
Legacy systems often have rigid significantly simplifying its tech stack.
data models and schemas, making
it challenging to adapt to changing By eliminating sharding, the company enabled
business requirements and evolve simple, no downtime scale in and scale out with no
alongside modern applications. application refactoring. Its new distributed SQL
architecture also provided strong consistency
• Data Silos: through full ACID compliance.
Legacy databases may suffer from
data silos, where different With a distributed SQL database, this company
applications maintain their own achieved better performance results—with more
isolated data stores, leading to data data—and no sharding, including:
duplication, inconsistencies, and
difficulties in data sharing. • 4,000 queries per second (QPS)
• 5ms response times on 2TBs of data

© 2023 PingCAP. All rights reserved. 27

Tech Stack Successfully meeting project requirements on one
data platform meant that the company could turn
Unification its attention to the innovation needs of the business
at-large.

Companies often find themselves managing a complex and disparate tech stack comprising
multiple databases, data processing frameworks, and data integration tools. This fragmented
infrastructure can lead to inefficiencies, increased maintenance costs, and challenges in data
management. Distributed SQL databases offer a compelling solution for tech stack unification,
enabling organizations to streamline their data infrastructure, simplify operations, and achieve
greater efficiency.

Challenges of a Disparate Tech Stack

A disparate tech stack often arises as organizations adopt various technologies to meet different
data processing requirements. However, this fragmentation can create several challenges:

• Data Inconsistency: • Increased Complexity:

Managing data across multiple systems Maintaining and operating a diverse tech
can result in data inconsistencies, as stack requires specialized skills and
each database may have its own data resources, leading to increased complexity
model and storage format. Synchronizing and overhead costs.
and reconciling data becomes complex
and error-prone. • Integration Complexity:
Integrating data from different systems
often requires custom solutions or ETL
processes, making data integration
complex and time-consuming.

© 2023 PingCAP. All rights reserved. 28

• Data Bottlenecks: Industry Use Case:

SaaS Providers
Moving data between different
systems can introduce latency and
performance bottlenecks, negatively
impacting the overall system
performance. Distributed SQL databases offer a unified solution for
SaaS providers to streamline their tech stack and
address the challenges posed by a disparate

Operational Data infrastructure.

Management A popular Customer Success Platform (CSP)

provider used PostgreSQL to handle all the data it
collected externally. However, as its business grew
and data sources rapidly expanded, PostgreSQL
wasn’t able to keep up with its needs. Additionally,
due to the increased amount of data being stored,
costs skyrocketed.

To handle these increasing demands, this

company redesigned its entire data processing and
storage system from the ground-up. A distributed
Efficient and effective management of
SQL database was chosen to power this new
operational data is a crucial aspect of
architecture’s data serving layer for pre-processing
any organization’s success. It involves
data for real-time customer queries.
handling real-time data ingestion,
processing, storage, and retrieval to
By adopting a distributed SQL database, the
support critical business operations and
company’s CSP now provides:
decision-making processes. Distributed
SQL databases offer a powerful
• A better customer experience with 60x faster
solution for operational data
query responses
management, enabling organizations
• A more resilient system
to handle large-scale data workloads,
• Amplified data storage and processing
ensure data integrity, and achieve high
• Real-time analytical capabilities
performance.

The company also reduced its overall storage,

operation, and maintenance costs.

© 2023 PingCAP. All rights reserved. 29

The Importance of Operational Industry Use Case:
Data Management
Enterprise Software
Operational data serves as the lifeblood
of organizations, fueling various business
Distributed SQL databases excel in managing
processes, applications, and analytics.
operational data by providing robust features and
Effective operational data management
capabilities. Here’s how a distributed SQL database
is essential for the following reasons:
enhanced operational data management for a
leading enterprise software company.
• Real-Time Decision Making:
Operational data provides valuable
This company was using MySQL to manage its
insights and supports real-time
control plane containing multiple cloud services in
decision-making processes,
AWS, GCP, and Azure. This control plane supports
allowing organizations to respond
users, cluster management, and web applications.
swiftly to changing market conditions
However, as the company’s Azure cloud usage grew
and customer needs.
by 3x, Azure MySQL was unable to handle the
increased load: queries became slow or even
• Business Process Optimization:
unresponsive for large customers.
Proper management of operational
data enables organizations to
The company evaluated a distributed SQL database
streamline their business processes,
against Azure MySQL and found the former
identify bottlenecks, and improve
outperformed the latter in latency and QPS with
operational efficiency.
comparable hardware resources. Development
teams were also impressed by distributed SQL’s
• Customer Experience:
horizontal scalability, no manual sharding, and zero
Access to accurate and up-to-date
downtime for scale-in and scale-out.
operational data enables organizations
to provide personalized and
With a distributed SQL database, this company has
exceptional customer experiences.
seen better performance with no MySQL scalability
issues. Additional benefits include:

• Lower average P99 and p999 latencies

• More than 10x QPS compared to Azure MySQL
• Reduced hardware costs and maintenance
burden as the company can now host several
control plane services in a single cluster

© 2023 PingCAP. All rights reserved. 30

 4
Choosing a
Distributed
SQL Database

As organizations embrace the benefits of distributed SQL databases, selecting the right database
becomes crucial for successful implementation. Choosing a distributed SQL database involves
considering several key factors, evaluating various criteria, and following best practices to ensure
the selected database aligns with your organization’s requirements.

In this chapter, we will explore the process of choosing a distributed SQL database, covering key
factors to consider, evaluation criteria to assess, and best practices to follow during the selection
process.

Key Factors
When choosing a distributed SQL database, it is essential to consider the following key factors:

1. Scalability:
Evaluate the database’s scalability capabilities to ensure it can handle the anticipated
growth in data volume and user traffic. Consider factors such as horizontal scaling, automatic
data partitioning, and the ability to add or remove nodes from the cluster seamlessly.

© 2023 PingCAP. All rights reserved. 31

2. Fault Tolerance and High Availability:
Consider the database’s fault tolerance and high availability features to ensure data
durability and minimize downtime. Look for features such as automatic data replication,
failover mechanisms, and efficient disaster recovery options.

3. Data Consistency and Integrity:

Examine the database’s data consistency and integrity mechanisms to ensure accurate and
reliable data management. Features like distributed ACID transactions, strong consistency
models, and conflict resolution strategies are crucial considerations.

4. Performance:
Assess the performance capabilities of the database, including query execution times,
throughput, and latency. Look for features such as distributed query processing to optimize
performance.

Evaluation Criteria
To effectively evaluate distributed SQL databases, consider the following criteria:

1. Architecture and Design:

Assess the database’s underlying architecture, distribution model, and data partitioning
strategies. Evaluate how well the architecture aligns with your organization’s requirements,
such as data distribution, fault tolerance, and scalability.

2. Performance Benchmarking:
Conduct performance benchmarking tests to evaluate the database’s performance under
realistic workloads. Compare query execution times, throughput, and latency across different
databases to identify the one that meets your performance expectations.

3. Data Management Features:

Evaluate the database’s data management features, such as indexing capabilities, data
compression, backup and restore mechanisms, and data security features. Consider how well
these features align with your organization’s data management needs.

4. Ecosystem Integration:
Assess the database’s compatibility and integration with your existing tech stack and
ecosystem tools. Consider its support for programming languages, frameworks, data
processing platforms, and data streaming frameworks.

© 2023 PingCAP. All rights reserved. 32

5. Open-Source Community and Support:
Evaluate the strength and responsiveness of the database’s community and support
channels. Consider factors such as active user communities, available documentation, the
type of licenses available, and the responsiveness of the vendor or open-source
community in addressing issues or providing support. Databases with deep open-source
roots have established trust with users, which is critical when considering a newer database
paradigm such as distributed SQL.

Best Practices
Follow these best practices when choosing a distributed SQL database:

1. Clearly-Defined Requirements:
Clearly define your organization’s requirements, including scalability needs, performance
expectations, data model support, and high availability requirements. This will help narrow
down the list of suitable distributed SQL databases.

2. Pilot Testing in Dev/Test Environment:

Conduct a pilot test phase where you can evaluate the shortlisted databases in a controlled
development or test environment. Use representative workloads and scenarios to assess the
performance, scalability, and ease of use of each database.

3. Consider Total Cost of Ownership (TCO):

Evaluate the TCO of each database, including licensing costs, hardware requirements,
operational overhead, and ongoing maintenance costs. Consider both the initial investment
and the long-term costs associated with the selected distributed SQL database.

4. Vendor or Community Support:

Consider the availability and quality of support options provided by the vendor or the open-
source community. Ensure that you have access to reliable support channels and resources
to address any issues or challenges that may arise during implementation and maintenance.

5. Scalability and Flexibility:

Choose a distributed SQL database that can scale with your organization’s growth and adapt
to evolving needs. Consider the flexibility to accommodate changes in data volumes, user
traffic, and future data model requirements.

© 2023 PingCAP. All rights reserved. 33

Choosing the right distributed SQL database is a critical decision that significantly impacts the
success of your data management initiatives. By considering key factors, evaluating various
criteria, and following best practices, you can navigate the selection process effectively.
Remember to align your requirements, conduct thorough evaluations, and consider long-term
implications to select a distributed SQL database that meets your organization’s needs and
supports its future growth and success.

© 2023 PingCAP. All rights reserved. 34

5
Evolving
Transactional
Data with TiDB

As we’ve shown throughout this eBook, selecting the right database to power modern
applications can be challenging. However, there’s a better option that can evolve alongside
your organization.

Introducing TiDB
TiDB, developed by PingCAP, is one of the most
advanced open-source, distributed SQL databases
that’s also MySQL compatible. TiDB powers business-
critical applications with a streamlined tech stack,
elastic scaling, real-time analytics, and continuous
access to data—all in a single database. With these
advanced capabilities, growing companies like yours
can focus on the future without worrying about
complex data infrastructure management or tedious
application development cycles.

© 2023 PingCAP. All rights reserved. 35

Origins of TiDB
Created in 2015, TiDB was born when three talented software engineers decided to build a new
database system that was more scalable and reliable than MySQL. At the time, they had grown
frustrated with how MySQL was managed and maintained. They also wanted a true open-source
solution that would maintain MySQL compatibility, as they enjoyed the database’s rich
ecosystem of tools and frameworks. What they didn’t know then is just how popular their creation
would become.

With over 34,000 GitHub stars and a growing community of contributors, TiDB is also one of the
world’s most adopted open-source distributed SQL databases. The “Ti” in TiDB represents the
symbol for Titanium from the Periodic Table of Elements. Found in nature as an oxide, Titanium
is a powerful chemical element that can produce highly elastic, versatile, and reliable Titanium
metal. With TiDB, some of the world’s largest companies across technology, financial services,
travel, Web3, and gaming are building modern applications as relentlessly powerful as Titanium.

Inside TiDB’s Distributed SQL

Architecture
What makes TiDB so advanced? At its core, its distributed SQL architecture provides horizontal
scalability, high availability, ACID transactions, and MySQL compatibility, while its unique mixed
workload processing layer enables real-time analytics.

© 2023 PingCAP. All rights reserved. 36

 Figure 5. A visual representation of TiDB’s architecture.

TiDB SQL Layer Placement Driver Layer

The TiDB SQL Layer separates compute from The Placement Driver (PD) Layer functions
storage to make scaling simpler, delivering a just like a full-time DBA, monitoring millions of
true cloud-native architecture. shards and performing hundreds of operations
per minute. It also handles the scale-in and
This stateless MySQL-compatible layer scale-out of clusters to meet demand.
provides uniformity into data access Additionally, this layer dynamically balances
without regard for sharding, or any other the data load in real time, mitigates hotspots,
underlying technical implementation, making and provides for the implementation of
applications easier to develop and easier to customized scheduling policies.
use.

Additionally, the TiDB SQL Layer functions as a

smart optimizer and parallel planning engine.

© 2023 PingCAP. All rights reserved. 37

TiDB Storage Layer

Consisting of row and column-based storage engines, the TiDB Storage Layer offers built-in high
availability and strong consistency that can auto-scale to hundreds of nodes and petabytes of
data.

This layer also offers a modern replication mechanism based on the Raft consensus protocol.

The Advantages of TiDB

TiDB can power all of your modern applications with elastic scaling, real-time analytics, and
continuous access to data. Companies using TiDB for their scale-out RDBMS and internet-scale
OLTP workloads benefit from a distributed SQL database that is:

• MySQL compatible: • Strongly consistent:

Enjoy the most MySQL compatible TiDB maintains ACID transactions when
distributed SQL database on the planet. distributing data globally.
TiDB is wire compatible with MySQL 5.7,
so developers can continue to enjoy the • Mixed workload capable:
database’s rich ecosystem of tools and A streamlined tech stack makes it easier to
frameworks. produce real-time analytics. TiDB’s Smart
Query optimizer chooses the most efficient
• Horizontally scalable: query execution plan, which consists of a
TiDB grants total transparency into data series of operators.
workloads without manual sharding. The
database’s architecture separates • Hybrid and multi-cloud enabled:
compute from storage to instantly scale With TiDB, IT teams can deploy database
data workloads out or in as needed. clusters anywhere in the world, in public,
private, and hybrid cloud environments on
• Highly available: VMs, containers, or bare metal.
TiDB guarantees auto-failover and
self-healing for continuous access to • Open source:
data during system outages or network Unlock business innovation with a
failures. distributed database that’s 100% open
source under an Apache 2.0 license.

© 2023 PingCAP. All rights reserved. 38

• Secure:
`TiDB protects data with enterprise-grade encryption both in-flight and at-rest.

Global organizations, such as those referenced in Chapter 3, are using TiDB for diverse database
solutions from large-scale transactional workloads to recommendation engines, data-intensive
applications, and more.

© 2023 PingCAP. All rights reserved. 39

Conclusion
We began by understanding the fundamental challenges faced
by organizations dealing with transactional data in today’s
rapidly changing landscape. The exponential growth in data
volume, the need for high-performance applications, and the
complexity of modern tech stacks necessitate a scalable,
reliable, and versatile solution. Distributed SQL databases provide
precisely that.

© 2023 PingCAP. All rights reserved. 40

Next, we explored the key factors that make distributed SQL databases essential for managing
transactional data in a modern context. From handling ever-increasing data requirements to
improving application performance and streamlining the tech stack, distributed SQL databases
address critical pain points faced by organizations across industries.

We then uncovered real-world use cases that demonstrate the practical applications of
distributed SQL databases across various industries. These examples showcased how
organizations have successfully leveraged distributed SQL to modernize their databases, unify
their tech stacks, and efficiently manage operational data management.

Distributed SQL databases such as TiDB provide a solid foundation for organizations who want to
evolve their transactional data management strategies. By embracing the modern distributed
database principles mentioned in this guide, organizations can unlock the full potential of their
data, leverage the scalability and reliability of distributed SQL architecture, and propel their
businesses forward in the digital economy.

If you want to learn more about TiDB, have questions about database modernization, or simply
want to better understand your options as you plan for the future, please don’t hesitate to
book a demo with one of our distributed SQL experts.

© 2023 PingCAP. All rights reserved. 41

 About
PingCAP is the company behind TiDB, the most advanced, open source, distributed SQL
database. TiDB powers modern applications with a streamlined tech stack, elastic scaling,
real-time analytics, and continuous access to data—all in a single database. With these
advanced capabilities, growing businesses can focus on their future growth. instead of complex
data infrastructure management. Some of the world’s largest companies across technology,
financial services, travel, Web3, and gaming trust TiDB to handle their business-critical workloads.
Headquartered in Silicon Valley, PingCAP is backed by Sequoia Capital, GGV Capital, Access
Technology Ventures, Coatue Management, and others. For more information, please visit
pingcap.com.