A Principled Technologies report: Hands-on testing. Real-world results.

2.4x the transactions

per minute (TPM)
for a TPROC-C,
online transaction
processing
(OLTP) workload
80vCore Business Critical on
Premium-series hardware vs.
db.m6i.32xlarge

Three Microsoft Azure SQL Managed

Instances offered better SQL Server
performance and value than their Amazon 5.5x the transactions
per second (tps)
RDS counterparts* in our tests for a complex
online order
Using managed Microsoft SQL Server databases in the cloud has the potential processing workload
to simplify scaling and performance optimization while saving administrators 16vCore Business Critical
time and effort in tasks such as provisioning infrastructure and updating on Premium Memory-
Optimized-series hardware vs.
software. Not all managed services offer the same operational and financial db.r5b.4xlarge

benefits, so it’s important to examine performance as well as price for the types
of SQL Server workloads you run.

Principled Technologies compared the cost and performance of three instances

Up to 5.5x as fast
each of two managed cloud instance services: Azure SQL Managed Instance
to analyze a single
and Amazon Relational Database Service (RDS) for SQL Server. Across those stream using a
three sets of instances using three common SQL Server database workloads— TPROC-H workload
two for online transaction processing and one for data analysis—Azure SQL 64vCore Business Critical
on Premium Memory-
Managed Instance significantly outperformed Amazon RDS. Offering between Optimized-series hardware vs.
double and five times the performance depending on the workload, the three db.r5b.16xlarge

SQL Managed Instances we tested also offered better overall value, reducing
price/performance by as much as 90.3 percent.

These three Azure SQL Managed Instances could offer better overall value in Get up to 90.3%
two ways. First, their significantly better price/performance ratio shrinks cloud better price/
performance
operating costs compared to their competitors in one-to-one comparisons.
using pay-as-
Second, these SQL Managed Instances could potentially reduce the number of
you-go pricing
instances an organization has to purchase to handle expected customer loads. 64vCore Business Critical
on Premium Memory-
Optimized-series hardware vs.
*
Using SQL Managed Instance on the business-critical service tier with premium-series hardware db.r5b.16xlarge
compared to R5b and M6i instances from Amazon RDS

Three Microsoft Azure SQL Managed Instances offered better SQL Server performance
May 2022
and value than their Amazon RDS counterparts in our tests
 About our testing approach
To compare performance of Azure SQL Managed Instance and Amazon RDS for SQL Server (on Amazon Web
Services, or AWS), we tested three instance types using three common SQL Server database workloads: online
transaction processing (TPROC-C), complex online transaction processing (MSOLTPE), and data analytics (TPROC-H).
In each test case, we tried to match Azure and AWS as close to equally as possible on either vCore count or memory
capacity, due to the 64K IOPS limit for AWS RDS SQL Server, which is lower than the capabilities of Azure SQL
Managed Instance. For the TPROC-H workload, we compared the 64vCore memory-optimized offerings for each
cloud service provider (CSP); for the MSOLTPE workload, we compared the 16vCore memory-optimized offerings for
each CSP; and for the TPROC-C workload, we compared the most powerful non-memory-optimized offerings from
each CSP (80vCore on Azure, 128vCore on AWS).

As you will note in each testing and results section, specifications differed across the competitors—but we used the
fairest comparison available at the start of each test, depending on what each instance offered. For example, for
TPROC-C testing, the Azure VM was capable of up to 320K IOPS, whereas Amazon RDS had a cap of only 64K IOPS.
To compensate, we configured the Amazon RDS instance with more vCPU, but the limitation in available IOPS led
the Azure VM with 80 vCore to outperform the 128vCPU Amazon RDS instance. While not an exact match, we chose
Multi-AZ from Amazon RDS (which is a considerable price increase) to match the high availability level that Azure
SQL Managed Instance provides.

In every testing case, the Azure SQL Managed Instance IOPS cap was higher than that of the Amazon instance it was
competing with, either due to the AWS IO1 limit of 64K IOPS, or in the case of the 16vCore AWS MSOLTPE instance,
due to the instance’s lower IOPS cap of 43.3K. To make these instances match on IOPS would have required a
mismatch in vCores, memory, or both, particularly in the case of the large VMs we used in TPROC-C and TPROC-H
testing. (Note: After we created our test plan and testing was well under way, Amazon RDS unveiled support for a
new memory-optimized instance type. To read more details about our testing, the configurations of each instance,
and what instances are available now, read the science behind the report.)

Note: Azure SQL Managed Instance service allows customers to select from three backing hardware series and does
not give a VM series name. In this study, we describe the Azure SQL Managed instances in terms of vCore count,
service tier (General Purpose or Business Critical), and hardware series (Standard, Premium, or Premium Memory-
Optimized.

In contrast, the Amazon RDS instance types have direct analogs in their AWS EC2 offerings; for example, Amazon
RDS hosted the TPROC-H instance on the db.r5b.16xlarge platform.

About Azure SQL Managed Instance

Azure SQL Managed Instance is a platform-as-as-service offering that provides fully managed SQL Server
applications in the Azure cloud. Because SQL Managed Instance is built on the SQL Server engine, Microsoft asserts
that software will always stay up to date without administrator intervention.

According to Microsoft, Azure SQL Managed Instance offers these attractive features:1

• Broad SQL Server compatibility on a fully managed • Secure data with layers of protection and intelligent
service requires minimal code changes threat detection
• Use the tools and experience you already have with • Native virtual network support functionality
SQL Server

With Azure SQL Managed Instance, organizations can also use Azure Hybrid Benefit pricing that allows them to use
existing SQL Server licensing to reduce costs in the Azure cloud. To learn more about Azure SQL Managed Instance,
visit https://azure.microsoft.com/en-us/products/azure-sql/managed-instance/#overview.

Three Microsoft Azure SQL Managed Instances offered better SQL Server performance
May 2022 | 2
and value than their Amazon RDS counterparts in our tests
 Handle more customer
TPROC-C on Microsoft SQL Server transactions (TPROC-C)
Azure SQL Managed Instance With an 80vCore Business Critical Azure SQL Managed Instance
on Premium-series hardware
• 80vCore Business Critical on Premium-
series hardware (non-memory If your organization plans to use your managed cloud
optimized) instances to host OLTP databases, maximizing the
number of customer transactions that each instance
• 560GB memory
can handle ensures you can support customers at
• 1.2TB database (13,000 warehouses) peak times and could reduce the number of instances
• 320K max IOPS you must purchase. Running a TPROC-C workload
that simulates customer order fulfillment, an 80vCore
Amazon RDS for SQL Server
Business Critical Azure SQL Managed Instance on
• m6i.32xlarge Premium-series hardware handled 2.4 times the
• 128 vCore transactions per minute that the db.m6i.32xlarge
• 512GB memory Amazon RDS instance did (see Figure 1). In these
tests, we compared the most powerful non memory-
• 1.2TB database (13,000 warehouses) optimized instances available for each cloud service
• 64K max IOPS provider at the time of testing.

Database performance
Transactions per minute (TPM) | TPROC-C workload | Higher is better

Figure 1: OLTP database performance, in

Azure SQL Managed Instance transactions per minute, that the instances
Instance name: 80vCore Business Critical Premium-series hardware achieved on the HammerDB TPROC-C
826,687 TPM benchmark. Higher numbers are better.
Source: Principled Technologies.
Amazon RDS for SQL Server
Instance name: db.m6i.32xlarge
334,228 TPM

That large increase in performance means that for this workload, you could reduce the chances that
customers bottleneck the system and slow down ordering at peak times, making a smoother consumer
experience. Or, when sizing your managed cloud instances to expected site visitors, you could keep
your cloud operating budget in check by purchasing fewer instances to handle your load with a SQL
Managed Instance.

Three Microsoft Azure SQL Managed Instances offered better SQL Server performance
May 2022 | 3
and value than their Amazon RDS counterparts in our tests
 Figure 2 compares the price/performance of an 80vCore Business Critical Azure SQL Managed Instance on
Premium-series hardware and an db.m6i.32xlarge Amazon RDS instance using results from our TPROC-C
workload.2 Though the 80vCore Business Critical SQL Managed Instance on Premium-series hardware
outperformed the Amazon RDS db.m6i.32xlarge instances dramatically, it didn’t come at a correlated price
increase. In fact, using pay-as-you-go pricing, Azure SQL Managed Instance would reduce cost by 79.1 percent.
If you have existing SQL Server licenses, you can use Azure Hybrid Benefit pricing, which drops cost even
further—reducing overall cost by 88.7 percent compared to Amazon RDS.

Price/performance comparison
Price/transactions per minute (TPM) | TPROC-C workload | Lower cost is better

Azure SQL Managed Instance (pay-as-you-go pricing) Figure 2: Price/performance comparison

Instance name: 80vCore Business Critical Premium-series hardware for running TPROC-C workloads using
$0.05 both cloud services. Lower costs are better.
Source: Principled Technologies.
Azure SQL Managed Instance (Azure Hybrid Benefits pricing)
Instance name: 80vCore Business Critical Premium-series hardware
$0.03

Amazon RDS for SQL Server

Instance name: db.m6i.32xlarge
$0.27

Who typically uses SQL Server for OLTP?

Online business services such as grocery stores, parcel delivery services, and retail organizations all rely on OLTP
databases. If you’ve ever made an online purchase, you’ve accessed an OLTP database. Customers can search,
make purchases, and complete transactions. Boosting OLTP database performance means that an instance can
handle more customers completing these actions at a time, which can ensure a speedy consumer experience even
at peak times.

The benchmark

To test OLTP performance, we used the TPROC-C workload from HammerDB—an open-source tool that tests the
performance of many leading databases. While HammerDB developers derived their TPROC-C workload from the
TPC-C standard, it is not a full implementation of TPC-C specifications. As such, the results we cite in this paper
are not directly comparable to those officially published by TPC. HammerDB reports results in both transactions
per minute (TPM) and new orders per minute (NOPM) that a system can handle. For more information about
HammerDB, visit www.hammerdb.com.

Three Microsoft Azure SQL Managed Instances offered better SQL Server performance
May 2022 | 4
and value than their Amazon RDS counterparts in our tests
 Complete more financial transactions
(MSOLTPE) MSOLTPE on Microsoft SQL Server
On a 16vCore Business Critical Azure SQL Managed Instance on Premium Azure SQL Managed Instance
Memory-Optimized-series hardware
• 16vCore Business Critical
Some workloads, such as financial applications, use OLTP on Premium Memory-Optimized-series
databases that have more diverse and complex actions. To hardware
show what the performance difference looks like between
• 217GB memory
Azure SQL Managed Instance and Amazon RDS for these types
of workloads, we used a benchmark developed by Microsoft • 80,000 customer database
to compare them. The benchmark, known as MSOLTPE, was • 64K max IOPS
specifically requested by Microsoft for purposes of comparing
Amazon RDS for SQL Server
our results to those produced by their own internal testing.
As Figure 3 shows, the 16vCore Business Critical Azure SQL • db.r5b.4xlarge
Managed Instance on Premium Memory-Optimized-series • 16 vCore
hardware handled 5.5 times the transactions per second (tps) of
• 128GB memory
the db.r5b.4xlarge Amazon RDS for SQL Server instance.
• 80,000 customer database
Again, for this type of workload, this performance increase
• 43.3K max IOPS
could mean that your organization spends less money on fewer
cloud instances to handle the load your customers place on
your databases. Or, it could mean that your business is ready to
comfortably support customers during peak financial transaction
times, such as tax season.

Database performance
Transactions per second (tps) | MSOLTPE workload | Higher is better
Figure 3: Complex OLTP database
Azure SQL Managed Instance performance, in tps, that the instances
Instance name: 16vCore Business Critical Premium Memory-Optimized-series hardware achieved on the MSOLTPE benchmark.
571.52 tps Higher numbers are better. Source:
Principled Technologies.
Amazon RDS for SQL Server
Instance name: db.r5b.4xlarge
102.36 tps

Three Microsoft Azure SQL Managed Instances offered better SQL Server performance
May 2022 | 5
and value than their Amazon RDS counterparts in our tests
 Figure 4 presents the price/performance comparison using results from our MSOLTPE testing. For financial
transaction workloads, the cost reduction was even greater than for our ecommerce workload: With pay-as-you-
go pricing, the 16vCore Business Critical Azure SQL Managed Instance on Premium Memory-Optimized-series
hardware would reduce costs by 89.5 percent over the db.r5b.4xlarge instance on Amazon RDS for SQL Server.
Taking advantage of Azure Hybrid Benefit pricing would reduce costs even further, by as much as 93.3 percent.

For organizations using their cloud managed database instances for financial transactions, choosing this
Azure SQL Managed Instance over a db.r5b.4xlarge instance on Amazon RDS can boost performance while
simultaneously helping their bottom line.

Price/performance comparison
Price/transactions per second (tps) | MSOLTPE workload | Lower cost is better

Azure SQL Managed Instance (pay-as-you-go pricing)

Instance name: 16vCore Business Critical Premium Memory-Optimized-series hardware
Figure 4: Price/performance comparison
$20.80
for running complex OLTP workloads using
both cloud services. Lower costs are better.
Azure SQL Managed Instance (Azure Hybrid Benefits pricing) Source: Principled Technologies.
Instance name: 16vCore Business Critical Premium Memory-Optimized-series hardware
$13.14

Amazon RDS for SQL Server

Instance name: db.r5b.4xlarge
$198.45

Who typically uses SQL Server for OLTP with more complex workloads?

Some OLTP databases are more complex than the standard type businesses use for ecommerce workloads. Financial
organizations use databases that are more structurally complex to handle a range of diverse transactions that occur
between customers and partners reporting and executing financial transactions. Some actions in these types of OLTP
databases are customer-driven, while other transactions occur at set price or time criteria.

The benchmark

For testing, Microsoft provided us with a benchmark kit they developed called MSOLTPE, or the Microsoft TPC-E
benchmark derived from TPC-E 1.14.0. Using the model of a brokerage firm while remaining applicable to any
organization executing financial transactions, the benchmark simulates both customer and partner transactions with
different run times. The benchmark reports in transactions per second (tps) that a system can handle. This workload
is derived from the TPC-E® Benchmark and is not comparable to published TPC-E Benchmark results, as this
implementation does not comply with all requirements of the TPC-E Benchmark.

Three Microsoft Azure SQL Managed Instances offered better SQL Server performance
May 2022 | 6
and value than their Amazon RDS counterparts in our tests
 Analyze your data faster (TPROC-H)
TPROC-H on Microsoft Using a 64vCore Business Critical Azure SQL Managed Instance on Premium
SQL Server Memory-Optimized-series hardware

Azure SQL Managed Instance Making sense of your piles of data faster can reduce the time to
insight, facilitating quicker business decisions that lead to real gains.
• 64vCore Business Critical on
Successfully running online analytical processing (OLAP) workloads
Premium Memory-Optimized-
on managed SQL Server databases requires knowing which type of
series hardware
managed service can help you cut through the data quicker. To find
• 870.4GB memory out, we ran a TPROC-H workload on comparable 64vCore memory-
• 256K max IOPS optimized instance types on both cloud service providers using both
Amazon RDS for SQL Server
a single stream of data and eight simultaneous data streams.

• db.r5b.16xlarge As Figure 5 shows, the 64vCore Business Critical Azure SQL

• 64 vCore Managed Instance on Premium Memory-Optimized-series hardware
analyzed a single data stream 5.5 times as fast as the db.r5b.16xlarge
• 512GB memory Amazon RDS instance, and analyzed eight streams 4.2 times as
• 64K max IOPS fast. Completing analytics in less time improves business agility by
delivering answers based on customer data sooner.

Time to complete data analysis

Seconds | TPROC-H workload | Lower is better

1 stream
Azure SQL Managed Instance Figure 5: Data analytics completion times,
Instance name: 64vCore Business Critical Premium Memory-Optimized-series hardware in seconds, that the instances achieved
669 on the HammerDB TPROC-H benchmark.
Lower numbers are better. Source:
Amazon RDS for SQL Server Principled Technologies.
Instance name: db.r5b.16xlarge
3,687

8 streams

Azure SQL Managed Instance

Instance name: 64vCore Business Critical Premium Memory-Optimized-series hardware
5,482

Amazon RDS for SQL Server

Instance name: db.r5b.16xlarge
23,083

Three Microsoft Azure SQL Managed Instances offered better SQL Server performance
May 2022 | 7
and value than their Amazon RDS counterparts in our tests
 Figure 6 shows the price/performance comparison using results from our TPROC-H tests. Again, Azure SQL
Managed Instances achieved better results at far less cost. With pay-as-you-go pricing, the 64vCore Business
Critical Azure SQL Managed Instance on Premium Memory-Optimized-series hardware would reduce costs by
90.3 percent over the db.r5b.16xlarge instance on Amazon RDS for SQL Server. Taking advantage of Azure
Hybrid Benefit pricing would reduce costs even further, by as much as 93.9 percent.

These results show that a 64vCore Business Critical Azure SQL Managed Instance on Premium Memory-
Optimized-series hardware could speed up data analytics workloads on SQL Server databases—and do so
at an attractive price point that can reduce ongoing cloud operating costs over db.r5b.16xlarge instances on
Amazon RDS.

Price/performance comparison (normalized)

Price/seconds | TPROC-H workload | Lower cost is better

Azure SQL Managed Instance (pay-as-you-go pricing)

Instance name: 64vCore Business Critical Premium Memory-Optimized-series hardware Figure 6: Price/performance comparison
(normalized to the lowest price/performance
$1.60 value for ease of reading) for running
TPROC-H workloads using both cloud
Azure SQL Managed Instance (Azure Hybrid Benefits pricing) services. Lower costs are better. Source:
Instance name: 64vCore Business Critical Premium Memory-Optimized-series hardware Principled Technologies.
$1.00

Amazon RDS for SQL Server

Instance name: db.r5b.16xlarge
$16.53

Who typically runs data analysis workloads on SQL Server data?

After collecting customer data, businesses are left with rows and rows of data with nothing to link them. By
analyzing that SQL Server data using TPC-H-like workloads, organizations can make meaningful relations between
data and find the answer to critical business questions. The insights gleaned from data analysis workloads can
include identifying peak buying times, suggesting other purchases to customers based on what they search for,
and more.

The benchmark

To test data analysis speeds for both a single stream of data and eight simultaneous data streams, we used the
TPROC-H benchmark from the HammerDB suite. The TPROC-H workload runs analytical queries about business-
related retail operations and reports the complete time, in seconds, that a system takes to complete these data
analytics tasks. As with the TPROC-C results, our test results do not represent official TPC results and are not
comparable in any manner to the official TPC-audited results. For more information about HammerDB,
visit www.hammerdb.com.

Three Microsoft Azure SQL Managed Instances offered better SQL Server performance
May 2022 | 8
and value than their Amazon RDS counterparts in our tests
 Bolster your bottom
line while serving more
database users
Selecting managed cloud instances for SQL
Server platform-as-a-service doesn’t have to
be a perplexing process. In our tests of three
popular database workloads that simulate
ecommerce databases, financial transaction
databases, and data analytics, we found that the
three Azure SQL Managed Instances we tested
offered significant performance increases over
the strongest comparable offerings available
from Amazon RDS at time of testing. Though
you might expect performance increases of
two to five times more to come with a related
price increase, our research shows this isn’t
the case. In fact, the 16vCore Business Critical
Azure SQL Managed Instance on Premium
Memory-Optimized-series hardware we tested
would shrink total cloud cost to run our TPC-E
workload using pay-as-you-go pricing by 89.5
percent—or 93.3 percent by taking advantage
of Azure Hybrid Benefit pricing—compared to
the db.r5b.4xlarge instance on Amazon RDS.

If your organization seeks the agility and

flexibility of managed cloud instances for SQL
Server, these test results show that choosing
these Business Critical Azure SQL Managed
Instances on Premium or Premium Memory-
Optimized-series hardware can help you
serve more database users per instance while
shrinking cloud operating costs compared to
M6i and R5b instances from Amazon RDS for
SQL Server.

1. Microsoft, “Azure SQL Managed Instance,”

accessed April 21, 2022, https://azure. Read the science behind this report at https://facts.pt/VE2gSX5
microsoft.com/en-us/products/azure-sql/
managed-instance/#overview.
2. To see pricing details and calculations, visit
the science behind the report. Principled Principled
Facts matter.® Facts matter.®
Technologies ®
Technologies ®

Principled Technologies is a registered trademark of Principled Technologies, Inc.

All other product names are the trademarks of their respective owners.
This project was commissioned by Microsoft. For additional information, review the science behind this report.

Three Microsoft Azure SQL Managed Instances offered better SQL Server performance
May 2022 | 9
and value than their Amazon RDS counterparts in our tests