BCR 2021 V1.1
BCR 2021 V1.1
BCR 2021 V1.1
BCR-2021-V1.1
2021 State of Cloud HPC Report
Forew
ts and engineers, and they deserve the best we can provide.
Edward Hsu,
Chief Product Officer, Rescale
2
2021 State of Cloud HPC Report
Big Compute
State of Cloud HPC Report
4
Big Compute is a global community of thought leaders who embrace the
freedom to think big in this era of virtually unlimited compute. From an annual
conference of inspiring speakers, to telling stories year-round of the many
“undercover superheroes” working behind the scenes to change the world,
5 Big Compute fosters innovation in aerospace, automotive, AI, life sciences,
new energy, and everything in between.
3
2021 State of Cloud HPC Report
“I used to ask companies how they’re going to get lot of data, but now I ask them
how they’re going to get a lot of compute…. It turns out that the most impressive
advances that we’ve had in the field of AI research I think have been more about
massive compute than massive data.”
- Sam Altman @ Big Compute 2020
4
1
2021 State of Cloud HPC Report
• Using the right architecture is often more impactful than Nissan leverages Rescale’s cost and performance
intelligence to select the optimal infrastructure based on cost
finding the ideal cloud provider and performance for their specific workloads. This modern
HPC approach ensures that they are always running the
latest, most performant hardware and software configuration.
Nissan
5
HARDWARE & INFRASTRUCTURE 2021 State of Cloud HPC Report
30%
32%
Percent of Organizations
200
28%
20%
100 18%
10%
11% 12%
0 0%
2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 Less than 1 year old 1-2 years old 2-3 years old 3-4 years old 4+ years old
Cloud providers have accelerated the rollout of new HPC architectures Many organizations begin testing and tuning new architectures during the
with exponential growth. first year of introduction, but fully adopt the following year after optimal
configurations are determined.
Source: Rescale Cloud HPC Platform Analytics. Market survey of publicly available cloud provider architectures. 6
HARDWARE & INFRASTRUCTURE 2021 State of Cloud HPC Report
An explosion of cloud-
3
0
overwhelmed requires
500 workload performance
100
50
analytics.
Core I/O (Gb/s)
10
5
1
10 Coretypes 5 Coretypes 1 Coretype
2014 2015 2016 2017 2018 2019 2020
DC HW purchase
10.00
for admin costs long
Cloud provider
(top 4)
enjoyed by cloud providers,
it is increasingly difficult
5.00 Data Center
(fully loaded) for enterprise data center
operations to compete
0.00
2015 2016 2017 2018 2019 2020 purely on performance/cost.
Date available
Notes: Performance based on benchmark CFD workload with STAR-CCM+. Cloud: Reserved Instance list prices used
Datacenter: Assumes 100M core-hours / year consumption at 80% HW utilization and fully loaded costs for HW, labor and facilities. Server to FTE ratio of 50:1. Assumes median US data center labor costs
DC 2015 HW: Per node (6 cores) cost of $3K for Intel Core I7 processors; DC 2019 HW: Per node (18 cores) cost of $10K for Intel Xeon 6154 processors
Source: Rescale Cloud HPC Platform Analytics 8
HARDWARE & INFRASTRUCTURE 2021 State of Cloud HPC Report
Source: Rescale Cloud HPC Platform Analytics, Ark Invest Big Ideas 2021 9
HARDWARE & INFRASTRUCTURE 2021 State of Cloud HPC Report
1.6
About RPI: The Rescale Performance Index (RPI) is a hardware scoring system that analyzes leading benchmarks on common
software to enable performance comparisons between coretypes, workloads, and core scale
10
HARDWARE & INFRASTRUCTURE 2021 State of Cloud HPC Report
50%
40%
43%
33%
20%
2% 1%
6% 0%
0%
1 CSP 2 CSPs 3 CSPs 4+ CSPs
2018 2020
• AI / ML workloads are increasingly common in HPC “Arrival works with Rescale to manage a broad software
portfolio and ensure engineers have access to the right
applications, the latest versions, and the most performant
hardware across each simulation requirement.”
12
SOFTWARE & WORKLOADS 2021 State of Cloud HPC Report
50% 50%
49%
40% 40% 45%
Percent of Organizations
Percent of Organizations
30% 30%
Most organizations run three or more software packages, in While many run any number of software versions, most
many cases to support multidisciplinary simulation efforts. organizations standardize on a small number of versions.
Source: Rescale Cloud HPC Platform Analytics, Note: Excludes In-House Software 14
SOFTWARE & WORKLOADS 2021 State of Cloud HPC Report
Source: Rescale Cloud HPC Platform Analytics, Note: Excludes In-House Software 15
3
2021 State of Cloud HPC Report
Performance Management
Key Findings Spotlight: Denso
16
PERFORMANCE MANAGEMENT 2021 State of Cloud HPC Report
17
PERFORMANCE MANAGEMENT 2021 State of Cloud HPC Report
AWS has an estimated ~50% share of the overall cloud computing market
Performance
and recently made $13B in cloud infrastructure investments in 2020 to fulfill
future growth and demand such as plans for a new region in Osaka. AWS
focus on HPC has been to deliver faster results with flexible configurations
and cost-effective operations.
CSP 1
Recent Developments
CSP 2
CSP 3 • Elastic Fabric Adapter (EFA), a unique OS bypass networking
mechanism that provides a low-latency, low-jitter channel for
AWS inter-machine communications
• Fully managed Lustre offering providing high performance and
scalable storage
Maturity & Scale Value
• Significant choice in architecture options (from high memory, high
storage, to compute-optimized) due maturity of HPC offerings
• Custom silicon with their 64-bit ARM Neoverse Graviton processors
that can deliver up to 40% better price performance
Source: Rescale Cloud HPC Platform Analytics, Using AWS list prices, Workload based on Caravan
benchmark on LS-DYNA
18
PERFORMANCE MANAGEMENT 2021 State of Cloud HPC Report
Azure has an estimated ~26% share of the overall cloud computing market
Performance and recently made $20B in cloud infrastructure investments in 2020 to
fulfill future growth. Recently Azure has announced investments in new
locations including Seoul, Delhi, Doha, Melbourne, and Toronto. Azure’s
focus on HPC has been to deliver optimized performance, cost control,
agile production-class platform, and coordinated partner go-to-market
CSP 1 strategies.
CSP 2
Recent Developments
CSP 3
Azure • Announced HBv2 VMs generally available in 2020, the first in the
public cloud with 200 Gbps HDR InfiniBand from Mellanox
• Released creative solutions for the HPC market such as convertible
Maturity & Scale Value reserved instances model which allows customers to use the latest
and greatest architectures while benefiting from reserve instance
pricing
• Hardware offerings include a wide variety of hardware offerings to
target your specific needs, including CPUs, GPUs, and FPGAs
Source: Rescale Cloud HPC Platform Analytics, Using Azure list prices, Workload based on Oil Rig
benchmark on Ansys Fluent
19
PERFORMANCE MANAGEMENT 2021 State of Cloud HPC Report
Source: Rescale Cloud HPC Platform Analytics, Using Oracle list prices, Workload based on
Molecular Dynamics LJ Liquid 2M on LAMMPS
20
PERFORMANCE MANAGEMENT 2021 State of Cloud HPC Report
Google Cloud Platform (GCP) estimates serving ~10% share of the overall cloud
computing market and recently made $26B in cloud infrastructure investments
Google Sets Six Large Scale Training Performance Records in MLPerf v0.7
in 2020 to fulfill growth. New regions planned include Israel, Mexico, and
Higher is better; results are normalized to fastest non-Google submission
Spain. Google Cloud’s flexible and scalable offerings help accelerate
3 time to completion, so organizations can convert ideas into discoveries
and inspirations into products. GCP’s focus on HPC market involves taking
advantage of open source, going to market with partners, and introducing
2 advanced technologies especially in the machine learning area with GPUs &
TPUs investments.
1
Recent Developments
Business Management
Key Findings Spotlight: Aerion
23
BUSINESS MANAGEMENT 2021 State of Cloud HPC Report
75% 15.0
prevent costly overruns.
50% 10.0
Separate and specific budgets
at a BU, team, project and user
25% 5.0 level help organizations adopt an
efficient cloud OPEX model.
0% 0.0
SMB Commercial Enterprise Organizations with financial
% Using Cost Controls # of Budgets Set
transparency and control
can more accurately forecast
resource needs and assess ROI.
80%
Organizations seek to
standardize optimal hardware
and software combinations for
60%
64% each specific workload.
52%
40%
45%
50% Larger organizations with more
40% HPC groups show increased
35%
20%
usage of hardware and software
policies.
0%
SMB Commercial Enterprise HPC managers are identifying new,
more performant configurations
Using Software Policies Using Hardware Policies
and setting policies to standardize
Company Size (SMB = 1 - 2 Groups, Commercial = 3 - 5 Groups, Enterprise = 6+ Groups) best practices.
Commercial and Enterprise organizations require Organizations with collaboration and multiple shared
additional security measures beyond the standard workspaces implement a high level of security to
cloud provider shared responsibility security model. protect intellectual property and user data.
26
5
2021 State of Cloud HPC Report
Practitioner Sentiment
Key Findings Spotlight: UWE Bristol
• HPC managers and users both predict aggressive Rescale modernized our School of Engineering’s HPC
capabilities with a control plane for hybrid and multi-cloud
shift to cloud in the next five years infrastructure to accelerate our simulations on one easy-to-
use platform.”
27
PRACTITIONER SENTIMENT 2021 State of Cloud HPC Report
50%
39% 39%
36% holistic R&D solutions such as
30% 33%
workflow automation is tempered
20%
25% by the HPC talent gap.
10%
To do more with less, many IT/HPC
leaders are looking to cloud HPC
0%
Increase
Capacity
Find More
Talent
Automate
Workflows
Upgrade
Infrastructure
Improve
Reporting
Migrate to the
Cloud
Reduce Overall
cost
management and automation.
Investment Category
50%
40%
42%
Percent of Respondents
0% 0%
Almost Entirely Mostly Equal Parts Mostly in the Almost Entirely in
On-Premises On-Premises On-Premises Cloud the Cloud
Predictions
50%
39%
30%
workloads to cloud.
0%
Cost Complexity Security No Concern Maintenance / Usability
Management
Cloud Concerns
About Rescale
Rescale helps organizations accelerate science and engineering Learn how you can modernize your high performance computing:
breakthroughs by eliminating computing complexity. From
supersonic jets to personalized medicine, industry leaders bring new
product innovations to market with unprecedented agility, speed,
and efficiency with the Rescale Platform - an intelligent full-stack Get in touch Start your View
automation HPC solution for digital R&D in the cloud. IT leaders use with our team free trial additional
Rescale to deliver high-performance computing-as-a-service to their of experts today resources
organization by harnessing the power of automation on a hybrid cloud
control plane with security, architecture, and financial controls.
www.rescale.com
32