Cloud Computing with FPGAs

EENG 428
ENAS 968

bit.ly/cloudfpga
 Course Introduction

Prof. Jakub Szefer

Dept. of Electrical Engineering
Yale University

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

2
bit.ly/cloudfpga © Jakub Szefer
Welcome to Cloud Computing with FPGAs

The course: This is a project-based course which focuses on Field Programmable Gate
Arrays (FPGAs) and their use in Cloud Computing
The goals: Main goal of the course is to teach students about FPGAs, their use in cloud
computing, security of FPGAs, as well as give a chance to work on novel, open-ended
research projects for the class
The staff: course staff for Fall 2023 are:

Instructor:
Prof. Jakub Szefer

Contact e-mail:
[email protected]

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

3
bit.ly/cloudfpga © Jakub Szefer
Course Resources

Class web sites:

• Web page: https://bit.ly/cloudfpga
• Canvas site: https://yale.instructure.com/courses/88438

Class materials:
• Class slides posted online
• Two books used during the semester:

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

4
bit.ly/cloudfpga © Jakub Szefer
Course Timeline: Projects

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

5
bit.ly/cloudfpga © Jakub Szefer
Course Timeline: Readings, Verilog

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

6
bit.ly/cloudfpga © Jakub Szefer
Course Timeline: Readings, Cloud FPGAs

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

7
bit.ly/cloudfpga © Jakub Szefer
Questions?

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

8
bit.ly/cloudfpga © Jakub Szefer
 Cloud Computing and Cloud-based FPGAs

Prof. Jakub Szefer

Dept. of Electrical Engineering
Yale University

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

9
bit.ly/cloudfpga © Jakub Szefer
 A Short Introduction to Cloud Computing

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

10
bit.ly/cloudfpga © Jakub Szefer
Cloud Computing

“Cloud computing is a model for enabling ubiquitous, convenient, on-demand network

access to a shared pool of configurable computing resources (e.g., networks, servers,
storage, applications, and services) that can be rapidly provisioned and released with
minimal management effort or service provider interaction.”
National Institute of Standardsand Technology (NIST),
“The NIST Definition of Cloud Computing”, Special Publication 800-145, Sept. 2011

• Users or devices use compute (storage and network)

resources located remotely to off-load local computation
• Need network connection, latency of connection
is sometimes important
• Remote resources are often more powerful than
local compute resources
• Backed up and %99.9999… available

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

11
bit.ly/cloudfpga © Jakub Szefer
Cloud Computing

• Compute resources in the cloud include: CPUs, Storage, GPUs, and now FPGAs
• The resources are provided as (from NIST’s definition):
• Software as a Service (SaaS) – The capability provided to the consumer is to use the provider’s
applications running on a cloud infrastructure
• Platform as a Service (PaaS ) – The capability provided to the consumer
is to deploy onto the cloud infrastructure consumer-created
or acquired applications created using programming
languages, libraries, services, and tools supported
by the provider.
• Infrastructure as a Service (IaaS ) – The capability
provided to the consumer is to provision processing,
storage, networks, and other fundamental computing
resources where the consumer is able to deploy
and run arbitrary software
• Cloud FPGA can be considered PaaS or IaaS, CPU GPU FPGA Storage Memory
focus on his course is on IaaS or FaaS: FPGA as a Service

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

12
bit.ly/cloudfpga © Jakub Szefer
Cloud Computing and Virtual Machines

In the PaaS model, users have access to Virtual Machines (VM):

• VM is a logical abstraction of a physical computer server
• Many VMs share one physical machine
• Physical resources are partitioned among the VMs on the machine, e.g., different users pay for
different types of VMs and get different amount of resources
• Access to the VM is through remote connection, e.g., SSH or remote desktop

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

13
bit.ly/cloudfpga © Jakub Szefer
Cloud Computing Deployment Models

• Deployment models defined by NIST:

• Private cloud – The cloud infrastructure is provisioned for exclusive use by a single organization
comprising multiple consumers (e.g., business units)
• Community cloud – The cloud infrastructure is provisioned for exclusive use by a specific
community of consumers from organizations that have shared concerns
Nimbix
• Public cloud – The cloud infrastructure is provisioned Xilinx
for open use by the general public OVH
Intel (Altera)
• Hybrid cloud – The cloud infrastructure is
a composition of two or more distinct cloud
Alibaba Cloud
infrastructures (private, community, or public) Baidu Cloud
Intel (Altera)
Xilinx
that remain unique entities, but are bound
together by standardized or proprietary
Huawei Cloud Alibaba Cloud
technology that enables data Xilinx Xilinx
and application portability Amazon AWS VMAccel
Xilinx Xilinx & Intel
• Focus on this course is public clouds, but same Microsoft Azure Tencent Cloud
tools and ideas apply to all types of clouds Intel (Altera) TACC Xilinx
Intel (Altera)
Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs
14
bit.ly/cloudfpga © Jakub Szefer
 FPGAs in Cloud Computing

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

15
bit.ly/cloudfpga © Jakub Szefer
Accelerating Computing in the Cloud with FPGAs

• Within last 6 to 7 years, there has been a significant increase

in the interest in, and deployment of, Field Programmable
Gate Arrays (FPGAs) in the cloud Video Processing

Genomics Analysis
• The FPGAs in cloud servers can be used to off-load and
accelerate high-performance computations, such as
video processing, genomics analysis, or machine learning
• They provide flexibility to load custom hardware
designed by users, or provided cloud providers,
or by 3rd party vendors Machine Learning

• However, sharing of resources within the cloud infrastructure

and the FPGAs themselves can lead to information leaks
• Security of FPGAs in the cloud needs to be considered
alongside performance and other metrics

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

16
bit.ly/cloudfpga © Jakub Szefer
Cloud FPGAs

Cloud FPGA describes a cloud computing infrastructures which allow users to access
FPGA resources in the cloud, on-demand, often proving whole tool-stack for
development and deployment of their hardware designs.
• Bare-metal Cloud FPGA – access FPGA resources directly, closest to IaaS model,
can program FPGA with (almost) any hardware design users wishes
• Accelerator Cloud FPGA – access FPGA resources through some framework,
closest to PaaS model, program in high-level language, e.g. use HLS, less control
over actual FPGA hardware

Components of Cloud FPGA:

• Servers with FPGA cards
• Development VMs (for writing code, simulation, testing, debug)
Focus of EENG 428
• Deployment VMs (which interact with actual FPGAs)
• Other cloud services, e.g., storage
• Other remote users and data center operator – sources of potential threats
Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs
17
bit.ly/cloudfpga © Jakub Szefer
FPGAs in the Cloud Datacenters

• Public Cloud FPGA is the computing paradigm where FPGAs

are made available to the users remotely
Cloud Provider
• On-demand, pay-as-you-go service
• Ability to quickly provision large
number of FPGAs
FPGA

PCIe

Server
FPGA boards
Share: Cloud FPGA Users
EENG 428 / ENAS 968 – Cloud Computing with FPGAs
18
bit.ly/cloudfpga © Jakub Szefer
Infrastructure of Cloud FPGAs

• Single-tenant Cloud FPGAs allow only one user to use each FPGA at a time
• Multi-tenant Cloud FPGAs allow multiple users to share an FPGA at the same time

Within a serer, FPGAs Within a server, FPGAs

share the PCIe bus share power, cooling, etc.

PCIe PCIe

Server Server
Different servers share Resources
same data center can be
infrastructure: shared
networking, power, within the
cooling, etc. FPGA FPGA boards FPGA boards

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

19
bit.ly/cloudfpga © Jakub Szefer
Pros and Cons of Cloud FPGAs

Advantages:
• On-demand access to FPGA boards
• No need to buy hardware, licenses, setup software
• Access to high-end FPGAs
• UltraScale+ development board is about 7,000$
• Rent UltraScale+ from Amazon F1 for 1.65$ per hour, includes all license fees, electricity, etc.
• No need to worry about system issues (Linux, PCIe drivers, C and Python support)
Disadvantages:
• Pay-per-use, with long-term use, cheaper to buy your own hardware
• Vendor lock-in, design may be specific to one FPGA offered by one cloud provider
• But most providers use similar FPGAs and similar setups
• No physical access, or control, over FPGAs
• Security issues due to other users
Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs
20
bit.ly/cloudfpga © Jakub Szefer
 Potential and Outlook for Cloud FPGAs

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

21
bit.ly/cloudfpga © Jakub Szefer
Potential for Cloud FPGAs

FPGA technology is not new, but keeps advancing in recent years,

with a growing market potential
• FPGA designs are typically faster than software
• Lower-cost alternative to making ASIC chips
• Increasing trend to heterogeneous
computing: CPUs, GPUs, and now FPGAs
• Biggest recent interest in leveraging FPGAs
for machine learning, as accelerators for inference
Cloud FPGAs lower barrier to entry for people
and companies wanting to make their own hardware
• On-demand access, eliminating up-front costs (have to be careful about long-term costs)
• Cloud providers already have other established services
Potential for Cloud FPGAs
• Cloud FPGAs are a new and developing field
• Companies using FPGAs: Amazon, Microsoft, Huawei, … all need engineers to make designs
MarketResearch.com Academic
Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs Source: The 2019-2024 World Outlook for Field-Programmable
Gate Arrays (FPGA); Icon Group International, Inc. 22
bit.ly/cloudfpga © Jakub Szefer
Outlook for FPGAs and Cloud Computing

Insights from Gartner, a global research and advisory firm:

• Cloud computing is reaching plateau

of productivity in Gartner’s hype cycle,
while FPGA accelerators have
still few years go to
• FPGAs can deliver extreme performance
and power efficiency for a growing
number of workloads and are
well-suited for AI inference workloads
• Programmability continues
to be a major challenge, limiting
broader adoption of FPGAs
• Market penetration of FPGAs:
less than 5%, much room for growth
(and jobs)

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

23
bit.ly/cloudfpga © Jakub Szefer
Questions?

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

24
bit.ly/cloudfpga © Jakub Szefer
 A Step Back: How FPGAs Work

Prof. Jakub Szefer

Dept. of Electrical Engineering
Yale University

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

25
bit.ly/cloudfpga © Jakub Szefer
 Overview of Working with FPGAs

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

26
bit.ly/cloudfpga © Jakub Szefer
What is an FPGA?

FPGA is the Field-Programmable Gate Array

• Integrated circuit made (mostly) of digital logic elements
• Flip-flops, simple logic gates, and memories (SRAM)
• Can have bigger elements such as DSPs (Digital Signal Processing elements)
• Can contain “Hard IP” elements, such as processor cores
• Additional elements: I/O, programming logic, unique device ID (OTP fuse or PUFs), etc.
• Gate Array: basic elements are arranged as an array of unconnected elements
• Before programming FPGA does not realize any particular digital logic function
• Connection and function of basic elements is not specified
• Field-Programmable: function of elements and interconnections is specified during
“programming” of the FPGA, in the field
• Fast prototyping of hardware circuits
• Low-cost alternative to making ASIC chip (but not lower power)
• Deployed in network routers, satellites, and now cloud computing data centers

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

27
bit.ly/cloudfpga © Jakub Szefer
Typical Textbook-Style FPGA Architecture

I/O: input/output cells

CLB: configurable logic block
SB: switch box
LUT: look-up table
FF: flip-flop
M: memory (SRAM)

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

28
bit.ly/cloudfpga © Jakub Szefer
Typical Column-Style FPGA Architecture

I/O connections

CLBs

Memory blocks

DSP blocks

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

29
bit.ly/cloudfpga © Jakub Szefer
Xilinx UltraScale Example

Xilinx UltraScale architecture:

• A CLB consists of 8 BLEs
• Each BLE has 2 LUTs and 2 FFs CK: clock
SR: set/reset
• 2 FFs share the same CK and SR signal, CE: clock enable
but CE signal can be different
• 2 LUTs can be used as a 6-input LUT
or 2 smaller LUTs with distinct inputs
no more than 5

Adapted from Li, et al. [2]

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

30
bit.ly/cloudfpga © Jakub Szefer
Xilinx UltraScale Example

Clock Region: dedicated circuits for carrying clock signals, e.g.,

to reduce clock skew; UltraScale has 24 clocks per region,
different regions can have different set of 24 clocks
Adapted from Chen, et al. [1]

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

31
bit.ly/cloudfpga © Jakub Szefer
“Programming” an FPGA

• FPGA is the Field-Programmable Gate Array,

it needs to be “programmed” to actually
perform useful operation
• The hardware designs are first written or described
in a Hardware Description Language (HDL)

Figure from Bacon, et al. [4]

• Verilog
• VHDL
• Also, can use High-Level Synthesis (HLS) with languages
such as C
• The design is then “synthesized” or “compiled” into
a bitstream that can be loaded or “programmed” onto
the FPGA chip
• Many steps (and tools) are used for the whole process,
example for Xilinx is on the right

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

32
bit.ly/cloudfpga © Jakub Szefer
Synthesis and Mapping to CLBs

• Example of synthesis of simple Verilog statement

and mapping to 2-input LUTs
Adapted from Chen, et al. [1]

assign out = a && b && c && d

Synthesis
A
2-Input
LUT 1
B
a 2-Input
O
Map LUT 3
b out
2-Input
c C LUT 2
d
D

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

33
bit.ly/cloudfpga © Jakub Szefer
Computing Contents of LUTs

• Given a Boolean expression, it is converted into a truth table

• The function described by the truth table (and thus the Boolean expression)
is realized by a decoder which is configured with the desired outputs from the truth table

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

34
bit.ly/cloudfpga © Jakub Szefer
Place and Route

• Example of place and route of mapped example

onto physical CLBs in the FPGA fabric
Adapted from Chen, et al. [1]

B 2
A

C 3 2-Input
LUT 1
B
2-Input
LUT 3 O
D 1
2-Input
C LUT 2

O
D

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

35
bit.ly/cloudfpga © Jakub Szefer
Typical Routing in FPGAs

Programmable
routing switch

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

36
bit.ly/cloudfpga © Jakub Szefer
Routing and Switch Boxes

• Switch boxes have wires which are connected with transistors, each can be
configured on or off:
• On: two wires are electrically connected
• Off: two wires are not connected

• Typically use SRAM cells to control

the transistors’ gates:
• Store logical 1: gate on
• Store logical 0: gate off

• Volatile, loose contents when power

is turned off
• High-power, SRAM dissipate power
to keep the transistors on or off

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

37
bit.ly/cloudfpga © Jakub Szefer
FPGA Bitstream

• Final bitstream are all the configuration bits used to specify the functionality of the FPGA
and the design that it realizes
• LUT entries are bits stored in SRAM
• Switch box configuration are bits stored in SRAM
• Memory (RAM) can be pre-initialized
• DSP and other components have configuration bits

...1010011100110000011100011011000001101001010111000110000111...

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

38
bit.ly/cloudfpga © Jakub Szefer
Storage of Bitstreams in FPGAs

The bistream needs to be stored in FPGA to maintain the configuration

SRAM-based
• Volatile memory But FPGA board can contain non-volatile
memory and re-program the FPGA chip
• Can be reprogrammed multiple times
on boot up
• Xilinx, Intel (formerly Altera), Lattice
Antifuse-based
• Non-volatile memory
• Can only be programmed once
• Microsemi
Flash-based
• non-volatile memory
• can be reprogrammed multiple times
• Microsemi, Lattice

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs Source: D. Göhringer, ACACES
39
bit.ly/cloudfpga © Jakub Szefer Summer School 2019
Tools and Languages for Programming FPGAs

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

40
bit.ly/cloudfpga © Jakub Szefer
Hardware Description Languages

Deploying hardware acceleration with FPGAs require users to first write the description of the
design, which is later synthesized to the bitstream
• VHDL (Very High Speed Integrated Circuits Hardware Description Language)
• IEEE Standard Since 1987
• Propagation: Mainly Europe
• Verilog
• IEEE Standard Since 1995
• Propagation: Mainly USA

Most FPGA tools can handle both languages (as well as others)
• Hardware effectiveness depends on the designer, not so much on language used
• Can mix modules written in different languages

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs Source: D. Göhringer, ACACES
41
bit.ly/cloudfpga © Jakub Szefer Summer School 2019
High Level Synthesis (HLS)

An alternative to use of Hardware Description Language (HDL) is use of HLS tools

• Design is specified in a high-level language, such as C, C++, SystemC
• Designers add “annotations” or “pragmas” to help specify which parts of the the design
can run in parallel, or which can be pipelined
• HLS tools convert the design into bitstream (or into HDL and then convert HDL to bitstream)
Many tools exist for HLS:
• Examples for commercial tools: Xilinx Vitis (formerly Xilinx Vivado HLS, former Xilinx
AutoESL), Calypto, CatapultC (formerly Mentor Graphics Catapult C), or ImpulseC
• Examples for open-source tools: ROCCC/ ROCCC 2.0, Trident, LegUp
Advantages:
• No need to know HDL, faster design time,
Disadvantages:
• Maybe not as efficient resulting design, still need to program in “hardware like” manner
Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs Source: D. Göhringer, ACACES
42
bit.ly/cloudfpga © Jakub Szefer Summer School 2019
Comparing FPGA “Programming” Methods

Programming Method Pros Cons

HDL: • Hardware skills necessary High
e.g. VHDL, Verilog, • Very flexible development time
SystemVerilog • Error-prone
• High abstraction level
Model-based Design: • High Costs
• Library with standard components
e.g. Matlab/Simulink • Less flexible
• Rapid Prototyping
• Only for IP-cores, not for a complete
FPGA-Design
HLS Tools: • High abstraction level
• Quality of results (resources,
e.g. Xilinx Vivado HLS, Calypto, • Fast design space exploration
performance) strongly depends on the C-
CatapultC, Impulse C • Rapid Prototyping
code
• Costs
• High costs
IP-Cores • Optimized modules • Source code often not available (only
netlists)

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs Source: D. Göhringer, ACACES
43
bit.ly/cloudfpga © Jakub Szefer Summer School 2019
Major FPGA Vendors and Tools

• Xilinx and Altera (now part of Intel) control majority of the market for FPGAs
• Each maintains its own design software (Vivado for Xilinx and Quartus for Intel)
• Run on Windows and Linux, Mac users need to use VMs or connect to remote server
• Can use any HDL
• Also have tools for HLS
• Other vendors, see reference [5]:
• Microsemi (now defunct)
• Microchip – low-end FPGAs, radiation-tolerant FPGAs, anitfuse FPGAs
• Lattice Semiconductor – low-power SRAM-based
• QuickLogic – extremely low powered, low density SRAM-based FPGAs
• Achronix – asynchronous FPGAs

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

44
bit.ly/cloudfpga © Jakub Szefer
Open-Source FPGA Resources

IceStorm (http://www.clifford.at/icestorm/)
Commercial FPGA architectures
• Tools: Yosys, Arachne-pnr, and IceStorm and bititstream formats are
• Verilog-to-Bitstream flow for iCE40 FPGAs typically industry secrets,
someone has to reverse-engineer
SymbiFlow (https://symbiflow.github.io/) them to make new tools.
• Verilog-to-Bitstream FPGA synthesis flow
• Currently targeting Xilinx 7-Series, Lattice iCE40 and Lattice ECP5 FPGAs

Archipelago (https://github.com/haojunliu/OpenFPGA)
• An open source FPGA chip architecture + related tools

TinyFPGA (https://tinyfpga.com/)
• Open-source FPGA boards (use commercial FPGA chips)

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

45
bit.ly/cloudfpga © Jakub Szefer
Questions?

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

46
bit.ly/cloudfpga © Jakub Szefer
References Used in Slides

1. Chen, Shih-Chun, and Yao-Wen Chang. "FPGA placement and routing." Proceedings of the 36th International Conference on
Computer-Aided Design. IEEE Press, 2017.
2. Li, Wuxi, and David Z. Pan. "A new paradigm for FPGA placement without explicit packing." IEEE Transactions on Computer-
Aided Design of Integrated Circuits and Systems(2018).
3. Li, Wuxi, Shounak Dhar, and David Z. Pan. "UTPlaceF: A routability-driven FPGA placer with physical and congestion aware
packing." IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 37.4 (2017): 869-882.
4. David F. Bacon, Rodric Rabbah, Sunil Shukla. “FPGA Programming for the Masses.” Communications of the ACM, April 2013,
Vol. 56 No. 4, Pages 56-63.
5. Wikipedia contributors. “Field-programmable gate array.” Wikipedia, The Free Encyclopedia. Available at:
https://en.wikipedia.org/w/index.php?title=Field-programmable_gate_array&oldid=909616311.

Share: EENG 428 / ENAS 968 – Cloud Computing with FPGAs

47
bit.ly/cloudfpga © Jakub Szefer