Increase Application

Performance and
Reduce Costs with
Memory-Based Storage
Tom Fenton

CONTENTS IN THIS PAPER

Introduction  2 This tech brief details how Formulus Black’s ground-breaking soft-
ware stack enables any workload to run in memory, without modifi-
Early Attempts at Improving
cation, and helps companies find new ways to speed up storage access


Performance for Data-Intensive
Workloads 2 and harness the full power of today’s processors.


Bringing Storage to the
Memory Channel 3


Forsa Memory-Based
Block-Level Device 3


Use Cases for Memory-Based
Storage Devices 4

Real Customers, Real Advantages  5

Taking Storage to the Next Level 5

I NCREAS E A P P L ICATI O N P E R FO R M A N CE A N D RE D UCE COSTS W IT H ME MORY-BAS E D STORAG E 1

INTRODUCTION EARLY ATTEMPTS AT IMPROVING
Artificial intelligence (AI), machine learning (ML), trans-
PERFORMANCE FOR DATA-INTENSIVE
actional and analytical databases, high-frequency trading WORKLOADS
(HFT), fraud detection, and other business-critical appli- Application and hardware vendors have come up with var-
cations need to access and process large amounts of data ious ways to deal with performance issues, ranging from
as quickly as possible. Unfortunately, today’s traditional using specialized hardware to creating new programs and
computer architecture and storage methods strangle such methodologies. The most exciting of these solutions is the
applications because they can’t process the information use of memory (either dynamic random-access memory
quickly enough. This can have detrimental consequences. [DRAM] or persistent memory [PMEM]) as the storage
media for hyperconverged infrastructure, also referred to
as Memory Converged Infrastructure (MCI). MCI, which
Timely access to processed relies on DIMM-based memory hardware and offers
data has a significant impact on memory bus speeds, bypasses the traditional (and much

how customers and end users slower) peripheral storage travel path to reach the CPU.
As an example, DRAM memory has a latency of about 51
perceive an organization. nanoseconds for reads and writes, PCIe NAND-Flash has a
latency of 47 microseconds for reads and 15 microseconds
In brokerage houses that do HFT, for example, a nanosec-
for writes, while traditional enterprise disk storage has a
ond of latency can mean the difference between making a
latency of 9 milliseconds for reads and writes. Relative to
profitable trade or not. Similarly, fraud detection applica-
non-memory channel storage technologies, throughput
tions rely heavily on data processing speed, as they must be
and bandwidth of even the highest-performing SSD is
able to comb through mountains of data to detect anomalies
several orders of magnitude lower.
that might indicate the misuse of a credit card or a fraud-
ulent credit application. Yet another study shows how data
Relative to non-memory channel
analysis and detection need to be extremely performant as
bot-driven ad-fraud can occur in less than a millisecond.
storage technologies, throughput
and bandwidth of even the
Timely access to processed data has a significant impact
on how customers and end users perceive an organization.
highest-performing SSD is several
About 10 years ago, we expected inventory stock reports to orders of magnitude lower.
be generated weekly; five years ago, the same reports were
Formulus Black wanted to see just what difference changing
generated nightly. Now, users expect to see the inventory
from NVMe to memory-based storage would have on da-
in stock in real-time, and executives want to be able to run
tabase performance. In one test the company found that a
ad hoc queries against internal databases and data ware-
database query that took 200 seconds on NVMe took only 2
houses instantaneously.
seconds using memory-based storage; a 100x improvement.
Application and hardware providers have come up with
An early example of MCI usage—and a prime example of
imaginative ways to try and satisfy our need for speed, but
the hazards involved with specialized hardware to accom-
new bottlenecks appear with every solution. Currently, one
plish MCI—is Fusion-io, an innovative startup company
of the biggest problems is that data in secondary storage
that designed and manufactured proprietary products that
can’t be processed quickly enough. Even non-volatile
used flash memory technology to increase the performance
memory express (NVMe) drives, which connect directly to
of servers. For a while, it was the darling of the data center;
the PCIe bus and are an order of magnitude quicker than
in the early 2010s, its ioDrive product was considered to be
solid-state drive (SSD) and hard-disk drive (HDD) devices,
one of the fastest (albeit most expensive) storage devices
still throttle mission-critical applications.
on the market.

I NCREAS E A P P L ICATI O N P E R FO R M A N CE A N D RE D UCE COSTS W IT H ME MORY-BAS E D STORAG E 2

Recently, other more established companies, such as stor- As the number of cores per CPU increased, the price of RAM
age goliath NetApp, have built MCI products, such as MAX dropped considerably, as well. By capitalizing on this cost
Data, that can increase application performance. MAX Data reduction, companies can now afford to equip their servers
uses a server’s DRAM; a non-volatile dual in-line memory with more memory than ever before. Additionally, in 2019
module (NVDIMM); or storage-class memory (SCM), such Intel released SCM devices based on XPoint memory media
as Intel Optane DCPMM, to create a high-performance that, when compared to DRAM, are less expensive, support
auto-tiering filesystem that can survive a system reboot. more capacity, and let data survive a reboot.
However, MAX Data has some limitations:
What was needed was a way to take advantage of the in-
• It has a highly restrictive hardware compatibility list creased cores and memory on commodity x86 servers to
(HCL) for the servers on which it can array improve application performance—without having to rely
on propriety hardware or software, and without having to
• It requires a NetApp array
rearchitect applications. One of the companies leading the
• It works only with NetApp ONTAP way in this transformation is Formulus Black.

• It requires a separate license

FORSA MEMORY-BASED
• Uses MAXFS—a NetApp proprietary file system BLOCK-LEVEL DEVICE
Formulus Black’s Forsa overcomes many of the limitations
Although some companies sought to improve application
of other MCI solutions by utilizing DRAM or SCM to unleash
performance through proprietary hardware, other vendors
the power of modern servers in the data center.
found innovative ways to improve performance by having
applications directly use the memory on a system. Perhaps Forsa takes a portion of the DRAM or SCM on a system
the best example of this is SAP HANA, an in-memory and creates Portable Operating System Interface (POSIX)-
relational database system proven to be very performant. compliant block devices—called Logical Extensions of
However, it requires rearchitecting of applications and mi- Memory (LEM)—that’s used like any other block device on
gration of existing data. Like the hardware approaches noted which you’d mount a filesystem such as HDFS, XFS, EXT3,
earlier, it’s expensive and also suffers from vendor lock-in. and so forth.

BRINGING STORAGE TO THE Formulus Black’s Forsa overcomes

MEMORY CHANNEL many of the limitations of other
Over the past decade, multiple factors have come into play MCI solutions by utilizing DRAM
that have radically changed the data center. For example,
CPUs have seen an explosion in the number of cores,
or SCM to unleash the power of
DRAM prices have dramatically dropped, and SCM has modern servers in the data center.
become available.
Forsa’s LEM is extremely robust and flexible, and can be
Furthermore, 16-core CPUs have become the most popular
resized on the fly if needed. Memory-based storage devices
servers in the data center, 32-core CPUs are no longer
behave just like any other block-level device (only with
considered exotic, and 64-core CPUs are available. The
much less latency): They can be cloned, snapshotted, and
proliferation of cores within servers means that data can
backed up using existing tools and methodologies. Forsa
now be processed quicker than ever. The problem is that
is currently supported on various Linux distributions, and
many companies that have shelled out for these processors
doesn’t require any propriety hardware.
find those cores sitting idle, as they can’t get data from
storage fast enough. And no organization can afford to You can migrate your applications or data to a memory-
waste money like that. based block-level device and enjoy the benefits of having

I NCREAS E A P P L ICATI O N P E R FO R M A N CE A N D RE D UCE COSTS W IT H ME MORY-BAS E D STORAG E 3

 will survive reboots. The point is that Forsa puts the choice
Application KVM in your hands—not a specific vendor’s.

Storage Service
Layer USE CASES FOR MEMORY-BASED
Virtual Block Device Provisioning
(Linux BDEV) STORAGE DEVICES
MCI solutions, such as Forsa, have some very interesting

Core Algorithm Engine use cases, and companies both big and small have exploit-
(I/O, Resource Management, Statistics, etc.) ed them to maximize their data center’s value.
Core Algorithm
& Device Driver Databases seem to be the first application IT managers
I/O Memory Driver (MMU) think of when considering an MCI solution, and many have
seen dramatic improvements in database performance
System Memory Memory Allocated for Forsa after moving them to Forsa. What surprised one IT man-
ager was that they were able to reduce costs; specifically,
Server Memory (DRAM or Persistent Memory) they reduced their database licensing cost by moving their
database from a 16-CPU license to a 4-CPU license—and
still had better performance.
your data processed at far greater speeds than if your data
was kept in traditional storage. Edge computing is becoming
By using Forsa, you don’t need to rearchitect your applica- increasingly essential to enterprises,
tions, except to put the data or program on the memory- and it’s often more important to be
based block storage device. Even though the price of DRAM able to process data at the edge
and PMEM has dropped dramatically, it’s still a relatively
than in the data center.
scarce resource that Forsa maximizes using Formulus Bit
Markers to expand memory capacity by eliminating dupli-
Enterprises are embracing the software defined data
cate copies of data.
center and new applications often run on virtual machines
One of the limitations of some of the early solutions that (VMs) or in containers. Forsa’s LEM supports both KVM-
attempted to create a memory-based storage device con- based VMs and Docker applications. These technologies
cerned data protection. Forsa overcame this issue with benefit from the same types of performance gains as non-
BLINK, which takes an application-consistent memory virtualized technologies and Formulus Black was able to
state capture and copies it to the SSD, allowing for recovery document impressive TPS/TPM figures on small (8 vCPUs/
should the need arise. BLINK even interfaces with UPS bat- 4GB RAM) and medium (32 vCPUs/ 16GB RAM) sized KVM
tery backup software, allowing you to back up data stored VMs running under Ubuntu.
in DRAM to persistent storage, such as an SSD, when a
Not all Forsa uses cases reside in the data center, as more
power failure occurs.
often we’re seeing servers running outside of the data
Closely related to data protection is data integrity, which center collecting information from a variety of sources:
Forsa supports with bad block replacement (BBR) to ensure video cameras, radar, high-definition microphones, and
that the data stored on the LEM is error-free. a multitude of sensors. This creates a torrent of data that
information must be extracted from locally or passed on
Because Forsa works with commodity (rather than pro-
to another application or end user who wants to analyze or
prietary) hardware, you can choose the right hardware
build reports off that data. Forsa can dramatically improve
for your data center. Some companies may choose DRAM
the efficiency and performance of these edge servers.
because it performs better than SCM, while others may
prefer SCM as it supports greater capacity, costs less, and

I NCREAS E A P P L ICATI O N P E R FO R M A N CE A N D RE D UCE COSTS W IT H ME MORY-BAS E D STORAG E 4

The consequences can be much worse with other VMs for its document-rendering application, etherFAX
edge-computing scenarios, such as autonomous vehicles effectively reduced the time needed to render documents
or edge devices used in healthcare settings. Poor latency by more than 70%—without having to rewrite code or
here can have more serious and impactful affects. rearchitect its application.

Forsa’s benefits in these situations become more obvious. Formulus Black sells Forsa as a software package that can
Formulus Black has documented actual use cases in which be licensed either on a per-socket basis, or, for users who
Forsa has been able to solve difficult problems or increase prefer the appliance model, through selected resellers that
the effectiveness of IT resources. sell systems with it preinstalled on reference architecture.

The appliance model appeals to users who want a single SKU

REAL CUSTOMERS, REAL ADVANTAGES
and support model, as well as to those who have existing
Packet is a bare-metal cloud provider with more than 20
hardware or whose preferred vendor doesn’t yet offer Forsa.
locations around the world that’s used by developers,
SaaS companies, and others who need to deploy physical Additionally, many vendors offer Forsa using DRAM for
infrastructure on a global scale. With support from Intel’s those who want the most performant block storage device
Non-Volatile Memory Solutions Group (NSG), Packet has or a PMEM solution, like Intel Optane DC persistent memory
teamed up with Formulus Black to use Forsa to create for those who want more memory-based storage capacity at
ultra-low latency, high-bandwidth solutions using Intel a lower cost relative to DRAM, and who want the added peace
Optane DC PMM. This makes it easier for developers and of mind that comes with using nonvolatile memory. Finally,
IT leaders to adopt persistent memory solutions and su- those who prefer using cloud services can use a provider like
percharge the performance of databases, ERP systems, and Packet to provision Forsa’s memory-based storage.
custom-built data-intensive applications without having
to use exotic software or expensive proprietary solutions. TAKING STORAGE TO THE NEXT LEVEL
It’s becoming increasingly clear that storage has developed
The appliance model appeals to users into the new bottleneck hampering performance for the

who want a single SKU and support most demanding workloads. CPUs are fast enough, with
enough cores. The problem is that the data can’t be served
model, as well as to those who have
up quickly enough to satisfy the application’s needs.
existing hardware or whose preferred
This issue will only continue to get worse, as digital transfor-
vendor doesn’t yet offer Forsa.
mation upends previous data center paradigms, forcing com-
Because Forsa runs on commodity x86 hardware and panies to find new ways to speed up storage access and harness
doesn’t require rearchitecting of applications, custom the full power of today’s processors. Formulus Black’s Forsa is
solutions that have been developed in-house, and which a ground-breaking software stack that enables any workload
would be difficult and costly to rewrite or port to other sys- to run in memory, without modification. Unlike legacy stor-
tems or the cloud, have seen substantial performance gains age solutions, Forsa enables memory to be provisioned and
by running on Forsa. A prime example of this is etherFAX. managed as a high-performance, low-latency storage media
for AI, ML, transactional and analytical databases, HFT, and
Many people think that faxed documents have been com-
other demanding operations.
pletely replaced by e-mail, but the reality is that heavily
regulated institutions such as healthcare, financial firms, If that’s what you need, your solution has arrived. Visit
and the education sector, often need to fax information to formulusblack.com/see-it-work/ to request a free trial.
stay legally compliant.

etherFAX fills this niche and transmits millions of pages

a day on behalf of these institutions. By using Forsa to
provision high-performance memory-based storage and

I NCREAS E A P P L ICATI O N P E R FO R M A N CE A N D RE D UCE COSTS W IT H ME MORY-BAS E D STORAG E 5