A MongoDB White Paper

How MongoDB Enables

Real-Time Data with
Event-Driven Architecture
October 2019
Table of Contents

Introduction 3

What Has Changed: Operating in Real Time 3

Bottom Line on Real Time: You Need Faster Data 4

The Major Driver of Real Time Today is Microservices 4

Core Building Block of Real Time is an Event 5

What Is Event-Driven Architecture? 5

Use Case Spotlight: Fraud 9

Detection & Remediation 9

Microservices and Event-Driven Architectures: Why MongoDB 12

How MongoDB Enables Event-Driven Architecture 13

Other Use Cases 17

MongoDB Event-Driven Architecture Customer Snapshots 21

Conclusion 22

We Can Help 23

Resources 23
Introduction What Has Changed:
Operating in Real Time
The world moves in real time, and the demands on
individuals, groups, and organizations are constantly Charles W. Bachman designed what is widely
changing. These truths may appear self-evident, but considered the first database management system
many enterprises are still trying to meet their data (DBMS) in 1960, his Integrated Database System. In
needs in ways that don’t match this new reality. To 1970, Edgar F. Codd wrote a series of papers on
keep up with the pace of business, companies must ways to construct databases that coalesced into a
become agile organizations fueled by applications paper titled A Relational Model of Data for Large
that access, analyze, and distribute data in real time. Shared Data Banks,1 and the years that followed saw
Those applications must run on a data fabric that the emergence of the first commercial relational
offers the elasticity to scale up and down as databases. For decades the purpose of databases
needed, efficiently and cost-effectively empowering was simply to collect data, and relational DBMSs did
people with the insights they need to do their jobs what they were designed to do: store structured
better and thus help their companies compete. data, such as simple customer records, that was
easily organized into tabular formats and readily
A traditional request-driven data architecture is, queried with SQL.
conceptually at least, an anti-pattern to this new
imperative. Request-driven architectures, true to Although its origins date back to the 1980s, the
their name, require users to make requests and wait Internet became widely available in the early to mid
until the requested information becomes available. 1990s. It revolutionized the way the world lives,
By contrast, an event-driven architecture proactively works, and plays, and in the process exposed the
makes a stream of data from source systems inherent limitations of traditional system
available in real time. Consuming applications and architectures, especially relational databases: they
services subscribe to topics of interest and consume could not keep up with the sheer amount of data
the data at their own pace. To put it another way, coming at them, and were never designed to
request-driven architectures are system-focused, manage the unstructured data generated from Web,
while event-driven architectures are user-focused, mobile, social and Internet of Things (IoT)
providing timely, contextual insights. applications. So the focus shifted to addressing the
limitations of the RDBMS by expanding to
This paper discusses why event-driven architectures accommodate these new data types and sources,
are the natural evolution of how the world stores thus addressing two of the so-called “three Vs of
and accesses data, shows how MongoDB can help data”: volume and variety. The first NoSQL
your organization establish event-driven databases emerged from places like Google and
architecture, and offers named examples of Amazon in 2004-05. Companies began collecting
customers working with MongoDB to adopt this as much data as they could afford to store, and
model and run their businesses in real time. shortly thereafter the term “big data” appeared,
describing massive datasets that were impossible to
process and manage using traditional data
management tools.

1
Dataversity, A Brief History of Database Management,
available here
3
Somewhere along the way the world figured out of business today, however, is clarifying matters in a
that ‘big data’ could also mean ‘slow data.’ Ironically, hurry. If your organization wants or needs to take
and ominously, while the pace of business was any action and that action is held up waiting for
accelerating, sitting on massive amounts of data data, you need faster data. Prosperity and survival
was slowing organizations down, leaving them less dictate that you can no longer afford to wait on data
equipped to keep up. Simply solving for volume and to arrive. Your people, wherever they are at any
variety left databases with a major blind spot: the moment, need real-time data, and real-time analytic
third of the three Vs, velocity. insights, to do their jobs better. Everything they do
contributes to (or detracts from) customer
So today the focus has shifted again, this time to experience – which is fast becoming the most
address all three Vs through stream processing of important competitive differentiator in the digital
all relevant data, of all structures and from all economy, surpassing price and product as the key
sources, to enable businesses to operate in real differentiator for your brand.2
time. General agreement exists that real time is one
of three data speeds, but definitions vary:

● The most commonly agreed definition is that

The Major Driver of Real Time
real time is the ability to ingest and act on Today is Microservices
data in 100-200 milliseconds; near real time,
anything up to 10 seconds; and batch, The key driver today in making the need for real-
anything over 10 seconds. Some, however, time data an organizational imperative is the
believe the bar for real time is (conceptually) emergence of microservices. Your ability to achieve
higher: 15-20 milliseconds. the benefits of real-time data depends on the ability
● The computer science definition of real time of your development organization to rapidly build
is merely that a given operation is and deploy new business applications by embracing
guaranteed to execute within a defined agile ways of working, such as creating an agile and
duration. DevOps culture, and adopting a microservices
● Beyond one-dimensional benchmarks of architecture.
speed, other considerations enter into the
picture. Chief among them: does a system Microservices architectures break up monolithic
deliver real-time data when it is being applications into small, discrete services or
accessed by a realistic number of functions, typically packaged and isolated within
concurrent users — or, using a convenient containers, which communicate with each other
benchmark favored by some providers, through well defined, network-based APIs. A
when accessed by a single user? microservices design approach creates self-
sufficient sprint teams that are empowered to bring
new capabilities online independently of each other,
Bottom Line on Real Time: then over time evolve and upgrade them without
impacting adjacent microservices. This helps you
You Need Faster Data simplify, migrate, and manage complex data
changes.
While speeds and feeds are important indicators of
the capabilities of a database or platform, these As beneficial as microservices can be, they require
varying definitions have clouded the issue. The pace the ability to work with large volumes of data that

2
Walker Research, Customers 2020: A Progress Report,
cited here and available here
4
change frequently, and that is a challenge many What Is Event-Driven
existing systems cannot meet. Adopting a
microservices architecture relies on a database with
Architecture?
a flexible data model that can support these rapid
As mentioned at the outset, conventional request-
changes, coupled with the scale needed to manage
driven architectures require users to make requests
ever faster and larger data sets.
and wait on the information they need. This often
In short: things happen in real time. Organizations creates a tightly-coupled system wherein one
must now operate in real time. The challenge facing application asks another for some needed
software architects is how to help their companies information, and that application provides it. By
do so. The need has long existed to move data contrast, in an event-driven architecture,
quickly between systems, but the emergence of applications and services publish events in real time
microservices means that the ability to do so has as they become available to a streaming event
never been more crucial than it is right now. And messaging queue. Instead of playing request-and-
“quickly” has now accelerated to real time. wait, other applications services subscribe
asynchronously to event types or topics of interest
and consume them at their own speed. An
Core Building Block of Real application or service publishing events to the
queue is termed a producer, and a subscribing
Time is an Event application or service is termed a consumer.
Although industry diagrams commonly portray them
While various definitions of an event exist, for data as such, there is not a fixed set of “producers” and
management purposes an event has three discrete another fixed set of “consumers;” one of the
characteristics: benefits of event-driven architecture is that it is a
decoupled ecosystem that allows applications and
1. Atomic: something happened, and the event
services to act interchangeably to provide and
is the atomic unit representing what
consume data when and where needed in real time.
occurred.
Indeed, many applications and services may both
2. Related: an event rarely stands alone; it is
produce and consume data.
nearly always part of a stream or sequence
of events.
3. Behavioral: these streams or sequences of
events create an accumulation of facts that
captures behavior.

The way, then, to enable companies to operate in

real time is to help them capture, manage, and act
on events as they occur. This is not to say that event
management is merely reactive; events also
become parts of broader sequences and patterns
that are indicators of other things that are about to
occur. The smart organization that captures and acts
on events can predict and proactively prepare for
what is to come. A cohesive model has emerged to
enable organizations to best leverage events: event-
driven architecture.
5
Elements of Event-Driven Architecture
Event-driven architecture requires a great many complex data movements and interactions, but in its
simplest form, it can be broken down as shown in Table 1:

Element What it Does

Event Producers Event producers generate events that place event messages into a
message queue or stream. Events can capture a myriad of occurrences
that are virtually limitless and are defined by the application:
● Purchases
● GPS positioning coordinates
● Periodic sensor readings such as current temperature
● A user sending a tweet with a specific hashtag
● User actions on a site, captured as a log entry for each click

Event Types and Topics The system shepherds events into logical streams called event types or
topics, and these are partitioned to support scale-out functionality.

Event Consumers Event consumers listen for, and react to, events appended to these
topics. A given application can and often does fulfill the role of both
producer and consumer.

Decoupled architecture A decoupled architecture is a framework for complex work that allows
components to remain completely autonomous and unaware of each
other. A decoupled system allows changes to be made to any one
system without having an effect on any other system. Producers and
consumers are decoupled from each other. This removes dependencies
between components, which creates a number of advantages, including:
● Making the system elastic so it can grow and shrink in response
to changing demand.
● Allowing each service to evolve independently, such as adding
new functionality without impacting the rest of an application.
That includes the ability to scale different microservices
independently of each other.
● Making it easy and risk-free to add new producer and (especially)
consumer apps.
● Enabling consuming apps to go offline – for example, overnight –
then reconnect and replay what they have missed, with no
additional coding or load on the producing apps.

Microservices A microservices architecture breaks up applications into discrete

services or functions, typically isolated within containers. Microservices
communicate with each other through well defined, network-based APIs.

Event streaming platform3 Each microservice will typically read and write directly to its own
database, but the event streaming platform is the glue that binds
microservices together, providing the channels that allow events to pass
between microservices and connecting microservices into complete

3 Technologists and market observers use various terms to describe this entity, but what is important is the
functionality it brings to the event-driven architecture equation
6
 Element What it Does

applications. Applications can query streams of data in-flight, then persist

both raw events and analytics into a database to make them available to
other apps.

Database In most microservices architectures, each microservice has its own

database, but the centralized database is the foundation for event-driven
architecture; this database:
● Aggregates and persists events.
● Enriches event streams with data from other sources, including
historical data.
● Provides a central repository for multiple event streams, enabling
applications and users to benefit from the data across all
microservices.
● Serves as a data source for operational and analytical apps.
● Provides a unified view of state across the business, without
having to replay and join multiple independent streams.

Table 1: Elements of Event-Driven Architecture

Event-driven architectures deliver significant value and microservices are a pattern based on The
to the business. There is tremendous pressure for Reactive Manifesto,4 which defines a reactive
applications to immediately react to changes as they system as one that is message-driven, remains
occur. Capturing and acting on events in real time responsive even as you elastically scale to meet
enables systems to react automatically and varying workload demands, and is resilient in the
immediately to events. Competitive tracking gives a face of failures.
company the opportunity to rapidly position itself to
outflank competitors. Detecting operational errors Detailed examples follow in this paper, but a quick
immediately lets the company minimize fallout and real-world example of event-driven architecture at
take quick corrective action. These benefits work in retail would be an e-commerce site. As
translate into not only operational excellence and shoppers place items into their carts and check out,
cost savings but also enhanced customer these events create a stream that invokes
experience as the company optimizes customer- microservices managing inventory, promotions &
facing processes. More broadly, an event-driven coupons, credit check, fraud detection, shipping,
architecture helps the organization improve post-purchase tracking and follow-up, invoicing,
business agility. marketing list building, retargeting, and more.

Event-driven architecture is innovative, but its Figure 1 shows event-driven architecture at work in
foundational principles are well established. an e-commerce application, where microservices
Microservices are the realization of the vision set are managing a range of functions such as those
forth in 2005 in Service-Oriented Architecture shown here.
(SOA), with its decoupled ecosystem of self-
describing, self-registering services and self-
subscribing consumers. Taken together, streaming

4
Available here

7
 Figure 1: Event-Driven Architecture in e-commerce

Figure 1 illustrates a price change scenario. Fuel [ 7 ] The centralized database aggregates and
costs to ship some items have gone up, which persists events, enriches event streams with data
could impact pricing: from other sources, including historical data, and
provides a central repository for multiple event
[ 1 ] This produces events about the cost increase streams. This enables applications and users to
and places them into the message stream where benefit from all data across all microservices and
the event queue makes them available. All provides a unified view of state across the e-
microservices are listening for messages of commerce provider’s enterprise.
interest.
Other examples of enterprise-driven architecture
[ 2 - 3 - 4 ] Pricing consumes the event, analyzes it in real-world situations include trading applications
against existing data, and produces events that need to be updated in real time as stock, ETF,
conveying the new pricing into the message and options prices change; creating an IoT data
stream. pipeline that generates alarms whenever a
[ 5 - 6 ] The database pushes those messages to connected vehicle moves outside of a geo-fenced
the event queue, which then makes them available area; or updating dashboards, analytics, and
to all consumers who are listening for messages of search engines as operational data changes. The
interest. Microservices directly impacted by pricing possibilities are endless.
changes, such as those that manage inventory,
marketing, promotions & coupons, point of sale
(POS), and the e-commerce provider’s order
management system (OMS), consume the price
change events and update their individual
databases accordingly.

8
The Database is Core to Event-Driven many providers issuing credit cards and creating a
range of financing options for their customers, this
Architecture
example can apply to many industries. With access
The database plays a central role in event-driven to more accurate and up-to-date data, systems can
architecture. An event messaging queue streams respond to emerging conditions and anomalous
data that is sequentially written as events into events as they occur; users get what they want
commit logs, which allows real-time data immediately; and businesses make better
movement between your services. decisions, gain operational simplicity, and deliver
better customer experiences.
While events flow through Kafka in an append-only
stream, a complete event-driven architecture can In our spotlight scenario, an identity thief has
aggregate, persist, and serve them to consuming stolen a legitimate customer’s credit card
apps and users in real time, enriching event information. The differences between how this
streams with associated data sources and plays out with legacy batch-loading data systems
historical data. The database is best at this, versus real-time treatment built on an event-driven
materializing views of the data and application architecture are striking and have real business
state to provide the lowest latency queries and implications.
deeper analysis to consuming applications,
without having to replay and regenerate data from THE OLD WAY: Batch Jobs Perform
multiple streams. Although each microservice is
Periodic Data Analysis
provisioned with its own database, the core
database provides a central repository for multiple In our scenario, an identity thief steals a legitimate
independent event streams. Thus the platform user’s credit card information and uses it, perhaps
serves as a data source driving operational and many times and likely up to the user’s remaining
analytical apps, and perhaps even more credit limit. Based on ongoing usage patterns and
importantly, provides a complete end-to-end view location data, after running a nightly batch job the
of the business. system flags fraudulent charges and locks the
user’s account. The next time the user tries to use
the card, it is mysteriously declined. The user calls
Use Case Spotlight: Fraud the bank, learns the bad news, gives the CSR

Detection & Remediation account verification information, waits three days

for a new card to arrive in the mail — and worries
what may happen next. While worrying, the user
The value of an event-driven architecture shines
may pull out another credit card they rarely use
through in many use cases across industry
and wonder if that card would be less risky to use
verticals, and a great example is a credit card fraud
going forward.
detection and customer care applications. With so

9
 Figure 2: The Old Way

The fundamental technology gap in legacy cover losses, and bears the added support burden
systems that allows this to happen is illustrated in of picking up the pieces with the customer.
Figure 2: components are not sending events to
one another in real time, but rather batching them THE NEW WAY: Real-Time Data
up and processing them all at once. This means
Changes the Game
suspicious activity is not detected until a batch
data job runs and performs its nightly If an identity thief obtains the user’s credit card
reconciliation with the charge network. Maintaining information, a charge is recorded as an event in a
transactional and historical data siloed in separate transaction stream in real time and touches off a
systems adds complexity and further slows the fraud detection algorithm. This enables a fraud
process. management service based on an event-driven
architecture to detect suspicious activity as it
The customer is surprised and worried over happens and apply various treatments, including
declined charges and wonders what is going on declining the transaction and notifying the
with the account. He or she also must go through merchant, card issuer, and user. The issuer may
extra effort, probably for some time into the future, notify authorities to initiate action against the thief
reviewing statements and disputing charges, while deciding whether to issue a new card and
addressing angry merchants who may blame the assuring the user the situation is under control.
user for the issue, and updating any accounts that
were set up to autobill the old card to the new Likewise, whenever the legitimate user goes to
card number. The business bears the cost of use the card, as shown in Figure 3, the system
fraudulent charges or raises an insurance claim to constantly checks for potential fraud. Depending

10
 on the issuer, the user can receive an alert, to effectively implement fraud detection and
confirm the legitimacy of the transaction, and use remediation. Detecting potentially anomalous
the card without incident. events, and analyzing them against account-
specific insights, in real time, significantly reduces
Here, too, event-driven architecture delivers the risk of fraud, and allows data scientists to
significant value to the business. As soon as a quickly update their models as new threat patterns
transaction event occurs, the fraud detection emerge. Immediate detection allows the company
service calls on multiple data sources to verify the to be proactive and positive.
integrity of the transaction, avoiding losses to the
credit card issuer. Finally, and from a business standpoint, perhaps
most importantly: with the new, the customer
In point of fact, not only is capturing and acting on experience is much better and far more likely to
events in real time a better way, it is the only way enhance customer satisfaction and retention.

Figure 3: The New Way

11
While our examples above are illustrative, many relational mapping (ORM) at the application layer.
companies beyond those who issue credit cards are Then your development team has to repeat the
in fact “financing” their users, which means fraud process, first for each microservice and then every
detection is important to a wide range of companies. change to the data model as application
For example, with cloud services, customers often functionality evolves.
consume a service and the provider bills them at the
end of the month. A company can find itself chasing With MongoDB, data modeling for microservices is
customers who commit intentional non-payment easy. MongoDB helps you move at the speed your
(INP) theft by using cloud services and disappearing, users demand. This gives you the power to launch
or simply refusing to pay. Event-driven architecture new digital initiatives and bring modernized
provides event streams that reveal user behavior applications to market faster, running reliably and
patterns much faster, which enables prevention, securely at scale, unlocking insights and intelligence
detection, enforcement, and containment. By ahead of your competitors. MongoDB is ideally
filtering on events such as usage above predefined suited to deploy your microservices architecture:
charging and time thresholds, and choice of 1. The document model is the most natural
payment method, some companies providing cloud way to work with data in your database. It is
services have reduced or eliminated their INP-based simple for any developer to learn how to
losses by adopting an event-driven architecture. code against MongoDB, and as a result,
Examples of other use cases illustrating the benefits industry surveys show it is wildly popular
of event-driven architecture follow further along in amongst developers.5 This makes it much
this paper. easier to hire and on-board new developers
as you add more microservice teams. JSON
documents are the language of APIs, which
Microservices and Event- are how events commonly enter the system,
and are also how event data is passed
Driven Architectures: around between the microservices within
the system. JSON documents are how you
Why MongoDB work with MongoDB, so there is no need to
convert between models.
Microservices are a common architectural pattern
2. The database-per-microservice model
for modern app development, and many are built
adopted by most organizations means you
around event-driven architecture, which allows data
may end up with scores of databases. With
to seamlessly move between the various
MongoDB that is no problem: MongoDB
microservices in an application. Microservices built
Atlas, the global cloud database for
with agile and DevOps methodologies, and
MongoDB, makes it easy to stand up fully
deployed to elastically scalable cloud platforms,
managed and fully automated databases.
enable business agility.
You save a huge amount of ops effort and
Trying to implement microservices in a legacy are not forcing every sprint team to include
relational database incurs the pain and friction of a DBA.
having to define a schema in the database and re- 3. That said, however, not all of the data in a
implement that same schema again to effect object- microservice’s database is of interest solely

5
Latest example at time of publication: Stackoverflow,
Developer Survey Results 2019, available here and cited
here: Mat Keep, MongoDB Blog, MongoDB: Most Wanted
Database by Developers for the 4th Consecutive Year
12
 to that microservice. MongoDB can serve as using a flexible document structure and rich
a centralized data source accessible to all query language to implement polyglot
microservices, so you can perform analytics persistence without having to rely on a
across data from all of them and benefit multitude of different database
from a unified view of your business. technologies.
4. In an event-driven microservices
architecture, you have many instances, often
running in a public cloud. This means it is a How MongoDB Enables
case of "when, not if" nodes will fail.
MongoDB replica sets give you redundancy
Event-Driven Architecture
and self-healing recovery out of the box. In
the cloud, each microservice, or each logical Today MongoDB is at the core of many event-driven
grouping of related microservices, is systems. MongoDB provides a database you can
provisioned with its own MongoDB replica consume as a service or run yourself on-prem or in
set, and deployed across multiple availability the cloud, with connectors and tools that make it
zones on any of the major cloud providers. incredibly easy to move data between systems, to
5. You can synchronize updates across analyze and visualize data, and to make data
serverless and microservices architectures accessible to users wherever they are at any
by triggering an API call when a document is moment.
inserted or modified. As shown in Figure 4, by starting with the core
6. Different types of data models are MongoDB data platform and binding in
sometimes appropriate for different complementary technologies, MongoDB provides
microservices; these data models, may the data persistence heart of an event-driven
include documents, tabular, key-value,
architecture.
geospatial, text, and graph. With MongoDB
you can work with all of these data models

13
 Figure 4: MongoDB At the Heart of Event-Driven Architecture

MongoDB Atlas MongoDB Change Streams: Addressing Change

Data Capture
MongoDB Atlas is our on-demand, fully-managed
and global database as a service (DBaaS) that runs MongoDB Change Streams enables applications to
on all leading cloud platforms to let you focus on stream real-time data changes in the database by
apps instead of ops. A leader in the DBaaS space,6 leveraging MongoDB’s underlying replication
Atlas radically simplifies development and capabilities. Change Streams can notify your
operations, provides a persistence layer for an application of all writes to documents (including
event-driven architecture, and distributes data to deletes) and provide access to all available
consumers (systems and users) whenever and information immediately as changes occur in the
wherever needed. Atlas lets you scale at the click of database. Downstream applications can register for
a button, in response to an API call, or fully notifications whenever a document or collection is
automatically as loads grow, to keep up with the modified, filtering on relevant events using
demands of your microservices, with no downtime, aggregation pipeline stages $match, $project, and
full security, and complete data protection. $redact. This enables them to act on new data in
Alternatively, you can deploy MongoDB Enterprise real time without constantly querying the entire
Advanced on your own self-managed on-premises collection to identify changes. It also avoids delays
or public cloud infrastructure.

6 Seong Park, MongoDB Blog, MongoDB Named A Leader in The

Latest example at time publication: The Forrester Wave™:
Database-As-A-Service, available here and cited here: Forrester Wave™: Database-As-A-Service Q2 2019

14
and higher system overhead that constant MongoDB’s Enterprise Operator for Kubernetes lets
request/response polling can introduce due to the you deploy and manage the full lifecycle of your
database being regularly checked even if nothing containerized application and MongoDB clusters all
has changed. from within your Kubernetes environment and get a
consistent experience wherever you run them: on-
Atlas Triggers and Functions: The Benefits of premises, hybrid, or public cloud. Kubernetes
Change Streams, Consumed Serverlessly Operators are application-specific controllers that
extend the Kubernetes API to create, configure, and
Business applications can consume change streams
manage instances of stateful applications such as
directly via a message queue, but MongoDB’s Atlas
databases. On self-managed infrastructure, whether
Triggers are a feature that provides a serverless way
on-premises or in the cloud, you can use our
of consuming change stream events. With Triggers,
Kubernetes Operator and the MongoDB Ops
you don’t have to stand up your own application
Manager to automate and manage MongoDB
server to run your change data capture process.
clusters. You have full control over your MongoDB
Change streams flow change data to Atlas Triggers
deployment from a single Kubernetes control plane.
to create responsive, event-driven pipelines.
You can use with the Operator with upstream
MongoDB Functions enable developers to run Kubernetes or with any popular Kubernetes
simple JavaScript code in a serverless platform, distribution such as Red Hat OpenShift and Pivotal
making it easy to implement application logic, Container Service (PKS).
securely integrate with cloud services and
microservices, and build APIs. Atlas Triggers can MongoDB Connector for Apache Kafka
execute Functions according to predefined
schedules or in real time in response to changes in Confluent Platform, including Apache Kafka and
the database or user authentication events. Kafka Connect, is designed as an event messaging
Examples include: queue for massive streams of data that sequentially
writes events into commit logs, allowing real-time
● Sending a text message or email to notify a data movement between your services and data
customer that his or her balance has fallen sources. The MongoDB Connector for Apache®
below a predefined threshold. Kafka® simplifies building robust, reactive pipelines
● Reacting to a customer order event by to move events between systems. As illustrated in
updating inventory levels, sending a delivery Figure 5, you can use MongoDB as:
request to your warehouse system, and
● A sink (consumer) to ingest events from
recording the order in the customer's profile.
Apache Kafka topics directly into MongoDB
MongoDB Enterprise Operator for Kubernetes collections, exposing the data to your
services for efficient querying, enrichment,
MongoDB gives you the freedom to run anywhere.
and analytics, as well as for long-term
Wherever you want to run your event-driven
storage.
architecture, we can be your database of choice.
● A source (producer) for Kafka topics; in this
Atlas is the preferred route for many, but we also
mode, data is captured via Change Streams
give you the flexibility to run MongoDB yourself,
within the MongoDB cluster and published
controlling it right alongside your microservices
straight into Kafka topics.
orchestrated with Kubernetes.

15
 Figure 5: MongoDB Can Serve as Both a Sink and a Source for Apache Kafka

These capabilities enable consuming apps to react build standalone dashboards or embed them
to data changes in real time using a fully event- straight into your web apps.
driven programming style. MongoDB and Apache
Kafka work together to enable you to build Charts supports workload isolation, working against
microservices and event-driven architectures with secondary replicas or dedicated analytics nodes in
ease. Our Kafka Connector is developed and MongoDB Atlas clusters to avoid contending for
supported by MongoDB engineers and verified by resources with operational apps. You either create
Confluent as a first-class component of Confluent standalone dashboards or embed charts directly
Platform. into your Web apps, building richer user
experiences while reducing development time and
Analytics and Visualization with MongoDB effort. All MongoDB Charts are live updated and
deeply customizable, making them the perfect fit for
Key features that enable your teams to obtain
your modern, styled web applications and sites.
analytic insights in MongoDB include:
MongoDB Charts also helps you make sense of
MongoDB Charts geospatial data, such as latitude and longitude
coordinates, or polygons on the earth’s surface.
MongoDB Charts is the fastest and easiest way to
Geolocation capabilities are already a part of the
visualize event data directly from MongoDB in order
MongoDB platform, and Charts extends and
to act instantly and decisively based on the real-time
enhances them by letting you perform geolocation
insights your event-driven architecture is delivering.
analysis without having to write any code.
Leveraging the analytical functionality of MongoDB’s
aggregation framework, MongoDB Charts lets you
MongoDB Connector for Apache Spark
connect to any MongoDB instance as a data source,
create charts and graphs, build dashboards, and Apache® Spark® performs analytics using a cluster
share them with other users for collaboration. of nodes and is used for batch workloads that
MongoDB's materialized views enable you to previously were run with Apache® Hadoop®.
maintain rolling snapshots of data, e.g., sales data, Because of Spark's ability to store the data in
that can be visualized in live dashboards in Charts to memory rather than on disk, users get much faster
monitor sales performance. Charts natively supports execution. Spark can also perform streaming
the richness of the document model, including analytics, reusing the same application analytics
nested and hierarchical data, making data code used in Spark batch jobs. The MongoDB
manipulation user-friendly by presenting operations Connector for Apache Spark lets you access Spark
that make sense for each data type. Charts lets you libraries and enables streaming, machine learning,
16
and real-time analytics to help you query streaming Time Series and IoT
data in real time, and persist results to MongoDB.
Time series applications capture and measure state
You can also use the Spark connector to extract
changes over time and are often used to forecast
data from MongoDB, and blend it with streaming
future changes and establish operational system
data being processed by Spark.
thresholds. Rather than appending only the most
MongoDB Connector for BI recent data and storing solely the latest application
state, time series applications store all changes of
If you’re sending data to third-party analysis tools via state (e.g., ` [x=150, x++, x+=12, x-=2]` ) in the
ETL, you’re operating with stale data and negating original order so they can be replayed to re-create
the real-time advantages that are the main reason to the final state and be analyzed to measure the
adopt event-driven architecture in the first place. changes that have occurred. IoT applications are the
The MongoDB Connector for BI creates a direct most widely deployed examples of time series
connection from MongoDB to third-party SQL-based applications. They must cope with massive numbers
analysis tools to ensure that your teams are always of events being generated by a multitude of
making the best decisions based on the freshest devices. Existing data systems are challenged
data, even if they are visualizing the data in third- simply to keep up, much less to capture events in
party reporting tools. sequence and make sense of not just the IoT-
specific data but what it means when analyzed
MongoDB Professional Services: along with all other relevant data across the
Accelerating Event-Driven Architecture enterprise.

MongoDB’s Professional Services team helps you Event-driven architecture creates a data pipeline
combine people, processes, and technology to where these producers – the devices, sensors, and
make the move to event-driven architecture and run gateways in your IoT networks – generate streams
your business in real time. MongoDB Professional of events, and event consumers listen and react to
Services works with you to build an in-house those events. MongoDB supports event-driven
Innovation Accelerator to achieve faster time-to- architecture that can accommodate the complex
market across multiple business applications and and quickly changing time series data generated by
use cases. Establishing event-driven architecture fleets of sensors and connected devices. MongoDB
requires streamlining your development processes and Kafka working together are well suited to this
with agile ways of working such as DevOps and design approach because MongoDB can store
microservices patterns. Professional Services arbitrarily long sequences of events, while Kafka can
delivers Innovation Solution Kits that provide design quickly and efficiently provide them in the correct
patterns for common use cases, with reference sequence to a consuming application via
architectures and technical standards to help you MongoDB’s Kafka Connector. Kafka can ingest data
make the move to microservices and establish your very quickly, acting as a buffer during spikes in
event-driven architecture. sensor data ingress, such that business apps can
process events at their own pace rather than having
to discard readings during busy times.
Other Use Cases MongoDB persists and aggregates events, serving
them to consuming apps in real time, using Atlas
Additional use cases offer examples of how
Triggers to flag data of interest. It can also enrich
MongoDB enables event-driven architecture to
event streams with associated data sources and
deliver technological advantages and business
historical data. This means that instead of having to
value. This section outlines just a few.
replay and regenerate data from multiple streams,
17
your existing apps can access and act on enriched in the average vehicle, and that number is expected
event streams in real time. When you add new apps, to double in the next two to three years. Diagnostics
you can so without changing anything else in your from a vehicle's sensors must be received and
architecture, and MongoDB can replay all events so processed back at base, in the backend, and
the new app can learn from historical events as well distributed in real time to microservices managing
as new ones. Two prominent IoT-focused use cases everything from vehicle systems to dealer and
illustrating the value of event-driven architecture are service records and more. And since connected
predictive maintenance and telematics. cars are quickly gaining the smartphone-like ability
to make payments, other microservices come into
Predictive Maintenance play to manage processes including credit check
and the transaction itself. Legacy data architectures
All machines eventually break down. When and their batch processing operations are no match
machines are at the heart of manufacturing and for the amount of data being generated by all of
other industrial processes, those breakdowns can these vehicles.
wreak havoc on production, schedules, resources,
delivery commitments, customer relationships, and Event-driven architecture supports telematics
more. Simply waiting until machines break down applications. Kafka streams event data from all of
and only then attempting repair or replacement is those sensors into MongoDB to drive machine
unacceptable, but predicting when it will happen, learning models monitoring performance of parts
with the specific knowledge to take evasive action, and equipment; real-time road, traffic, and weather
is incredibly difficult. The complexity of the systems, conditions; location and movement of the vehicle,
equipment, and parts involved in these processes and more. This helps to ensure driver safety,
can overload legacy data architectures, rendering optimize automotive performance, and keep
proactive approaches impossible. maintenance schedules on track.

Event-driven architecture can be deployed as the MongoDB’s IoT Reference Architecture provides a
foundation for predictive maintenance applications. deeper exploration of MongoDB at the core of
Kafka streams time series event data from assembly event-driven architecture. Learn more →
lines and parts inventories into MongoDB to drive
machine learning models that detect degrading E-commerce
conditions in equipment and systems that predict
problems. The company can either repair equipment Engaging buyers and keeping them engaged
or swap it out for replacement parts before it shuts throughout their buying journeys is critical.
down assembly lines. Atlas Triggers can alert MongoDB processes events in the customer journey
engineers to potential equipment failures, and so the system can respond instantly with
analysis of both real-time and historical trending personalized offers. The system’s responses are
data can help the company adopt optimal based on real-time inventory, available offers,
equipment operating modes and manufacturing customer behaviors and purchasing patterns,
parameters. seasons, time of day, weather, and other factors.
Accessing event data in real time enables the e-
commerce site to make offers with high takeup
Telematics
potential and update inventory the moment it
The average vehicle now has more than 30,000 changes to avoid disappointing shoppers with
parts, and every one of them can impact operation, stockouts on items, for example, that the system
safety, and performance. Parts and systems are now may have just shown as available.
monitored and managed by more than 100 sensors

18
When a shopper presses the ‘Buy Now' button, the messages and application state at any moment in
system must communicate with various time. This enables a wide range of actions, e.g.,
microservices managing inventory, credit check, notifying an Atlas Trigger when an order has been
ordering, shipping, and more to complete the completed or dispatched, and the Atlas Trigger
transaction and ensure a good customer calling a notification service to send an email, text,
experience. MongoDB provides the means for or social media message to a shopper.
microservices to pass messages to each other, with
multiple microservices publishing and subscribing to Figure 6 is the corollary to Figure 1, this time
the same topics so each microservice has the data it showing where MongoDB products and features fit
needs. The database persists the sequences of into event-driven architecture for e-commerce.

Figure 6: MongoDB in e-commerce Event-Driven Architecture

19
The MongoDB Connector for Apache Kafka first applications while mitigating the impact to legacy
serves as a Kafka sink: the events about the systems. Kafka captures any changes in source
increased fuel costs become part of a Kafka topic systems and streams them into MongoDB to keep
inserted into a MongoDB Collection. From there the ODL fresh, and conversely, if users write to the
the microservice that manages the pricing ODL, Change Streams and Kafka can push that
leverages MongoDB’s advanced analytical back in real time to update the source systems.
querying capabilities such as the MongoDB MongoDB’s white paper, Implementing an
Aggregation Framework to calculate new pricing, Operational Data Layer, discusses the topic in
and this new pricing data can be further analyzed depth. Learn more →
and visualized using MongoDB Charts, the
MongoDB Connector for Apache Spark, and the Artificial Intelligence
MongoDB Connector for BI. MongoDB Change
Streams flows change data to Atlas Triggers and Training AI models is time- and resource-intensive,
Functions to push the change data into a Kafka and at a time when concerns about climate
topic using the source capability of the Kafka change and sustainable practices are at all-time
Connector. This enables other microservices that highs, some assert that simply training a single AI
depend on the pricing data to consume events model creates what to many would be an
about the pricing changes, using a fully reactive, unthinkable carbon footprint. Slashing the amount
event-driven programming pattern. of time it takes to train AI models, then, may be the
single most important step toward making AI a
cost-effective (and palatable) proposition, and
Operational Data Layer
event-driven architecture is poised to do exactly
MongoDB and Kafka can work together in an that.
event-driven architecture to create an Operational
By enabling you to analyze data in real time,
Data Layer (ODL). An ODL is an architectural
event-driven architecture can accelerate by orders
pattern that centrally integrates and organizes all
of magnitude your assessments of how AI models
siloed enterprise data and makes it accessible to
are performing, so you can access additional
users or apps that need to access the data. An
datasets quickly and get models field-ready faster.
ODL can combine data from multiple legacy
You can access training and model data
sources into a single repository where new
programmatically via MongoDB’s R and Python
applications can access all data from all sources to
drivers, or via our Spark Connector, so you are
develop single views of customers, products,
always updating your models with the freshest
financial assets, or other relevant entities. Unifying
data. Machine learning algorithms can be
all data into one repository helps the organization
computationally expensive and may not be able to
make smarter decisions and take action faster.
keep up with incoming events at the busiest times.
An enterprise data warehouse (EDW) or Hadoop MongoDB and Kafka work together to act as a
platform can also unify data – for daily or weekly buffer when a learning algorithm is unable to keep
batch analysis. Both leave an organization up with spikes in event volumes.
operating with stale data. By contrast, an ODL
provides real-time data to support operational
processes and user queries. Similar to the way
microservices can operate independently of each
other, an ODL can expose existing data to new

20
Web Activity Tracking and Log powers Ticketek’s multi-channel ticket sales and
distribution network. Ticketek’s strategy is to
Aggregation
banish batch processing from the industry and
Website activity tracking was the original use case create an advanced data ecosystem through open
for Kafka when it was developed at LinkedIn. In integration and real-time data streaming with
event-driven architecture, site activity such as clients and partners. Every ticket sale passes
pages visited, engagement duration, ads through an e-commerce platform backed by fully
displayed, and many other metrics are captured in managed MongoDB Atlas databases. The platform
Kafka topics (one topic per data type). These feeds a network of real-time dashboards that
topics can be consumed by applications such as provide insights into ticket sales performance and
monitoring, real-time analysis, or archival for offline trends. Learn more →
analysis. Insights from the data are persisted in
MongoDB where they can be analyzed alongside
EG
data from all other sources.
EG, the UK’s leading commercial property data
Log aggregation traditionally involved collecting service, made MongoDB the center of its event-
physical log files from multiple servers and storing driven architecture. EG runs its business on a
them in a central location (e.g., HDFS), with the microservices-inspired architecture it terms “pods.”
data analyzed by periodic batch jobs. Modern Each pod maintains a local instance of the data it
event-driven architectures use MongoDB and needs, pulling from a centralized system of record
Kafka to combine real-time log feeds from multiple that stores all property details, transactions,
sources so the data can be continuously locations, market data, availability, and more. All
monitored and analyzed. Events can be logged data product pods and the system of record run on
and their insights and consequences can be MongoDB. With its legacy relational databases, EG
immediately understood and acted upon. could deploy only one application update per
month, but with MongoDB, it is deploying new
business functionality 50-60 times per month.
MongoDB Event-Driven Learn more →
Architecture Customer
ao.com
Snapshots
ao.com, a leading online electrical retailer, uses
Following are examples of companies working event-driven architecture built on MongoDB. AO
with MongoDB to run their businesses in real time has many sources of customer information spread
through event-driven architectures. across many different departments, each using its
own technology. The company recognized the
need to bring this data together into an
Ticketek
Operational Data Layer to provide a 360-degree
Ticketek, Australia’s largest sports and live single customer view. Now apps in its source
entertainment ticketing company, chose MongoDB system update customer data, which is written as
over a range of databases including Oracle, events to Kafka and pushed to MongoDB Atlas to
Microsoft SQL Server, MySQL, and several NoSQL create one source of truth for all customer data.
providers to enhance the customer experience for MongoDB then serves as a source for apps
millions of patrons. MongoDB Atlas serves as the including customer service, fraud detection, and
data layer for the event-driven architecture that GDPR compliance. With MongoDB, ao.com can
21
scale up and out on demand, with no application event-driven architecture enables them to get
downtime; it can perform complex queries in there. By starting with the MongoDB platform and
milliseconds improving customer experience and binding in complementary technologies,
accelerating fraud detection from hours to companies can avail themselves of all of the
seconds. With MongoDB, AO can react in real time advantages of an event-driven architecture,
to anything that happens in its production helping them align teams effectively, innovate
environment. Learn more → rapidly, and meet the challenges of ultra-
competitive global markets.
Man AHL
Safe Harbor Statement
AHL, a subsidiary of The Man Group, which is one
of the world’s largest hedge fund investment firms, This document contains “forward-looking
used MongoDB to create a single platform for all of statements” within the meaning of Section 27A of
its financial data. The system receives data for up the Securities Act of 1933, as amended, and
to 250 million ticks per second from multiple Section 21E of the Securities Exchange Act of
financial sources and writes it to Kafka. Kafka 1934, as amended. Such forward-looking
consolidates and buffers events before they are statements are subject to a number of risks,
stored in MongoDB, where the data can be uncertainties, assumptions and other factors that
analyzed by quantitative researchers. could cause actual results and the timing of certain
Learn more → events to differ materially from future results
expressed or implied by the forward-looking
statements. Factors that could cause or contribute
comparethemarket.com
to such differences include, but are not limited to,
comparethemarket.com, a leading price those identified our filings with the Securities and
comparison provider, uses MongoDB as the Exchange Commission. You should not rely upon
default operational database across its forward-looking statements as predictions of future
microservices architecture. While each events. Furthermore, such forward-looking
microservice uses its own MongoDB database, the statements speak only as of the date of this
company needs to maintain synchronization presentation.
between services, so key data changes are written
In particular, the development, release, and timing
to a Kafka topic. Those events are written to
of any features or functionality described for
MongoDB to enable real-time personalization and
MongoDB products remains at MongoDB’s sole
optimize the customer experience. Learn more →
discretion. This information is merely intended to
outline our general product direction and it should
not be relied on in making a purchasing decision
Conclusion nor is this a commitment, promise or legal
obligation to deliver any material, code, or
The world runs in real time, and businesses must
functionality. Except as required by law, we
match that pace by becoming agile organizations.
undertake no obligation to update any forward-
Their ability to do so rests on empowering their
looking statements to reflect events or
development teams to rapidly build new business
circumstances after the date of such statements.
functionality through agile ways of working, such
as DevOps and microservices patterns.
Microservices are now the major driver in
companies making the move to real time, and
22
We Can Help infrastructure for handling requests and
coordinating cloud service and database
interactions. Realm takes care of the infrastructure,
We are the company that builds and runs
allowing you to focus on the high-value parts of
MongoDB. More than 14,200 organizations rely on
your applications.
our commercial products. We offer software and
services to make your life easier: MongoDB Cloud Manager is a cloud-based tool
that helps you manage MongoDB on your own
MongoDB Atlas is the database as a service for
infrastructure. With automated provisioning, fine-
MongoDB, available on AWS, Azure, and GCP. It
grained monitoring, and continuous backups, you
lets you focus on apps instead of ops. With
get a full management suite that reduces
MongoDB Atlas, you only pay for what you use
operational overhead, while maintaining full
with a convenient hourly billing model. Atlas auto-
control over your databases.
scales in response to application demand with no
downtime, offering full security, resilience, and MongoDB Consulting packages get you to
high performance. production faster, help you tune performance in
production, help you scale, and free you up to
MongoDB Enterprise Advanced is the best way to
focus on your next release.
run MongoDB on your own infrastructure. It's a
finely-tuned package of advanced software, MongoDB Training helps you become a MongoDB
support, certifications, and other services expert, from design to operating mission-critical
designed for the way you do business. systems at scale. Whether you're a developer,
DBA, or architect, we can make you better at
MongoDB Atlas Data Lake allows you to quickly
MongoDB.
and easily query data in any format on Amazon S3
using the MongoDB Query Language and tools.
You don’t have to move data anywhere, you can
work with complex data immediately in its native
Resources
form, and with its fully-managed, serverless
architecture, you control costs and remove the For more information, please visit mongodb.com
operational burden. or contact us at [email protected].

MongoDB Charts is the best way to create Case Studies

visualizations of MongoDB data anywhere. Build
Presentations
visualizations quickly and easily to analyze
complex, nested data. Embed individual charts into White Papers
any web application or assemble them into live
dashboards for sharing. Free Online Training

MongoDB Realm is a serverless application Webinars and Events

platform that provides an elastically scalable
Documentation

US +1 866.237.8815 • Rest of World +1 650.440.4474 • [email protected]

© 2019 MongoDB, Inc. All rights reserved.
23