Definitive Guide Graph Databases For RDBMS Developer
Definitive Guide Graph Databases For RDBMS Developer
Ebook
neo4j.com
Introduction
Introduction:
Query Languages:
SQL vs. Cypher
18
Deployment Paradigms:
Bringing in Graphs
26
When two database technologies share space in the same book title, theres bound to be
confusion as to the motives of its writing.
Drivers: Connecting to
a Graph Database
30
Conclusion
33
First things first: We didnt write this book to bash relational databases or to criticize a stillvaluable technology. Without relational databases, many of todays most mission-critical
applications wouldnt run, and without the early innovation of RDBMS pioneers, we would
have never gotten to where we are with todays database technology.
Other Resources
34
Rather, we wrote this book to introduce you a developer with RDBMS experience to a
database technology that changed not only how we see the world, but how we build the
future. Todays business and user requirements demand applications that connect more and
more of the worlds data, yet still expect high-levels of performance and data reliability.
We believe those applications of the future will be built using graph databases, and we dont
want you to be left behind. In fact, were here to help you through every step of the learning
process.
While other NoSQL (Not only SQL) databases advertise themselves with in-your-face defiance
of RDBMS technology, we prefer to be a helpful resource in helping you add graph databases
to your professional skillset.
Relational databases still have their perfect use cases. But for those cases when you need a
different solution, we hope this book helps you recognize when and how to use a graph
database to tackle those new challenges.
As you read and peruse these pages, feel free to reach out to us with your questions. You can
most commonly find us on Stack Overflow, our Google Group or our public Slack channel.
Happy graphing,
neo4j.com
Chapter 1:
Why Relational Databases Arent Always Enough
Relational databases are powerful tools.
Since the 80s, they have been the power-horse of most software applications and continue
to be so today. Relational databases (RDBMSs) were initially designed to codify paper forms
and tabular structures, and they do that exceedingly well. For the right use case and the right
architecture, they are one of the best tools for storing and organizing data.
Because relational databases store highly structured data in tables with predetermined
columns and many rows of the same type of information, they require developers and
applications to strictly structure the data used in their applications.
But todays user requirements and applications are asking for more. More features, more
data, more agility, more speed and most importantly more connections.
These types of complex, real-world data are increasing in volume, velocity and variety. As a
result, data relationships which are often more valuable than the data itself are growing at
an even faster rate.
The problem: Relational databases arent designed to capture this rich relationship
information.
The bottom line: Applications (and the enterprises that create them) are missing out on
critical connections essential for todays high-stakes, data-driven decisions.
neo4j.com
Slow and expensive schema redesigns also hurt the agile software development process by
hindering your teams ability to innovate quickly a significant opportunity cost no matter the
size of your bottom line.
The verdict: Relational databases arent engineered for the speed of business agility.
Applications
(and the enterprises
that create them) are
missing out on critical
connections essential
for todays highstakes, data-driven
decisions.
In relational databases, references to other rows and tables are indicated by referring to their
primary-key attributes via foreign-key columns. (See Figure 1 below for an example.)
Figure 1: A JOIN table between the Persons and Departments tables in a relational
database using foreign key constraints.
These references are enforceable with constraints, but only when the reference is never optional. JOINs are then computed at query time by matching primary- and foreign-keys of the
many (potentially indexed) rows of the to-be-JOINed tables. These operations are computeand memory-intensive and have an exponential cost as queries grow.
Consequently, modeling and storing connected data becomes impossible without extreme
complexity. That complexity surfaces in cases like SQL statements that require dozens of
lines of code just to accomplish simple operations. Overall performance also degrades from
query complexity, the number and levels of data relationships and the overall size of the
database.
With todays real-time, always-on expectations of software applications, traditional relational
databases are simply inappropriate whenever data relationships are key to success.
neo4j.com
neo4j.com
Chapter 2:
Why Graph Databases?
We already know that relational databases arent enough (by themselves) for handling the
volume, velocity and variety of todays data, but whats the clear alternative?
There are a lot of other database options out there including a number of NoSQL data
stores but none of them are explicitly designed to handle and store data relationships.
None, that is, except graph databases.
The biggest value that graphs bring to the development stack is their ability to store
relationships and connections as first-class entities.
For instance, the early adopters of graph technology reimagined their businesses around the
value of data relationships. These companies have now become industry leaders: LinkedIn,
Google, Facebook and PayPal.
As pioneers in graph technology, each of these enterprises had to build their own graph
database from scratch. Fortunately for todays developers, thats no longer the case, as graph
database technology is now available off the shelf.
Lets take a further look into why you should consider a graph database for your next
connected-data application. Well start with some basic definitions.
What Is a Graph?
If youre already
familiar with relational
databases, youll find
graphs to be a breeze.
You dont need to understand the arcane mathematical wizardry of graph theory in order
to understand graph databases. On the contrary, if youre already familiar with relational
databases, youll find graphs to be a breeze.
First thing: A graph
in mathematics is not the
same as a chart, so dont
picture a bar or line chart.
Rather, picture a network
or mind map, like in the
example to the right.
neo4j.com
Unlike other
databases,
relationships take
first priority in
graph databases.
This means your
application doesnt
have to infer data
connections using
foreign keys or out-ofband processing, such
as MapReduce.
A graph database is an online database management system with Create, Read, Update and
Delete (CRUD) operations working on a graph data model. Graph databases are generally
built for use with transactional (OLTP) systems. Accordingly, they are normally optimized
for transactional performance, and engineered with transactional integrity and operational
availability in mind.
Unlike other databases, relationships take first priority in graph databases. This means your
application doesnt have to infer data connections using foreign keys or out-of-band processing, such as MapReduce.
By assembling the simple abstractions of nodes and relationships into connected structures,
graph databases enable us to build sophisticated models that map closely to our problem
domain.
There are two important properties of graph database technologies:
Graph Storage
Some graph databases use native graph storage that is specifically designed to store and
manage graphs, while others use relational or object-oriented databases instead. Non-native
storage is often much more latent, especially as data volume and query complexity grow.
Graph Processing Engine
Native graph processing (a.k.a. index-free adjacency) is the most efficient means of processing graph data because connected nodes physically point to each other in the database.
Non-native graph processing uses other means to process CRUD operations that arent
optimized for graphs, often involving an index lookup which results in reduced performance.
neo4j.com
Minutes-to-Milliseconds Performance
Query performance and responsiveness are at the top of many organizations concerns with regard to their data platforms. Online
transactional systems large web applications in particular must respond to end users in milliseconds if they are to be successful.
In the relational world, as an applications dataset size grows, JOIN pains begin to manifest themselves, and performance deteriorates.
Using index-free adjacency, a graph database turns complex JOINs into fast graph traversals which are constant time operations
thereby maintaining millisecond performance irrespective of the overall size of the dataset.
Drastically Accelerated Development Cycles
The graph data model reduces the impedance mismatch that has plagued software development for decades, thereby reducing the
development overhead of translating back and forth between an object model and a tabular relational model.
More importantly, the graph model reduces the impedance mismatch between the technical and business domains. Subject matter
experts, architects and developers can talk about and picture the core domain using a shared model that is then incorporated into
the application itself.
Extreme Business Responsiveness
Successful applications rarely stay still. Changes in business conditions, user behaviors, and technical and operational infrastructures
drive new requirements. In the past, this has required organizations to undertake careful and lengthy data migrations that involve
modifying schemas, transforming data and maintaining redundant data to serve old and new features.
Developing with graph databases aligns perfectly with todays agile, test-driven development practices, allowing your graph database
to evolve in step with the rest of the application and any changing business requirements. Rather than exhaustively modeling a domain ahead of time, data teams can add to the existing graph structure without endangering current functionality.
Enterprise Ready
When employed in a mission-critical application, a data technology must be robust, scalable and more often than not transactional. Although some graph databases are fairly new and not yet fully mature, there are graph databases on the market that provide all
the -ilities needed by large enterprises today:
ACID transactionality
High availability
Horizontal read scalability
Storage of billions of entities
These characteristics have been an important factor leading to the adoption of graph databases by large organizations, not merely in
modest offline or departmental capacities, but in ways that truly transform the business.
neo4j.com
Chapter 3:
Data Modeling: Relational vs. Graph Models
In some regards, graph databases are like the next generation of relational databases, but
with first class support for relationships, or those implicit connections indicated via foreign
keys in traditional relational databases.
Each node (entity or attribute) in the graph database model directly and physically contains a
list of relationship-records that represent its relationships to other nodes. These relationship
records are organized by type and direction and may hold additional attributes.
Graph databases
like Neo4j provide
a minutes-tomilliseconds
performance
advantage of several
orders of magnitude,
especially for JOINheavy queries.
Figure 3: A graph/JOIN table hybrid showing the foreign key data relationships
between the Persons and Departments tables in a relational database.
Whenever you run the equivalent of a JOIN operation, the database just uses this list and has
direct access to the connected nodes, eliminating the need for an expensive search-andmatch computation.
This ability to pre-materialize relationships into database structures allows graph databases
like Neo4j to provide a minutes-to-milliseconds performance advantage of several orders of
magnitude, especially for JOIN-heavy queries.
The resulting data models are much simpler and at the same time more expressive than
those produced using traditional relational or other NoSQL databases.
Figure 4: A graph data model of our original Persons and Departments data.
Nodes and relationships have replaced our tables, foreign keys and JOIN table.
8
neo4j.com
Graph databases support a very flexible and fine-grained data model that allows you to
model and manage rich domains in an easy and intuitive way.
You more or less keep the data as it is in the real world: small, normalized, yet richly
connected entities. This allows you to query and view your data from any imaginable point of
interest, supporting many different use cases (see Chapter 2 for more information).
The fine-grained model also means that there is no fixed boundary around aggregates,
so the scope of update operations is provided by the application during the read or write
operation. Transactions group a set of node and relationship updates into an Atomic,
Consistent, Isolated and Durable (ACID) operation.
Graph databases like Neo4j fully support these transactional concepts, including write-ahead
logs and recovery after abnormal termination, so you never lose your data that has been
committed to the database.
If youre experienced in modeling with relational databases, think of the ease and beauty of a
well-done, normalized entity-relationship diagram: a simple, easy-to-understand model you
can quickly whiteboard with your colleagues and domain experts. A graph is exactly that: a
clear model of the domain, focused on the use cases you want to efficiently support.
Lets take a model of the organizational domain and show how it would be modeled in a
relational database vs. the graph database.
First up, our relational database model:
Graph databases
support a very flexible
and fine-grained data
model that allows you
to model and manage
rich domains in an
easy and intuitive way.
Figure 5: A relational database model of a domain with Persons and Projects within an Organization with several Departments.
neo4j.com
If we were to adapt this (above) relational database model into a graph database model, we would go through the following checklist
to help with the transformation:
Each entity table is represented by a label on nodes
Each row in a entity table is a node
Columns on those tables become node properties.
Remove technical primary keys, but keep business primary keys
Add unique constraints for business primary keys, and add indexes for frequent lookup attributes
Replace foreign keys with relationships to the other table, remove them afterwards
Remove data with default values, no need to store those
Data in tables that is denormalized and duplicated might have to be pulled out into separate nodes to get a cleaner model
Indexed column names might indicate an array property (like email1, email2, email3)
JOIN tables are transformed into relationships, and columns on those tables become relationship properties
Once weve taken these steps to simplify our relational database model, heres what the graph data model would look like:
Figure 6: A graph data model of the same domain with Persons and Projects within an Organization with several
Departments. With the graph model, all of the initial JOIN tables have now become data relationships.
This above example is just one simplified comparison of a relational and graph data model. Now its time to dive deeper into a more
extended example taken from a real-world use case.
Relational vs. Graph Data Modeling Case Study: A Data Center Management Domain
To show you the true power of graph data modeling, were going to look at how we model a domain using both relational- and graphbased techniques. Youre probably already familiar with RDBMS data modeling techniques, so this comparison will highlight a few
similarities and many differences.
In particular, well uncover how easy it is to move from a conceptual graph model to a physical graph model, and how little the graph
model distorts what were trying to represent versus the relational model.
To facilitate this comparison, well examine a simple data center management domain. In this domain, several data centers support
many applications on behalf of many customers using different pieces of infrastructure, from virtual machines to physical load
balancers.
10
neo4j.com
A graph is a clear
model of the domain,
focused on the use
cases you want to
efficiently support.
11
neo4j.com
As the operators of such a data center domain, we have two primary concerns:
Ongoing provision of functionality to meet (or exceed) a service-level agreement,
including the ability to perform forward-looking analyses to determine single
points of failure, and retrospective analyses to rapidly determine the cause of any
customer complaints regarding the availability of service.
Billing for resources consumed, including the cost of hardware, virtualization,
network provisioning and even the costs of software development and operations
(since these are simply logical extensions of the system we see here).
If we are building a data center management solution, well want to ensure that the
underlying data model allows us to store and query data in a way that efficiently addresses
these primary concerns. Well also want to be able to update the underlying model as the
application portfolio changes, the physical layout of the data center evolves and virtual
machine instances migrate.
Given these needs and constraints, lets see how the relational and graph models compare.
(Its worth noting that adept RDBMS developers often skip directly to table design and
normalization without using an intermediate E-R diagram.)
Heres our sample E-R diagram:
neo4j.com
Now with a logical model complete, its time to map it into tables and relations, which are normalized to eliminate data redundancy. In many cases, this step can be as simple as transcribing the E-R diagram into a tabular form and then loading those tables via SQL commands into
the database.
But even the simplest case serves to highlight the idiosyncrasies of the relational model. For
example, in the figure below we see that a great deal of accidental complexity has crept into
the model in the form of foreign key constraints (everything annotated [FK]), which support
one-to-many relationships, and JOIN tables (e.g., AppDatabase), which support many-to-many relationships and all this before weve added a single row of real user data.
Despite being
graphs, E-R diagrams
immediately show the
shortcomings of the
relational model. E-R
diagrams allow only
single, undirected
relationships between
entities. In this respect,
the relational model
is a poor fit for realworld domains
where relationships
between entities are
both numerous and
semantically rich.
Figure 9: A full-fledged relational data model for our data center domain.
These constraints and complexities are model-level metadata that exist simply so that we
specify the relations between tables at query time. Yet the presence of this structural data
is keenly felt, because it clutters and obscures the domain data with data that serves the
database, not the user.
neo4j.com
One of the challenges of the relational paradigm is that normalized models generally arent fast enough for real-world needs. In
theory, a normalized schema is fit for answering any kind of ad hoc query we pose to the domain, but in practice, the model must be
further adapted for specific access patterns.
In other words, to make relational databases perform well enough for regular application needs, we have to abandon any vestiges of
true domain affinity and accept that we have to change the users data model to suit the database engine, not the user. This approach
is called denormalization.
Denormalization involves duplicating data (substantially in some cases) in order to gain query performance.
For example, consider a batch of users and their contact details. A typical user often has several email addresses, which we would
then usually store in a separate EMAIL table. However, to reduce the performance penalty of JOINing two tables, its quite common to
add one or more columns within the USER table to store a users most important email addresses.
Assuming every developer on the project understands the denormalized data model and how it maps to their domain-centric code
(which is a big assumption), denormalization is not a trivial task.
Often, development teams turn to an RDBMS expert to munge a normalized model into a denormalized one that aligns with the characteristics of the underlying RDBMS and physical storage tier. Doing all of this involves a substantial amount of data redundancy.
14
neo4j.com
In the early stages of graph modeling, the work is similar to the relational approach: Using
lo-fi methods like whiteboard sketches, we describe and agree upon the initial domain. After
that, our data modeling methodologies diverge.
Once again, here is our example data center domain modeled on the whiteboard:
15
neo4j.com
The Catch
So, whats the catch? With a graph database, what you sketch on the whiteboard is what you
store in the database. Its that simple.
No catch.
After adding properties, labels and relationships, the resulting graph model for our data
center scenario looks like this:
Figure 11: A full-fledged graph data model for our data center domain.
Note that most of the nodes here have two labels: both a specific type label (such as Database, App or Server), and a more general-purpose Asset label. This allows us to target
particular types of assets with some of our queries, and all assets irrespective of type
with other queries.
Compared to the finished relational database model (included again on the next page),
ask yourself which of the two data models is easier to evolve, contains richer relationships
and yet is still simple enough for business stakeholders to understand. We thought so.
16
neo4j.com
Figure 12: A full-fledged relational data model for our data center domain. This data
model is significantly more complex and less user friendly than our graph model
on the preceding page.
Conclusion
Dont forget: Data models are always changing. While it might be easy to dismiss an upfront
modeling headache with the idea that you wont ever have to do it again, todays agile
development practices will have you back at the whiteboard (or worse, calling a migration
expert) sooner than you think.
Even so, data modeling is only a part of the database development lifecycle. Being able to
query your data easily and efficiently which often means real time is just as important
as having a rich and flexible data model. In the next chapter, well examine the differences
between RDBMS and graph database query languages.
17
neo4j.com
Chapter 4:
Query Languages: SQL vs. Cypher
When it comes to a database query language, linguistic efficiency matters.
Querying relational databases is easy with SQL. As a declarative query language, SQL allows
both for easy ad hoc querying in a database tool as well as specifying use-case related
queries in your code. Even object-relational mappers use SQL under the hood to talk to the
database.
But SQL runs up against major performance challenges when it tries to navigate connected
data. For data-relationship questions, a single query in SQL can be many lines longer than the
same query in a graph database query language like Cypher (more on Cypher below).
Lengthy SQL queries not only take more time to run, but they are also more likely to include
human coding mistakes because of their complexity. In addition, shorter queries increase
the ease of understanding and maintenance across your team of developers. For example,
imagine if an outside developer had to pick through a complicated SQL query and try to
figure out the intent of the original developer trouble would certainly ensue.
But what level of efficiency gains are we talking about between SQL queries and graph
queries? How much more efficient is one versus another? The answer: Fast enough to make
a significant difference to your organization.
The efficiency of graph queries means they run in real time, and in an economy that runs at
the speed of a single tweet, thats a bottom-line difference you cant afford to ignore.
18
neo4j.com
Graph database models, on the other hand, not only communicate how your data is related, but they also help you clearly communicate the kinds of questions you want to ask of your data model. Graph models and graph queries are just two sides of the same coin.
The right database query language helps us traverse both sides.
If we want to express the pattern of this basic graph in Cypher, we would write:
(emil)<-[:KNOWS]-(jim)-[:KNOWS]->(ian)-[:KNOWS]->(emil)
This Cypher pattern describes a path which forms a triangle that connects a node we call jim to the two nodes we call ian and emil,
and which also connects the ian node to the emil node. As you can see, Cypher naturally follows the way we draw graphs on the
whiteboard.
19
neo4j.com
(emil:Person {name:'Emil'})
<-[:KNOWS]-(jim:Person {name:'Jim'})
-[:KNOWS]->(ian:Person {name:'Ian'})
-[:KNOWS]->(emil)
(a:Person {name:'Jim'})
(b:Person)
(c:Person)
(a)
20
neo4j.com
We draw relationships using using pairs of dashes with greater-than or less-than signs (-->
and <--) where the < and > signs indicate relationship direction. Between the dashes, relationship names are enclosed by square brackets and prefixed by a colon, like in this example
from the query above:
-[:KNOWS]->
Node labels are also prefixed by a colon. As you see in the first node of the query, Person is
the applicable label:
(a:Person )
Cypher matches
the remainder of
the pattern to the
graph immediately
surrounding this anchor
point based on the
provided information
on relationships and
neighboring nodes.
Node (and relationship) property key-value pairs are then specified within curly braces, like in
this example:
( {name:'Jim'})
In our original example query, were looking for a node labeled Person with a name property
whose value is Jim. The return value from this lookup is bound to the identifier a. This
identifier allows us to refer to the node that represents Jim throughout the rest of the query.
Its worth noting that this pattern
(a)-[:KNOWS]->(b)-[:KNOWS]->(c), (a)-[:KNOWS]->(c)
could, in theory, occur many times throughout our graph, especially in a large user dataset.
To confine the query, we need to anchor some part of it to one or more places in the graph.
In specifying that were looking for a node labeled Person whose name property value is
Jim, weve bound the pattern to a specific node in the graph the node representing Jim.
Cypher then matches the remainder of the pattern to the graph immediately surrounding
this anchor point based on the provided information on relationships and neighboring
nodes. As it does so, it discovers nodes to bind to the other identifiers. While a will always be
anchored to Jim, b and c will be bound to a sequence of nodes as the query executes.
21
neo4j.com
RETURN
This clause specifies which expressions, relationships and properties in the matched data should be returned to the client. In our
example query, were interested in returning the nodes bound to the b and c identifiers.
22
neo4j.com
SQL vs. Cypher Query Examples: The Good, the Bad and the Ugly
Now that you have a basic understanding of Cypher, its time to compare it side by side with SQL to realize the linguistic efficiency of
the former and the inefficiency of the latter when it comes to queries around connected data.
Our first example uses the organizational domain (from Chapter 3) pictured below as a relational data model:
Cypher Statement:
MATCH (p:Person)<-[:EMPLOYEE]-(d:Department)
WHERE d.name = "IT Department"
RETURN p.name
23
neo4j.com
In this example on the previous page, the Cypher query is half the length of the SQL statement and significantly simpler. Not only
would this Cypher query be faster to create and run, but it also reduces chances for error.
Now for another example, this one more extreme. Well start with the Cypher query:
Cypher Statement:
This Cypher query says that for each customer who bought a product, look at the products that peer customers have purchased and
add them as recommendations. The WHERE clause removes products that the customer has already purchased, since we dont want
to recommend something the customer already bought.
Each of the arrows in the MATCH clause of the Cypher query represents a relationship that would be modeled as a many-to-many JOIN
table in a relational model with two JOINs each. So even this simple query encompasses potentially six JOINs across tables. Heres the
equivalent query in SQL:
SQL Statement:
This SQL statement is three times as long as the equivalent Cypher query. It will not only suffer from performance issues due to the
JOIN complexity but will also degrade in performance as the dataset grows.
24
neo4j.com
Conclusion
If application performance is a priority, then your database query language matters.
SQL is well-optimized for relational database models, but once it has to handle complex,
relationship-oriented queries, its performance quickly degrades. In these instances, the root
problem doesnt lie with SQL but with the relational model itself, which isnt designed to
handle connected data.
For domains with highly connected data, the graph model is a must, and as a result, so is a
graph query language like Cypher. If your development team comes from an SQL background,
then Cypher will be easy to learn and even easier to execute.
When it comes to your next graph-powered, enterprise-level application, youll be glad that
the query language underpinning it all is built for speed and efficiency.
25
neo4j.com
Chapter 5:
Deployment Paradigms:
Bringing Graphs into Your Architecture
Whether youre ready to move your entire legacy RDBMS into a graph database, youre
syncing databases for polyglot persistence or youre just conducting a brief proof of concept,
at some point youll want to bring a graph database into your organization or architecture.
Once youve decided on your deployment paradigm, youll then need to move some (or all) of
your data from your relational database into a graph database. In this chapter, well show you
how to make that process as smooth and seamless as possible.
Your first step is to ensure you have a proper understanding of the native graph property
model (i.e., nodes, relationships, labels, properties and relationship-types), particularly as it
applies to your given domain.
In fact, you should at least complete a basic graph model on a whiteboard before you begin
your data import. Knowing your data model ahead of time and the deployment paradigm in
which youll use it makes the import process significantly less painful.
Figure 15: The three most common paradigms for deploying relational and graph
databases.
First, some development teams decide to abandon their relational database altogether and
migrate all of their data into a graph database. This is typically a one-time, bulk migration.
Second, other developers continue to use their relational database for any use case that
relies on non-graph, tabular data. Then, for any use cases that involve a lot of JOINs or data
relationships, they store that data in a graph database.
26
neo4j.com
Third, some development teams duplicate all of their data into both a relational database and
a graph database. That way, data can be queried in whatever form is the most optimal for the
queries theyre trying to run.
The second and third paradigms are considered polyglot persistence, since both approaches
use a data store according to its strengths. While this introduces additional complexity into an
applications architecture, it often results in getting the most optimized results from the best
database for the query.
None of these is the correct paradigm for deploying an RDBMS and a graph. Your team
should consider your application goals, frequent use cases and most common queries and
choose the appropriate solution for your particular environment.
Another facet to consider is that many relational databases arent designed or optimized for
exporting large amounts of data within a short time period. So if youre trying to migrate data
directly from an RDBMS to a graph, the process might stall significantly.
For example, in one case a Neo4j customer had a large social network stored in a MySQL
cluster. Exporting the data from the MySQL database took three days; importing it into Neo4j
took just three hours.
One final tip before you begin: When you write to disk, be sure to disable virus scanners and
check your disk schedule so you get the highest disk performance possible. Its also worth
checking any other options that might increase performance during the import process.
neo4j.com
The LOAD CSV command is a powerful way of converting flat data (i.e., CSV files) into connected graph data. LOAD CSV works both
with single-table CSV files as well as with files that contain a fully denormalized table or a JOIN of several tables.
LOAD CSV allows you to convert, filter or de-structure import data during the import process. You can also use this command to split
areas, pull out a single value or iterate over a certain list of attributes and then filter them out as attributes.
Finally, with LOAD CSV you can control the size of transactions so you dont run into memory issues with a certain keyword, and you
can run LOAD CSV via the Neo4j shell (and not just the Neo4j browser), which makes it easier it script your data imports.
In summary, you can use Cyphers LOAD CSV command to:
Ingest data, accessing columns by header name or offset
Convert values from strings to different formats and structures (toFloat, split, )
Skip rows to be ignored
MATCH existing nodes based on attribute lookups
CREATE or MERGE nodes and relationships with labels and attributes from the row data
SET new labels and properties or REMOVE outdated ones
name;email;dept
"Lars Higgs";"[email protected]";"IT-Department"
"Maura Wilson";"[email protected]";"Procurement"
Cypher Statement:
You can import multiple CSV files from one or more data sources (including your RDBMS) to enrich your core domain model with
other information that might add interesting insights and capabilities.
Other, dedicated import tools help you import larger volumes (10M+ rows) of data efficiently, as described below.
28
neo4j.com
The transactional Cypher HTTP endpoint is available to all drivers. You can also use the HTTP
endpoint directly from an HTTP client or an HTTP library in your language.
Using the HTTP endpoint (or another API), you can pull the data out of your relational
database (or other data source) and convert it into parameters for Cypher statements. Then
you can batch and control import transactions from there.
From Neo4j 2.2 onwards, Cypher also works really well with highly concurrent writes. In one
test, one million nodes and relationships per second were inserted with highly concurrent
Cypher statements using this method.
The Cypher-based loading method works with a number of different drivers, including the
JDBC driver. If you have an ETL tool or Java program that already uses a JDBC tool, you can
use Neo4js JDBC driver to import data into Neo4j because Cypher statements are just query
strings (more on the JDBC driver in Chapter 6). In this scenario, you can provide parameters
to your Cypher statements as well.
neo4j.com
Chapter 6:
Drivers: Connecting to a Graph Database
Up to this point in the book, weve covered graph databases in general, only using Neo4j
when examples were required. However, when it comes to connecting your application to a
graph database, specifics are essential.
In this chapter, well discuss the language drivers and APIs specific to Neo4j with plenty of
resources for further exploration. At this point, if you are curious about other, non-Neo4j
graph databases, we encourage you to explore the available drivers within their respective
communities.
neo4j.com
Another example: An HTTP request that executes Cypher to create a Person would look like this, here shown with the plain JSON
response:
:POST http://localhost:7474/db/data/transaction/commit
{"statements":[
{"statement":"CREATE (p:Person {name:{name}}) RETURN p",
"parameters":{"name":"Daniel"}}
]}
->
{"results":[{"columns":["p"],"data":[{"row":[{"name":"Daniel"}]}]}],
,"errors":[]}
The language drivers discussed below use the same APIs under the hood, but make them available in a convenient way.
31
neo4j.com
Lets look at an example query using the JDBC driver. Working from the organizational data
model example in Chapter 3, heres what a query for finding Johns department would look
like using the Neo4j-JDBC driver:
Connection con =
DriverManager.getConnection("jdbc:neo4j://
localhost:7474/");
String query =
"MATCH (:Person {name:{1}})-[:EMPLOYEE](d:Department) RETURN d.name as dept";
try (PreparedStatement stmt = con.prepareStatement
(QUERY)) {
stmt.setString(1,"John");
ResultSet rs = stmt.executeQuery();
while(rs.next()) {
String department = rs.getString("dept");
....
}
}
More details on how to use the Neo4j-JDBC driver can be found with our example project
for Java-JDBC. There, we implement the backend as a minimal Java web app that leverages
Neo4j-JDBC to connect to Neo4j Server.
32
neo4j.com
Conclusion:
Never Write Another JOIN
The recent proliferation of NoSQL database technologies is a testament to the fact that
relational databases arent the right tool for every job.
While relational databases are powerful tools for the right use case and the right architecture,
todays user and business needs are changing. As weve seen above, todays complex, realworld data is increasing in volume, velocity and variety, and the data relationships which are
often more valuable than the data itself are growing at an even faster rate.
Leveraging those connected data relationships are where graph databases shine. When you
make the switch to graph databases whether as your main or secondary data store you
can capture that rich relationship information in a way your RDBMS never can.
Furthermore, using a graph database for your next application allows you to match changes
in your database schema with the speed of business agility. And once your application
is in production, a graph database allows you to query connected data at speeds and
performance levels that no RDBMS can match.
Because of the natural fit of the graph database model to business domains and processes,
Forrester Research estimates that at least 25% of enterprises worldwide will use graph
databases by 2017, while Gartner Research reports that graph databases are the fastestgrowing category in database management systems, and that "by the end of 2018, 70
percent of leading organizations will have one or more pilot or proof-of-concept efforts
underway utilizing graph databases."
Gartner Research
reports by the end of
2018, 70 percent of
leading organizations
will have one or
more pilot or proofof-concept efforts
underway utilizing
graph databases.
Instead of imposing a connected data model onto a traditional RDBMS with the
accompanying struggle against intensive and frustrating JOINs its time to use a graph
database for every use case where data relationships matter most.
In the end, business wont wait and neither will users. When you choose the right tool for
your connected-data application, you never have to write another JOIN. Even better: You
reduce query times from minutes to milliseconds.
With so many ways to get started quickly, mastering graph database development is one of
the best time investments you can make.
33
neo4j.com
Other Resources:
More on Graph Databases for the RDBMS Developer
This ebook only scratches the surface when it comes to how todays RDBMS developer can
best take advantage of graph databases.
For more insights, tips and tricks on the intersection of relational and graph databases,
explore any of the many resources below:
Articles and Presentations:
The Graph Databases for Beginners blog series
Neo Technology exists to help the world make sense of data. Thats why Neo Technology created Neo4j, the worlds
leading graph database. Organizations worldwide use Neo4j to extract real-time insights not only from their data
but also data relationships. Neo Technology has brought the power of graph databases to organizations both
large and small from enterprises like Walmart, eBay, UBS, Cisco, HP, Telenor and Lufthansa to young startups like
Polyvore, Medium and Zephyr Health. These businesses have used Neo4j to build solutions as varied as personalized
recommendation engines, identity and access management models, social networks, network monitoring tools and
master data management applications. From its inception, Neo Technology has always valued relationships, and now
organizations worldwide use the Neo4j graph database to unlock the business value from their data relationships.
neo4j.com