Improve

Your RAG
Performance
with Graph-
Based AI

WHITEPAPER — September 2024

Content
Introduction 3

Comparison with Traditional RAG 4

Traditional RAG Limitations 4

Why GraphRAG is Different 6

How GraphRAG Works 7

Core Process 7

Role of Ontologies 12

Challenges of Implementing GraphRAG 13

Complex Implementation 13

Resource Intensive 13

Why Lettria is the Solution 14

Comprehensive Expertise 14

Efficient Integration 14

Scalable Solutions 14

Use Cases 15

Healthcare 15

Financial Services 15

Industrial Maintenance 16

Legal & Compliance 16

Conclusion 17

Strategic Importance 18

Why Choose Lettria’s GraphRAG Solution? 19

1
Introduction
As enterprises increasingly adopt artificial intelligence to enhance decision-
making, optimize operations, and deliver personalized experiences, the limitations
of traditional AI approaches are becoming more apparent. While AI has made
significant strides, particularly in natural language processing (NLP) and large
language models (LLMs), these advancements often fall short when dealing with
complex, unstructured data. The challenge lies in maintaining the context,
accuracy, and explainability of the information processed by these systems.
Traditional AI models, particularly those based on vectorization, tend to flatten
data into simplified representations. This process, while efficient for certain tasks,
strips away the nuanced relationships and context that are vital for understanding
complex information. As a result, enterprises face a significant barrier in extracting
actionable insights from their vast, unstructured data sources—ranging from
technical documentation to financial reports and beyond.

This is where GraphRAG (Graph-based Retrieval-Augmented Generation) comes

into play. GraphRAG represents a paradigm shift in how AI systems handle and
process data. By leveraging the power of graphs, this approach preserves the
intricate relationships between data entities, ensuring that the context is
maintained and that the AI outputs are both accurate and explainable. GraphRAG
allows enterprises to unlock the full potential of their data, transforming it into a
strategic asset that drives innovation and competitive advantage.

At Lettria, we are at the forefront of this revolution. Our commitment to advancing

AI capabilities led us to develop a comprehensive GraphRAG solution that not only
addresses the limitations of traditional AI models but also sets a new standard for
contextual intelligence. This white paper explores the transformative power of
GraphRAG, delving into its core mechanisms, advantages, and practical
applications across various industries.

In the following sections, we will compare GraphRAG with traditional RAG

systems, explain how GraphRAG works, and demonstrate its value through
real-world use cases. By the end of this paper, you will have a clear
understanding of why GraphRAG is the future of enterprise AI and how Lettria
is leading the charge in this exciting field.

3
Comparison with
Traditional RAG
As enterprises strive to harness the power of AI to transform their operations, the
limitations of traditional Retrieval-Augmented Generation (RAG) systems have
become increasingly apparent. While traditional RAG has brought valuable
advancements in AI’s ability to reference external data, it falls short in several
critical areas, particularly when dealing with complex, unstructured data that is
rich in context and relationships.

Traditional RAG Limitations

Traditional RAG systems typically rely on vector-based methods to process and
retrieve information. In these systems, data is converted into numerical vectors,
which are then used to find and generate relevant responses. While this approach
offers speed and efficiency, it also introduces significant limitations:

• Flattening of Data and Loss of Context:

The process of vectorization inherently simplifies data by reducing it to
numerical values. This flattening process often results in the loss of rich,
contextual information that is embedded within the original data. Complex
relationships between entities, such as hierarchical structures, temporal
sequences, and nuanced dependencies, are often not preserved. As a result,
the AI system may miss critical subtleties, leading to outputs that are less
accurate and potentially misleading.

• Inadequate Representation of Complex Concepts:

Vector-based methods struggle to accurately represent and distinguish
between complex concepts, particularly in domains where precision is
paramount. For instance, in legal or technical documents, where specific
terminology and intricate relationships are key, traditional RAG systems may
fail to capture the full meaning, resulting in responses that lack depth and
relevance.

• Reduced Explainability and Traceability:

Vector-based RAG systems operate largely as "black boxes," making it
difficult to trace how specific outputs were generated. This lack of
transparency can undermine trust, especially in high-stakes environments
where decision-making processes need to be clear and justifiable.
Stakeholders in sectors such as healthcare, finance, and legal services
require AI systems that not only deliver accurate information but also provide
a clear rationale for their conclusions.

4
• It mainly addresses extractive questions
where relevant passage is quite short and contains all keywords in the
question. For instance, a questions such as : “what are the four most relevant
ideas in this corpus?” may not be well tackled because relevant contexts will
not be retrieved successfully.

Over the past several months, Lettria has been approached by dozens of AI
leaders in large enterprises across multiple industries who are struggling with
these exact limitations. Many of them report that their Q&A bots, designed to
enhance customer interactions or assist in complex decision-making, are proving
to be unreliable. These bots, powered by traditional RAG systems, often produce
responses that lack accuracy, depth, and the contextual relevance required to
deliver meaningful insights.

More troubling is the fact that half of these enterprises’ GenAI projects stall at
the prototype stage. Despite significant investments in AI development,
companies find that their systems fall short due to inaccurate or incomplete
information retrieval. Without the ability to retrieve the right data in a contextually
aware manner, AI models fail to produce the high-quality outputs that are
expected, leading to frustration and disillusionment among stakeholders.

For instance, leaders from sectors such as finance and legal services have
expressed concerns that their RAG-powered AI systems do not handle the
nuanced relationships between entities like contracts, regulatory documents, or
case law, resulting in poor decision support. Similarly, healthcare organizations
face challenges where their AI systems cannot provide trustworthy, explainable
recommendations, leading to doubts about deploying these solutions in sensitive,
real-world scenarios.

These AI leaders came to Lettria because they needed a solution that goes
beyond the limitations of traditional RAG. They recognize the need for an
approach that can preserve context, accurately represent complex concepts, and
deliver explainable, traceable results—an approach that is critical for scaling AI
projects beyond the prototype stage and ensuring their success in production
environments.

5
Why GraphRAG is Different
GraphRAG represents a fundamental shift in how AI systems process and retrieve
information, addressing many of the shortcomings of traditional RAG.
By leveraging graph structures, GraphRAG maintains the richness of the original
data and ensures that the relationships between entities are preserved.

GraphRAG : The best of two worlds

Unstructed Data

Leverage ontology model to

Produce embedding
produce RDF (node, edges)

Hybrid database merging

knowledge graph and vectors

6
• Preservation of Relationships and Context:
Unlike vector-based methods, GraphRAG transforms data into nodes and edges,
representing entities and the relationships between them. This graph structure
allows the system to maintain the context and intricacies of the original data. For
example, in a knowledge graph, each entity is connected to related entities, and
these connections can represent various types of relationships, such as "is a part
of," "is related to," or "occurs before." This approach ensures that the AI system
understands and respects the complex interdependencies within the data, leading
to more accurate and contextually aware outputs.

• Enhanced Accuracy and Relevance:

By preserving the original relationships within the data, GraphRAG is better
equipped to handle complex queries and provide more relevant responses.
Whether the task involves analyzing financial trends, interpreting legal contracts,
or providing medical recommendations, GraphRAG’s ability to consider the full
context of the data results in outputs that are more accurate and tailored to the
specific needs of the user.

• Transparency and Traceability:

One of the key advantages of GraphRAG is its ability to provide explainable AI
outputs. Because the system retains the structure and relationships within the
data, it is possible to trace how a particular conclusion or recommendation was
reached. This transparency is crucial in industries where decisions must be
justified, and it helps build trust in the AI system’s outputs. Users can follow the
connections and logic used by the AI, ensuring that the results are not only
accurate but also understandable and reliable.

In summary, while traditional RAG systems offer efficiency, they often do so at the
expense of accuracy, context, and transparency. GraphRAG, on the other hand,
preserves the richness of the original data, leading to more meaningful, relevant,
and explainable AI outputs. As enterprises increasingly demand AI systems that
can handle complex, unstructured data with nuance and precision, GraphRAG
emerges as the superior solution.

7
How GraphRAG Works
GraphRAG represents a transformative approach in AI, where the complexity and
richness of unstructured data are not only preserved but leveraged to provide
contextually accurate and explainable insights. This section delves into the core
processes that power GraphRAG, from the extraction of data into graph structures
to the role of ontologies in enhancing AI’s understanding.

Core Process
The GraphRAG process involves several critical stages, each designed to handle
complex data in a way that traditional methods cannot. These stages ensure that
every piece of information, no matter how nuanced, is captured, contextualized,
and made accessible for AI-driven analysis.

User Context Retrieval LLM

Asks question Sends question with context Generates Answer

Sends nodes description as context

Hybrid Database

Vector similarity + Nodes identification

8
• Extracting Graphs from Documents
At the heart of GraphRAG is the ability to convert unstructured data into structured
graph formats. Lettria’s approach begins with the extraction of data from various
document types, including textual content and complex tabular data. This process
involves breaking down the information into its constituent parts—entities,
relationships, and attributes—and organizing them into a graph structure where
nodes represent entities and edges represent the relationships between them.

One of the significant challenges in this process is handling multimodal documents,

particularly those containing complex tables. Tables in documents, especially in
fields like finance, industry and healthcare, often include multiple layers of headers,
cross-references, and nested information. Traditional methods that attempt to
linearize this data often fail to preserve the relationships and context inherent in
the table format.

To address this, Lettria has developed a specialized table-parsing framework that

effectively converts tables into graph structures. For example, in a financial report,
a table listing assets might include various categories, subcategories, and
corresponding values. Lettria’s approach identifies these hierarchical relationships
and translates them into a graph, where each category becomes a node, and
relationships like "is part of" or "has value" are represented as edges connecting
the nodes. This ensures that the context and meaning of the table are fully
preserved, allowing for more accurate querying and analysis later in the process.

Goal:
To create a graph for each document

Method:
LLMGraphTransformer from langchain, improved to add properties to relationships
(containing event times, values, …)

9
• Merging with Existing Graphs
Once the data is extracted and converted into a graph structure, it must be
integrated with existing knowledge graphs. This step is crucial for maintaining
consistency across the entire dataset and ensuring that new information is
appropriately contextualized.

Lettria’s process for merging graphs involves aligning the newly created nodes and
edges with the existing graph’s structure. This includes identifying overlapping
entities, resolving conflicts between differing data sources, and ensuring that the
relationships between entities are accurately represented. The goal is to create a
unified knowledge graph that seamlessly incorporates new information without
losing the context or introducing inconsistencies.

This integration allows for a dynamic and continually evolving knowledge base,
where each piece of data enriches the overall understanding and capability of the
AI system.

Nodes that are created from multiple

documents are merged in the
FullPropertyGraph automatically by
adding them into the GraphDB

10
• Extracting Triplets from the Graph
Once the data has been fully integrated into the graph structure, the next step is to
extract relevant subgraphs—often represented as triplets (subject-predicate-
object)—that are directly pertinent to the user's query. This extraction is guided by
the embedding of relations in the graph. In this process, the system identifies the
most relevant nodes and relationships within the graph that align with the user’s
question, ensuring that the output retains critical context and connections.

For example, a query in a financial knowledge graph might involve identifying

relationships between market events, asset prices, and company reports. The
extraction of triplets helps narrow down the vast dataset into focused subgraphs
that directly address the question. This allows for efficient and targeted
information retrieval, ensuring that only the most relevant data is selected from
the larger graph.

By embedding relations, the system ensures that the extracted triplets maintain the
integrity of the graph’s original structure, preserving the nuanced interconnections
necessary for accurate responses.

• Feeding Triplets into the LLM with Customized Prompts

Once the relevant triplets are extracted, they are then fed into LLM as part of the
query’s context. Alongside the triplets, a customized system prompt may be
added, depending on the client or use case, to provide additional necessary
context for generating a high-quality final response.

This approach enhances the LLM's understanding by ensuring it has access to the
most critical data points from the graph, and the system prompt further refines the
query’s scope or intent. For certain clients—such as FIVES or Euronext—these
custom system prompts provide indispensable context that guides the model to
focus on the specific domain, ensuring that responses are tailored and aligned with
the industry or task requirements.

By using both extracted triplets and tailored system prompts, the LLM is able to
generate highly qualitative, context-aware answers. This approach allows
GraphRAG to go beyond simple retrieval, offering a richer, more nuanced
understanding of complex queries that require deep, domain-specific knowledge.

11
Role of Ontologies
Ontologies play a critical role in the success of GraphRAG, providing a structured
framework that defines the relationships and hierarchies between concepts within
the graph. In essence, an ontology serves as the backbone of the knowledge
graph, ensuring that the AI system understands the meaning and context of the
data it processes.

• Ontologies as a Foundation
Ontologies enable the creation of a common language that is understandable by
both humans and machines. By defining the classes, properties, and relationships
between different entities, ontologies allow the AI to navigate the data with a clear
understanding of how concepts are related. This is particularly important for tasks
that require nuanced understanding, such as word sense disambiguation or
contextual interpretation of complex documents.

For example, in the context of GraphRAG, Lettria might use an ontology to

represent the relationships within a body of legal texts. The ontology would define
the various legal concepts (such as "contract," "precedent," "clause") and their
interrelationships (such as "is part of," "refers to," "contradicts"). This structured
understanding allows the AI to accurately interpret and retrieve information based
on these relationships.

• Customizing Ontologies for Domain-Specific Needs

While general-purpose ontologies like DBpedia offer a broad framework for
understanding data, they may not always be suitable for specialized applications.
Lettria recognizes the need for customization and offers tools within its platform to
build or tailor ontologies that better fit the specific requirements of different
domains.

For instance, in the financial sector, a customized ontology might include specific
terminology and relationships unique to financial instruments, regulations, and
market behaviors. By using Lettria’s platform, enterprises can either create new
ontologies from scratch or modify existing ones to better capture the intricacies of
their data.

This customization ensures that the AI system not only understands the general
relationships within the data but also grasps the specific, domain-related nuances
that are critical for accurate analysis and decision-making.

12
Challenges of Implementing
GraphRAG
GraphRAG represents a powerful leap forward in AI, but building and maintaining
such systems is not without its challenges. Understanding these complexities is
crucial for enterprises considering implementation, as they underscore the need
for specialized expertise and infrastructure.

Complex Implementation
While the benefits of GraphRAG are undeniable, setting up and maintaining a
GraphRAG system can be technically challenging. Building the underlying
knowledge graphs requires specialized skills in areas like graph theory, ontology
management, and advanced natural language processing (NLP).

One of the first hurdles enterprises face is the need to structure data correctly—
defining relationships, resolving inconsistencies, and ensuring that the graph
reflects the real-world connections between entities. This task requires a deep
understanding of the domain, along with technical expertise, which may not be
readily available in many organizations.

Furthermore, integrating GraphRAG into existing systems can be difficult,

especially for enterprises with well-established legacy infrastructure. Aligning new
graph-based data with older systems, ensuring data quality, and maintaining
seamless integration over time can be resource-intensive and technically
demanding.

Resource Intensive
In addition to the complexity of setup, GraphRAG systems require significant
computational resources. Knowledge graphs can grow to substantial sizes,
particularly in data-heavy industries like healthcare, legal, and finance. The larger
and more complex the graph, the more memory and processing power is required
to manage it effectively. Querying these large-scale graphs—while ensuring
performance remains high—adds to the resource burden.

Moreover, maintaining such a system requires ongoing investment in specialized

personnel, hardware, and software. These resource demands are compounded
when scaling the system, particularly for organizations handling growing data
volumes and increasingly complex relationships.

13
Why Lettria is the Solution
The complexities of building and maintaining a GraphRAG system make expertise
and proper infrastructure critical to success. Lettria is uniquely positioned to help
enterprises navigate these challenges and unlock the full potential of GraphRAG.
Here's why:

• Comprehensive Expertise
Lettria brings extensive knowledge in graph theory, NLP, and ontology
management, ensuring that your GraphRAG implementation is not just functional
but optimized for your specific needs.

• Efficient Integration
We understand the intricacies of integrating new graph-based systems with legacy
infrastructure. Lettria simplifies this process, helping you seamlessly incorporate
GraphRAG into your existing workflows without the usual friction.

• Scalable Solutions
Our platform is designed to handle the scaling complexities associated with
GraphRAG, ensuring high performance as your data and graph complexity grow.

By choosing Lettria, you gain a trusted partner that can manage the technical
complexities and resource demands, allowing your team to focus on strategic
initiatives while we handle the heavy lifting. Lettria’s end-to-end approach ensures
that your enterprise maximizes the value of GraphRAG technology without being
overwhelmed by its inherent challenges.

14
Use Cases
GraphRAG has broad applications across a variety of industries, offering
transformative potential for enterprises dealing with complex and unstructured
data. By preserving relationships, context, and nuance, GraphRAG enhances the
accuracy and relevance of AI-generated insights in fields that demand precision,
accountability, and contextual awareness. Below are some of the key use cases
for GraphRAG.

Healthcare
In the healthcare sector, data is often vast, complex, and highly fragmented,
ranging from medical records to clinical trial results and treatment protocols. The
ability to extract meaningful insights from this diverse data is critical for improving
diagnostics, treatment outcomes, and personalized medicine.

GraphRAG’s Role
GraphRAG excels at integrating complex patient data into a coherent, queryable
knowledge graph. Medical records, lab results, imaging reports, and genetic data
can be organized in a way that maintains the relationships between various data
points—such as symptoms, diagnoses, treatments, and outcomes. This contextual
understanding is vital for making accurate diagnoses and developing personalized
treatment plans.

Financial Services
The financial services industry deals with an overwhelming amount of data, much
of which is highly complex and interconnected. Whether it’s monitoring
transactions for fraud, assessing credit risk, or managing investments, accuracy
and context are crucial for making sound decisions. Traditional systems often
struggle to handle this complexity, particularly when it comes to integrating
diverse financial documents and datasets.

GraphRAG’s Role
GraphRAG can be deployed to analyze complex financial documents—such as
transaction records, balance sheets, or regulatory filings—by converting these
into structured, queryable graphs. In fraud detection, for instance, GraphRAG
allows institutions to map out relationships between transactions, accounts, and
individuals, identifying suspicious patterns or anomalies in real-time.

15
Industrial Maintenance
Industries reliant on heavy equipment and machinery—such as manufacturing, oil
and gas, and transportation—depend on efficient and timely maintenance to
ensure operational continuity. Maintenance operations are typically informed by a
range of complex technical documentation, including equipment manuals,
maintenance schedules, and sensor data from connected devices. However,
managing and analyzing this diverse data to prevent downtime or costly repairs
can be a challenge.

GraphRAG’s Role
In industrial maintenance, GraphRAG can optimize operations by organizing vast
amounts of technical documentation into a coherent knowledge graph.
Maintenance teams can query the system to identify specific procedures, part
specifications, or historical performance data for a particular machine, with the
graph ensuring that all relevant information is connected and easily accessible.

Legal & Compliance

Legal departments and compliance teams face the daunting task of managing
vast repositories of legal documents, regulations, contracts, and case precedents.
Ensuring compliance with industry regulations and staying updated with evolving
legal standards requires AI systems that can handle not only the volume of
information but also the intricate relationships between legal concepts.

GraphRAG’s Role
GraphRAG is particularly effective in managing and querying legal documents. By
converting legal texts into a graph structure, where clauses, contracts, and
precedents are represented as interconnected nodes, the system can ensure that
all relevant legal references and relationships are preserved.

16
v
Strategic Importance
In today’s data-driven landscape, the ability to effectively harness and interpret
vast amounts of unstructured information is paramount for enterprises seeking to
maintain a competitive edge. GraphRAG stands out as a transformative
technology that addresses the limitations of traditional AI models by preserving
the intricate relationships and contextual nuances within data. This preservation
leads to enhanced accuracy, enabling more precise and relevant AI outputs that
drive informed decision-making.

The strategic advantages of adopting GraphRAG are multifaceted:

• Enhanced Accuracy
By maintaining the context and relationships within data, GraphRAG ensures that AI
models produce more accurate and reliable results. This is crucial for industries
where precision is essential, such as healthcare, finance, and legal services.

• Transparency and Explainability

GraphRAG’s structured approach allows for greater transparency in AI processes.
Enterprises can trace the origins of AI-generated insights, fostering trust and
accountability. This transparency is particularly important in regulated industries
where explainability is a legal and ethical requirement.

• Flexibility and Scalability

GraphRAG is designed to handle increasingly complex and diverse data sets. Its
scalable architecture ensures that as an enterprise’s data grows, the system can
expand without compromising performance or accuracy. This flexibility makes
GraphRAG a sustainable solution for long-term AI strategy.

Adopting GraphRAG enables enterprises to transform their data into a strategic

asset, driving innovation and operational excellence. By leveraging the full
potential of their complex, unstructured data, organizations can unlock deeper
insights, optimize processes, and gain a significant competitive advantage in their
respective markets.

18
Why Choose Lettria’s GraphRAG Solution?
• Expertise and Innovation
Lettria is at the forefront of AI innovation, dedicated to developing cutting-edge
solutions that address the complex needs of modern enterprises. Our expertise in
graph theory, ontology management, and AI integration ensures that our
GraphRAG solution is both robust and effective.

• Customized Solutions
Recognizing that each industry has unique requirements, Lettria offers
customizable ontologies and tailored graph structures. This ensures that the
GraphRAG implementation aligns perfectly with your specific data and business
needs.

• Seamless Integration
Lettria’s GraphRAG solution is designed to integrate seamlessly with your existing
systems and workflows. Our comprehensive support and implementation services
ensure a smooth transition, minimizing disruption and maximizing value from day
one.

Take the Next Step Towards AI Excellence

Embrace the future of enterprise AI by integrating Lettria’s GraphRAG
solution into your data strategy. By doing so, you can unlock the full
potential of your complex, unstructured data, drive more accurate and
insightful decision-making, and position your organization as a leader
in AI-driven innovation.

19
Contact
Lettria Today

Discover how Lettria’s GraphRAG can

transform your enterprise AI capabilities.
Contact us to schedule a demo, discuss
your specific needs, and learn how our
solutions can help you achieve your
strategic objectives.

Charles Borderie
CEO @Lettria
[email protected]