CHECKLIST 2023

The Semantic Layer’s

Critical Roles in Modern
Data Architectures
By Philip Russom
 The Semantic Layer’s Critical Roles
in Modern Data Architectures
By Philip Russom

M
any of the most exciting innovations and
advancements in data management today Understanding the semantic layer’s critical roles:
are occurring within the semantic layer
of data architectures and data stacks. For example, 1 Understand what a semantic layer should
we are witnessing new or improved approaches be and do
to semantic modeling, data cataloging, and data
lineage. Even older forms of managing semantics— 2 Consider why your data architecture needs a
such as metadata and virtualization—are being modern semantic layer
infused with new techniques for agile modeling,
performance optimization for logical and virtual 3 Recognize where a semantic layer fits in
data environments, and intelligent augmentation today’s data architectures
(i.e., tool algorithms driven by machine learning and
graph analytics). 4 Understand how the semantic layer helps
remodel distributed data
The innovations of the semantic layer also play a
role in improving large-scale data and analytics 5 Appreciate how a semantic layer contributes
architectures. For example, the new definition of data to common data architectures
fabric is not possible without a modern semantic
layer, and the semantic layer can be a backbone for
unifying new data and analytics architectures in the

1   TDWI RESEARCH tdwi.org

 TDWI CHECKLIST REPORT: THE SEMANTIC LAYER’S CRITICAL ROLES IN MODERN DATA ARCHITECTURES

cloud. Furthermore, a well-designed semantic layer tool types, data platforms, use cases, and business
allows analytics teams to define business metrics, departments without favoring one over another.
hierarchies, and dimensions on top of big data while
providing a means to centrally govern data access The data descriptions (or “data about data”) created
and deliver high-performance interactive queries. and managed by a semantic layer may take the form
of older techniques (such as metadata management,
This TDWI Checklist educates data and analytics federation, and virtualization) or newer ones (such
leaders about modern platforms and practices for the as data catalogs, data lineages, dimensional models,
semantic layer. It does so by discussing five beneficial or automated generation of data descriptions via
characteristics of the modern semantic layer but in knowledge graphs). Whether the semantic layer is
the context of the semantic layer’s critical roles in a unified environment from a vendor or assembled
modern data architectures. by technical users as a “best of breed” collection
of multiple tools from multiple vendors, it should
NOTE: This report assumes the reader is familiar support many semantic tool functions.
with data architectures. Readers needing more
details can read the TDWI Checklist Report: Six A semantic layer platform must go beyond data
Requirements for the Modern Data and Analytics definitions to provide rich capabilities in semantic
Cloud Stack. modeling and data modeling. In other words, a tool
for the semantic layer should actively support the
creation of new data structures and data products
(whether federated, virtual, logical, or data sets in
storage), not just descriptions of source data and its
Understand what a
1 semantic layer should
characteristics.

be and do The semantic layer manages the interactions

between data consumers (whether they be humans
The semantic layer has been around for many years or AIs) and enterprise data assets (typically stored
in many guises. The earliest ones originated as in the cloud). This interaction hinges on a semantic
functionality embedded and buried within larger model that maps the language of raw data (e.g., field
tools, typically for reporting, enterprise business and table names, file and database formats) with
intelligence, data integration, and database business concepts (e.g., revenue, ship quantity, fiscal
management. The catch was that such semantic quarter). The structure of the semantic model may
layers were usually optimized for their parent tool take different forms, defined using various ways of
and not much else. describing data.

Today, a truly modern semantic layer is a standalone A semantic layer must translate data consumer
tool type that provides data semantics services for requests to the flavor of SQL preferred by the source
multiple tools within a multitool and multiplatform data platform. It must accommodate multiple
data architecture. This gives the modern semantic inbound protocols (not just SQL) because tools
layer the ability to serve many architectural layers, themselves support different protocols. For example,

2   TDWI RESEARCH tdwi.org

 TDWI CHECKLIST REPORT: THE SEMANTIC LAYER’S CRITICAL ROLES IN MODERN DATA ARCHITECTURES

to support Excel, the semantic layer should support and other security for systems the semantic layer
MDX. For data science, it should support Python. For accesses
application developers, it should support REST.
• Friendly descriptions of data that simplify and
In production, a semantic model solution must improve modern data practices (e.g., self-service,
deliver “speed of thought,” direct query performance dashboard customization)
through automated performance optimization. Query
performance is imperative because without it, many • Reuse of composable data objects and data
end users will make redundant and non-governed products listed in the semantic layer
copies of data in the form of data extracts (TDEs in
Tableau) or data imports (Power BI, Qlik, etc.). One • Automation for data governance, monitoring via
of the greatest benefits of a modern semantic layer, data, audit, and data observability
extended with query optimization and virtualization,
is that the semantic layer serves as an abstraction • When done well, can elevate data literacy and
layer for governance, security, and “single source democracy
of truth.”
Today’s data architectures are trending toward
centralized data organization paradigms (databases,
data lakes, data warehouses, data science labs)
within both cloud and on-premises architectures.
Consider why your data
2 architecture needs a
Even when physically consolidated, many data
environments are not logically organized to support
modern semantic layer direct analytics use.

The semantic layer provides a collection of data A semantic layer helps to unify far-flung data
descriptions (and tools to create and maintain them) architectures in that it:
to make a single, centralized, and standardized
architectural layer for most data semantics. Being • Describes data consistently for all layers of the
centralized, a semantic layer delivers a standardized architecture and beyond
and consistent way of representing enterprise data to
different types of users, tools, and data management • Reaches all platforms: multiple brands, on
processes. multiple clouds or on premises, etc.

This centralized approach simplifies many things, • Can provide data views that incorporate data
and it delivers important architectural benefits: from many architectural layers and elsewhere

• Provides a process that is a governed, • Facilitates data access and interfacing for many
standardized, and consistent way of representing users and tools
distributed enterprise data
• Can enable virtualization for the logical data
• Can be a single point of entry, with single sign-on warehouse and virtual data lake

3   TDWI RESEARCH tdwi.org

 TDWI CHECKLIST REPORT: THE SEMANTIC LAYER’S CRITICAL ROLES IN MODERN DATA ARCHITECTURES

Because it is an abstracted layer, the semantic layer layers; the data fabric gets the advanced semantic
creates a kind of future proofing. By decoupling the functionality and automation it needs; but the whole
layers of data consumption tools and data storage data architecture still has access to standalone
platforms, a semantic layer provides IT with the and independent semantic layer tooling. This also
freedom to consolidate, move, or transition its data creates a unification effect, which is beneficial to
without disrupting end-user data consumption. A multiplatform architectures.
semantic layer also provides an open platform for
DATA CONSUMPTION
plugging in new or different data consumption tools
as they arise. Data Science Business Intelligence
Self-Service

As you can see, the business and technology DATA FABRIC

benefits of a semantic layer are many, which is why Semantic Layer
organizations are turning to it. Real-Time Data Layer
Pipelines, ETL, Quality, etc.
DATA STORAGE

Recognize where a Data Lake Data Warehouse

3 semantic layer fits in
Data Lakehouse
SOURCE: Philip Russom

today’s data Figure 1. The semantic layer’s strategic position in

architectures the data architecture.

The modern semantic layer can support multiple

A modern semantic layer can be deployed to serve
styles of data architecture, including those that are
as an independent layer near the top of today’s data
centralized, distributed, or a combination of both.
architectures (see Figure 1). It fits just below the
For example:
data consumption layer (for reporting, analytics, and
self-service) and just above the data integration and
• A semantic layer provides centralized data
streaming functions of the data fabric. It represents
semantics, which in turn can support the
both data traveling into the data fabric and later
centralized data that many users want in new
being processed by the fabric, as well as data at rest
data architectures (especially in the cloud)
in data storage (both on premises and in the cloud),
typically organized as data lakes, warehouses, and
• An independent semantic layer also supports
other storage options.
decentralized and distributed design patterns,
including the data mesh
This is a strategic position for the semantic layer,
between the data consumption and data fabric
• The semantic layer can support hybrid
layers. This location enables it to serve multiple
combinations of multiple architectural variations,
functions: users and their tools for BI and analytics
even when in the cloud or on premises
get a broader enterprise view of data compared
to the limited views from their internal semantic

4   TDWI RESEARCH tdwi.org

 TDWI CHECKLIST REPORT: THE SEMANTIC LAYER’S CRITICAL ROLES IN MODERN DATA ARCHITECTURES

Essentially, a modern, independent semantic exposed multiple ways to multiple users, teams, and
layer can support just about any variation of data tools. This is so that users can avail themselves of
architecture available today. multiple data consumption styles, unlike the limited
approaches typical of embedded semantic layers or
traditional data warehouses.

As another example, data lakes are known for their

Understand how the
4 semantic layer helps
massive stores of raw data. The semantic layer can
make this data far more useful and valuable by
remodel distributed data creating a variety of views into the lake. This creates
a logical data lake that complements and adds value
As we just saw, a modern semantic layer can to the actual lake in storage but without creating
be applied to either a physically centralized redundant copies and without having to maintain
environment (data lake or warehouse) or to a data aggregates in storage.
distributed environment that includes many data
sets and platform types (as results from many
unplanned or uncontrolled data programs).
Appreciate how a
In these cases, a semantic layer often acts as an 5 semantic layer
abstraction layer that unifies disparate layers of a
data architecture and its diverse tools and platforms. contributes to common
It does this by supporting business metadata and
data architectures
other approaches to creating business-friendly views
of data and data products. The resulting business-
A semantic layer can add many types of technical
friendly data models (created at the semantic layer,
and business value to a variety of local data
not physically in storage) hide the complexity of
architectures across an enterprise. This is true
data in the data fabric and storage layers of the
whether the semantic layer is retrofitted to a mature
architecture, thus making distributed data easier to
deployment or utilized from the beginnings of that
understand and access for a wide range of users and
architecture. In particular, the semantic layer can be
applications.
beneficial to data warehouses, data lakes, data fabric,
and DataOps.
Ideally, data views may be constructed as semantic
models, relational tables, OLAP dimensions, metric
hierarchies, or time series. This means that a modern THE SEMANTIC LAYER HELPS THE
semantic layer tool should be capable of modeling DATA WAREHOUSE (DW)
most of the data structures common in data The semantic layer modernizes the data
architecture usage, especially for analytics use cases. semantics that a DW depends on. For example,
most DWs suffer from metadata that is sparse,
For example, many organizations are hoping for incorrect, and nonstandard; a semantic layer tool
one data source (or collection of data sources) helps to correct these problems and give metadata a

5   TDWI RESEARCH tdwi.org

 TDWI CHECKLIST REPORT: THE SEMANTIC LAYER’S CRITICAL ROLES IN MODERN DATA ARCHITECTURES

higher quality. Similarly, some DWs operate almost THE SEMANTIC LAYER HELPS THE
exclusively with technical metadata; a semantic layer DATA FABRIC
helps the DW team embrace business metadata and
The most recent definition of the data fabric is “an
more advanced forms of semantics, such as the data
architecture for unifying and governing multiple data
catalog and data lineage.
management and data semantics disciplines, from
data integration and quality to metadata and data
The business-friendly semantics of the semantic
cataloging.” Among other requirements, a data fabric
layer enable DWs to participate in more use
requires sophisticated semantics that are centralized,
cases. This is particularly useful when less-technical
standardized, and shared for the many tool types
users want to access the data in a DW for self-service
found in modern data fabrics. The semantic layer
or when users need to personalize their management
satisfies this requirement.
dashboards with metrics and KPIs from a DW.

The semantic layer is a key enabler for the logical THE SEMANTIC LAYER HELPS DATAOPS
DW. A logical DW is inherently multiplatform, All data-driven development processes benefit from
hybrid, and distributed. To make this complicated better semantics and DataOps is such a process. In
microarchitecture seem more unified and usable, fact, the semantic layer can help DataOps achieve
DW professionals use data virtualization and data many of its key objectives by providing:
views. Because a semantic layer is inherently logical,
virtual, and view-driven, it can be a natural addition • Centralized, standardized, and shared data
to a DW to make it a true logical DW. semantics for data engineering but with more
features than a metadata repository or catalog
THE SEMANTIC LAYER HELPS THE
DATA LAKE • Automation for data semantics to accelerate the
delivery of data products
The semantic layer helps a data lake avoid
becoming a data swamp. A lack of metadata
• Semantic modeling (faster than models based on
and other data semantics is the leading cause of
aggregation)
swamps; a semantic layer provides ample metadata
management for this situation.
• Reduced data prep and design work

The semantic layer provides logical structures

• Quick turnaround for the CI/CD processes of data
for unstructured data lakes. Most lake data is raw,
products
file-based, and multistructured. The semantic layer
gives this environment much-needed structure via
logical data views without creating redundant data
copies and aggregations. In turn, the views make
the raw and unimproved data of a lake far more
accessible for self-service, exploration, and analytics.

6   TDWI RESEARCH tdwi.org

 TDWI CHECKLIST REPORT: THE SEMANTIC LAYER’S CRITICAL ROLES IN MODERN DATA ARCHITECTURES

• Look for semantic layer tools with innovative

Concluding Thoughts and advanced functions. A comprehensive tool

will support multiple forms of data semantics and
be open and tool-agnostic. It will also support
Organizations wishing to modernize their data
performance optimization, data virtualization,
semantics by adopting a semantic layer should heed
and multiple approaches to modeling.
the following key takeaways and recommendations:

• Take data semantics more seriously, in

general. Make it a priority, from storage to
data engineering to data consumption.

• Do more than just create and manage

technical metadata. Modernize with business
metadata, cataloging, data lineage, and tools
optimized for the unified semantic layer.

• Recognize that today’s semantic layer is a

standalone architectural layer. It is not buried
inside a single tool. It is standalone, so it can
be agnostic and support many tools, users, and
use cases.

• Adopt the semantic layer concept as a best

practice and architecture standard. As a
best practice, it improves data access, reuse,
standards, and engineering. As an architectural
layer, it unifies the whole architecture, enables
interoperability, and presents architecture-wide
observability.

• Deploy a modern semantic layer for its

business and technical benefits. For example, it
enables new practices, such as the data fabric and
next-level data virtualization. It also modernizes
and elevates common data solutions, namely the
warehouse, lake, fabric, and DataOps.

7   TDWI RESEARCH tdwi.org

 TDWI CHECKLIST REPORT: THE SEMANTIC LAYER’S CRITICAL ROLES IN MODERN DATA ARCHITECTURES

About our sponsor About the author

Philip Russom has 25 years of
experience as an IT industry analyst
atscale.com researching user best practices, vendor
products, and market trends in data
AtScale enables smarter decision-making by management and analytics. This
accelerating the flow of data-driven insights. includes data warehousing, data lakes,
The company’s semantic layer solutions simplify, data integration, data quality, hybrid data
accelerate, and extend business intelligence and architectures, cloud data management, data
data science capabilities for enterprise customers governance, analytics, and data platforms.
across all industries. With AtScale, customers
are empowered to democratize data, implement He has worked at most of the world’s leading IT
self-service BI, and build a more agile analytics analyst firms, namely: Gartner Inc., Forrester
infrastructure for better, more impactful Research, Giga Information Group, TDWI, and
decision-making. Hurwitz Group. In those positions and others, he
produced over 650 research reports, magazine
For more information, visit atscale.com. articles, speeches, and webinars. Before becoming
an industry analyst, he worked for database software
vendors as a product manager, product marketer, and
documentation writer.

8   TDWI RESEARCH tdwi.org

 TDWI CHECKLIST REPORT: THE SEMANTIC LAYER’S CRITICAL ROLES IN MODERN DATA ARCHITECTURES

About TDWI Research About TDWI Checklist Reports

TDWI Research provides industry-leading research TDWI Checklist Reports provide an overview of
and advice for data and analytics professionals success factors for a specific project in business
worldwide. TDWI Research focuses on modern data intelligence, data warehousing, analytics, or a related
management, analytics, and data science approaches data management discipline. Companies may use
and teams up with industry thought leaders and this overview to get organized before beginning a
practitioners to deliver both broad and deep project or to identify goals and areas of improvement
understanding of business and technical challenges for current projects.
surrounding the deployment and use of data and
analytics. TDWI Research offers in-depth research
reports, commentary, assessments, inquiry services,
and topical conferences as well as strategic planning
services to user and vendor organizations.

A Division of 1105 Media

6300 Canoga Avenue, Suite 1150
© 2023 by TDWI, a division of 1105 Media, Inc. All rights reserved. Reproductions in whole or
Woodland Hills, CA 91367 part are prohibited except by written permission. Email requests or feedback to [email protected].

Product and company names mentioned herein may be trademarks and/or registered trademarks
E [email protected] of their respective companies. Inclusion of a vendor, product, or service in TDWI research does
not constitute an endorsement by TDWI or its management. Sponsorship of a publication should
tdwi.org not be construed as an endorsement of the sponsor organization or validation of its claims.

9   TDWI RESEARCH tdwi.org