Power BI Architecture

End-to-End
Melissa Coates
Coates Data Strategies

March 10, 2021

Download slides: CoatesDS.com/Presentations

Melissa Coates
Data architect | Technical trainer | Consultant
Specialist in Power BI governance & administration
Microsoft Data Platform MVP

Owner of Coates Data Strategies

@SQLChick | @CoatesDS

Power BI Deployment & Governance Workshop

Power BI Data Culture & Architecture Security & Adoption &

Governance Center of Excellence Decisions Data Protection Administration
Power BI Architecture End-to-End
Agenda

Overview, Options Other Key

Authoring for Components in
Tools & Content Delivery the Architecture
Data Storage
Modes Data Gateway
Power BI Service
Power BI Report Server Power BI Premium
Embedding Dataflows
Other Other
Where to Download Materials

Slides:
CoatesDS.com/Presentations

Diagram:
CoatesDS.com/Diagrams
Power BI Overview,
Authoring Tools &
Data Storage Modes
What is Power BI?
A suite of tools to support both self-service BI and enterprise BI:
• Data acquisition • Calculations
• Data preparation & cleansing • Reports & visuals
• Data modeling • Content delivery & viewing

Power BI is a broad collection

of tools, services, and features
Data Architecture Components
Ingest Prepare Model Serve

Access to Data cleansing, Data warehouse Reports

original preparation & and/or &
source data transformations semantic model dashboards

Power BI can be used for some or all of these components

Authoring Tools
Power BI Authoring

Source Query Data Reports &

Data Editor Calculations Visuals
Relationships
Skills: Data Data Data
preparation modeling visualization
Languages: Power Query Data Analysis
Formula eXpressions
Language (“M”) (“DAX”)
Native Tools for Creating Power BI Content
Power BI Desktop (.pbix) +
Power BI Desktop for Report Server (.pbix)
Power BI Paginated Report Builder (.rdl)

Excel (.xlsx)

Power BI Service
Azure Synapse Studio

Visual Studio Code

Visual Studio SQL Server Data Tools
Tools for Creating Content
Native tools intended for self-service authors:
Power BI Paginated Power BI
Excel
Desktop Report Builder Service

Native tools intended for more advanced authors:

Azure Synapse Visual Studio Visual Studio
Studio Code SQL Server Data Tools
External tools for advanced dataset authoring & deployment:
DAX Tabular ALM + many more
Studio Editor Toolkit community tools
Data Storage Modes
Dataset Storage Modes Data Data
stored in remains
Power BI in source
Data is imported to Power BI’s in-
Import memory columnar data storage

Queries are passed at execution

DirectQuery time to a relational data source Forms of
data
Queries are passed at execution virtualization:
Live
time to an Analysis Services model querying
Connection
or a Power BI dataset the data
where it lives
Composite A combination of Import, Live
Model Connection and/or DirectQuery
Terminology Clarifications
Live Connection -vs- DirectQuery
Conceptually similar in that they both query data which resides elsewhere.
Examples:
• Live Connection from a report to a Power BI dataset or Azure AS
• DirectQuery from a report to an Azure SQL Database

Two Types of DirectQuery

(1) DirectQuery to (usually) a relational database Standard DirectQuery mode
(2) DirectQuery for Power BI datasets & Azure One type of composite model
Analysis Services models
Data Storage Modes
Local model Remote model
Data Import Mode Live Connection Mode
stored in A PBIX file which contains A PBIX file with reports that
Power BI both the schema & the obtain data by querying a
data Power BI shared dataset
Data DirectQuery Mode Live Connection Mode
remains A PBIX file which contains a A PBIX file with reports that
in source model that is schema only; obtain data by querying an
data is obtained by querying Analysis Services Tabular
the data source when report Model
is executed
Using a Shared Dataset w/ Live Connection

Sales Analysis
Shared dataset: Dataset

Reports make a live connection to

the remote model to access data

MoM
YTD Sales Daily Sales Sales by
Sales
Reports: Revenue Targets Product
Trends
Extending & Customizing a Shared Dataset
DirectQuery for Power BI Datasets and Azure Analysis Services

Organizational
Shared Sales Analysis Dataset
dataset:
Live Connection is converted to a DirectQuery connection

Composite Organizational data:

model: Departmental DirectQuery to remote model
Sales Analysis
Dataset Departmental data:
Local model
Customize & extend organizational model
with additional data sources & calculations
Options for
Content Delivery
with Power BI
Power BI Service
Publishing to the Power BI Service
Power BI
Service
Power BI publish
Desktop .pbix file My Workspace
publish view
Workspace App content
Excel .xlsx file
A A

Paginated publish Workspace App

.rdl file B B
Report Builder
Collaborate on content
creation & validation
Storing Content in the Power BI Service
My Workspace
Personal Content & Data
Content
& Data Workspace
Organizational
Dashboards,
Content & Data
reports,
datasets,
dataflows
Viewing Content in the Power BI Service
My Workspace
Content
Personal BI Sharing
Distribution
& Data Workspace of individual
Team collaboration & pieces of
Dashboards,
small team distribution content
reports,
datasets,
dataflows
App
Broad content distribution
Objects in the Power BI Service
Dashboard 1 Dashboard 2

Workspace
Report 1 Report 2 Report 3 Report 4

Dataset 1 Dataset 2

Dataflow A
Licenses for Using Power BI Service
User Licenses:
User
Free Personal analytics (my workspace) identity
Team collaboration and content from Azure
Pro Active
distribution (workspaces, apps, sharing)
Directory
PPU Add-on to Pro for most Premium features

Core-Based Capacity Licenses (optional):

Premium Broad content distribution scenarios
+ additional features
Top Reasons for Using the Power BI Service
Collaboration, Sharing & Content Distribution

Delivering reports using Multiple methods & security

Power BI Desktop on a file options to collaborate with both
system is difficult to do well internal and external users based
& is not user-friendly on Azure Active Directory
Top Reasons for Using the Power BI Service
Collaboration, Sharing & Content Distribution

Friendler experience Integration with Microsoft Teams:

for report consumers full capabilities of Power BI Service
with apps + easier to so users can be productive where
discover content they already are working
Top Reasons for Using the Power BI Service
Collaboration, Sharing & Content Distribution

Row-level security Use of the Power BI Report & dashboard

works mobile app & the comments
Windows application
Top Reasons for Using the Power BI Service
Automation

Subscriptions Scheduled Integration with

data refresh Power Automate
& Power Apps
Top Reasons for Using the Power BI Service
Centralization & Reusability

Centralized dataset Certified & promoted Centralized data

reusability across (endorsements) for cleansing with Power
numerous reports datasets & dataflows BI dataflows
with shared datasets
& live connections
Top Reasons for Using the Power BI Service
Governance, Compliance & Administration

Tenant settings to manage Administrator visibility into

features & user experience usage & activity logging
+ administrator access to all for adoption,
workspaces to support user administration & support
population
Top Reasons for Using the Power BI Service
Governance, Compliance & Administration

Integration with Integration with Microsoft

Microsoft 365 information Cloud App Security to
protection & sensitivity labels implement data loss
prevention policies
Power BI Report Server
Power BI Report Server (PBIRS)

Power BI Power BI collaborate

Desktop for Report Server on content
publish creation &
Report Server .pbix file validation

publish
Excel .xlsx file view
Web Server content

Paginated publish Report Server

.rdl file Report Server TempDB
Report Builder
Top Reasons for Using PBIRS

Need an alternative to Existing significant Desire a simple folder-

the Power BI Service investment in based reporting portal;
for delivery of reports Reporting Services a slower release
cadence & a subset of
feature support is ok
Embedding Solutions
Embedding Content in a Custom App
Power BI
collaborate
Service on content
Power BI publish Workspace creation &
.pbix file validation
Desktop Report
JavaScript +
REST APIs

Custom App view

content
iFrame
Report
No-Code Embedding
Power BI
collaborate
Service on content
Power BI publish Workspace creation &
.pbix file validation
Desktop Report
Generate Secure
Embed Code

SharePoint view
content
iFrame
Report
Options for Embedding Content
Internal Embedding External Embedding
Embed for Embed for
Embed for productivity Public
(no-code) your org customers
(user owns data) (app owns data)

URL Embedding Embedding Embedding REST APIs: REST APIs: Publish to web:
for internal websites in in Custom portals ISV apps and public portals, open
and portals Teams SharePoint and custom apps services data initiatives

Secured by organizational Azure AD Secured by No user

application authentication
authentication
Image adapted from: Planning a Power BI Deployment Whitepaper
Using Other Tools
Using the XMLA Endpoint
XMLA = XML for Analysis
An industry standard protocol supported by BI vendors
Interacts with dataset via the Tabular Object Model (TOM)

Power BI Power BI Service

Desktop XMLA endpoint Premium or PPU
for data
Workspace
Tabular management &
Editor deployment Dataset

ALM XMLA endpoint

Toolkit for querying
Other Key Components
in a Power BI Architecture
On-Premises Data Gateway
Three Types of Gateways
Virtual Network
Data Gateway
Power BI
Service Personal
Mode
Data
Gateway
Standard Mode
Data Gateway Cluster
Three Types of Gateways

VNet Standard Mode Personal Mode

For use by Many users Many users One user
Managed by Microsoft Customer (Admin) Customer (User)
Supports:
Data Refresh Yes Yes Yes
AAD SSO (DQ-PBI dset) Yes --No-- --No--
Standard DQ (+src SSO) Yes Yes --No--
Live Connection Yes Yes --No--
R & Python --No-- --No-- Yes
When is a Gateway Needed?
In the Power BI Service: AND Data Source Is Located:
Refreshing imported Data center within corporate network or
datasets on-premises
Refreshing dataflows Cloud-based virtual machine (IaaS:
infrastructure as a service)
Using DirectQuery
Cloud-based database in a VNet (PaaS:
Using Live Connection platform as a service in a virtual network)
for Analysis Services
OR Certain Functionality is Used:
Web.Page() function
Single M query combines cloud & on-
prem data
Standard Mode Data Gateway
Administrators

Gateway Data Source 1 Users

Cluster Data Source 2 Users

Data Source 3 Users

Dataset
Virtual Network Data Gateway
Administrators

VNet Data Data Source 1 Users

Gateway Data Source 2 Users

Data Source 3 Users

<Virtual Network>

Dataset
Power BI Premium & Premium Per User
 4 Workspace Types are Based on Licensing
User-based My Workspace Power BI Free license
named licensing Pro Workspace Power BI Pro license

+ User-based PPU Workspace Premium Per User (PPU)

Premium licensing
license
Gen 2

Capacity-based Power BI Premium (P or EM)

Gen 1

Premium
Premium licensing Workspace or
Power BI Embedded (A SKUs)
 Who Can Access a Workspace? Free Pro PPU
User User User

User-based My Workspace X X X
named licensing Pro Workspace X X

+ User-based PPU Workspace X

Premium licensing
Gen 2

Capacity-based
Gen 1

Premium X X X
Premium licensing Workspace
Top Reasons for Using Power BI Premium
Unlimited Content Distribution to Free Users

Large number of Free viewers

can be more cost-effective
than Pro licenses
Not available with PPU
Top Reasons for Using Power BI Premium
Features to Bring Enterprise BI Capabilities to Self-Service BI

XMLA Paginated reports in

Deployment endpoint the Power BI Service
pipelines read/write

Full featureset for Near real-time reporting:

dataflows change detection with
automatic page refresh
Top Reasons for Using Power BI Premium
Integration with Azure Apps

Azure Cognitive Azure Machine

Services Learning
Top Reasons for Using Power BI Premium
Scalability

Auto-scale v-cores for 24

More frequent hours to handle
data refreshes Large datasets occasional bursts of traffic
Up to 100GB if PPU;
Gen 2 only
400GB if Premium
Top Reasons for Using Power BI Premium
Regulatory & Privacy Concerns

Bring-your-own Data sovereignty:

encryption key (BYOK) specific geography for
with Azure Key Vault data storage
Available with PPU only if the
Not available with PPU
entire tenant is BYOK
Top Reasons for Using Power BI Premium
Hybrid Cloud

Use of Power BI Report Server as an

alternative report portal
Not available with PPU
Power BI Dataflows
Dataflows in the Power BI Service
Dashboard 1 Dashboard 2

Workspace
Report 1 Report 2 Report 3 Report 4

Dataset 1 Dataset 2

Dataflow A
Dataflows
Power BI
Service
create
dataflow in Workspace
Power Query
Online Dataflow

use dataflow as data

source for datasets

Power BI Power BI
Desktop Desktop
Top Reasons for Using Power BI Dataflows

Reusable self-service Standardized, Reduce load &

data cleansing & consistent data queries sent to
preparation reduces consumed by many source systems
repetitive work by different datasets
data modelers
Other Components
Integration with Microsoft 365
M365 Audit Log → Power BI Activity Log
User licenses & Power BI Premium licenses
OneDrive for Business and SharePoint
Teams
Integration with Power Platform

Power Automate (Flow)

Power Apps
Common Data Service
Common Data Model
Power Platform Admin Center
Integration with Azure Services
Azure Active Directory:
Users, Groups, AAD Apps, Service Principals, B2B
Power BI Embedded (A-series SKUs)
Azure Data Lake Storage Gen 2 (dataflows storage)
Azure Premium Files (large dataset storage)
Azure Cognitive Services Azure Machine Learning
Azure Synapse Analytics Azure Purview
Integration with Azure Security &
Governance Services
Microsoft Cloud App Security
Microsoft Information Protection
Azure Key Vault
InTune (mobile app management)
Wrap-Up,
Q&A,
Links to More Info
Q&A
Finding More Info from Melissa
Slides: Diagrams:
CoatesDS.com/Presentations CoatesDS.com/Diagrams

YouTube: Blog:
YouTube.com/CoatesDataStrategies CoatesDS.com/Blog-Posts

Power BI Governance Training: Twitter:

CoatesDS.com/Workshop @SQLChick | @CoatesDS