This document provides a cheatsheet of common data transformation and analysis functions that can be used in Power BI. It includes functions for loading, cleaning, and preparing data as well as performing calculations on dates, text, numbers and conditional logic.
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0 ratings0% found this document useful (0 votes)
102 views10 pages
Power BI Data Storytelling
This document provides a cheatsheet of common data transformation and analysis functions that can be used in Power BI. It includes functions for loading, cleaning, and preparing data as well as performing calculations on dates, text, numbers and conditional logic.
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 10
# [ Power BI Data Storytelling ] [ cheatsheet ]
Data Loading and Transformation
● Load data from CSV: Source = Csv.Document(File.Contents("data.csv"),
[Delimiter=",", Columns=10, Encoding=1252, QuoteStyle=QuoteStyle.None]) ● Load data from Excel: Source = Excel.Workbook(File.Contents("data.xlsx"), null, true) ● Load data from SQL Server: Source = Sql.Database("server", "database", [Query="SELECT * FROM table"]) ● Load data from web: Source = Web.Page(Web.Contents("https://example.com")) ● Remove columns: Table.RemoveColumns(Source, {"Column1", "Column2"}) ● Rename columns: Table.RenameColumns(Source, {{"OldName1", "NewName1"}, {"OldName2", "NewName2"}}) ● Filter rows: Table.SelectRows(Source, each [Column] = "Value") ● Sort rows: Table.Sort(Source, {{"Column", Order.Ascending}}) ● Group by and aggregate: Table.Group(Source, {"GroupColumn"}, {{"AggColumn", each List.Sum([ValueColumn]), type number}}) ● Merge queries: Table.NestedJoin(Source1, {"KeyColumn"}, Source2, {"ForeignKeyColumn"}, "NewColumn", JoinKind.LeftOuter)
Data Cleaning and Preparation
● Remove duplicate rows: Table.Distinct(Source)
● Replace values: Table.ReplaceValue(Source, "OldValue", "NewValue", Replacer.ReplaceText, {"Column"}) ● Fill down missing values: Table.FillDown(Source, {"Column"}) ● Fill up missing values: Table.FillUp(Source, {"Column"}) ● Pivot data: Table.Pivot(Source, List.Distinct(Source[PivotColumn]), "PivotColumn", "ValueColumn", "AggregateFunction") ● Unpivot data: Table.UnpivotOtherColumns(Source, {"KeyColumn"}, "Attribute", "Value") ● Split column by delimiter: Table.SplitColumn(Source, "Column", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv), {"NewColumn1", "NewColumn2"}) ● Merge columns: Table.CombineColumns(Source, {"Column1", "Column2"}, Combiner.CombineTextByDelimiter(",", QuoteStyle.None), "NewColumn") ● Extract text before delimiter: Table.TransformColumns(Source, {{"Column", each Text.BeforeDelimiter(_, " "), type text}})
By: Waleed Mousa
● Extract text after delimiter: Table.TransformColumns(Source, {{"Column", each Text.AfterDelimiter(_, " "), type text}})
Date and Time Operations
● Convert to date: Table.TransformColumns(Source, {{"Column", each
Date.From(_), type date}}) ● Convert to time: Table.TransformColumns(Source, {{"Column", each Time.From(_), type time}}) ● Extract year: Table.TransformColumns(Source, {{"Column", each Date.Year(_), type number}}) ● Extract month: Table.TransformColumns(Source, {{"Column", each Date.Month(_), type number}}) ● Extract day: Table.TransformColumns(Source, {{"Column", each Date.Day(_), type number}}) ● Extract hour: Table.TransformColumns(Source, {{"Column", each Time.Hour(_), type number}}) ● Extract minute: Table.TransformColumns(Source, {{"Column", each Time.Minute(_), type number}}) ● Extract second: Table.TransformColumns(Source, {{"Column", each Time.Second(_), type number}}) ● Calculate age from birth date: Table.AddColumn(Source, "Age", each Date.Year(Date.From(DateTime.LocalNow())) - Date.Year([BirthDate]), type number) ● Calculate difference between dates: Table.AddColumn(Source, "DaysDiff", each Duration.Days(Duration.From([EndDate] - [StartDate])), type number)
Text Operations
● Convert to uppercase: Table.TransformColumns(Source, {{"Column",
Text.Upper, type text}}) ● Convert to lowercase: Table.TransformColumns(Source, {{"Column", Text.Lower, type text}}) ● Trim whitespace: Table.TransformColumns(Source, {{"Column", Text.Trim, type text}}) ● Remove numbers: Table.TransformColumns(Source, {{"Column", each Text.Remove(_, {"0".."9"}), type text}}) ● Remove punctuation: Table.TransformColumns(Source, {{"Column", each Text.Remove(_, {"!"..".", ";".."@", "[".."", "{".."}"}), type text}})` ● Extract text between delimiters: Table.TransformColumns(Source, {{"Column", each Text.BetweenDelimiters(_, "<", ">"), type text}})
By: Waleed Mousa
● Extract first n characters: Table.TransformColumns(Source, {{"Column", each Text.Start(_, 5), type text}}) ● Extract last n characters: Table.TransformColumns(Source, {{"Column", each Text.End(_, 5), type text}}) ● Replace text: Table.TransformColumns(Source, {{"Column", each Text.Replace(_, "old", "new"), type text}}) ● Concatenate columns: Table.AddColumn(Source, "CombinedColumn", each [Column1] & " " & [Column2], type text)
Numeric Operations
● Round numbers: Table.TransformColumns(Source, {{"Column", each
Number.Round(_, 2), type number}}) ● Calculate percentage: Table.AddColumn(Source, "Percentage", each [Value] / List.Sum(Source[Value]), type number) ● Calculate running total: Table.AddColumn(Source, "RunningTotal", each List.Sum(Table.Range(Source, 0, [Index])[Value]), type number) ● Calculate rank: Table.AddColumn(Source, "Rank", each Table.RowCount(Table.Sort(Table.FirstN(Source, [Index] + 1), {{"Value", Order.Descending}})), type number) ● Calculate percentile: Table.AddColumn(Source, "Percentile", each Number.Percentile([Value], 0.5), type number) ● Calculate moving average: Table.AddColumn(Source, "MovingAverage", each List.Average(List.LastN(Table.Range(Source, 0, [Index])[Value], 3)), type number) ● Calculate standard deviation: Table.AddColumn(Source, "StandardDeviation", each List.StandardDeviation(Source[Value]), type number) ● Calculate correlation: Table.AddColumn(Source, "Correlation", each Table.CorrelationCoefficient(Source, "Column1", "Column2"), type number) ● Calculate linear regression: Table.AddColumn(Source, "Regression", each Table.LinearRegression(Source, "Column1", "Column2"), type number) ● Calculate logarithm: Table.TransformColumns(Source, {{"Column", each Number.Log(_, 10), type number}})
Conditional Operations
● Add conditional column: Table.AddColumn(Source, "NewColumn", each if
[Column1] > 10 then "High" else "Low", type text) ● Filter based on condition: Table.SelectRows(Source, each [Column1] > 10) ● Replace values based on condition: Table.ReplaceValue(Source, each if _ = "OldValue" then "NewValue" else _, Replacer.ReplaceValue, {"Column"}) By: Waleed Mousa ● Apply conditional formatting: Table.TransformColumns(Source, {{"Column", each if _ > 10 then "Red" else if _ < 5 then "Green" else "Blue", type text}}) ● Calculate conditional sum: Table.AddColumn(Source, "ConditionalSum", each List.Sum(Table.SelectRows(Source, each [Column1] > 10)[Value]), type number) ● Calculate conditional count: Table.AddColumn(Source, "ConditionalCount", each Table.RowCount(Table.SelectRows(Source, each [Column1] > 10)), type number) ● Calculate conditional average: Table.AddColumn(Source, "ConditionalAverage", each List.Average(Table.SelectRows(Source, each [Column1] > 10)[Value]), type number) ● Calculate conditional min/max: Table.AddColumn(Source, "ConditionalMin", each List.Min(Table.SelectRows(Source, each [Column1] > 10)[Value]), type number) ● Calculate conditional percentile: Table.AddColumn(Source, "ConditionalPercentile", each Number.Percentile(Table.SelectRows(Source, each [Column1] > 10)[Value], 0.5), type number) ● Calculate conditional rolling average: Table.AddColumn(Source, "ConditionalRollingAverage", each List.Average(List.LastN(Table.SelectRows(Table.Range(Source, 0, [Index]), each [Column1] > 10)[Value], 3)), type number)
each ([Value] - Table.SelectRows(Source, each [Date] = Date.AddYears([Date], -1))[Value]{0}) / Table.SelectRows(Source, each [Date] = Date.AddYears([Date], -1))[Value]{0}, type number) ● Calculate month-over-month growth: Table.AddColumn(Source, "MoMGrowth", each ([Value] - Table.SelectRows(Source, each [Date] = Date.AddMonths([Date], -1))[Value]{0}) / Table.SelectRows(Source, each [Date] = Date.AddMonths([Date], -1))[Value]{0}, type number) ● Calculate quarter-over-quarter growth: Table.AddColumn(Source, "QoQGrowth", each ([Value] - Table.SelectRows(Source, each [Date] = Date.AddQuarters([Date], -1))[Value]{0}) / Table.SelectRows(Source, each [Date] = Date.AddQuarters([Date], -1))[Value]{0}, type number) ● Calculate running total by year: Table.AddColumn(Source, "YearRunningTotal", each List.Sum(Table.SelectRows(Table.Range(Source, 0, [Index]), each Date.Year([Date]) = Date.Year([Date]))[Value]), type number)
By: Waleed Mousa
● Calculate running total by month: Table.AddColumn(Source, "MonthRunningTotal", each List.Sum(Table.SelectRows(Table.Range(Source, 0, [Index]), each Date.Year([Date]) = Date.Year([Date]) and Date.Month([Date]) = Date.Month([Date]))[Value]), type number) ● Calculate running total by quarter: Table.AddColumn(Source, "QuarterRunningTotal", each List.Sum(Table.SelectRows(Table.Range(Source, 0, [Index]), each Date.Year([Date]) = Date.Year([Date]) and Date.QuarterOfYear([Date]) = Date.QuarterOfYear([Date]))[Value]), type number) ● Calculate year-to-date sum: Table.AddColumn(Source, "YTDSum", each List.Sum(Table.SelectRows(Source, each Date.Year([Date]) = Date.Year([Date]) and [Date] <= [Date])[Value]), type number) ● Calculate quarter-to-date sum: Table.AddColumn(Source, "QTDSum", each List.Sum(Table.SelectRows(Source, each Date.Year([Date]) = Date.Year([Date]) and Date.QuarterOfYear([Date]) = Date.QuarterOfYear([Date]) and [Date] <= [Date])[Value]), type number) ● Calculate month-to-date sum: Table.AddColumn(Source, "MTDSum", each List.Sum(Table.SelectRows(Source, each Date.Year([Date]) = Date.Year([Date]) and Date.Month([Date]) = Date.Month([Date]) and [Date] <= [Date])[Value]), type number) ● Calculate rolling average by period: Table.AddColumn(Source, "RollingAverage", each List.Average(List.LastN(Table.SelectRows(Table.Range(Source, 0, [Index]), each Date.Year([Date]) = Date.Year([Date]) and Date.Month([Date]) = Date.Month([Date]))[Value], 3)), type number)
Advanced Analytics
● Calculate forecast using exponential smoothing: Table.AddColumn(Source,
"Forecast", each Table.ExponentialSmoothing(Source, "Value", 0.2, 3), type number) ● Calculate forecast using ARIMA: Table.AddColumn(Source, "Forecast", each Table.ARIMA(Source, "Value", 1, 0, 1), type number) ● Calculate forecast using ETS: Table.AddColumn(Source, "Forecast", each Table.ETS(Source, "Value", "AAA", 3), type number) ● Calculate forecast using linear regression: Table.AddColumn(Source, "Forecast", each Table.LinearRegressionForecast(Source, "Date", "Value", [Date]), type number) ● Calculate forecast using neural network: Table.AddColumn(Source, "Forecast", each Table.NeuralNetwork(Source, "Value", 3, "sgd", 100, 10), type number)
By: Waleed Mousa
● Detect outliers using Z-score: Table.AddColumn(Source, "Outlier", each if (([Value] - List.Average(Source[Value])) / List.StandardDeviation(Source[Value])) > 2 then "Yes" else "No", type text) ● Detect outliers using IQR: Table.AddColumn(Source, "Outlier", each if [Value] < (List.Percentile(Source[Value], 0.25) - 1.5 * (List.Percentile(Source[Value], 0.75) - List.Percentile(Source[Value], 0.25))) or [Value] > (List.Percentile(Source[Value], 0.75) + 1.5 * (List.Percentile(Source[Value], 0.75) - List.Percentile(Source[Value], 0.25))) then "Yes" else "No", type text) ● Calculate sentiment score: Table.AddColumn(Source, "Sentiment", each Text.Sentiment([Comment]), type number) ● Perform text clustering: Table.TextCluster(Source, "Comment", 5) ● Perform text classification: Table.TextClassify(Source, "Comment", "Category", {"Positive", "Negative"})
Data Visualization
● Create a bar chart: BarChart = Table.Visualizations(Source, "BarChart",
● Create a new table: NewTable = Table.FromRows(Source, {"Column1", "Column2", "Column3"}) ● Append queries: AppendedTable = Table.Combine({Table1, Table2, Table3}) ● Merge queries: MergedTable = Table.NestedJoin(Table1, "Key", Table2, "ForeignKey", "NewColumn", JoinKind.LeftOuter) ● Create a calculated table: CalculatedTable = Table.AddColumn(Table1, "NewColumn", each [Column1] * [Column2], type number) ● Create a calculated column: Table.AddColumn(Source, "CalculatedColumn", each [Column1] + [Column2], type number) ● Define a measure: Measure = CALCULATE(SUM(Table[Column]), Table[FilterColumn] = "Value") ● Create a date table: DateTable = CALENDAR(DATE(YEAR(MIN(Table[Date])), 1, 1), DATE(YEAR(MAX(Table[Date])), 12, 31)) ● Create a hierarchy: Hierarchy = Table.FromColumns({Table[Country], Table[State], Table[City]}, {"Country", "State", "City"}) ● Create a parent-child hierarchy: ParentChildHierarchy = Table.FromColumns({Table[Employee], Table[Manager]}, {"Employee", "Manager"}) ● Create a many-to-many relationship: ManyToManyRelationship = Table.AddJoinColumn(Table1, "Key", Table2, "ForeignKey", "Relationship")
Data Validation and Quality
● Check for null values: Table.AddColumn(Source, "IsNull", each if [Column]
= null then "Yes" else "No", type text) ● Check for empty values: Table.AddColumn(Source, "IsEmpty", each if [Column] = "" then "Yes" else "No", type text) ● Check for duplicates: Table.AddColumn(Source, "IsDuplicate", each if Table.RowCount(Table.SelectRows(Table.PrefixColumns(Source, {"Index"}), each [Column] = [Column])) > 1 then "Yes" else "No", type text) ● Check for valid email format: Table.AddColumn(Source, "IsValidEmail", each if Text.Contains([Email], "@") and Text.Contains([Email], ".") then "Yes" else "No", type text) ● Check for valid phone number format: Table.AddColumn(Source, "IsValidPhone", each if Text.Length([Phone]) = 10 and Text.Start([Phone], 2) = "08" then "Yes" else "No", type text) ● Check for values within range: Table.AddColumn(Source, "IsInRange", each if [Value] >= 0 and [Value] <= 100 then "Yes" else "No", type text) ● Check for values in list: Table.AddColumn(Source, "IsInList", each if List.Contains({"A", "B", "C"}, [Category]) then "Yes" else "No", type text)
By: Waleed Mousa
● Check for consistent formatting: Table.AddColumn(Source, "IsConsistentFormat", each if Text.Length([Column]) = 5 and Text.Start([Column], 1) = "X" then "Yes" else "No", type text) ● Calculate data quality score: Table.AddColumn(Source, "DataQualityScore", each if [IsNull] = "No" and [IsEmpty] = "No" and [IsDuplicate] = "No" and [IsValidEmail] = "Yes" then 1 else 0, type number) ● Filter rows with data quality issues: Table.SelectRows(Source, each [DataQualityScore] = 1)
Geospatial Analysis
● Create a geographic hierarchy: GeographicHierarchy =
Table.FromColumns({Table[Country], Table[State], Table[City]}, {"Country", "State", "City"}, "Location") ● Calculate distance between points: Table.AddColumn(Source, "Distance", each Table.DistanceInMiles([Latitude], [Longitude], [OtherLatitude], [OtherLongitude]), type number) ● Calculate area of polygon: Table.AddColumn(Source, "Area", each Table.AreaInSquareMiles([Polygon]), type number) ● Find nearest location: Table.AddColumn(Source, "NearestLocation", each Table.NearestPoint(Source, [Latitude], [Longitude], [OtherLatitude], [OtherLongitude]), type text) ● Perform spatial join: Table.SpatialJoin(Table1, Table2, [Polygon1], [Polygon2], JoinType.Intersects) ● Create a density map: DensityMap = Table.Visualizations(Source, "DensityMap", [Latitude], [Longitude], [Value], [Color]) ● Create a choropleth map: ChoroplethMap = Table.Visualizations(Source, "ChoroplethMap", [Location], [Value], [Color], [Tooltips]) ● Create a route map: RouteMap = Table.Visualizations(Source, "RouteMap", [StartLatitude], [StartLongitude], [EndLatitude], [EndLongitude], [Color]) ● Create a flow map: FlowMap = Table.Visualizations(Source, "FlowMap", [SourceLocation], [DestinationLocation], [Value], [Color]) ● Create a spatial clustering: SpatialClustering = Table.Visualizations(Source, "SpatialClustering", [Latitude], [Longitude], [Cluster], [Color], [Size])
Statistical Analysis
● Calculate covariance: Table.AddColumn(Source, "Covariance", each
Table.Covariance(Source, "Column1", "Column2"), type number)
By: Waleed Mousa
● Calculate correlation matrix: CorrelationMatrix = Table.CorrelationMatrix(Source, {"Column1", "Column2", "Column3"}) ● Perform t-test: Table.AddColumn(Source, "TTestResult", each Table.TTest(Table.SelectRows(Source, each [Category] = "A"), Table.SelectRows(Source, each [Category] = "B"), "Value", 0.05), type text) ● Perform ANOVA: Table.AddColumn(Source, "ANOVAResult", each Table.ANOVA(Source, "Value", "Category", 0.05), type text) ● Perform chi-square test: Table.AddColumn(Source, "ChiSquareResult", each Table.ChiSquareTest(Table.Pivot(Source, "Category", "Result", "Value", List.Sum), 0.05), type text) ● Calculate linear regression coefficients: Table.AddColumn(Source, "RegressionCoefficients", each Table.LinearRegression(Source, "X", "Y"), type any) ● Calculate logistic regression coefficients: Table.AddColumn(Source, "LogisticRegressionCoefficients", each Table.LogisticRegression(Source, "X", "Y"), type any) ● Calculate decision tree: Table.AddColumn(Source, "DecisionTree", each Table.DecisionTree(Source, "Features", "Target"), type any) ● Calculate random forest: Table.AddColumn(Source, "RandomForest", each Table.RandomForest(Source, "Features", "Target"), type any) ● Calculate K-means clustering: Table.AddColumn(Source, "KMeansClustering", each Table.KMeansClustering(Source, "Features", 3), type any)
Time Series Analysis
● Decompose time series: TimeSeriesDecomposition =
Table.TimeSeriesDecompose(Source, "Date", "Value", TimeSeriesComponents.Trend) ● Calculate moving average: Table.AddColumn(Source, "MovingAverage", each Table.MovingAverage(Source, "Value", 7), type number) ● Calculate exponential moving average: Table.AddColumn(Source, "ExponentialMovingAverage", each Table.ExponentialMovingAverage(Source, "Value", 0.2), type number) ● Calculate rolling standard deviation: Table.AddColumn(Source, "RollingStandardDeviation", each Table.RollingStandardDeviation(Source, "Value", 30), type number) ● Calculate rolling correlation: Table.AddColumn(Source, "RollingCorrelation", each Table.RollingCorrelation(Source, "Column1", "Column2", 30), type number) ● Detect seasonality: Table.AddColumn(Source, "Seasonality", each Table.DetectSeasonality(Source, "Date", "Value"), type number)
By: Waleed Mousa
● Detect trend: Table.AddColumn(Source, "Trend", each Table.DetectTrend(Source, "Date", "Value"), type number) ● Perform Dickey-Fuller test: Table.AddColumn(Source, "DickeyFullerTestResult", each Table.DickeyFullerTest(Source, "Value"), type text) ● Perform Granger causality test: Table.AddColumn(Source, "GrangerCausalityTestResult", each Table.GrangerCausalityTest(Source, "Column1", "Column2", 5), type text) ● Forecast using Holt-Winters: Table.AddColumn(Source, "HoltWintersForecast", each Table.HoltWinters(Source, "Value", 12, 0.2, 0.2, 0.2), type number)
Data Storytelling and Communication
● Create a KPI visual: KPI = Table.Visualizations(Source, "KPI", [Value],