9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

Exploring Lesser Known SQL

Functions for Data Analysis — Part I
Rasiksuhail · Follow
Published in Towards Dev · 4 min read · Aug 17

72 4

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 1/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

Photo by James Harrison on Unsplash

In the realm of SQL, there are several advanced functions that often remain
underutilized despite their potential to streamline data analysis, improve
performance, and enable more sophisticated queries. Let’s delve into some
of these lesser-known SQL functions that can elevate your data analysis
capabilities:

Lets begin

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 2/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

1. Window Functions:
Window functions allow you to perform calculations across a set of rows
related to the current row. These functions can significantly simplify
complex queries that involve ranking, cumulative sums, moving averages,
and more.

Example: Calculating Running Total

SELECT
date,
revenue,
SUM(revenue) OVER (ORDER BY date) AS running_total
FROM
sales_data;

2. Recursive Common Table Expressions (CTEs)

Recursive CTEs enable you to traverse hierarchical or recursive data
structures like organizational charts, bill of materials, and more.

Example: Recursive Factorial Calculation

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 3/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

WITH RECURSIVE factorial(n, value) AS (

SELECT 1, 1
UNION ALL
SELECT n + 1, n * value FROM factorial WHERE n < 10
)
SELECT value FROM factorial;

3. JSON Functions
Modern databases support JSON data types, and SQL provides functions to
manipulate and query JSON data directly within your queries.

Example: Extracting JSON Data

SELECT
json_data->>'$.name' AS name,
json_data->>'$.age' AS age
FROM
json_table;

4. STRING_AGG Function
STRING_AGG allows you to concatenate values from multiple rows into a
single string, useful for generating comma-separated lists or concatenated
https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 4/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

text.

Example: Concatenating Employee Names

SELECT
department,
STRING_AGG(employee_name, ', ') AS employees
FROM
employee_data
GROUP BY
department;

5. LAG and LEAD Functions

LAG and LEAD functions enable you to access values from preceding or
succeeding rows within a result set, aiding in time-based analysis and trend
identification.

Example: Calculating Month-over-Month Growth

SELECT
date,
revenue,

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 5/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

LAG(revenue) OVER (ORDER BY date) AS prev_month_revenue,

(revenue - LAG(revenue) OVER (ORDER BY date)) / LAG(revenue) OVER (ORDER BY
FROM
sales_data;

6. FILTER Clause with Aggregate Functions

The FILTER clause allows you to apply aggregate functions only to specific
rows that meet a certain condition, enhancing the flexibility of your
aggregations.

Example: Average Revenue for High-Value Customers

SELECT
customer_id,
AVG(revenue) FILTER (WHERE revenue > 1000) AS avg_high_value_revenue
FROM
sales_data
GROUP BY
customer_id;

7. ARRAY Functions

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 6/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

Some databases support ARRAY data types and provide functions to

manipulate arrays directly within SQL queries.

Example: Finding Common Elements in Arrays

SELECT
user_id,
ARRAY_INTERSECT(purchased_items, recommended_items) AS common_items
FROM
user_purchase_history;

8. Percentile Cont and Percentile Disc

These functions allow you to calculate percentile values for a given dataset,
helping you analyze distribution characteristics.

Example: Calculating Median and 75th Percentile

SELECT
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY score) AS median,
PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY score) AS 75th_percentile

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 7/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

FROM
exam_scores;

9. STRING_TO_ARRAY
When working with arrays, these functions facilitate converting strings to
arrays and vice versa, enabling more versatile data manipulation.

Example: Converting Comma-Separated Values to Array

SELECT
student_name,
STRING_TO_ARRAY(subjects, ', ') AS subject_list
FROM
student_subjects;

10. Regular Expressions (REGEXP)

Regular expressions allow for powerful pattern matching within text data,
aiding in data extraction and cleansing.

Example: Extracting Phone Numbers

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 8/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

SELECT
customer_name,
REGEXP_SUBSTR(contact_info, '\d{3}-\d{3}-\d{4}') AS phone_number
FROM
customer_contacts;

11. LATERAL Joins

LATERAL joins enable you to correlate results from a subquery with the
rows in the main query, useful for complex calculations involving correlated
subqueries.

Search Medium Write

Example: Calculating Moving Averages

SELECT
timestamp,
value,
AVG(value) OVER (
ORDER BY timestamp
RANGE BETWEEN INTERVAL '1 day' PRECEDING AND CURRENT ROW
) AS moving_average
FROM
sensor_data;

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 9/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

12. Common Table Expressions (CTEs) for Complex Queries

CTEs can be used for more than just recursive queries. They can also
simplify complex queries by breaking them down into smaller, more
manageable parts.

Example: Analyzing Sales Trends

WITH sales_summary AS (
SELECT
date,
SUM(revenue) AS daily_revenue
FROM
sales_data
GROUP BY
date
)
SELECT
date,
daily_revenue,
AVG(daily_revenue) OVER (
ORDER BY date
RANGE BETWEEN INTERVAL '7 days' PRECEDING AND CURRENT ROW
) AS 7_day_avg_revenue
FROM
sales_summary;

13. Combining HAVING Clause & Window Functions

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 10/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

Combine the HAVING clause with window functions to filter aggregated

results based on window calculations.

Example: Identifying Customer Behavior Changes

SELECT
customer_id,
AVG(order_amount) AS avg_order_amount,
MIN(order_amount) AS min_order_amount
FROM
orders
GROUP BY
customer_id
HAVING
MAX(order_amount) - MIN(order_amount) > 100;

These advanced SQL functions extend your capabilities in data

manipulation, analysis, and transformation. Incorporating these functions
into your toolkit empowers you to handle intricate scenarios and glean
deeper insights from your data.

Happy Learning

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 11/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

Stay tuned for my next blog.

Exploring Lesser Known SQL Functions for Data Analysis — Part

II
In the world of SQL, there are some fancy tools that many folks don’t
use much, even though they can make working with…
towardsdev.com

Sql Data Analysis Data Programming Querying

Written by Rasiksuhail Follow

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 12/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

272 Followers · Writer for Towards Dev

Exploring Data!!

More from Rasiksuhail and Towards Dev

Rasiksuhail Franklyne Kibet in Towards Dev

Orchestrating dbt with Airflow: A Data Modeling in the Modern Data

Step by Step Guide to Automating… Stack
In today’s data-driven world, organizations Data modeling is arguably the most impactful
rely heavily on automated data pipelines to… decision for a data team. It determines your…

10 min read · Jul 23 6 min read · Aug 20

82 281 3

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 13/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

Avi Chawla in Towards Dev Rasiksuhail

Streamlit vs. Taipy — The Ultimate Orchestrating dbt with Airflow: A

Comparison Step by Step Guide to Automating…
In two of my recent articles, I presented a In today’s data-centric landscape,
comprehensive overview of Taipy — an open… organizations heavily rely on automated dat…

5 min read · Sep 3 10 min read · Jul 29

265 99 1

See all from Rasiksuhail See all from Towards Dev

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 14/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

Recommended from Medium

Sam Zamany Arthur De Kimpe in Deezer I/O

Unit Testing in Data Engineering: A Rethinking Your Data Platform

Practical Guide Documentation So That People…
In the realm of data engineering, one often Some tips to structure your documentation
overlooked practice is unit testing. Many… and empower your employees to understan…

10 min read · Jul 12 9 min read · Sep 6

206 3 486 5

Lists
https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 15/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

It's never too late or early to General Coding Knowledge

start something 20 stories · 350 saves
15 stories · 127 saves

New_Reading_List Coding & Development

174 stories · 108 saves 11 stories · 178 saves

Nam Huynh Thien Jeferson Marques

Building a Dimensional Data Leveraging Subqueries in SQL:

Warehouse Using dbt Enhancing Data Aggregation with…
Introduction Structured Query Language (SQL) is the
backbone of data manipulation and retrieval…

10 min read · Aug 5 4 min read · Sep 5

31 91

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 16/17
9/19/23, 6:52 PM SQL functions useful for Data analysis | Towards Dev

Nishchay Agrawal Gunjan Sahu in Nerd For Tech

Data Engineer Morgan Stanley Spotify ETL using Python and AWS
Interview Experience This arcticle focuses on the steps I took and
Morgan Stanley is a leading global investment the valuable insights I gained during my…
bank and wealth management firm and it…

8 min read · Jun 16 12 min read · Jul 10

101 2 22

See more recommendations

https://towardsdev.com/exploring-lesser-known-sql-functions-for-data-analysis-5cb007c93e44 17/17