Attendance-taking Lucky draw

Scan for a chance to win $30 Grab vouchers! 3 winners to be decided by next week
NOTE: this QR code will only be shown twice...
 NOTE: If you are
watching this on Zoom,
this session will NOT be

Product Analytics recorded. The Zoom

chat will also not be

with Tableau
monitored.

This workshop will NOT

teach you to code in your
typical languages!

Low/no-code
presentation focused
NUS Statistics and
NUS PRODUCT CLUB more on concepts
Data Science Society

Do remember to install
Tableau for the hands-on
segment!
13 February 2025 (Thursday)
But first! Some past history...

2022: Power BI 2023: PME by NUS 2024: First Product

workshop Product Club Analytics workshop
Collaboration between NUS Metrics session taught by Collaboration between NUS
SDS, NUS CFG & BIPO guest product managers Product Club & NUS SDS
Workshop outline
1 Introduction to product analytics 3 Hands-on with Tableau

What is product management? Introduction to Tableau and dataset

What is product analytics? Visualisations for predictive analytics

Why is product analytics important? Exploring advanced features

2 Techniques for product analytics 4 Sample dashboards

Predictive analytics Dashboard objectives and key features

A/B testing Examples of dashboards

Funnel analysis and cohort analysis Designing dashboards

Harry Chang
Part 1: Introduction to product analytics

Data Analyst, PayPal

Co-Founder, NUS Product Club
Data Science & Analytics Graduate
What is
product UX TECH

management?
Business process of planning, PRODUCT
developing, launching, and BUSINESS MANAGEMENT
managing a product or service -
requiring collaborations with
designers, developers, marketers etc
What is
product
lifecycle?
After successfully ideating and
developing a product, we analyse its
performance to assess the usefulness of
its current features before constantly
updating the product
What is product analytics?
In three points
Analyse how users engage with a
product or service
Track, visualise, analyse user
engagement and behaviour data
Aims to improve and optimise a
product or service

The insights we derive from product analytics TL;DR

tells us how users actually use the product
A process that dissects how customers
engage with digital products
Comparing two similar terms

Product Analytics Marketing Analytics

Focuses on optimising customer Deals with everything pre-purchase:
experience post-purchase enhancing marketing campaign to
increase conversions
Aspects of product analytics

Product improvement Product engagement Customer retention User experience

Focus on retention: optimising the post-purchase customer experience

Aspects of marketing analytics

Customer Marketing Return on Customer

acquisition campaigns investments segmentation

Emphasis on acquisition: enhancing pre-purchase metrics to generate high-

quality leads
Importance of product analytics

Optimising user experience Measuring success Identifying new opportunities

to improve user flows, by defining and tracking for innovation and growth,
streamline interfaces, enhance KPIs - acquisition, especially features that
overall usability retention, revenue etc resonate well with users
Optimising user experience
TikTok - AI-driven content recommendations
By analysing watch time, likes, comments,
shares, and replays, TikTok's algorithm curates a
personalized "For You" page that continuously
adapts to user preferences.
This optimization leads to higher user
engagement and longer session durations,
making TikTok an addictive social platform.
Measuring success
Netflix - “Top 10” trending list
When Netflix introduced its "Top 10" trending list,
it measured how it influenced viewing behavior.
Data showed that users were more likely to
watch content ranked in the top 10, leading to
increased engagement.
By continuously tracking viewing patterns, drop-
off points, and completion rates, Netflix can
determine what content resonates best and what
adjustments are needed to improve engagement.
Identifying new opportunities
Spotify - "Discover Weekly" playlist
Popular feature providing users with
personalised playlist they might enjoy based
on listening history
By identifying that users often enjoy
discovering new music, Spotify provided a
fresh and personalised experience, leading to
increased user satisfaction and engagement
Lee Kai Rong
Part 2: Techniques for Product Analytics (1)

Curriculum Executive, NUS Product Club

Workshops Executive, NUS SDS
Year 2 Data Science & Analytics
Predictive Analytics
the practice of using historical data, statistical algorithms, and
machine learning techniques to forecast future events or
outcomes.
 Types of Predictive Analytics

and more!

Classification Regression Time Series

How is Predictive Analytics
used in Product Analytics?
 Examples of Predictive Analytics in PM

CLASSIFICATION:
Anticipating Customer Behaviour

Will a customer churn or continue using our product?

Analyzes past customer interactions, purchase history, and feedback.
Uses classification to classify customers as likely to churn or likely to stay.
Helps in personalizing offers to retain high-risk customers.

Streaming platforms (like Netflix) predict which users may cancel

their subscriptions and offer them tailored content or discounts.
 Examples of Predictive Analytics in PM

REGRESSION:
Pricing Optimization

What is the ideal price point for maximizing revenue?

Analyzes historical sales, competitor prices, and market conditions.
Uses regression to find the optimal price where demand and revenue are maximized.
Helps businesses adjust pricing dynamically based on customer behavior.

Ride-sharing apps (Grab, Uber) use dynamic

pricing to adjust fares based on demand.
 Examples of Predictive Analytics in PM

TIME-SERIES:
Demand Forecasting

How many units of a product will we sell next month?

Uses historical sales data, seasonal trends, and external factors (holidays, economy).
Predicts future demand to optimize inventory and production.
Prevents overstocking (waste) and stockouts (missed sales).

Amazon uses demand forecasting to predict holiday

season sales and adjust inventory accordingly.
Predictive Analytics - Common Tools

Visualization &
Low-Code Code-based
Analytics
A/B Testing
a method of experimentation where you compare
two or more versions of a product to see which
performs better.

A = Control (current version)

B = Treatment/ Variation (new version being tested)
How is A/B Testing used in PM?
✔ Test new features before building
✔ Optimize landing pages, emails, pricing
✔ Improve onboarding flows & user engagement
✔ Increase conversion rates
The A/B Testing Process
1️⃣ Define Goals: What do you want to improve? (Conversions, engagement, etc.)
2️⃣ Choose a Metric: Click-through rate, time on site, purchases
3️⃣ Create Variants: Control (A) vs. Treatment (B)
4️⃣ Run the Test: Show different versions to different user groups
5️⃣ Analyze Results: Use statistical methods (Chi-Square, ANOVA)
6️⃣ Implement the Winner!
The A/B Testing Process
Goal:
1️⃣ Increase conversion rate
2️⃣ Improve user engagement
3️⃣ Personalize the shopping experience
4️⃣ Reduce bounce rate
(keeping users on the site longer)

Metrics:
Conversion Rate (%)
Click-Through Rate (CTR) (%)
Bounce Rate (%)
Time on Site (Seconds/Minutes)
Tools for A/B Testing

No-code Enables in-app messages &

user onboarding flow test

Allows diff versions of web

Integrates with Google
page test and personalizes UX
Analytics
Common A/B Testing Mistakes
🚫 Testing tiny changes that don’t matter
🚫 Running tests with too small a sample size
🚫 Stopping a test too early before it reaches
statistical significance
Su Shwe Yee
Part 2: Techniques for Product Analytics (2)

Curriculum Executive, NUS Product Club

Year 3 Information Systems
What is
funnel
analysis?
Funnel analysis is a tool that helps us understand
the steps required to achieve a specific outcome
in a product and the number of users that get
through each step.
Why use funnel analysis?

Identify drop-off points for Improve user experience by Increase number of desired
users eliminating friction outcome
Conducting a
funnel analysis
Conducting a
funnel analysis

Step 1

Define the desired outcome

For a user to make a purchase
Conducting a
funnel analysis
Step 2

Map out the user’s journey

to the desired outcome
What are the key actions that a user has to
take before making a purchase?
Visit a clothing category page
OR
Search for a clothing item

Complete Payment
Add an item to bag

Visit an item page Click checkout

Conducting a
funnel analysis
Step 3

Record the number of users

who has taken each step
Custom event tracking, 3rd party product
analytics solutions, etc.
Conducting a
funnel analysis
Step 4

Visualise the funnel

Data visualisation tools, 3rd party product
analytics solutions, etc.
Conducting a
funnel analysis Visit a clothing category page OR
Search for a clothing item

Visit an item page

Step 5
Add an item
to bag
Draw insights from your
visualisations Click
Checkout

Where are the highest user drop-offs

happening?
Conducting a
funnel analysis Visit a clothing category page OR
Search for a clothing item

Visit an item page

Step 5
Add an item to bag

Draw insights from your Click Checkout

visualisations
Where are the highest user drop-offs
happening?
What is
cohort
analysis?
Cohort analysis is a method for
tracking different groups of users
over time
What are cohorts?
A cohort is a user group that shares a common
characteristic.

Subscription start date

Subscription tier
User behaviour (active or inactive)
Demographic
Cohort Retention Analysis
Horizontal
Analyse retention of a single cohort over
different months

Vertical
Compare retention across different cohorts
after a particular time period

Diagonal
Analyse retention in a single month
What does cohort analysis tell us?

User Behaviors driving Critical Drop-Off Point Impact of Product Changes

Retention
At what point in the user journey do we see a Has our new onboarding process
Do referred users demonstrate higher
significant drop-off—such as users engaging (launched in March) led to improved
retention than self-registered users?
consistently for three weeks before their long-term retention?
activity declines?
Harish M.
Part 3: Hands-on with Tableau

Workshops Executive, NUS SDS

Year 2 Data Science & Analytics
Gangjoon Lee
Part 3: Hands-on with Tableau

Workshops Executive, NUS SDS

Year 2 Data Science & Analytics
Tableau Interface: Start Page
Tableau Interface: Data Source Page
Tableau Interface: Data Source Page

Data Source
Tableau Interface: Data Source Page
Data Model
Tableau Interface: Data Source Page

Data Grid

Meta Data Grid

Tableau Interface: Work Space Page
Tool Bar
Tableau Interface: Work Space Page

Shelves & Cards

Our Dataset:
Industry Overview
The dataset focuses on an online sporting goods store selling:
Products: Clothing, shoes, accessories, and sports nutrition.

Problem Statement
The store displays one of five banners randomly on the main page to stimulate sales.
Each banner promotes a specific product category or the entire brand.
The effectiveness of these banners might vary by segment and characteristics of user behaviors

Primary Goal: Evaluate the effectiveness of banner advertisements.

Our Dataset:
order_id: Unique identifier for each purchase transaction.
user_id: Unique identifier to track individual customer behavior.
page_id: Unique page or event bundle interacted with during a session.
product: Type of product or banner displayed (e.g., Clothes, Sports nutrition).
site_version: Platform used to access the website (Mobile or Desktop).
time: Timestamp of the event or purchase for tracking activity patterns.
title: Type of event (banner display, click, or purchase).
target: Indicates if the event resulted in a purchase (1 for Yes, 0 for No).
Plotting Data: Total Purchases by Day
Adding trendline: linear
Adding a trendline: polynomial
Forecast Feature
Purchase by Product Category
User Interaction with Banner
Using the Filter Feature
 Cluster Groups in Tableau: The Concepts
Data Clustering : Statistical technique to group similar data points together
powerful tool for analysis and pattern recognition, enabling data-driven decisions
many cluster algorithms, Tableau uses k-means algorithm by default

There are many applications of Clustering, including:

Customer Segmentation
Product Analysis
Usage Pattern Detection
K-means Algorithm
What is K-means Algorithm?
Grouping ‘similar’ data points
‘K’ represent the number of groups we want
‘Means’ refer to averaging within groups

How the Algorithm Works

Picks K random center points
Each point is assigned to the closest center (using Euclidean distance)
Center point is recalculated to the average of the points
Steps 1~3 is repeated until it is stable
Example: Customer Segmentation
Segmenting customers based off of profit and sales to identify high-value customers
Building Decision-Making Visualizations in Tableau

Why Decision-Making Visualizations Matter:

Decision trees and Sankey diagrams enable organizations to simplify complex data into actionable insights,
supporting data-driven strategies

Objectives:
Learn to create decision trees and Sankey diagrams in Tableau.
Understand their use cases in predictive modeling and business decisions.
Apply these tools to real-world scenarios.
Decision Trees
Definition:
A graphical representation that outlines decisions and their possible consequences.
Key Components:

Root Node: The starting point (e.g., a question or initial condition)

Branches: Decision paths.
Leaf Nodes: Final outcomes or results.
Decision Trees
Year
Month
Site
Is the
Product banner
effective?
Decision Trees
Decision Trees
Decision Trees
EDA Data Cleaning
Sankey Diagrams
Definition:
A visualization tool used to display the flow of quantities between nodes.

Key Components:
Nodes: Represent entities or categories.
Flows: Represent relationships or transitions between nodes.
Thickness: Indicates magnitude.
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Sankey Diagrams
Use Cases in Predictive Modeling

Content:
Decision Trees:
Classify customers for targeted marketing.
Predict credit risk or loan defaults.
Sankey Diagrams:
Analyze customer journey paths.
Optimize resource allocation in supply chains.
Cohort Analysis
Definition: Examines groups of users (cohorts) based on acquisition date or
behavior to track trends and performance.

Purpose:
Identify trends and patterns
Optimize marketing strategies
Improve user retention

Example:
Email Campaign:
Send a campaign to 1,000 users and track purchases on Day 1, Day 2, etc.
Comparison:
Send a similar campaign to a new cohort after a few weeks.
Compare purchase patterns to evaluate the impact of timing or messaging.
Cohort Analysis
Cohort Analysis - Hands on
1. Our Goals
Measure user engagement over time after they first see a banner on our site
Identify how many users return or perform any key actions (clicks or purchases) in subsequent
months
Compare different monthly cohorts of first-time banner viewers to see if later cohorts behave
differently than earlier ones

2. Key Metrics
User Engagement: Number of unique users performing any site action (click, purchase)

3. Cohort Definition
The month of the user’s first banner_show
Cohort Analysis - Hands on
Cohort Analysis - Hands on
Cohort Analysis
Conversion Rate Analysis
Conversion Rate Analysis
Final Dashboard
From this..
Final Dashboard
Other Examples
Other Examples
Final Notes
Attendance-taking Lucky draw

Scan for a chance to win $30 Grab vouchers! 3 winners to be decided by next week
NOTE: this QR code will only be shown twice...
@nusproductclub @nus.sds

NUS Product Club NUS Statistics and Data Science Society

t.me/nuspc t.me/nussds

[email protected] [email protected]

www.nusproductclub.com www.nussds.com