This. is. Insurjo!

You know the drill! Tag @TheProductfolks on Twitter to show us some ♥

Shoutout to our Partners 📢
Product Coach #3
Analytics: Measure what Matters
Moinak Bandyopadhyay
Staff Product Manager, Mixpanel
 Who am I ?

● Staff Product Manager

 Mixpanel
● Interests: Light roast
coffee, Bollywood and
(really) bad puns.
 Agenda

01 02 03 04

Why measure Common pitfalls North Star Metrics Metrics

anything? Framework

05 06 07 08

Counter metrics Analyzing Hands-on tutorial Q&A

product-market fit
Why measure anything?
Why analytics when you have user interviews?
Why analytics when we have user interviews?

Question User interviews Analytics

What problems to solve? ✅ ✅

Which ones to solve first? ❌ ✅

Why is that a problem? ✅ ❌

Is this a possible solution? ✅ (usability calls ❌ (possible AFTER building and

with figma) running a/b tests, betas..)

Did this solve the problem? ❌ ✅

 Truth > Kernel of Truth
User interviews capture what users SAY they would do.

Analytics captures what users ACTUALLY do.

 Influence without authority

Job of PM Deeply understand customer pain/opportunities to

propose solutions and then rally different teams to solve that
problem.

User Interviews -> why is there a problem

Analytics → do enough people feel the pain?

Why + Does it matter -> Trust + Influence to make decisions

10
Recap: Why measure anything?

★ Complements user interviews

★ Helps teams understand what people ACTUALLY do in the wild
★ Helps with opportunity assessment and influence teams to care about problems.

11
Common pitfalls! 🚩
What can go wrong with analytics?
Metrics not tied to real customer value

13
Too many metrics → Analysis Paralysis

14
Misaligned Incentives!

15
Missed Opportunities!

16
Recap: Common pitfalls

★ Metrics not tied to real customer value

★ Too many metrics -> analysis paralysis
★ Wrong metrics -> Misaligned incentives -> wrong outcome
★ Wrong metrics -> Missed opportunities -> suboptimal outcome

17
WHERE WE ARE HEADED
Our goal for today!

Focus metric
Matters most to your
business

Level 1 metrics
Complement the focus metric

Level 2 metrics
More specific and drive the
L1 and focus metrics

v i deo ct
: u
m ple prod
Exa ming
ea
str 18
🌟North Star Metrics
(aka Focus Metric)
What is a North Star Metric?

What is it? Metric that’s most predictive of company’s long-term success.

Key characteristics:

★ Captures core value proposition (value moment) delivered to user

★ Predictive of company success
★ Simple (your NSM is only as good as your company's ability to internalize and
repeat it)

20
Identifying your product’s Value Moment
What’s the primary value moment?

22
What’s the primary value moment?

23
 Industry Example Value Moment

Media Played video / media

Finance Completed transaction

VALUE MOMENT Item ordered and

eCommerce
delivered (on-time)
The way products create Shared file / chat started /
SaaS
value for users is different etc.

in each industry. HR/Payroll Running employee payroll

Social Start Session

networking

Vacation Bookings
Rental

24
Identifying your product’s
natural usage frequency
 Daily: social media, gaming

NATURAL USAGE FREQUENCY

Match it to your product’s Weekly: business software, video

on-demand service
natural usage patterns.
Monthly: online publication
subscription, clothing retailer, banking
app

Annually: tax software, holiday card

app

26
What is my product’s natural usage
frequency?

27
 Metric Types
Options to choose North Star Metric from)
Metrics should address the full customer
lifecycle

Reach Activation Active Usage Engagement Retention

Addressable Value Frequent Deep / power Long-term
audience discovery usage usage usage

Business Specific
Customer satisfaction, Costs, efficiency

29
Reach

What is it? Total number of people who have used the product in a recent time period /
maximum number of users who could reasonably be activated.

Examples:

★ Total registered accounts

★ Paid monthly users (Dropbox)
★ Market share (e.g. % of Fortune 1000 as customers)

30
Activation

What is it? Activation is a foundational step that primes a new user to become an active
user.

31
Active Users

What is it? Counts the number (or % of users that have taken a key action and received
value from your product within a recent time period.

Active Users = Value Moment + Natural Usage Frequency

Examples:

- Monthly active riders (Uber)

- Daily Active People (Instagram)
- % of members that watch 15 minutes per month (Netflix)

32
Engagement

What is it? It accounts for depth (more intense usage), breadth (more varied features)
and frequency (more # of days) of completing key actions.

Examples:

○ Number of videos watched

○ Messages sent
○ Chat sessions with >10 messages
○ Number of orders (Cameo)
○ Number of days active (in previous 7 days)
○ Cart conversion rate

33
Retention

What is it? Rate at which users come back to use a product within a certain time period.

Examples:

★ Day-7 retention of watching videos

★ 3-month retention rate of meditators
★ Week-4 retention for riders

34
Revenue

What is it? Amount of money being made by the business.

Examples:

★ GMV (marketplace)
★ ARR/MRR SaaS
★ Transaction volume ($
★ Ads revenue
★ Affiliate revenue
★ Net Revenue Retention (Starting revenue + expanded revenue - contracted revenue
★ churned revenue/Starting revenue)

35
Business-Specific

What is it? These are the less generalizable metrics that are still very relevant to a
particular business.

Examples:

★ Data center burn

★ NPS
★ CSAT survey
★ Referral rate
★ Duolingo Learning Competency (CEFR

36
Recap - Metric types

★ Reach (total user-base that can be engaged)

★ Activation (rate at which new users → activated users)
★ Active Users (# of users that do value moment at a natural usage frequency)
★ Engagement (depth, breadth and frequency of usage - not all users are created
equal)
★ Retention (rate at which users are still using the product days,weeks later)
★ Revenue (total $ made by the business)
★ Business-specific

37
🌟North Star Metrics
choosing a North Star Metric
North Star Metric

39
North Star Metric

40
North Star Metric

41
North Star Metric

42
 Counter Metrics 📈
Aligning metrics to customer value isn’t enough. You still need checks and
balances.
Evaluating New
Features
01 Delight
02 Hard to copy
03 Margin-enhancing

44
Counter Metrics
Ask yourself:
★ If I am fixated on the north star metric only, how can I unintentionally end up hurting the user
experience and the business?
★ What checks do I need to add to avoid such an occurrence?

Metric Counter metric

Subscription purchase Activation rate, Retention
Ad Revenue NPS, Retention
Visits Conversion rate
LTV CAC

45
Product Market Fit 🦄
Analyzing PMF
 Analyzing Product-Market fit

100%
Retention curves

% users still active

0%

1 week Active weeks later 8 weeks

PC: Rocio Lopez 47

Analyzing Product-Market fit
Sean Ellis Survey

48
Analyzing Product-Market fit

Net Promoter Score

49
Analyzing Product-Market fit

Net Promoter Score

50
 Measurement framework example→ Mixpanel
ACTIVE USAGE

Weekly Learners
# of users who segmented a metric
on 3 out of the previous 7 days

Reach Activation Engagement Retention Business-Specific

Prod & Mkt Product Product Product Engineering

Accounts live 1st Answers per user Weekly Data center

in last 12 mos segmentation retention burn

Marketing Product Marketing Product Product Product Product Product CSM Eng

New Existing Resurrected Integration Segmentati Questions / Answer By plan By Fast ARR
users users users rate on rate user success type account
Analyzing Product-Market fit→ Airbnb

FOCUS METRIC

Nights booked

Reach Engagement Business-Specific

Unique visitors to Guest conversion Supply of homes

site rate
Thank You
Q&A 🎲
Assignment - Week 3
Assignment - Week 3 | Part 1

VALUE MOMENT NATURAL USAGE FREQUENCY

List the actions in your product that How often should a typical user be in
create value for users. Then circle the your product? Be specific (daily,
most important one. weekly, etc).

The more specific you are, the better you can measure Remember: Frequent usage may be unhealthy.
important usage patterns. (ex: completed deposit vs. (ex: daily use of an app for managing a cell phone plan)
transacted)

56
NORTH STAR METRIC

Assignment - Week 3 | Part 2

List what could be the north star metric for your company / Insurjo project.

Drop the name of company/project, 1-line description, value moment, North Star Metric, what were alternatives considered, and
why did you choose this.

57
Assignment - Week 3 | Part 3
Build your own metrics framework

Team responsible

Metric
FOCUS METRIC definition

Reach Activation Engagement Retention Business-Specific

58
The Lounge is now LIVE!