LECTURE 1

1. Product Analytics: Introduction

2. Product Analytics: Definitions by Eminent People

3. Importance of Product Analytics
4. Product Analytics Metrics
5. Product Analytics in Action
6. Who Decides? Hippo
7. Data Bias
8. Variables: Independent vs. Dependent vs. Interdependent
9. Research Workflow in Product Analytics
10. HANDSON: improving e-commerce shopping experiences
through a structured approach, using the principles of Product
Analytics. The key areas are:

 Decision Statements
 Research Objective
 Research Questions
 Hypothesis

1|Page
Product Analytics: Introduction

What is Product Analytics?

 Product Analytics is the practice of analyzing how users interact with a product. It helps businesses
understand user behavior, measure product performance, and make data-driven decisions to
improve the product.

Why is Product Analytics Important?

 It helps identify which features users love, which they ignore, and where they face issues.
 Companies use it to enhance user experience and build products that better meet customer needs.

Key Dimensions of Product Analytics

1. User Engagement

 Measures how much users interact with the product.

 Metrics:
o Active users: Daily/Monthly Active Users (DAU/MAU)
o Session duration: How long a user stays on the app.
o Clicks and taps: Which features users click on most.
 Example: If a food delivery app finds users spend more time browsing restaurants than ordering, it
indicates a need to streamline the process.

2. Feature Usage

 Tracks how frequently users use specific product features.

 Metrics:
o Feature adoption rate: Percentage of users using a feature.
o Frequency of use: How often a feature is used.
 Example: A payment app may see higher use of UPI payments than debit card payments, showing
user preference.

3. Retention and Churn

 Retention shows how many users return after using the product, while Churn refers to those who
stop using it.
 Metrics:
o Retention rate: Percentage of users returning after a specific time.
o Churn rate: Percentage of users leaving the product.
 Example: A streaming app may see a drop in user retention after a price hike, signaling
dissatisfaction.

4. Conversion Rate

 Measures how many users complete a desired action (like making a purchase).
 Metrics:
o Conversion funnel: Tracks the user journey toward conversion.
2|Page
 oDrop-off points: Identifies where users leave without completing the action.
 Example: An e-commerce site can analyze where users abandon their cart.

5. Customer Satisfaction

 Gauges user sentiment toward the product.

 Metrics:
o Net Promoter Score (NPS): Measures user willingness to recommend the product.
o Customer reviews and ratings: Direct feedback from users.
 Example: A cab service may see low ratings if drivers are consistently late.

Case Study: Indian Fintech App

Problem:

A popular Indian fintech app noticed a decline in user activity. Despite a rise in new sign-ups, few were
making transactions. The company needed to understand why users weren't engaging with its features.

Solution using Product Analytics:

1. User Engagement: The app tracked user sessions and found that many users spent time on the
homepage but didn’t navigate to the transaction section.
2. Feature Usage: Analytics showed that while users explored the app, they didn’t use the UPI
payment option, which was one of the key features.
3. Retention and Churn: The app noticed high churn after the first week of use. Users were dropping
off due to a confusing interface for UPI activation.
4. Conversion Rate: There was a significant drop-off right before payment confirmation. Many users
reported issues with the app not accepting certain bank accounts.
5. Customer Satisfaction: Negative reviews highlighted these issues, leading to a low Net Promoter
Score (NPS).

Action Taken:

 The app simplified the UPI activation process and addressed technical glitches for certain banks.
 A tutorial was added to guide users through the first transaction.
 After these improvements, user engagement increased, churn decreased, and the NPS score
improved.

Uncommon Insight:

 False Belief: "More features make a product better."

 Truth: Fewer, well-optimized features that users need can boost engagement and satisfaction. Too
many features often overwhelm users and reduce product value.

Product analytics offers powerful insights to make informed decisions, helping businesses evolve products
to meet user needs effectively.

Product Analytics: Definitions by Eminent People

3|Page
Introduction to Product Analytics

 Product Analytics is the process of gathering and analyzing data to understand how users interact
with a product. This helps businesses optimize the product and improve the user experience.

Several industry leaders have provided definitions of product analytics, each highlighting a different
perspective. Let’s explore their views.

Definitions by Eminent People

1. Brian Balfour (Founder of Reforge)

 Definition: "Product analytics helps businesses understand user behavior and use that data to drive
product growth."
 Key Takeaway:
o Focus on Growth: The primary goal is to use insights to help the product grow.

2. Julie Zhuo (Former VP of Product Design, Facebook)

 Definition: "Product analytics is about learning from data to build products that people love."
 Key Takeaway:
o User-Centric: The focus is on improving the user experience to make products more
engaging and enjoyable.

3. Ken Rudin (Former Head of Analytics, Facebook)

 Definition: "Product analytics turns data into insights that drive actionable decisions."
 Key Takeaway:
o Actionable Insights: It focuses on making data-driven decisions that improve the product.

4. Ben Horowitz (Co-founder, Andreessen Horowitz)

 Definition: "Product analytics is essential for making informed product decisions."

 Key Takeaway:
o Informed Decisions: It’s crucial for product teams to make decisions backed by data.

Dimensions and Sub-dimensions of Product Analytics

1. User Behavior

 Sub-dimensions:
o Clickstream Analysis: Tracks user interactions such as clicks and page visits.
o Session Duration: Measures how long users stay on a platform.
 Example: Flipkart tracks which product pages users visit the most during festive sales.

2. Engagement Metrics

 Sub-dimensions:
o Daily Active Users (DAU): Measures how many users engage daily.
4|Page
 Churn Rate: Tracks how many users stop using the product over time.
o
 Example: Paytm monitors DAU to track the impact of new wallet features.

3. Conversion Tracking

 Sub-dimensions:
o Conversion Rate: Percentage of users who complete desired actions (like purchasing).
o Funnel Analysis: Identifies where users drop off during their journey.
 Example: Myntra analyzes why users drop off during checkout after adding items to their cart.

4. Feature Adoption

 Sub-dimensions:
o Adoption Rate: Measures how quickly users start using new features.
o Feature Utilization: Tracks how frequently key features are used.
 Example: Swiggy tracks how quickly users switch to new payment methods like UPI.

5. A/B Testing

 Sub-dimensions:
o Hypothesis Testing: Experimenting with different versions of features.
o Outcome Evaluation: Deciding which version performs better based on data.
 Example: Zomato tests different home screen layouts to see which increases order rates.

Case Study: An Indian EdTech Platform

Problem:

 An Indian edtech platform noticed that many users signed up for free courses but very few upgraded
to paid subscriptions.

Solution:

1. User Behavior Analysis: Tracked user interactions and discovered that most users dropped off
during the payment step.
2. Engagement Metrics: Found that users weren’t engaging with premium course features before
being prompted to pay.
3. Conversion Tracking: Used funnel analysis to identify where users lost interest.
4. A/B Testing: Tested two different versions of the payment flow—one with a demo of premium
features and one without.
5. Results: Offering a demo improved conversion by 25%, leading to a rise in paid subscriptions.

Uncommon Insight:

 False Belief: Adding more features will improve user satisfaction.

 Truth: Sometimes too many features confuse users. Simplified experiences often lead to better
engagement and conversion.

5|Page
This overview of product analytics demonstrates how businesses can leverage data to make informed
product decisions and improve user experiences.

Importance of Product Analytics

Introduction

Product Analytics helps businesses track and analyze how users interact with their product. It informs key
decisions, improves user experience, and drives revenue growth. Based on the attached image, the critical
dimensions of product analytics include Improved Decision Making, Revenue Growth, Predictive
Analytics, Customer Understanding, and Product Optimization.

1. Predictive Analytics

 Definition: Using historical data to predict future trends and customer behaviors.
 Sub-dimensions:
o Forecasting: Estimating future sales or user behavior based on past patterns.
o Trend Analysis: Identifying emerging patterns in user behavior or product usage.
 Example: An Indian telecom company using past user data to predict demand for mobile data plans
during cricket seasons.

2. Customer Understanding

 Definition: Deep insights into how users think, behave, and engage with the product.
 Sub-dimensions:
o User Segmentation: Grouping users based on common characteristics (e.g., demographics,
behavior).
o Customer Feedback: Using surveys or direct feedback to understand user needs.
 Example: Flipkart analyzing shopping behavior during Diwali to provide customized product
recommendations.

3. Product Optimization

 Definition: Improving the product based on user data to enhance performance and user satisfaction.
 Sub-dimensions:
o Feature Enhancement: Identifying which features need improvements based on usage.
o User Experience (UX) Design: Optimizing design for easier and faster navigation.
 Example: Swiggy optimizing its checkout flow to reduce time taken for food orders during peak
hours.

4. Improved Decision Making

 Definition: Using analytics to make more informed, data-driven decisions.

 Sub-dimensions:
o A/B Testing: Comparing two versions of a feature to see which performs better.
6|Page
 o Data-Driven Insights: Making strategic choices backed by actual user data.
 Example: Amazon India using A/B testing to determine which product descriptions lead to higher
conversion rates.

5. Revenue Growth

 Definition: Leveraging product analytics to identify opportunities for increasing revenue.

 Sub-dimensions:
o Monetization: Identifying the right pricing strategies or upselling techniques.
o Customer Lifetime Value (CLV): Maximizing the value of each customer over their
lifetime.
 Example: Paytm increasing revenue by promoting its wallet services based on usage patterns.

Case Study: A Food Delivery App in India

Problem:

A popular food delivery app in India noticed high bounce rates during checkout, leading to a loss of
potential orders.

Solution:

1. Customer Understanding: The company analyzed user behavior and found that users were
abandoning orders due to unclear delivery fees.
2. Product Optimization: Introduced a transparent breakdown of charges at the start of the checkout
process, simplifying the interface.
3. Improved Decision Making: The team used A/B testing to experiment with different ways of
displaying the fee structure.
4. Revenue Growth: As users completed more orders, the app saw a 20% increase in daily
transactions.
5. Predictive Analytics: Predicted future spikes in orders during weekends, allowing better preparation
for peak demand.

Uncommon Insight:

 False Belief: Many assume that all customer data is equally valuable.
 Truth: Only specific, well-analyzed data leads to actionable insights. Collecting too much irrelevant
data can overwhelm and confuse decision-making.

Product analytics is essential for businesses to thrive in today’s data-driven world. By understanding user
behavior, predicting future trends, and optimizing the product, businesses can grow and create products that
meet real user needs.

Product Analytics Metrics

7|Page
Introduction

Product analytics metrics help businesses understand how customers interact with their products and where
they can improve. These metrics are crucial for making informed decisions that enhance customer
experience, increase retention, and drive revenue growth. In this discussion, we will explore key metrics
including Customer Engagement, Customer Churn, Customer Acquisition Cost (CAC), Customer
Lifetime Value (CLV), and Customer Retention.

1. Customer Engagement

 Definition: This measures how actively users interact with a product. It tracks user actions, such as
clicks, page views, and feature usage, to understand the level of user interest.
 Sub-dimensions:
o Daily Active Users (DAU): The number of unique users interacting with the product daily.
o Session Duration: The average time users spend on the platform in each visit.
 Example: Flipkart monitors customer engagement by tracking how often users browse product
categories during sales.

2. Customer Churn

 Definition: Churn measures the percentage of users who stop using the product over a given period.
A high churn rate indicates that the product is not meeting user needs.
 Sub-dimensions:
o Churn Rate: The percentage of customers who do not return to the product after a set time.
o Reasons for Churn: Identifying why users stop engaging (e.g., poor experience, high
prices).
 Example: Zomato tracks churn by analyzing how many users uninstall the app or stop placing orders
after their first experience.

3. Customer Acquisition Cost (CAC)

 Definition: CAC refers to the total cost incurred to acquire a new customer, including marketing,
sales, and promotions.
 Sub-dimensions:
o Marketing Spend: The budget allocated for advertising and promotional campaigns.
o Sales Efforts: The cost associated with converting leads into paying customers.
 Example: Paytm calculates how much it spends on digital marketing campaigns during festival
seasons to attract new users for its payment app.

4. Customer Lifetime Value (CLV)

 Definition: CLV is the total revenue a business can expect from a customer over their entire
relationship with the company. It helps businesses focus on retaining valuable customers.
 Sub-dimensions:
o Average Order Value: The average amount a customer spends on a purchase.
8|Page
 oRetention Period: How long a customer stays engaged with the product.
 Example: Swiggy tracks CLV by analyzing how much loyal customers order over several months
and how often they return.

5. Customer Retention

 Definition: This measures how well a business retains customers over time. High retention means
users find value in the product and are coming back.
 Sub-dimensions:
o Retention Rate: The percentage of customers who continue using the product after their
initial use.
o Incentive Programs: Offering loyalty points, discounts, or special offers to retain users.
 Example: Amazon Prime’s subscription model is an excellent retention strategy, offering faster
delivery, exclusive deals, and other perks to keep users engaged.

Case Study: An Indian EdTech Platform

Problem:

An Indian EdTech platform that offers online courses noticed that although many students signed up for free
courses, very few upgraded to paid courses. They experienced high churn in free-tier users, especially after
the first few days of course consumption.

Solution:

1. Customer Engagement: The platform used analytics to track how students interacted with their free
courses. They found that users stopped engaging because they weren’t aware of the value of the paid
features.
2. Customer Retention: To retain free-tier users, the platform introduced a freemium model where
users could experience premium features for a limited time. They also sent notifications to students
who hadn’t completed their free courses, reminding them of the benefits of upgrading.
3. Customer Acquisition Cost (CAC): They optimized their ad spend by targeting users who had
completed more than 50% of a free course, thus lowering acquisition costs and focusing on higher
conversion rates.
4. Customer Lifetime Value (CLV): They calculated that students who upgraded were 3 times more
likely to buy additional courses. By improving the user experience and adding value to paid tiers, the
platform increased CLV.
5. Outcome: After introducing these changes, conversion rates from free to paid courses increased by
25%, and the churn rate dropped significantly, leading to higher long-term retention.

Uncommon Insight:

 False Belief: Businesses often think that increasing marketing spend is the best way to acquire new
customers.
 Truth: Reducing churn and increasing customer retention is often more cost-effective than acquiring
new customers. Retained customers can provide more long-term revenue than new ones, especially
when companies focus on CLV.
9|Page
Product analytics metrics offer a comprehensive way to understand customer behavior, improve retention,
and optimize business strategies. By focusing on engagement, retention, and customer value, businesses can
make data-driven decisions that drive growth.

Product Analytics in Action

Product analytics involves collecting and analyzing data from product usage to guide decisions about
features, user experience, and overall product improvements. The goal is to help businesses make data-
driven decisions that enhance the value for customers and increase profitability. Now, let’s dive into each
image step by step.

1. Casino Slot Machines

 Dimension: Behavioral Analytics

o Casinos heavily use product analytics to influence user behavior. Slot machines are
strategically placed and designed to keep users engaged for longer periods.
o Subdimension: Customer Engagement
 Data from slot machines help casinos understand which games are played the most,
how long people stay engaged, and what factors lead to higher spending.
o Example: Casinos might adjust the lighting, sounds, or rewards structure based on this data
to increase engagement. For instance, they may notice that brighter lights and celebratory
noises increase the likelihood of continued play.
 Subdimension: Personalization
o Personalized offers are made to players based on their gambling behavior. For instance, if
someone plays certain games regularly, they may receive offers to come back with free
credits or room stays.
o Example: A regular player at a casino might receive personalized email offers based on how
much money they've spent or how often they've played certain games.

Insight: It’s often believed that casinos purely rely on luck, but they use sophisticated analytics to guide
customer behavior and maximize profits.

2. Netflix User Interface

 Dimension: User Experience (UX) Analytics

o Netflix optimizes its user interface based on data from how people interact with its platform.
This includes what people click on, how much time they spend on different sections, and
what makes them continue watching.
 Subdimension: A/B Testing
o Netflix often tests different user interfaces to see which one keeps users watching for longer
or makes them click “Next Episode” more often. Product analytics help decide which version
is better.
o Example: The placement of the "Next Episode" button is a product of analyzing how many
users want to skip credits and jump right into the next show.
 Subdimension: Recommendation System
o Netflix’s powerful recommendation system is driven by product analytics. Data is used to
recommend shows based on viewing history, preferences, and even the time of day.
10 | P a g e
 o Example: Netflix suggests a thriller show to someone who watched several similar movies.
The accuracy of these recommendations increases user satisfaction and retention.

Insight: While many people think Netflix’s recommendations are based solely on genres, it also considers
many hidden factors, like your watching habits at specific times or how long you pause shows.

3. Amazon Cart

 Dimension: E-commerce Analytics

o Amazon relies heavily on product analytics to enhance its e-commerce platform. Every click,
search, and purchase is analyzed to optimize the user experience and increase sales.

 Subdimension: Cart Abandonment

o Product analytics help Amazon understand why users abandon their carts. This could lead to
targeted reminders or discounts to bring users back to complete their purchase.
o Example: If a user adds a product to their cart but doesn’t check out, they might receive an
email reminder offering a discount on that product.

 Subdimension: Product Recommendations

o Similar to Netflix, Amazon also uses analytics to recommend products. It looks at purchase
history, browsing patterns, and the behavior of similar users to suggest items.
o Example: A customer who buys a phone might be recommended a phone case or earphones
based on past purchase data.

Insight: While most people believe that Amazon's "frequently bought together" section is purely based on
popularity, it's actually driven by complex machine learning models that predict what customers are most
likely to buy next.

4. Frequently Bought Together Section

 Dimension: Cross-Selling and Up-Selling

o This section of Amazon uses product analytics to bundle related items that users are likely to
buy together. The goal is to increase the average order value through smart cross-selling
techniques.
 Subdimension: Bundling Strategies
o Amazon analyzes user data to figure out which products should be recommended together.
This not only helps in making informed decisions but also enhances customer satisfaction by
offering useful suggestions.
o Example: A person buying a musical instrument might be shown related accessories, such as
cables or headphones, which enhances the customer experience and increases sales.
 Subdimension: Pricing Analytics
o Amazon uses product analytics to determine pricing strategies that will maximize revenue
while still offering competitive prices.
o Example: The combination of a discount when purchasing multiple items may prompt a
customer to buy all of them, raising the overall transaction value.

Insight: Some think that “frequently bought together” is just a reflection of customer behavior. However,
it’s actually a result of deliberate data-driven choices to boost sales.

11 | P a g e
5. Member Rewards

 Dimension: Loyalty Analytics

o Loyalty programs are often enhanced using product analytics. Companies track which
rewards keep users coming back and what types of offers are most successful in retaining
customers.

 Subdimension: Reward Optimization

o By analyzing user engagement with different rewards, companies can refine their loyalty
programs. This involves tweaking point systems, reward tiers, and special promotions.
o Example: A supermarket chain might offer double points on weekends after analyzing that
foot traffic is lower during those times.

 Subdimension: User Retention

o Loyalty programs are designed to retain customers over the long term. Product analytics
allow businesses to evaluate how well the program is working and what changes could be
made to improve it.
o Example: A clothing brand might notice that customers are more likely to use loyalty points
on new arrivals, leading them to offer more points on new stock to increase early purchases.

Insight: Many people assume loyalty programs are about giving away discounts. In reality, they are
powerful tools driven by data to increase customer lifetime value.

Case Study: Flipkart's Use of Product Analytics

Problem

Flipkart, one of India’s leading e-commerce platforms, noticed that a significant number of users were
abandoning their shopping carts, leading to missed sales. Additionally, their recommendation system was
not as effective in keeping users engaged with the platform compared to other competitors like Amazon.

Solution

1. Behavioral Analytics: Flipkart began analyzing user behavior, tracking every interaction with the
site, from what users clicked on to how long they spent on each page.
2. Cart Abandonment Analysis: Through data analysis, they discovered that users often abandoned
carts because of shipping fees shown at the last step of the purchase process. To fix this, Flipkart
started showing estimated shipping costs earlier in the process, and even offered discounts on
shipping fees for certain thresholds.
3. Personalization and Recommendation Engine: Flipkart revamped its recommendation engine
based on advanced product analytics, focusing on cross-selling and upselling. They began to
recommend products based on real-time browsing history and similar customer behavior.
4. Loyalty Program: Flipkart Plus, their loyalty program, was refined using product analytics. They
discovered that offering free delivery was a huge motivator for users to join the program. Data
showed that customers in the program had higher purchase frequencies and higher average order
values than those who weren't members.

Results

12 | P a g e
  Increase in Sales: Cart abandonment dropped by 15% in the first quarter after implementing these
changes.
 Improved Customer Satisfaction: User surveys showed increased satisfaction with the platform’s
transparency about shipping fees.
 Higher Engagement: The improved recommendation engine kept users on the platform longer, and
cross-sell purchases increased by 20%.

Flipkart’s success in using product analytics demonstrates the power of understanding user behavior and
continuously optimizing the product experience to meet customer needs.

Conclusion

Product analytics is a powerful tool that helps businesses enhance customer experience, drive sales, and
improve product offerings. From casinos optimizing slot machine placements to Amazon's cross-selling
strategies, the examples show how analytics shape modern business practices. By continuously analyzing
user behavior, businesses can make informed decisions to stay competitive and keep customers engaged.

Who Decides? HippO

Overview

Decision-making within organizations can often be influenced by the loudest voice or the "HiPPO" (Highest
Paid Person’s Opinion). This dynamic can sometimes override data-driven insights, leading to suboptimal
outcomes. In the context of product analytics or business strategy, it’s critical to understand the pitfalls of
this approach and the importance of using data over individual opinions.

Dimensions and Subdimensions

1. The Loudest Voice

 Dimension: Influence of Authority or Dominant Personalities

o In some organizations, decision-making is driven by individuals who assert dominance
through strong personalities, regardless of whether their insights are backed by data.
o Example: A CEO or manager with a persuasive style might push for a marketing strategy
they prefer, ignoring feedback from product analytics that suggests a different direction.

 Subdimension: Groupthink
o This happens when team members agree with the loudest voice to avoid conflict, leading to a
lack of critical thinking and innovation.
o Example: A marketing team might go along with the loudest person in the room, even if
analytics suggest their campaign won’t work.

2. HiPPO (Highest Paid Person's Opinion)

 Dimension: Authority-Driven Decisions

o HiPPOs are senior executives or individuals in leadership roles who influence decisions by
virtue of their position, even when their opinion may not align with the data.
13 | P a g e
 o Example: An organization’s senior VP may decide to launch a new product feature based on
personal experience or preference, disregarding the analytics that indicate a different need for
users.

 Subdimension: Bias in Decision-Making

o HiPPO decisions often stem from cognitive biases, such as overconfidence in personal
experience or resistance to new information that contradicts past successes.
o Example: A HiPPO might insist on sticking to traditional methods, ignoring data suggesting
that customers are more receptive to digital marketing strategies.

Why This Approach Can Be Problematic

 Lack of Data-Driven Decisions

o Decisions based on the loudest voice or HiPPO can lead to misguided strategies that are not
rooted in factual insights or analytics.
 Suboptimal Outcomes
o When data is ignored in favor of authoritative opinions, companies may miss opportunities
for optimization and innovation.
 Limited Innovation
o Groupthink and HiPPO dominance stifle creativity, as teams become less likely to challenge
decisions, even when data supports a different direction.

Uncommon Insight

 False Belief: The HiPPO Always Knows Best

o Many believe that senior leadership always makes the best decisions, but this is false. Data-
driven approaches often outperform decisions made solely on experience or intuition.

Indian Case Study: Decision-Making at a Local E-Commerce Startup

Problem

An Indian startup specializing in online fashion was facing decreasing customer engagement on their
platform. The HiPPO, the company's founder, believed the issue was a lack of product variety and pushed
the team to focus on expanding inventory. However, the analytics team suggested that the real issue was
slow website performance and complicated checkout processes, which were driving customers away.

Solution

Despite initial resistance from the HiPPO, the team presented clear data showing that users abandoned the
site due to loading delays, not product variety. The team A/B tested faster page loading times and simpler
checkout options. As a result:

 Website speed improved by 40%, reducing bounce rates.

 Checkout process simplification led to a 25% increase in completed purchases.

14 | P a g e
Eventually, the HiPPO acknowledged the data-driven insights, and the startup shifted its focus to optimizing
user experience rather than expanding inventory, leading to improved customer satisfaction and revenue
growth.

Conclusion

Decisions made by the loudest voice or HiPPO can sometimes lead to suboptimal results. Data-driven
decision-making, based on product analytics and user behavior insights, tends to lead to better outcomes. By
relying on evidence rather than opinions, companies can drive innovation, avoid biases, and optimize their
strategies for long-term success.

Data Bias

Overview

Data bias occurs when there is a systematic error in the collection, processing, or interpretation of data,
leading to misleading conclusions. The image humorously depicts a form of bias where respondents are
coerced into answering a specific question, illustrating how data collection methods can lead to skewed
outcomes. In the real world, data bias can significantly impact decision-making, especially in product
development and marketing.

Dimensions and Subdimensions of Data Bias

1. Data Collection Bias

 Dimension: Sampling Bias

o Sampling bias occurs when the sample used for data collection is not representative of the
broader population. This leads to skewed results, as the data only reflects a specific subset.
o Example: A retail company surveys only its high-spending customers about their
satisfaction, ignoring the majority of mid-range and low-spending customers. This leads to an
overestimation of customer satisfaction.
 Subdimension: Question Framing Bias
o The way a question is asked can bias the responses. Leading questions or loaded questions
can prompt respondents to answer in a particular way.
o Example: In the image, the researchers ask, "Would you pay $2.99 for toothpaste?" in a way
that makes the respondent feel pressured to agree. A more neutral approach would be, "What
price would you consider reasonable for toothpaste?"

2. Analysis Bias

 Dimension: Confirmation Bias

o Analysts may focus on data that confirms their preconceptions while ignoring data that
contradicts their views. This can lead to one-sided interpretations.
o Example: A product team might focus on positive feedback from customer surveys while
disregarding critical feedback, leading them to mistakenly believe their product is flawless.
 Subdimension: Data Interpretation Bias
o Misinterpreting the data or drawing incorrect conclusions due to preconceived notions or
flawed methodologies.

15 | P a g e
 o Example: A marketing team may see an uptick in website visits and assume a successful
campaign, when in reality, the bounce rate is high, indicating that users are not staying on the
site.

3. Technological Bias

 Dimension: Algorithmic Bias

o In machine learning and AI, if the training data is biased, the algorithms will make biased
predictions. This is a growing concern in automated decision-making systems.
o Example: A recommendation system on an e-commerce platform may favor products that
historically performed well, thereby reinforcing the popularity of those products while
ignoring newer or niche items.

4. Presentation Bias

 Dimension: Visualization Bias

o The way data is presented visually can influence the interpretation. Charts and graphs can be
manipulated by adjusting scales or focusing on selective data points.
o Example: A company might show only the best months of sales in a graph to give the
impression that business is booming, while omitting months with poor performance.

Uncommon Insight

 False Belief: "More Data Means Better Insights"

o Many people believe that collecting more data automatically leads to better insights. This is
false. If the data is biased, larger quantities only amplify the bias. Quality and representation
matter far more than sheer volume.

Indian Product Case Study: Paytm's Data Collection Bias

Problem

Paytm, a leading digital wallet company in India, wanted to increase user engagement on its platform. They
conducted surveys to understand why users were not utilizing certain features, such as bill payments and
financial services. However, their surveys were only targeted at tech-savvy users in urban areas, leading to a
biased view of customer needs.

Solution

1. Sampling Bias: Paytm recognized that their survey sample was not representative of their entire user
base. They expanded their surveys to include rural users, small business owners, and individuals
from lower socio-economic backgrounds. These groups had different needs and faced unique
challenges, such as limited access to smartphones or the internet.
2. Question Framing: Initially, the surveys framed questions in a way that assumed users were
familiar with digital wallets. By changing the question framing to ask more basic questions about
digital literacy and mobile usage, Paytm got a clearer understanding of why some features were
underused.
3. Data Analysis: After analyzing the broader dataset, Paytm realized that many rural users needed
simpler, more accessible features like cash-in-hand payment options, or offline support for
16 | P a g e
 transactions. They also learned that their urban, tech-savvy user base had different priorities, like
cashback offers or instant loan services.
4. Outcome: By addressing these biases, Paytm redesigned their app to offer simpler user interfaces,
multilingual support, and financial education for new users. This led to an increase in engagement
from rural users by 30% and improved adoption of their financial services.

Conclusion

Data bias can distort decision-making and lead to flawed product strategies. By addressing biases in data
collection, analysis, and presentation, companies can make more informed, accurate decisions. The case of
Paytm highlights the importance of considering diverse user groups and ensuring data represents the broader
population for successful product development.

Variables: Independent vs. Dependent vs. Interdependent

Overview

In data analysis, variables play a critical role in understanding relationships and making predictions.
Variables can be classified as independent, dependent, and interdependent. In this context, the slide shows a
used car listing with multiple variables, such as car name, year, selling price, and more. Let’s break down
these variables, explain their roles, and provide an understanding of their interrelationships using clear
examples.

Dimensions of Variables

1. Independent Variables

 Definition: Independent variables are inputs or factors that influence the outcome but are not
affected by other variables in the model. These are the "cause" in a cause-effect relationship.
 Example from Image:
o Year of the car, Fuel Type, and Transmission are independent variables in the car selling
process. These factors exist independently of other variables and can influence the selling
price.
o Explanation: The year of the car (how old it is) directly affects the selling price. A newer car
will likely have a higher value compared to an older model.

2. Dependent Variables

 Definition: Dependent variables are the outcomes or results that are influenced by the independent
variables. They are the "effect" in a cause-effect relationship.
 Example from Image:
o The Selling Price is the dependent variable. It depends on several factors, such as the car's
age, kilometers driven, and fuel type.
o Explanation: If the car has been driven a lot (high kilometers), the selling price may
decrease, making the selling price dependent on how many kilometers the car has been
driven.

3. Interdependent Variables

17 | P a g e
  Definition: Interdependent variables are those that influence each other. In some cases, variables do
not have a strict one-way relationship but instead impact one another in a cyclic or feedback manner.
 Example from Image:
o Present Price and Kms Driven can be considered interdependent. A car’s value on the
market is determined by how much it has been driven, and how much it has been driven
affects its current value.
o Explanation: The more kilometers a car has been driven, the lower its present price, but the
present price also affects the demand, which in turn may affect how often the car is driven
before a sale.

Detailed Example with a Popular Indian Product: OLX Car Listings

Problem

OLX, a popular platform for buying and selling used cars in India, often faces the challenge of helping
buyers and sellers determine the right selling price. Several variables, such as the car's year, kilometers
driven, and transmission type, all play a role in determining the final selling price. However, sellers often
overestimate the value of their vehicles, leading to overpriced listings.

Solution

1. Independent Variables Analysis: OLX conducted data analysis to understand how independent
variables like car year, transmission type, and fuel type affected car prices. They found that manual
transmission cars typically had lower selling prices than automatic ones, and diesel cars held their
value better than petrol cars.
2. Dependent Variables Analysis: The Selling Price was analyzed as a dependent variable. By
analyzing data from thousands of listings, OLX developed a pricing tool that could predict the
appropriate selling price based on factors such as car year and kilometers driven. This tool helped
sellers set realistic prices, improving the chances of a successful sale.
3. Interdependent Variables Consideration: The platform also recognized that factors like Seller
Type (individual or dealer) and Owner History (first or second owner) were interdependent. Cars
with multiple owners or sold by dealers tended to have a lower selling price due to perceived wear
and tear or distrust in dealers.
4. Outcome: By using the pricing tool, sellers were able to set fair market prices, leading to quicker
sales. Buyers also benefited from transparency, as they could see how various factors influenced the
price of a vehicle.

Uncommon Insight

 False Belief: The Selling Price Is Only Influenced by Age

o Many believe that the age of a car is the most important factor in determining its price.
However, this is false. Other variables, such as the number of kilometers driven, the condition
of the car, and the type of transmission (manual vs. automatic), can play an equally important
or even larger role in determining the price.

Conclusion

18 | P a g e
Understanding the roles of independent, dependent, and interdependent variables is crucial for making
informed decisions, whether in buying a used car or analyzing any other product. The case of OLX’s used
car listings illustrates how proper analysis of these variables leads to better pricing strategies and market
outcomes. By applying data-driven insights, both buyers and sellers can optimize their decisions.

Research Workflow in Product Analytics

1. Identifying the Idea / Problem Symptom

 Description: This step focuses on recognizing what issue or opportunity you're trying to address
with a product. It could stem from low user engagement, decreasing sales, or new market demands.
 Example: Imagine an e-commerce platform observing a drop in customer retention. This is the
symptom that requires further investigation.

2. Interview Analysts

 Description: Once the problem is identified, gather insights from analysts, data scientists, or domain
experts to frame the problem accurately. Analysts help translate raw data into meaningful insights.
 Example: For the e-commerce platform, interview customer behavior analysts to understand
potential reasons for low retention, like poor user experience, missing features, or technical glitches.

3. Formulating the Management Decision

 Description: At this stage, based on insights from analysts, the management decides the focus areas
for solving the problem. The decisions typically align with the company’s strategic objectives.
 Example: The management might decide to focus on enhancing the product’s user experience to
solve the customer retention issue.

4. Arrive at Research Objectives

 Description: Here, concrete research goals are established. These could involve hypotheses around
potential solutions, specific customer behaviors to analyze, or product features to improve.
 Example: A research objective could be: "Investigate how streamlining the checkout process can
increase retention by 20%."

5. Formulate Multiple Research Questions

 Description: This step involves asking detailed questions to uncover the root causes and potential
solutions. These questions guide the direction of data collection and analysis.
 Example:
o What percentage of customers drop off at the payment page?
o How does page loading time affect customer engagement?

6. Testing Hypotheses

 Description: After formulating hypotheses based on research questions, they are tested using
quantitative data (such as user metrics) or qualitative methods (like user interviews or A/B testing).
 Example: Implement an A/B test to compare different versions of the checkout process and
determine which version leads to higher customer retention.

19 | P a g e
Key Dimensions and Sub-dimensions in Product Analytics Research Workflow

A. Data Collection

 Sub-dimensions:
o User behavior tracking (e.g., clicks, page visits)
o Sales data (e.g., purchase history)
o Feedback surveys (e.g., customer satisfaction)

B. Data Analysis

 Sub-dimensions:
o Descriptive analytics (What happened?)
o Diagnostic analytics (Why did it happen?)
o Predictive analytics (What will happen next?)
o Prescriptive analytics (What should be done?)

C. Decision-Making

 Sub-dimensions:
o Strategic alignment (How does it fit the broader business goals?)
o Cost-benefit analysis (Is the solution cost-effective?)
o Feasibility (Can the solution be implemented with available resources?)

Case Study: Swiggy’s Product Analytics Journey

Problem Symptom

 Swiggy noticed a significant drop in user engagement on their app, especially during the payment
stage, resulting in high cart abandonment rates.

Interview Analysts

 Swiggy's product team interviewed data analysts who identified that customers were dropping off
during the payment process due to a complicated multi-step checkout flow and slow payment
gateway loading.

Formulating the Management Decision

 The leadership team decided to improve the user experience specifically around checkout, focusing
on simplifying the payment process and speeding up transactions.

Arriving at Research Objectives

 The key research objective was to increase successful transactions by reducing checkout
abandonment by 15%.

Formulate Multiple Research Questions

 Swiggy's team formulated specific questions like:

o How long do users spend on the payment page before abandoning the cart?

20 | P a g e
 o What percentage of users experience errors during payment?

Testing Hypotheses

 Swiggy ran A/B tests comparing the original multi-step checkout to a streamlined one-click payment
process. They also analyzed user feedback on the new version.

Solution and Results

 By optimizing the checkout process (integrating one-click payments and improving gateway speed),
Swiggy saw a 20% reduction in cart abandonment. This also led to a 10% increase in daily
transactions, boosting revenue.

Uncommon Insights in Product Analytics

 False Belief: "More data equals better decisions."

o Truth: It’s not about the volume of data but about identifying the right data. Too much data
can cloud decision-making unless it's focused on key performance indicators (KPIs).
 False Belief: "All problems can be solved with A/B testing."
o Truth: A/B testing is not always feasible, especially for long-term changes. Some decisions
require deeper insights from user behavior trends and qualitative analysis, like interviews or
usability testing.

By understanding each step of the research workflow and applying it systematically, companies can better
address product challenges and make data-driven decisions to optimize their product experience and user
engagement.

HANDSON:

improving e-commerce shopping experiences through a structured approach, using the principles of Product
Analytics. The key areas are:

1. Decision Statement: "What should be done to increase shopping on the eComm portal?"
2. Research Objective: "Factors influencing college students to buy from the portal."
3. Research Questions: This includes exploring user experience, ease of product search, and the
checkout process.
4. Hypothesis: The hypothesis focuses on the impact of more search filters in helping shoppers find
specific products faster, which could result in a 30% improvement in users finding products within
one minute.

Product Analytics refers to the process of analyzing how users interact with your product or service, with the
goal of understanding behavior, improving experiences, and driving outcomes like increased sales.

I. Decision Statement

 Dimension: This sets the core problem that needs to be addressed.

 Sub-dimensions:
21 | P a g e
 o User Retention: How do we make sure customers return to the portal?
o Increased Conversions: What steps can be taken to encourage more purchases?
o Customer Satisfaction: Are the users happy with their experience? What will make their
journey smoother?

Analogy: Imagine a retail store struggling to increase footfall. The same principles apply to your
eComm portal: how do you get more people in, keep them browsing, and ensure they leave with a
purchase?

II. Research Objective

 Dimension: This focuses on identifying the key factors that influence college students to purchase
from the portal.
 Sub-dimensions:
o Target Audience Understanding: College students may have unique preferences, budgets,
and priorities when shopping online.
o Price Sensitivity: Students often prioritize affordability. Hence, offering competitive prices
or discounts can influence their decisions.
o User Experience: The overall navigation and layout of the portal must appeal to students
who are often tech-savvy but have low patience for poorly designed sites.
o Product Range: A broad and relevant range of products is essential. If students can find what
they need (e.g., stationery, gadgets, fashion), they are more likely to make a purchase.
o Trust Factors: Does the platform feel secure for students to trust with their personal and
payment information?

Example: For college students, an intuitive platform that makes it easy to buy textbooks or
electronics with student discounts will likely perform well.

III. Research Questions

To enhance the shopping experience, ask the right research questions. Let’s analyze the questions provided:

1. How easy is it to find a product?

o Dimension: Ease of navigation and product discovery.
o Sub-dimensions:
 Search Functionality: How well does the search feature work? Are the results
relevant?
 Filters and Sorting: More search filters allow customers to narrow down their
choices quickly.
 Product Categorization: Are products properly categorized, and can they be found
easily?

Analogy: Think of a bookstore with shelves labeled incorrectly. Customers would find it frustrating
to locate the right book. Similarly, a disorganized eComm portal makes users abandon their cart.

2. How is the checkout experience?

o Dimension: Efficiency of the purchase process.
o Sub-dimensions:

22 | P a g e
  Simplicity of Checkout: A smooth, multi-step checkout process without glitches.
 Payment Options: Offering multiple payment options (UPI, cards, wallets) for
convenience.
 Cart Abandonment Rate: Measuring how many users leave without purchasing and
understanding why.

Analogy: In a physical store, if the cashier counter is too crowded or slow, customers may leave
their items and walk away. Similarly, an online store with a poor checkout flow will lose sales.

IV. Hypothesis Development

 Dimension: Hypothesis testing in analytics is the process of forming educated guesses and validating
them through data.
 Sub-dimensions:
o Search Filters Hypothesis: With additional filters, shoppers will find specific products
faster.
o Time Reduction Goal: 30% more users will find what they need within a minute of
searching.

Uncommon Insight: Many eCommerce platforms wrongly assume that offering too many filters
complicates the experience. In reality, targeted filters (like by price range, brands, or student-specific
deals) simplify decision-making, especially for students with tight budgets.

Example: Let’s say a student is looking for a specific brand of laptop within a set budget. If the
portal has a “student discount” filter or a “laptops under ₹40,000” filter, the student will appreciate
the ease of search, thus leading to a faster purchase.

Key Dimensions and Sub-dimensions

A. User Experience (UX)

 Dimension: This is the most critical aspect influencing purchase decisions.

 Sub-dimensions:
o Intuitive Design: The design should be clean, engaging, and require minimal clicks to reach
checkout.
o Mobile Optimization: Since most students shop using smartphones, ensuring the site is
mobile-friendly is crucial.
o Loading Speed: If the site takes too long to load, users may leave out of frustration.

Example: Think of using Google. The search results load almost instantly. If your eComm portal can
replicate this speed, you will see happier users.

B. Personalization

 Dimension: Tailoring the shopping experience to individual users.

 Sub-dimensions:
o Product Recommendations: Use algorithms to suggest products based on previous
purchases or browsing history.

23 | P a g e
 o Dynamic Pricing: Offer exclusive discounts to students or show products within their price
range.

Example: Amazon's "customers also bought" feature is a prime example of effective personalization
that boosts sales.

C. Trust and Security

 Dimension: Students need to feel their data is secure, and the platform is trustworthy.
 Sub-dimensions:
o Secure Payment Gateways: SSL certificates and a secure payment infrastructure.
o Reviews and Ratings: Displaying verified customer reviews adds credibility.
o Clear Return Policies: Transparent return and refund policies increase buyer confidence.

Example: Flipkart introduced “no questions asked” returns, which helped increase trust among
users.

D. Marketing and Promotions

 Dimension: Effective marketing campaigns can drive more traffic.

 Sub-dimensions:
o Targeted Email Campaigns: Reach students with discount offers or seasonal sales.
o Referral Programs: Encourage students to refer their friends for additional rewards.
o Social Media Marketing: Use platforms like Instagram or Facebook, where students spend
time, to promote products.

Example: Zomato’s discount offers targeted at students during peak exam seasons boosted their
student user base significantly.

Case Study: Flipkart’s Student-Centric Approach

Problem: Flipkart noticed that although they had a large number of users, student engagement was
relatively low. Students are a major demographic with high purchasing power, especially for electronics,
textbooks, and fashion, but they were not utilizing Flipkart’s portal effectively.

Solution:

1. Student Pricing & Discounts: They introduced the Flipkart "Back to College" campaign, offering
exclusive discounts on laptops, mobile phones, and other student essentials.
2. User Experience Enhancements:
o They revamped their search functionality to introduce more specific filters like “Best Laptops
under ₹50,000” or “Mobiles for Students,” making the portal more intuitive.
o They focused heavily on mobile optimization, ensuring the site worked seamlessly on
smartphones.

3. Improved Checkout Experience:

o A one-step checkout process was designed with UPI and digital wallets integrated for faster
payments.

4. Trust & Security:

24 | P a g e
 o Flipkart worked on enhancing trust by introducing easier returns, student-friendly EMI
options, and secured payment gateways.

5. Targeted Marketing:
o Flipkart used social media platforms popular with students and ran targeted ads and
promotions during the exam season when students are looking for electronic gadgets and
study materials.

Results:

 Increase in Student Engagement: The platform saw a 40% increase in student traffic within the
first quarter of the campaign.
 Higher Conversion Rate: The changes in search filters helped students find products 20% faster,
which resulted in higher conversion rates.
 Increased Loyalty: The referral program incentivized students to bring in friends, increasing the
overall number of users

Uncommon Insight

One common misconception is that lowering prices is the only way to increase sales among students. In
fact, value-driven benefits like offering better product filters, improving mobile user experience, and
creating a faster checkout process often have a more significant impact than mere price reductions.
Students value convenience, speed, and trust just as much as affordability.

25 | P a g e