lOMoARcPSD|40030935

Compendium IIM Shillong Analytics and Prod Man

Interview Prep (Indian Institute of Management Kashipur)

Scan to open on Studocu

Studocu is not sponsored or endorsed by any college or university

Downloaded by alex sharma ([email protected])
 lOMoARcPSD|40030935

bITeSys Compendium

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Contents
1. Overview ...................................................................3
2. Analytics .................................................................. 45
3. Industry Applications of Data Science....................... 21
4. Sample Puzzles ......................................................... 23
5. Product Management .............................................. 26
7. PM Frameworks ....................................................... 46
8. Metrices ................................................................... 46
9. Types of Product Questions...................................... 52
10.Useful Resources...................................................... 68

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Overview
Every day the world around us creates about 2.5 quintillion bytes of data, around 2,500,000
TBs of data. Even though most of the data is junk (not used again), we can easily guess that
data will play a vital role in every aspect of our life. The ads served to you on your
mobile/laptop device are personalised because of the data created by none other than you.
A local shop can reduce churn prevention by looking up at data coming from Google Analytics
and fixing the issues on their website. The best part about all of this is that we would be
making critical decisions based on factual data and not just hypotheses or gut feelings of the
owner.

As an effort in this direction, bITeSys is proud to present to you a compendium that can help
you prepare for your placement interviews. It is divided into two segments – Analytics and
Product Management. Each part is further divided into relevant sub-categories. As always,
no one document/book is the source of all knowledge, so don’t limit yourself to just this
compendium but look out for other credible sources. We have also included a collection of
sources here as well; feel free to explore them.

All the best for your placement process!

Regards,

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Analytics
1. Difference between supervised and unsupervised Learning

Supervised Learning Unsupervised Learning

Supervised learning is where you have Unsupervised learning is where you only
input variables (x) and an output variable have input data (X) and no corresponding
(Y) and you use an algorithm to learn the output variables.
mapping function from the input to the
output.
Y = f(X)
Further grouped into regression and Further grouped into clustering and
classification problems. association problems.
• Classification: A classification problem is • Clustering: A clustering problem is
when the output variable is a category, where you want to discover the inherent
such as “red” or “blue” or “disease” and groupings in the data, such as grouping
“no disease” customers by purchasing behaviour
• Regression: A regression problem is • Association: An association rule learning
when the output variable is a real value, problem is where you want to discover
such as “dollars” or “weight” rules that describe large portions of your
data, such as people that buy X also tend
to buy Y

Few use cases of Supervised learning Few use cases of unsupervised learning
• Predicting the price of a house based on • Grouping customers into different
attributes like sq. foot of house, no of buckets based on their purchasing
rooms, locality etc pattern
• Image classification - Classifying • Image categorization - Categorize image
whether an image is a cat image into cat image, dog image, or lion image
• Spam email classification - Identifying
and classifying whether an email is a
spam or not

Few supervised learning algorithms Few unsupervised learning algorithms

• Linear Regression for regression • k-means for clustering problems
problems • Apriori algorithm for association rule
• Random Forest for classification and learning problems
regression problems
• Support Vector Machines for
classification problems

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

2. Difference between Data Mining and Data Analysis

Data Mining Data Analysis

A hypothesis is not required for Data Mining Data analysis begins with a hypothesis.
Data Mining demands clean and well- Data analysis involves data cleaning.
documented data.
Results of data mining are not always easy to Data analysts interpret the results and
interpret. present it to the stakeholders.
Data mining algorithms automatically Data analysts have to develop their own
develop equations. equations.

3. Steps in an Analytics Project

4. What is data exploration?

Data exploration is done to become familiar with the data. This step is especially important when
dealing with new data. There are a number of things you will want to do in this step –
What is there in the data – look at the list of all the variables in the data set. Understand the
meaning of each variable using the data dictionary. Go back to the business for more information
in case of any confusion.
How much data is there – look at the volume of the data (how many records), look at the time
frame of the data (last 3 months, last 6 months etc.)
Quality of the data – how much missing information, quality of data in each variable. Are all fields
usable? If a field has data for only 10% of the observations, then maybe that field is not usable
etc.
You will also identify some important variables and may do a deeper investigation of these. Like
looking at averages, min and max values, maybe 10th and 90th percentile as well…

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

5. What does Data Preparation entail?

In data preparation, you will prepare the data for the next stage i.e. the modelling stage. What
you do here is influenced by the choice of technique you use in the next stage. But some things
are done in most cases – example identifying missing values and treating them, identifying outlier
values (unusual values) and treating them, transforming variables, creating binary variables if
required etc.
This is the stage where you will partition the data as well i.e. create training data (to do modelling)
and validation (to do validation)

6. How are missing values treated?

The first step is to identify variables with missing values. Assess the extent of missing values. A
few possible methods are –

7. How are outliers treated?

You can identify outliers using graphical analysis and univariate analysis. If there are only a few
outliers, you can assess them individually. If there are many, you may want to substitute the
outlier values with the 1st percentile or the 99th percentile values. If there is a lot of data, you may
decide to ignore records with outliers.
Not all extreme values are outliers. Not all outliers are extreme values.

8. Correlation
Correlation analysis is a method of statistical evaluation used to study the strength of linear
relationship between two, numerically measured, continuous variables.
There are a number of ways to calculate the correlation value. The most commonly used formula
for Calculating correlation coefficient is the Pearson’s Correlation coefficient formula which is as
follows.

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Another correlation formula which is used is as follows

Where 𝜎𝑥 and 𝜎𝑦 are the standard deviation for the two data sets and 𝜎𝑥𝑦 is the covariance.

9. Covariance
Covariance is the expected value of variations of two random variates from their expected values.
It is a measure of how changes in one variable are associated with changes in a second variable.
A positive correlation means that higher values of one variable are associated with higher values
of the other variable.

Where,
𝑋𝑖 = the values of the X variable
𝑌𝑗 = the values of the Y variable
𝑋 = the mean of the X-variable
𝑌 = the mean of the Y-variable
n = the number of data points

10. What is Hypothesis Testing?

Hypothesis testing is the use of statistics to determine the probability that a given hypothesis is
true. The usual process of hypothesis testing consists of four steps.
a. Formulate the null hypothesis H0 (commonly, that the observations are the result of
pure chance) and the alternative hypothesis Ha (commonly, that the observations show
a real effect combined with a component of chance variation)
b. Identify a test statistic that can be used to assess the truth of the null hypothesis
c. Compute the P-value, which is the probability that a test statistic at least as significant
as the one observed would be obtained assuming that the null hypothesis were true.
The smaller the P-value, the stronger the evidence against the null hypothesis
d. Compare the p-value to an acceptable significance value alpha (sometimes called an
alpha value). If p<=alpha, that the observed effect is statistically significant, the null
hypothesis is ruled out, and the alternative hypothesis is valid

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

A few related concepts:

• Directional vs Non-Directional Hypothesis - In the Directional Hypothesis, the null
hypothesis is rejected if the test score is too large (for right-tailed and too small for left
tailed). Thus, the rejection region for such a test consists of one part, which is left/right
from the center. In a Non-Directional Hypothesis test, the Null Hypothesis is rejected if
the test score is either too small or too large. Thus, the rejection region for such a test
consists of two parts: one on the left and one on the right.

• Z-Test – This is a statistical way of testing a hypothesis when either o We know

the population variance, or
o We do not know the population variance but our sample size is large n ≥
30
• T-test – This is a statistical way of testing a hypothesis when: o We do not know
the population variance
o Our sample size is small, n < 30

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Example - A principal at a certain school claims that the students in his school are above average
intelligence. A random sample of thirty students IQ scores have a mean score of 112. Is there
sufficient evidence to support the principal’s claim? The mean population IQ is 100 with a
standard deviation of 15.

Null hypothesis. The accepted fact is that the population mean is 100, so: H0: μ=100
Alternate Hypothesis. The claim is that the students have above average IQ scores,
so: H1: μ > 100

The fact that we are looking for scores “greater than” a certain point means that this is a one-
tailed test (directional hypothesis)

Selecting an alpha level of 0.05 (standard value of 5%), we find the rejection region area from the
z-table. An area of .05 is equal to a z-score of 1.645. If the test statistic is greater than this
threshold (critical value), we would reject the null hypothesis.

Next, we find the test statistic using this

formula –

For this set of data: z= (112.5 – 100) / (15/√30) =

4.56.
In this case, since the test statistic is greater than
the critical value (4.56 > 1.645), so you can reject
the null.

11. R2 value
Coefficient of determination is the proportion in the variance of the dependent variable that can
be predicted from the independent variable. In regression, the R2 coefficient of determination is
a statistical measure of how well the regression predictions approximate the real data points. An
R2 of 1 indicates that the regression predictions perfectly fit the data.

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Where,
N = No. of observations
Σ xy = Sum of paired product
Σ x = x score sum
Σ y = y score sum
Σ x2 = square of x score sum
Σ y2 = square of y score sum

12. P value
A probability that provides a measure
of the evidence against the null
hypothesis given by the sample.
Smaller value indicate more evidence
against H0, i.e. stronger evidence to
reject the null hypothesis(H0).

When you run a hypothesis test, you

compare the p value from your test to
the alpha level you selected when you
ran the test. Graphically, the p value is
the area in the tail of a probability
distribution. It is the area to the right
of the test statistic (if you’re running a
two-tailed test, it’s the area to the left
and to the right).

13. What is Bias and Variance in a model? What is Bias Variance Trade off?
Bias refers to model error and variance refers to the consistency in predictive accuracy of models
applied to other data sets. The best models have low bias (low error, high accuracy) and low
variance (consistency of accuracy from data set to data set).
Unfortunately, there is always a tradeoff between these two building predictive models. You can
achieve low bias on training data, but may suffer from high variance on held-out data because the
models were overfit.
Error due to bias: The error due to bias is the difference between the expected (or average)
prediction of our model and the correct value which we are trying to predict. Imagine you could
repeat whole model building process more than once: each time you gather new data and run
anew analysis creating a new model. Due to randomness in the underlying datasets, the resulting
model will have range of predictions. Bias measures how far off in general these model's
prediction are from the correct value. A high bias error means we have an under-fitting model
which keeps on missing important labels.

10

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Error due to variance: The error due to variance is taken as the variability of a model prediction
for a given data point. Again, imagine you can repeat the entire model building process multiple
times. The variance is how much the predictions for a given point varies between different
iterations of the model. A high variance model will over-fit on your training population and
perform very badly on any observation outside the training data.

The below diagram might oversimplify things e.g. models with just one or two features, but this
helps to conceptually understand what it means for a model to have high bias (underfitting) or
high variance (overfitting). The concept is much the same just much harder to visualize when
there are many features.

14. Understanding Linear Regression

Linear Regression uses a linear function to map input variables to continuous

response/dependent variables. Once fitted, a Linear Regression model can be used to predict the
values of response/dependent variables for new values of the input variables. An example
application, might be to predict the total value of trades that will occur by end of day based on
the number and size of orders that have been submitted so far. The output of Linear Regression
is a continuous value, and due to the use of a straight line to map the input variables to the
dependent variables, the output can be any one of an infinite number of possibilities. This means
that outputs can be positive or negative, with no maximum or minimum bounds.

11

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

A linear regression line has an equation of form:

Y = β0 + β1X1 + β2x2 + β3x3+… + E,

where,

Xs are the independent variables

Y is the dependent variable
E is the error (or residual) term
β0 is intercept and β1 to βn are the
regression coefficients

12

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Understanding Regression Output:

i. Check R square value to see how well the model explains the variance around the mean;
generally, more than 80% is acceptable
ii. Identify variables will P-value < 0.05, indicating statistically significant variables for 95%
significance
iii. Intercept (β0) coefficients (β1 to βn) are used to determine the Regression equation
Yp = -6260.09 + 0.88161* X1 + 613.97* X2
iv. Residuals (E) can be also calculated using:
Actual Y (Y) – Predicted Y (Yp)
v. Correlation Analysis is also done to remove highly correlated variables. Generally, values
> 70% are considered to be highly correlated and thus, one among the two variables is
removed

Model & its Uses

• Generates a best fit straight line between the variables by minimizing sum of squared
residuals
• Helps yield valuable and actionable business insights like Predictable product cost, degree
of influence of one variable on another, interpolation or extrapolation to identify
appropriate pricing, etc.

15. Understanding Logistic Regression

Logistic Regression uses a logistic function to map the input variables to categorical
response/dependent variables.

In contrast to Linear Regression,

Logistic Regression outputs a
probability between 0 and 1. In
essence, Logistic Regression estimates
the probability of a binary outcome,
rather than predicting the outcome
itself.

Logistic Regression is typically used for

binary classification problems, where
the output is the probability of the
given input belonging to a categorical
target class.

For example, threat detection for cybersecurity aims to distinguish between benign and
suspicious patterns of host and network activity; this is a binary classification problem that can
potentially be tackled using logistic regression (among other techniques).

13

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

16. What is a confusion matrix?

A confusion matrix is a summary of prediction results on a classification problem. The number of

correct and incorrect predictions are summarized with count values and broken down by each
class.
The confusion matrix shows the ways in which your classification model is confused when it makes
predictions. It gives us insight not only into the errors being made by a classifier but more
importantly the types of errors that are being made. Let’s understand the matrix in terms of
pregnancy analogy.

Definition of the Terms:

Positive (P): Observation is positive (for example: is pregnant)
Negative (N): Observation is not positive (for example: is not pregnant)
True Positive (TP): Observation is positive, and is predicted to be positive i.e.
predicted that a woman is pregnant, and she actually is
False Negative (FN): Observation is positive, but is predicted negative i.e. predicted
that a woman is not pregnant, but she actually is
True Negative (TN): Observation is negative, and is predicted to be negative i.e.
predicted that a man is not pregnant, and he actually is not
False Positive (FP): Observation is negative, but is predicted positive i.e. predicted
that a man is pregnant, but he actually is not

14

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Some Accuracy Metrics associated with the confusion matrix are explained below.

Consider the following confusion matrix –

• Classification Rate/Accuracy: This measures an overall

correctness of the classifier i.e. out of all the classes, how
much we predicted correctly.
In the above example, it would be -
(TP+TN)/total = (100+50)/165 = 0.91

• Misclassification Rate: Also called error rate, measures overall incorrectness of classifier

(1 - Accuracy) = (FP+FN)/total = (10+5)/165 = 0.09

• Recall or True Positive Rate: This gives how good the model is in
recognizing a positive class correctly i.e. sensitive to positive
observations. Hence it is also called sensitivity –

TP/actual yes = 100/105 = 0.95

• Precision: This measures the number of times the correctly

predicted cases actually turned out to be positive.

TP/predicted yes = 100/110 = 0.91

• False Positive Rate: This measures the number of times the classifier predicts a positive
observation when it was actually negative

FP/actual no = 10/60 = 0.17

• Specificity or True Negative Rate: This measures the number of times the classifier
predicts a negative observation when it was actually negative

15

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

(1 – False Positive Rate) = TN/actual no = 50/60 = 0.83

• F1-Score: In practice, when we try to increase the precision,
the recall goes down, and vice-versa. F1-score captures both
the trends in a single value.

F1-score is a harmonic mean of Precision and Recall, and so it gives a combined idea about
these two metrics. It is maximum when Precision is equal to Recall. However, the
interpretability of the F1-score is poor, since there is not main objective of the classifier.
So, we use it in combination with other evaluation metrics which gives us a complete
picture of the result.

17. What are Decision Trees?

Decision tree is a type of supervised learning

algorithm (having a pre-defined target
variable) that is mostly used in classification
problems.

It works for both categorical and continuous

input & output variables. In this technique,
we split the population (or sample) into two
or more homogeneous sets (or sub-
populations) based on most significant
splitter / differentiator in input variables.

In Decision Tree the major challenge is to

identification of the attribute for the root node in each level. This process is known as attribute
selection. ‘Information Gain’ is a popular attribute selection measure.

Information Gain
When we use a node in a decision tree to partition the training instances into smaller subsets the
entropy changes. Information gain is a measure of this change in entropy.

Entropy is the measure of uncertainty of a random variable, it characterizes the impurity of an

arbitrary collection of examples. The higher the entropy more the information content.

16

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Formula definition: Suppose S is a set of instances, A is an attribute, Sv is the subset of S with A =

v, and Values (A) is the set of all possible values of A, then:

Illustration

For the set X = {a,a,a,b,b,b,b,b}

Total instances: 8
Instances of b: 5
Instances of a: 3

𝐸𝑛𝑡𝑟𝑜𝑝𝑦 𝐻(𝑋)= −[(3/8)𝑙𝑜𝑔2(3/8)+(5/8)𝑙𝑜𝑔2(5/8)]

= -[0.375 * (-1.415) + 0.625 * (-0.678)]
=-(-0.53-0.424)
= 0.954

Building Decision Tree using Information Gain

Steps:
a) Start with all training instances associated with the root node
b) Use info gain to choose which attribute to label each node with
c) Note: No root-to-leaf path should contain the same discrete attribute twice
d) Recursively construct each subtree on the subset of training instances that would be
classified down that path in the tree

The border cases:

a) If all positive or all negative training instances remain, label that node “yes” or “no”
accordingly
b) If no attributes remain, label with a majority vote of training instances left at that node
c) If no instances remain, label with a majority vote of the parent’s training instances

Limitations of Decision Trees:

a) Decision Trees are known to overfit data
b) DTs don’t scale well to very large parameter sets

Further Reading:
The following resource can be used to understand the concepts and applications of Decision Trees
better - https://towardsai.net/p/programming/decision-trees-explained-with-a-practical-
example-fe47872d3b53

17

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

18. What are Model Ensembles?

Model Ensembles, or simply “ensembles,” are combinations of two or more predictions from
predictive models into a single composite score. Advantages of ensembling -

• Improved model accuracy (low error) - Ensembles nearly always improve model predictive
accuracy and rarely predict worse than single models
• Improved model robustness (less overfitting) - averaging multiple models into a single
prediction, no single model dominates the final predicted value of the models, reducing
the likelihood that a flaky prediction will be made (so long as all the models don’t agree
on the flaky prediction)

19. What are the common Statistical Data Distributions?

Distributions provide a parameterized mathematical function that can be used to calculate the
probability for any individual observation from the sample space. This distribution describes the
grouping or the density of the observations, called the probability density function. We can also
calculate the likelihood of an observation having a value equal to or lesser than a given value. A
summary of these relationships between observations is called a cumulative density function.

Binomial distribution – This measures the probabilities of the number of successes over a given
number of trials with a specified probability of success in each try. In the simplest scenario of a
coin toss (with a fair coin), where the probability of getting a head with each toss is 0.50 and there
are a hundred trials, the binomial distribution will measure the likelihood of getting anywhere
from no heads in a hundred tosses (very unlikely) to 50 heads (the most likely) to 100 heads (also
very unlikely). The binomial distribution in this case will be symmetric, reflecting the even odds;
as the probabilities shift from even odds, the distribution will get more skewed.

Poisson distribution - This distribution measures the likelihood of a number of events occurring
within a given time interval, where the key parameter that is required is the average number of
events in the given interval (l). The resulting distribution looks similar to the binomial, with the
skewness being positive but decreasing with l.

Negative Binomial distribution - Returning again to the coin toss example, assume that you hold
the number of successes fixed at a given number and estimate the number of tries you will have
before you reach the specified number of successes. The resulting distribution is called the
negative binomial and it very closely resembles the Poisson. In fact, the negative binomial
distribution converges on the Poisson distribution, but will be more skewed to the right (positive
values) than the Poisson distribution with similar parameters.

Geometric distribution - Consider again the coin toss example used to illustrate the binomial.
Rather than focus on the number of successes in n trials, assume that you were measuring the
likelihood of when the first success will occur. For instance, with a fair coin toss, there is a 50%
chance that the first success will occur at the first try, a 25% chance that it will occur on the second
try and a 12.5% chance that it will occur on the third try. The resulting distribution is positively
skewed and looks as follows for three different probability scenarios

18

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Normal distribution - The normal distribution is a probability function that describes how the
values of a variable are distributed. It is a symmetric distribution where most of the observations
cluster around the central peak and the probabilities for values further away from the mean taper
off equally in both directions. Extreme values in both tails of the distribution are similarly unlikely.
It has the following properties –

• symmetric bell shape

• mean and median are equal; both located at the centre of the distribution
• ~68% of the data falls within 111 standard deviation of the mean
• ~95% percent of the data falls within 222 standard deviations of the mean
• ~99.7% of the data falls within 333 standard deviations of the mean

19

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Industry Applications of Data Science

1. Finance

With the introduction of quantitative analytical models, large financial institutions and hedge
funds have shifted from manual trading to trading backed by technology. These models can be
used to analyse big data to

• Make accurate enter/exit trade decisions

• Risk mitigation
• Gauge market sentiment using opinion mining

For example, The Securities Exchange Commission (SEC) is using big data to monitor financial
market activity. They are currently using network analytics and natural language processors to
catch illegal trading activity in the financial markets.

2. Banking

The amount of data in the banking sector is skyrocketing every second. Proper study and analysis
of this data can help detect any and all illegal activities that are being carried out such as

• Misuse of credit/debit cards

• Venture credit hazard treatment
• Business clarity
• Money laundering
• Reduce NPA and increase profitability

For example, banks could use predictive analytics to calculate makeshift “credit scores” for people
that don’t have a credit history based on behavioural traits such as social media posts and
spending habits. They could use these scores to determine whether or not to lend to someone

3. Communications, Media and Entertainment

• Create content for different target audiences
• Recommend content on demand
• Measure content performance
• Effective targeting of the advertisements
• Optimized or on-demand scheduling of media streams in digital media distribution
platforms

For example, Spotify, an on-demand music service, uses Hadoop big data analytics, to collect data
from its millions of users worldwide and then uses the analysed data to give informed music
recommendations to individual users.

20

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

4. Retail

Retail data, derived from customer loyalty cards, POS scanners, RFID etc. is now being used
enough to improve customer experiences on the whole. Analytics in retail can be used for -

• Optimized staffing through data from shopping patterns, local events, and so on
• Reduced fraud
• Timely analysis of inventory

Demand forecasting is another application of big data. For example, retailers like Walmart and
Walgreens regularly analyse changes in weather to see any patterns in product demand.

5. Transportation

Since the rise of big data, it has been used in various ways to make transportation more efficient
and easy. Following are some of the areas where big data contributes to transportation.

• Route planning - Big data can be used to understand and estimate users’ needs on
different routes and on multiple modes of transportation and then utilize route planning
to reduce their wait time
• Congestion management and traffic control - Using big data, real-time estimation of
congestion and traffic patterns is now possible. For examples, people are using Google
Maps to locate the least traffic-prone routes
• Safety level of traffic - Using the real-time processing of big data and predictive analysis
to identify accident-prone areas can help reduce accidents and increase the safety level
of traffic

6. Telecom

• Customer retention/improving customer loyalty

There is an increased focus on improving user experience. To do so, analysts are creating
sophisticated 360-degree profiles assembled from a number of sources like, voice, SMS and
data usage patterns, video choices, customer care history, social, consumer demographics,
service usage, etc. This allows telecom companies to offer personalized services or products
at every step of the purchasing process. Businesses can tailor messages to appear on the right
channels (e.g., mobile, web, call centre, in-store), in the right areas and in the right words and
images.

• Network optimization
Telecommunication companies tend to regard the customer's engagement process and
internal channels as a guarantee of smooth functioning of the operations. Network
management and optimization gives an opportunity to define the score points in operations
to identify the root causes of these complications. Looking into historical data and predicting
possible future problems or, on the contrary, beneficial scenarios is a great benefit for the
telecom providers.

21

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

7. Health care

A few examples of applying big data and predictive analytics in healthcare are
• Big data can be used to predict negative health events that seniors could experience from
home-care. At AlayaCare, the analysis reduced hospitalizations and ER visits by 73
percent, and 64 percent amongst chronically ill patients
• Historical big data from healthcare providers can be used to identify and analyse certain
risk factors in patients. This is useful for earlier detection of diseases, allowing doctors and
their patients to take action sooner
• Big data can identify disease trends as a whole based on demographics, geographies,
socio-economics, and other factors

8. Manufacturing

• Supply chain management and big data go hand-in-hand, which is why manufacturing is
one of the top industries to benefit from the use of big data.
• Monitoring the performance of production sites is more efficient with big data analytics.
The use of analytics is also extremely useful for quality control, especially in large-scale
manufacturing projects.
• Big data analytics plays a key role in tracking and managing overhead and logistics across
multiple sites. For example, being able to accurately measure the cost of shop floor tasks
can help reduce labour costs.
• Then there’s predictive analytics software, which uses big data from sensors attached to
manufacturing equipment. Early detection of equipment malfunctions can save sites from
costly repairs capable of paralyzing production.

22

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Sample Puzzles

1. Bag of Coins
You have 10 bags full of coins. In each bag are infinite coins. But one bag is full of forgeries, and
you can’t remember which one. But you do know that a genuine coins weigh 1 gram, but
forgeries weigh 1.1 grams. You have to identify that bag in minimum readings. You are provided
with a digital weighing machine.

Answer
1 reading
Explanation - Take 1 coin from the first bag, 2 coins from the second bag, 3 coins from the third
bag and so on. Eventually, we’ll get 55 (1+2+3…+9+10) coins. Now, weigh all the 55 coins
together. Depending on the resulting weighing machine reading, you can find which bag has the
forged coins such that if the reading ends with 0.4 then it is the 4th bag, if it ends with 0.7 then it
is the 7th bag and so on.

2. Prisoners and hats

There are 100 prisoners all sentenced to death. One night before the execution, the warden
gives them a chance to live if they all work on a strategy together. The execution scenario is as
follows –

On the day of execution, all the prisoners will be made to stand in a straight line such that one
prisoner stands just behind another and so on. All prisoners will be wearing a hat either blue or
red in color. The prisoners don’t know what color of hat they are wearing. The prisoner who is
standing at the last can see all the prisoners in front of him (and what color of hat they are
wearing). A prisoner can see all the hats in front of him. The prisoner who is standing in the
front of the line cannot see anything.

The executioner will ask each prisoner what color of hat they are wearing one by one, starting
from the last in the line. The prisoner can only speak “Red” or “Blue”. He cannot say anything
else. If he gets it right, he lives otherwise he is shot instantly. All the prisoners standing in front
of him can hear the answers and gunshots.

Assuming that the prisoners are intelligent and would stick to the plan, what strategy would the
prisoners make over the night to minimize the number of deaths?

Answer
The strategy is that the last person will say ‘red’ if the number of red hats in front of him are odd
and ‘blue’ if the number of red hats in front of him are even. Now, the 99th guy will see the if the
red hats in front of him are odd or even. If it is odd then obviously the hat above him is blue, else
it is red. From now on, it’s pretty intuitive.

23

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

3. Blind games

You are in a dark room where a table is kept. There are 50 coins placed on the table, out of
which 10 coins are showing tails and 40 coins are showing heads. The task is to divide this set of
50 coins into 2 groups (not necessarily same size) such that both groups have same number of
coins showing the tails.

Answer
Divide the group into two groups of 40 coins and 10 coins. Flip all coins of the group with 10
coins.

4. Time and tide waits for none

You have two sand timers, which can show 4 minutes and 7 minutes respectively. Use both the
sand timers (at a time or one after other or any other combination) and measure a time of 9
minutes.

Answer
a) Start the 7 minute sand timer and the 4 minute sand timer
b) Once the 4 minute sand timer ends turn it upside down instantly
c) Once the 7 minute sand timer ends turn it upside down instantly
d) After the 4 minute sand timer ends turn the 7 minute sand timer upside down(it has now
minute of sand in it)

So effectively 8 + 1 = 9

5. Chaotic Bus

There is a bus with 100 labeled seats (labeled from 1 to 100). There are 100 persons standing in
a queue. Persons are also labelled from 1 to 100.
People board on the bus in sequence from 1 to n. The rule is, if person ‘i’ boards the bus, he
checks if seat ‘i’ is empty. If it is empty, he sits there, else he randomly picks an empty seat and
sit there. Given that 1st person picks seat randomly, find the probability that 100th person sits
on his place i.e. 100th seat.

Answer
The final answer is the probability that the last person ends in up in his proper seat is exactly 1/2
Explanation - First, observe that the fate of the last person is determined the moment either the first or the last
seat is selected! This is because the last person will either get the first seat or the last seat. Any other seat will
necessarily be taken by the time the last guy gets to ‘choose’. Since at each choice step, the first or last is
equally probable to be taken, the last person will get either the first or last with equal probability: ½

24

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

6. Shooters in a circle

N persons are standing in a circle. They are labelled from 1 to N in clockwise order. Every one of
them is holding a gun and can shoot a person on his left. Starting from person 1, they starts
shooting in order e.g. for N=100, person 1 shoots person 2, then person 3 shoots person 4, then
person 5 shoots person 6……..then person 99 shoots person 100, then person 1 shoots person 3,
then person 5 shoots person 7……and it continues till all are dead except one. What’s the index
of that last person?

Answer
Write 100 in binary, which is 1100100 and take the complement which is 11011 and it is 27.
Subtract the complement from the original number. So 100 – 27 = 73.
Try it out for 50 people. 50 = 110010 in binary.
Complement is 1101 = 13. Therefore, 50 – 13 = 37.
For the number in form 2^n, it will be the first person. Let’s take an example:
64 = 1000000
Complement = 111111 = 63.
64-63 = 1.
You can apply this for any ’n’

25

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Product Management
What is a Product Manager?
A PM is responsible for making sure that a team ships a great product.

Some people will say that the product manager (sometimes called the program manager or
project manager) is like a mini-CEO of their product. That’s accurate in some ways, since a PM
takes holistic responsibility for the product, from the little details to the big picture. The PM needs
to set vision and strategy. The PM defines success and makes decisions.
But in one of the most important ways, the description of a product manager as CEO misses the
boat: product managers don’t have direct authority over the people on their team.

As a PM, you’ll need to learn to lead your team without authority, influencing them with your
vision and research. Product managers are highly respected at most companies, but not more so
than engineers. If you show up and start bossing people around, you’ll probably find it hard to get
things done. After all, engineers are the ones actually building the product. You need them on
your side.
One reason product management is such an appealing career is you get to sit at the intersection
of technology, business, and design. You get to wear many hats and learn multiple points of view.
As a product manager, you’ll be the advocate for the customer. You’ll learn their needs and
translate those needs into product goals and features. Then you’ll make sure those features are
built in a cohesive, well-designed way that actually solves the customer’s needs. You’ll focus on
everything from the big picture to the small details. One day you might brainstorm the three-year
vision for your team, while the next day you work through the details of the buttons in a dialog.

Product management is a highly collaborative role. The product manager usually serves as the
main liaison between the engineering and other roles such as design, quality assurance, user
research, data analysts, marketing, sales, customer support, business development, legal, content
writers, other engineering teams, and the executive team. It’s usually the job of the product
manager to identify times when one of those teams should be brought in, and to fill in for them
if they don’t exist.

Functions of a PM
The day-to-day work of a product manager varies over the course of the product life cycle. At the
beginning, you’ll be figuring out what to build; in the middle you’ll help the team make progress;
at the end you’ll be preparing for the launch.

26

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

While the product life cycle varies by company (and sometimes even by team), it usually follows
a general pattern of Research & Plan, Design, Implement & Test, and Release. Of course, these
frequently overlap and feed back into each other.

Some companies or teams split the product manager role across two people: the more business
focused person and the more engineering-focused person. When companies make this split, they
call the engineering-focused person the technical program manager or technical product manager
(TPM), and they call the business-focused person the product manager (PM).

When a team has a TPM and a PM, the product manager focuses on Research & Planning and

Release, while the technical product manager is more focused on Design and Implement & Test.
For example, the product manager will research the market and define the requirements. The
TPM will work with the PM to translate those requirements into the specific feature work
required, and then facilitate the engineering team as they build it.

Research & Planning

All products and features start with research and planning. This is the time when the PM is starting
to think about what to build next. The next idea may come from a customer request, competitive
analysis, new technology, user research, the sales or marketing teams, brainstorming, or the big
vision for the product.

Depending on the scope of the role, a big part of the product manager’s job in this phase is
creating or proposing a roadmap. This means figuring out a cohesive long-term plan for the team.
The PM talks to all possible sources to create a large list of potential features or development
work. Then, based on factors like customer needs, the competitive landscape, business needs,
and the team’s expertise, he prioritizes the features and scenarios.
Once the PM has a proposed roadmap, he needs to bring other people on board. Some
companies, such as Microsoft, Apple, and Amazon, have a top-down approval process, where
executives and directors get involved very early. Other companies, such as Google, Facebook, and
many startups, have a more bottom-up approach, where the PM focuses on winning over the
engineers.

Once he’s chosen a feature set, the product manager becomes the expert on them. He’ll think
deeply about the problems he’s trying to solve and the goals of the features. In the coming phases,
everyone on the team will have questions, including “why are we working on this?”, and the PM
will need to have answers.

This is also the time when the PM starts defining success. He’ll envision what the world looks like
if the team is successful. Many companies use the model of Objectives and Key Results (OKRs) to

27

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

communicate the most important goals of the team. In this model, the PM works with the team
to come up with measurable results that it can commit to.

Design

Once the PM has formed an agreement on what the team is going to build, it’s time to design the
product and features.

Product design does not just mean user interface (UI) design or drawing out what the product will
look like. Product design is defining the features and functionality of the product. The PM’s role
in product design varies substantially between companies and teams.

On some teams, especially shipped software (as opposed to online software) teams at Microsoft,
the PM will write out a detailed functional specification (spec) that includes things like:

• Goals
• Use Cases
• Requirements
• Wireframes
• Bullet points describing every possible state of the feature
• Internationalization
• Security

The spec will then spend weeks being inspected, reviewed, and iterated on by developers, testers,
and other PMs. On these teams, the PM is expected to make every user-facing decision.

Other teams have much looser specs and a more rapid design process. The PM might sit down
with an interaction designer, chat about the goals, brainstorm on a whiteboard, and then iterate
by giving feedback on the designer’s mock-ups. When the mocks are ready, the PM might send
them to the engineers with just a few sentences in an email. On these teams, the engineers will
generally make easy product decisions as they come up and call the PM over to ask about the
more difficult ones.

And for some teams, especially at Apple, the design is done mostly by a dedicated design team
with minimal input from the PM. On those teams, the PM might focus more on project
management and fighting fires as they come up.
Since the product manager’s role in product design can vary so much across teams, it’s a great
thing to ask about during your interview. Ask about who you’ll be working with on your core and
extended team. Find out how much of your time will be spent writing specs and how much you’ll
be working with designers. Learn where the balance is between PMs, designers, and engineers in
making product decisions.

28

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Implement & Test

A PM’s work isn’t done once engineers start coding. During the implementation stage, the product
manager keeps track of how the project is going and makes adjustments.

During implementation, one of the most important parts of the job is helping the engineers work

efficiently. The product manager will check in regularly with his team and learn how things are
going.

Often an engineer will be blocked because she’s waiting for work from another team. In this case,
the PM will need to find other tasks for the engineer and, in the meantime, work with the other
team to get the work finished more promptly.

Sometimes, implementation of a feature will turn out to be harder than anticipated, and the PM
will look for ways to change the feature to make it easier to implement. If an engineer is running
behind schedule, the product manager can review the scheduled work carefully and cut lower
priority work.

During implementation, the product manager will also start gathering feedback and reporting
bugs on the early versions of the product. Sometimes, a feature that looked good during the
design phase will not work as well as expected once it’s used in the real world. To find problems
like this, teams will do usability studies, run experiments, and do internal “dogfooding.”

Dogfooding comes from the term “Eat your own dogfood,” and simply means using your own
product yourself. For example, people at Microsoft run early versions of the next Windows release
on their computers every day. Facebook employees use Facebook Groups to communicate.

Sometimes teams need to be more creative to find ways to use their own products. For example,
Google gives employees an AdWords budget and encourages people to create advertising
campaigns to make sure they get enough dogfooding.

Another way to find out if a product will work before it’s launched is through usability studies. In
a usability study, participants try out early prototypes of a new product or feature. Usually the

participants are given a scenario or goal, and then they’ll try to use the prototype to accomplish
the goal.

Larger companies usually have a user researcher, who develops and runs the study with some
input from the product manager. At smaller companies, the PM might run the studies. In both
cases, by watching a handful of studies, PMs can see where people struggle and identify key
usability issues.

While dogfooding and usability studies are great for getting qualitative feedback, running
experiments is a way to get quantitative feedback when you have online software. In an
experiment, the new feature is turned on for a percentage of users (the experiment group), while
the rest of users (the control group) keep using the product without the new feature. For online
software, it’s common for all new features to be launched gradually as an experiment.

In an experiment, you can measure specific metrics for the new feature, such as how many people
click a new button you added, as well as overall success metrics like user engagement, retention,

29

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

and revenue. By comparing the success metrics between the experiment and control groups, you
can tell how successful the new feature is.

As feedback comes in through dogfooding, usability studies, and experiments, the PM identifies
the most important issues and iterates on the feature design to find better solutions. Prioritization
is one of the product manager’s most important functions at this point; if the team were to fix
every bug and build every new feature idea, the product would never launch. The PM needs to
consider all of the new requests and decide if they should be prioritized for the current release or
punted to a later time.

Release

When the development process is finished, the product manager needs to make sure the launch
goes smoothly. The launch process varies from team to team but usually involves things like:

• Running through a launch checklist. There might be final approvals from key stakeholders
like Legal or coordination steps with teams like Marketing and Operations.
• Making sure that the teams who will support the product going forward are prepared.
For a web product this might be the customer service team; for a hardware product it
could be the manufacturing team; for a service-based product it could be the operations
team.
• Preparing for all the things that could go wrong. As the release nears, urgent issues
inevitably pop up, and the PM thinks on her feet to fight the fires.

After a successful launch, the PM usually announces the launch to the rest of the company,
celebrates with the team, and then prepares to do it all over again. Depending on the team, the
PM might continue to support the product after the launch, gathering metrics and iterating on
user feedback, or the product might be handed over to another team for operations and
maintenance.

Project Manager and Program Manager

Note: Microsoft has a role called Program Manager that is different from the Program Manager
role at most other software companies, and is similar to the Product Manager role described
earlier.

There are many roles that are related to product management, and the lines between the roles
can be blurry. If you’re looking to apply for a project manager or program manager role, see if you
can talk to people who have done the role at the companies where you’re applying to make sure
you understand the nuances of the requirements.

Project managers make sure that their projects get delivered on time and on budget, to the
satisfaction of the customer, whoever that may be. Program managers (except for Microsoft
program managers) are similar but are usually in charge of a long-running program instead of a
series of projects with set end dates.

30

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Software companies often have project managers leading teams that are internal-facing, like
infrastructure projects and operations programs, or leading consulting teams that are focused on
a single customer. This is in contrast to product managers, who usually lead teams building
customer facing products. However, this isn’t a hard and fast rule.

The Job of a Project Manager

Two things are unique about these roles. First, the project manager has a very clear and specific
customer or goal.

Project managers working on infrastructure and operations teams have another employee of the

company as their customer. The other employee often has a clear and accurate idea of what they
want out of the project.

Project managers working with a consulting team serve an individual customer who has already

signed a statement of work that includes the details of each project.

In all of these cases, the research, planning, and design stages are fairly straightforward, or
already done by someone else. For project managers, product design is not usually a big part of
their job, so you may or may not get product design questions.

Also, because the customer is so deeply invested in the outcome of the project, communication
and expectation setting are very important. Project managers need to be comfortable reporting
on the team’s progress. Being detail oriented and closely involved with the team are important
so that you can answer questions from the customer and explain when things go wrong.

The second unique part of project and program management is that budgeting and resource
management is a big part of the job. Project managers often work in cost centers, so they focus a
lot on operational efficiency and reducing costs while keeping quality up to par, unlike product
managers who are often shielded from those aspects.

Here are some types of budgeting and resource management work that project managers and
program managers do:

• Clarify goals and gather satisfaction metrics.

• Determine the people and skills needed to complete a project.
• Set up project management tools, plans and processes.
• Run status meetings and gather status reports.
• Analyze data to identify opportunities.
• Identify & implement changes to improve efficiency.
• Manage changes that come in from the customer.
• Find ways to keep the project on track even when things go wrong.
Requirements for project managers vary from company to company and role to role. Some teams
look for project management certifications, while others do not. Generally, companies look for
project manager candidates who have prior experience managing projects and excellent
communication skills.

31

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Frameworks
1. RICE

Known as Intercom's internal scoring system for prioritizing ideas, RICE allows product teams to
work on the initiatives that are most likely to impact any given goal.

This scoring system measures each feature or initiative against four factors: reach, impact,
confidence and effort (hence the acronym RICE). Here’s a breakdown of what each factor stands
for and how it should be quantified:

Then, those individual numbers get turned into one overall score using a formula. This formula
gives product teams a standardized number that can be applied across any type of initiative that
needs to be added to the roadmap.

After running each feature by this calculation, you’ll get a final RICE score. You can then use that
final score to rank the order in which you’ll tackle the idea, initiative or feature. Here’s an
example:

32

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Pros of using the RICE method

• It requires that product teams make their product metrics SMART before they quantify
them. SMART stands for specific, measurable, attainable, relevant and—as seen in the
parameter for the Reach score—time-based.

It lessens the influence that inherent biases have on prioritization. By adding a confidence
dimension in the calculations, prioritization moves away from attempting to predict success to
measuring the level of assertion each team member has for the features. This shifts the
prioritization discussions from “Here’s how much this feature is worth“ to “Here is how we are
quantifying our level of confidence for each of these qualitative, speculative scores.”

Cons of using the RICE method

• RICE scores don’t take dependencies into account. There are times when an initiative with
a high RICE score needs to be deprioritized over something else, so product teams need
to treat the score as an exercise, rather than the end-all, be-all answer to “What should
we build next?”
• Estimations will never be 100% accurate. RICE prioritization is simply an exercise in
quantifying features in a way that takes into account the level of confidence teams have
on their estimations.

2. Value vs. Effort

This simple approach to prioritization involves taking your list of features and initiatives and
quantifying them using value and effort scores.

With this method, it’s important to keep in mind that the final scores are just an estimation. A lot
of guesswork and opinions (backed with as much applicable data as possible) are involved in the
process of quantifying the big question prioritization aims to answer: “Will this feature/update
push our goals and metrics forward if we build it? Can we feasibly build it with the resources we
have?”

33

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Scoring methods like value vs. effort give product teams a quick and easy way to visualize a set of
quantified priorities.

This method of prioritization makes room for healthy discussions among stakeholders on what
they believe value and effort means, which in turn helps product managers find the strategic
alignment holes and fix them. Here's what a value vs. effort scorecard looks like:

Pros of using value vs. effort

• What constitutes value or effort is flexible. For some organizations, effort could just mean
development effort, in others it could be the implementation cost. A flexible prioritization
framework can be used by any type of company, industry or type of product.
• It’s a good tool for alignment. By encouraging teams to quantify and numerically score
features, product teams can agree on which initiatives have more weight than others. It
leaves vague guesswork and assumptions out of the prioritization discussions.
• In companies where resources are extremely limited, something as simple as a value vs.
effort analysis allows teams to focus only on the things that will have the biggest impact
on their business and product goals.
• It’s easy to use because it doesn’t involve any complex formulas or models. All it requires
is an agreed-upon numerical value that gets added into one overall total number.

34

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Cons of using value vs. effort

• Like all prioritization exercises, it’s a game of estimation and guessing. This leaves a lot of
room for cognitive bias at the hands of the people doing the estimating. The final score
for each feature might be too inflated, or not accurate enough.
• When it’s time for product and development to vote on how high or low the value/effort
scores should be, the disagreements can take a while to resolve.
• It can be hard to use with large teams with multiple product lines, components, and
product teams that oversee each of those.

3. Kano Model

The Kano model plots two sets of parameters along a horizontal and a vertical axis. On the
horizontal axis, you have the implementation values (to what degree a customer need is met).
These values can be classified into three buckets:

• Must-haves or basic features: If you don’t have these features, your customers won’t even
consider your product as a solution to their problem.
• Performance features: The more you invest in these, the higher the level of customer
satisfaction will be.
• Delighters or excitement features: These features are pleasant surprises that the
customers don’t expect, but that once provided, create a delighted response.

On the vertical axis, you have the level of customer satisfaction (the satisfaction values). They
range from the needs not being met on the left, all the way to the needs being fully met on
the right (the implementation values).

The way you get this customer insight is by developing a Kano questionnaire where you ask
your customers how they’d feel with or without any given feature.
The core idea of the Kano model is that the more time you spend investing resources (time,
money, effort) to create, innovate and improve the features in each of those buckets, the
higher the level of customer satisfaction will be.

35

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Pros of using this prioritization framework:

• The Kano model questionnaire can teach teams not to overestimate excitement
features and to stop underestimating must-haves.
• It can help teams make better product decisions and market predictions for feature
success and your audience’s expectations for those features.

Cons of using this prioritization framework:

• The Kano questionnaire can be time-consuming. In order to get a fair representation

of all your customers, you need to carry a number of surveys that are proportionate
to the number of customers you have.
• Your customers might not fully understand the features you’re surveying them about.

4. Story Mapping

The beauty of this product prioritization framework is in its simplicity. It also puts the focus
on the user’s experience, rather than on the internal opinions of your team and stakeholders.
Along a horizontal line, you create a series of sequential buckets or categories that represent
each stage of the user’s journey through your product. This allows you to think about the way
your customers navigate your product from signing up, to setting up their profile, to using
specific features.

Along a vertical line, you then place these tasks in order of importance, from top to bottom.
This allows you to prioritize the order of the features you’ll work on. In some cases, the
bottom part of the axis is labeled “Backlog items” for the tasks that you decide to put on hold.

Finally, you draw a line across all these stories to divide them into releases and sprints.

36

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Pros of using this prioritization framework:

• It helps you quickly—and efficiently—identify your MVP.

• It’s centered around the user’s experiences. You get to write user stories.
• It’s collaborative. Story mapping is a group activity that involves the whole team.
Cons of using this prioritization framework:

• It doesn’t take into account external product prioritization factors like business
value and complexity.

5. The MoSCoW Method

The MoSCoW method allows you to figure out what matters the most to your stakeholders and
customers by classifying features into four priority buckets. MoSCoW (no relation to the city—the
Os were added to make the acronym more memorable) stands for Must-Have, Should-Have,
Could-Have, and Won’t-Have features.

• Must-Have: These are the features that have to be present for the product to be
functional at all. They’re non-negotiable and essential. If one of these requirements
or features isn’t present, the product cannot be launched, thus making it the most
time-sensitive of all the buckets.
Example: “Users MUST log in to access their account”
• Should-Have: These requirements are important to deliver, but they’re not time
sensitive.
Example: “Users SHOULD have an option to reset their password”
• Could-Have: This is a feature that’s neither essential nor important to deliver within
a timeframe. They’re bonuses that would greatly improve customer satisfaction, but
don’t have a great impact if they’re left out.
Example: “Users COULD save their work directly to the cloud from our app”
• Won’t-Have: These are the least critical features, tasks or requirements (and the first
to go when there are resource constraints). These are features that will be
considered for future releases.

37

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

The MoSCoW model is dynamic and allows room for evolving priorities. So a feature that was
considered a “Won’t-Have” can one day become a must-have depending on the type of product.

Pros of using this prioritization framework:

• It’s good for involving stakeholders without a technical background in the product
prioritization process.
• Quick, easy and intuitive way of communicating priorities to the team and the
customers.
• It allows you to think about resource allocation when you classify your features and
requirements into each bucket.

Cons of using this prioritization framework:

• It’s tempting for teams and stakeholders to overestimate the number of Must-Have
features.
• It’s an exercise in formulating release criteria more than a prioritization method.

6. Opportunity scoring

Also known as opportunity analysis, this prioritization method comes from Anthony Ulwick’s
Outcome-Driven Innovation concept. His theory states that customers buy products and
services to get certain jobs done. The idea is that, while customers aren’t very good at coming
up with the solutions to their problems, their feedback is still important. This feedback is what
the product team will use to come up with the desired outcomes for a product or feature.

Opportunity scoring uses a Satisfaction and Importance graph to measure and rank
opportunities. After you come up with a list of ideal outcomes, you then survey your customers
to ask them the following questions:

• How important is this outcome or feature? Ask your customers to rank them.

38

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

• How satisfied is the customer with the existing solutions?

After you plot these answers along the chart, you should be able to see the features that matter
the most to the customers (the outcomes) yet currently have low satisfaction scores within your
product. These are the features you’ll prioritize for your next sprint.

Pros of using this prioritization framework

• It’s a simple framework for quickly identifying the most innovative solutions to a
common problem.
• It’s easy to visualize and categorize on a graph.

Cons of using this prioritization framework

• In the survey or questionnaire, customers might overestimate or underestimate the

importance of a feature.

7. The Product Tree

Also known as pruning the product tree, this collaborative innovation game was developed by
Bruce Hollman. The focus of this activity is to shape the product so it matches the customer
outcomes that will bring the highest value to the company. The game aims to prune product
backlog items to ensure that innovative ideas aren’t being left behind.

Here’s how it goes:

• First, draw a large tree with a few big branches on a whiteboard or a piece of paper.
• The trunk of your tree represents the features your product already has.
• The outermost branches represent the features that will be available in the next release.
• The other branches represent the features that aren’t available yet.

39

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Ask your participants (in this case, your customers) to write some potential features on sticky
notes. These will be the leaves of your tree.

Then, ask your customers to place their feature leaves on a branch.

By asking customers to place their desired features on the tree, you can identify the biggest
clusters or branches. This will allow you to determine which areas of your product need more
work, which features need to be changed, and what product feature areas can be deprioritized
from all immediate future releases.

Pros of using this prioritization framework:

• It gives you a visual sense of how well balanced your product’s features are.
• It’s highly collaborative, allowing you to tap directly into the insight of your customers
without relying on a rigid survey.

Cons of using this prioritization framework:

• This method doesn’t give product managers any quantitative values for how to rank
each feature, only a general visual guide.
• Features aren’t separated into any sort of grouping bucket, making the exercise time-
consuming.

8. Cost of Delay

Joshua Arnold defined Cost of Delay as “a way to communicate the impact of time on the
outcomes [the company wishes] to achieve.” Specifically, cost of delay allows you to ask the
questions:

• What would this feature be worth if the product had it right now?
• How much would it be worth it if this feature gets made earlier?

40

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

• How much would it cost if it was made later than planned?

The way you assign this monetary value to each feature is by calculating how much time and
team effort they will take to build. You and the team can also assign value to the features in
terms of how much they will be worth after they’re built.

So, let’s say you have one feature that costs you $30,000 per each week that it’s delayed, and it
will only take three months to build. On the other hand, you have a feature that costs $10,000
per each week that it’s delayed and it will take you the same amount of time to build. Within
this prioritization framework, the first feature would be the one your team focuses on first.

Pros of using this prioritization framework:

• It allows you to quantify your product backlog in terms of money.

• It helps product managers make better decisions based on the value that matters the
most to the company.
• It changes the team’s mindset around features from cost and efficiency metrics, to
speed and value.

Cons of using this prioritization framework:

• The parameters for determining the monetary value of a feature are based on gut-feel
and intuition. This can lead to internal disagreements regarding the arbitrary value of
any given feature.

9. Buy a feature

Buy a feature is an innovation game that can involve customers and stakeholders (it’s up to you
and the needs of your product). When you use it as an exercise with your customers, this
method can quantifiably tell you how much a feature or an idea is worth to the people who’ll
end up using it.

41

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

The game goes like this:

Choose a list of features, ideas, or updates and assign a monetary value to each one. This value
isn’t arbitrary—it should be based on how much time, money and effort each feature will cost
you and the team.

Put together a group of people (up to 8 customers or your own internal team). Give them a set
amount of money to “spend” on these features.

Ask your participants to buy the features they like. Some customers will put all of their money
on only one feature they’re passionate about, others will spread their cash around multiple
different features. Ask the participants to explain why they spent money on the feature they
picked.

Then, reorganize that list of features in order of how much money your customers were willing
to spend on them.

Innovation games suggests that you price some of the features high enough that no one can buy
them. This forces your customers to team up and negotiate which feature they’d be willing to
pool their money on.
Pros of using the buy a feature method:

• If you believe in that Steve Jobs quote (“People don't know what they want until you
show it to them”) but you also would like to tap into the wisdom of your customers, this
method is perfect.
• It replaces the stuffy, old-schooled customer questionnaire with a collaborative, fun
exercise that forces your customers to rationalize why they think they need a feature.

Cons of using the buy a feature method:

• This prioritization method can only include features that you’ve already decided to
include in a product development roadmap—the results just tell you what features
customers value the most.
• Ideally, the activity requires you to get a group of customers in one place at the same
time, which can be difficult to coordinate.

10. The HEART framework

The HEART framework is a methodology to improve the user experience (UX) of software. The
framework helps a company evaluate any aspect of its user experience according to five user-
centered metrics. These metrics, which form the acronym HEART, are:

1. Happiness
2. Engagement
3. Adoption
4. Retention
5. Task success

42

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

How does the HEART framework work?

To use this framework, a team will identify goals, signals, and metrics for each of HEART’s five
categories, as the above visual shows. This Goals-Signals-Metrics framework also originated at
Google, and it is very straightforward:

Goals: Broad objectives. For Happiness, a goal might be to “increase user satisfaction.”

Signals: Indicators that your team is making progress toward its goals. For Engagement, a sign
might be “users are spending more time per session in our software.”

Metrics: Quantifiable data points indicating success or failure. For Retention, a useful metric
might be “reduced churn.”

When they have chosen goals, identified the signals to look for, and set quantifiable success
metrics for each of the five elements of HEART, a UX design team will have an objective way of
learning whether their user experience is positive or needs improvement.

Note: Not all five elements of HEART will be relevant to every feature, product, or website that a
UX team wants to quantify. In some cases, the team will use only four or even three of the
categories.

HEART framework template

What are the Benefits of the HEART Framework?

There are many advantages to using HEART, such as:

1. Valuable trends and business intelligence

Because this framework tracks and measures the same user experience from multiple angles—
user happiness, retention, etc.—it can help the company identify essential patterns such as how
improving one metric might weaken another.

43

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

The team might learn, for example, that when it focuses resources on increasing user adoption,
it also lowers the product’s happiness score. This information can help the team make
adjustments in how it builds and markets its products.

2. More strategic focus

A key reason Kerry Rodden created HEART was to help Google’s UX designers focus on a handful
of important areas of the user experience and tune out the rest. She saw that many designers
and researchers were becoming overwhelmed with the massive amounts of raw usage data.

By using this framework, a team can devote its attention and energy to just those areas of the
user experience they believe will have the most significant strategic impact on the product and
the company’s bottom line.

3. A more predictable ROI

The data that UX and product teams will generate over time using the HEART framework can
uncover valuable insights about which of these five elements most significantly and consistently
lead to increased revenue.

If the company sees over time that investing in boosting retention adds more to the bottom line
than investing in engagement, the UX or product team will know where to places its resources
on a new product if both engagement and retention rates are low.

Who Should Use the HEART Framework?

The HEART framework was designed for software UX teams. The framework’s five elements and
the Goals-Signals-Metrics model that it works with are still most logically suited to user-
experience designers and researchers.

But HEART can also be a useful framework for product managers—specifically to help them
weigh competing projects to determine which offer the most strategic value. Product teams can
use HEART as a prioritization framework when they have more ideas or requests for features
and enhancements than their cross-functional team can work on in a given timeframe.

44

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Metrices

1. MAUs / DAUs

Monthly Active Users (MAU) and Daily Active Users (DAU) are a great overview of a digital
product’s overall health. If you’re thinking about your long-term growth, these are metrics which
you can’t afford to miss! They help to track whether your user base is growing or not, and how
‘sticky’ your product is for end-users.

Determining what a DAU is will depend entirely on your product. It’s very rarely as simple as
someone who just opens the app and then closes it again. What many product people try to do is
determine a minimum action taken to get value from a product. For a music streaming service,
for example, this might be playing a song. For a messaging app it might be sending one message.

2. Churn & Customer Retention Rate

Customer retention rate designates the percentage of customers the company has retained over
a given time period. Retention rate is a reverse side of churn rate, which shows the percentage of
customers a company has lost over a specific period.

Customer Retention Rate = ((EC-NC)/SC)*100, where:

EC - number of customers at the end of a period

NC - number of new customers during that period

SC - number of customers at the start of that period

Let's say you released a mobile game. On September 1st you had 1000 players. You got 500 new
players by September 30, however 200 players stopped playing the game. So, at the end of a
period (in our case one month) you had 1300 playing customers. Let's calculate the retention rate:
((1300-500)/1000)*100=80

3. NPS

Your NPS score, in a nutshell, tells you how well your product is loved by users. It helps you
segment your users into 3 categories based on how they rate your product out of 10.

1-6 = Detractors

These people use your product, but only out of necessity/lack of alternative, and wouldn’t
recommend it to a friend.

7-8 = Passives

These people like your product, but you haven’t delighted them.

9-10 = Promoters

45

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

These people are gold dust. They’re your number one fans and will actively promote your product
within their circles.

Once you have your promoters, you can leverage them in your product-led growth strategy, and
in your marketing.

4. CSAT Score

CSAT is a simpler score, and can be used to measure how happy users are with individual
processes and features. While NPS is more often used to measure how happy customers are with
the entire user journey, CSAT allows you to be more specific. For example, you can ask users to
score your onboarding experience out of ten, once they’ve finished the tutorial. It’s a one-tap
survey which is a common feedback collection method within customer service.

5. CLTV

CLTV, or Customer Lifetime Value, helps you to put a price tag on your users.

It’s very simple to work out, and can be made more effective by working out the CLTV for each
segment of your users. For example, the value of a B2B/enterprise customer could be higher than
your average B2B customer.

For example, if you run a subscription service that the average customer pays $10 per month for.
If the average customer lifetime is 2 years, that gives each customer in that segment a CLTV of
$240.

6. CAC

CAC, or Customer Acquisition Cost, is a great metric to keep track of alongside your CLTV. Your
customers may be expensive to acquire, but if the CLTV is high enough, it’ll be worth it. However
if you find that your average CLTV is pretty low (let’s say you’re operating on a freemium model),
a high acquisition cost could put you out of business!

46

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

7. MRR / ARR

Monthly Recurring Revenue (MRR) and Annual Recurring Revenue (ARR) are important numbers
for two reasons. They’re numbers that may fluctuate, but represent reliable income. That is to
say, they’re the amount of revenue you can reasonably predict to be coming in every month and
every year.

For subscription services, this means knowing that you have X amount of customers spending Y
amount every month, leading to a predictable income stream. The number will shift as people
sign up and cancel, but let’s say you have between 100,000 and 110,000 subscribers in your first
quarter. As long as you maintain 100,000 users, that’s 100,000 user’s worth of reliable revenue.

Awareness

Awareness metrics include:

• Website visits: how many people are visiting your website on a daily, weekly and monthly
basis.
• Click-through rate (CTR): the percentage of people who click on a hyperlink vs. the number
of people who see that hyperlink.

47

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

• Impressions: the number of people who could have seen your content. Impressions
indicate potential reach.
• Social impressions: the number of people who had your social post appear in their feed;
sometimes called “reach.”

User-behavior metrics

● Time on page: the average amount of time site visitors spend on specific webpages.
● Bounce rate: the number of site visitors who immediately leave your site.
● Pages per session: the average number of pages viewed by visitors before they leave your
site.
● Percentage of new vs. returning visitors: the ratio of how many of your site visitors who
are new vs. how many are returning

Acquisition

Acquisition metrics include:

• Visit-to-lead conversion rate: the percentage of your site visitors who become leads.
• Contacts generated: how many contacts you’ve created over a given time period.
• Leads generated: how many leads you’ve created over a given time period.
• MQLs, SQLs, CQLs and opportunities generated: how many marketing qualified leads,
sales qualified leads, chat qualified leads and opportunities you’ve created over a given
time period.
• Social engagement rates
• Shares: how many shares you receive on a single post or across a platform.
• Likes: how many “likes” or “favorites” you receive on a single post or across a platform.
• Comments: how many comments you receive on a single post or across a platform.
• Email click-through rate: the percentage of people who opened an email that clicked on
a hyperlink within it.
• Cost of Acquisition (CAC): the cost of acquiring a single customer.
• Average deal cycle: how long it takes on average for a prospect to progress from their
first interaction with you to closing the deal.

Activation

These metrics include:

• Product qualified leads (PQLs): how many people are using your product in a way that
indicates they’re ready to upgrade or become paid customers.
• Lead-to-sign-up conversion rate: how many leads are signing up to use your product.
• Number of sign-ups: how many people are signing up to use your product in a given time
period.
• Sign-up-to-PQL conversion: how many people who signed up for your product became
PQLs.

48

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

• Time from sign-up to active usage: how much time passes on average between a person
signing up to use your product to meeting your definition of an active user.

Revenue

Revenue metrics include:

• Average deal size: the average dollar value of your closed deals.
• Monthly recurring revenue (MRR): how much revenue you can expect to earn on a
monthly basis from both new and existing business.
• Annual Recurring Revenue (ARR): how much revenue you can expect to earn on a yearly
basis from both new and existing business.

Retention

Retention metrics include:

• Customer lifetime value (CLV): the average amount of revenue you receive per customer
over the entirety of your engagement with them.
• Retention rate: the percentage of customers who stay with your company over a given
time period; the inverse of churn rate.
• Churn rate: how many customers your business loses in a given time period.
• Payback period: how long it takes to recover the cost of acquiring a customer.
• Product usage: how much and to what extent a customer is using your product. High
usage is an indicator of retention.
• Customer satisfaction: do your customers feel like your product or service is fulfilling the
purpose they purchased it for? If customers are unsatisfied, they are likely to churn.

Referral

Referral metrics include:

• NPS score: how willing your customers are to recommend your product or service to
others.
• Social shares: how often your customers are sharing your content with their networks on
social media.
• Review rates: how many reviews you receive in a given time period.
• Referred business: the amount of business that comes from referrals, usually through
word of mouth.
• Customer satisfaction: do your customers feel like your product or service is fulfilling the
purpose they purchased it for? If they are satisfied, they’re more likely to recommend
your product or service to others.

49

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

You can measure reviews and social shares through review sites and social platforms. NPS scores
and referred business should be tracked in your CRM, but that can require integration between
referral channels and your CRM.

50

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Types of Product Questions

About the Product Question

The product question is the heart and soul of the PM interview. It directly gets at what a PM does:

design, build, and improve products.

These questions come in three common forms:

1. How would you design _________? For example, you might be

asked to design an alarm clock for the blind.

2. What would you improve about ____________?

3. What’s your favorite product and why?

While these questions sound different, they have one very important aspect in common:

You need to understand and focus on the goal. The goal might be to design the best product for
the user or it might be to increase revenue or another metric.

Type 1: Designing a Product

Question: Design an Alarm Clock for the Blind

Let’s walk through this problem step by step. Note that this is only one solution. There are many
ways to respond to this question.

Step 1: Ask questions to understand the problem

In this situation, we’re told explicitly who the user is. However, this information may be only half
true.

What kind of blind person? Many people who are blind can detect light or even see blurry shapes.

Second, blind people can be children, adults, or elderly people. They may even have additional
disabilities. If we’re designing for a specific type of blind person, this will affect our product.

We should also understand where the blind person is using the alarm clock. Will this be an alarm
used at home, or perhaps one for travel? Is it even a physical alarm clock, or could the interviewer
be asking about a mobile app designed for blind people?

Let’s assume our interviewer confirms he’s thinking of a fully blind person—zero ability to see—
and that he would like us to design for a blind adult who will be using a physical alarm clock at
home.

Step 2: Provide a structure

The approach in this problem is essentially the structure we offer to the interviewer.

51

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Okay, now that I understand the problem, I’m going to tackle this in a few parts. First, I’ll think
about who the users are and what they’re using the alarm clock for. Second, I’m going to compare
existing alarm clocks against these use cases to see where the gaps are. Then finally, I’m going to
discuss how we can fill in these gaps. As you go through these steps, make it clear to your
interviewer when you’re transitioning from one step to the next. You could say something like,
“Now that we’ve identified the users, let’s move on to evaluating existing alarm clocks.”
Phrasing Questions

Be careful about how you phrase your questions. A question like, “What does the blind person
do?” can sound like you want your interviewer to solve problems for you. However, if you word
it as “Is there anything we know about what a blind person does?” makes it clear you’re trying to
work collaboratively with your interviewer.

If your interviewer pushes back with a question such as “What do you think?” that’s a sign to stop
pushing with the questions.

Step 3: Identify the users and customers

We have one user in mind already: a fully blind adult.

Who else might use the alarm clock? A blind person is likely around non-blind people, such as
their spouse, children, or healthcare workers. Could you imagine if your spouse bought an alarm
clock that you couldn’t turn off? Yikes! An alarm clock for the blind still needs to be moderately
usable by nonblind people.

Step 4: What are the use cases? Why are they using this product?

The core use case here is presumably to wake up from sleep, usually in the morning but possibly
for naps as well. This means he’ll need to check if the alarm is set, configure the time, enable the
alarm, be woken up by it, snooze the alarm, and disable it.

Could there be other reasons why the blind person uses the product? Sure!

He could be using it to check the current time.

He could be using it to time something (e.g., boiling an egg). After all, not all his appliances will be
adapted for his disability, so he might use the alarm clock to compensate.

He could be using it to remind himself to take medicines. In this case, he’ll probably need some
sort of recurring alarm.He may be using it to listen to music or the radio, as many people do with
their alarm clocks. I’ll assume the primary function we want to design for is the wake-up-from-
sleep one.

Checking the current time is certainly an important use case for alarm clocks in general, but it may
not be an essential one for blind people. People use an alarm clock to check the time because it’s
highly convenient (that is, the time is right there, staring at them from across the room). Unless
we can achieve similar convenience, blind people probably wouldn’t use the alarm clock for this
purpose. They would default to using whatever they otherwise use to check the time.
Step 5: How well is the current product doing for their use cases? Are there obvious weak spots?

52

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

A standard alarm clock relies on visual indicators for almost everything. We know if the alarm is
on or off based on a light on the alarm clock. We know the current time because we can read it
on the display. We just look at the alarm clock to know if the device is plugged in. Essentially, the
only thing we don’t use a visual indicator for is waking up. This poses a number of challenges for
a blind person, since he won’t be able to see the visual Indicator.

Enabling / disabling the alarm: Alarms commonly have a single button to toggle the alarm; press
the “alarm set” button to flip the state. This is fine when you can see if the alarm is set, but it’s
likely to be problematic for a blind person. Toggling the state of something only works well if the
current state is very obvious. Ensuring the alarm is plugged in: Electronics can get unplugged for
any number of reasons. Simply giving the blind person a way to check if the alarm is plugged in
might not be enough, since they probably won’t remember to check this option before going to
bed. Rather, we need to make it obvious that the alarm is unplugged.

Checking if the alarm is set: A standard alarm clock has a light on the display to indicate whether
the alarm is enabled. This won’t work for blind people since they can’t see the indicator light.

Setting the alarm: A standard alarm clock is set by pushing an “up” or “down” button on the time,
and waiting until the time is set correctly. Since a blind person can’t read the clock, the visual
feedback on the time doesn’t really help them.

Setting / checking the current time: Although we don’t need to use the alarm clock to check the
time, we do need to ensure the current time is set correctly. Since a standard alarm clock relies
on a visual display for the time, this doesn’t work well for blind people.
We now need to resolve these issues.

Step 6: What features would improve those weak spots?

We want to keep the alarm clock simple to use. Complicated designs are no fun for anyone, blind
or not.
Design 1: Audio Playback

The major issue we’ll need to design around is the lack of visual display. As a very simple approach,
we can use audio playback, in addition to a visual display.

The benefit here is it’s easy to build audio playback, and the use of a visual display would make
the design usable for non-blind people as well. However, it has a major drawback: it might wake
up the blind person’s partner or spouse. We’ll work with this approach for now, though.

Turning on / off alarm + Ensuring the alarm is plugged in like a standard alarm clock, this alarm
clock could have a button that turns the alarm on and off. We will provide audio feedback when
we set the alarm to indicate either “the alarm is on” or “the alarm is off.” This also makes it easy
to ensure the alarm didn’t get secretly unplugged.

Checking if the alarm is set

To keep things simple, we can just reuse the button that turns the alarm on and off. We don’t
really need a special button for this.

Snoozing the alarm

53

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

To snooze, we will have a separate “snooze” button, much like a normal alarm.

Setting the alarm time

To set the current alarm, we can offer “up” and “down” buttons to change the hour and minute
hands. When we press a button, it announces the current time.

Doing this on every single button press might be a bit slow though, particularly when changing
the minute. Instead, we can design it so that if you hold down the “up” or “down” buttons, it
suppresses the audio playback until the button is released.

Setting the current time

We can set the current time essentially the same way that we would set the alarm time.

Design 2: Limited Audio Feedback

As we said earlier, the issue with using audio playback is that it might disturb others in the room.
We may want to design a more advanced alarm clock that doesn’t rely on audio playback for the
most common functions (particularly those that might be done while the user’s partner is asleep):
turning on and off the alarm and setting the alarm time.

Turning on and off the alarm

Instead of audio feedback, we could use a physical switch (much like a light switch) to turn the
alarm clock on and off. The risk here is that the alarm clock could wind up unplugged and the user
wouldn’t know.

Instead, we can still use a single button to toggle the alarm. Instead of audio feedback, the clock
will vibrate once to say “alarm is on” and twice to say “alarm is off.”

Checking if the alarm is set

Again, we will just reuse the button which turns the alarm on and off. The vibration feedback will
tell us if the alarm is set.

Setting the alarm time

To set the alarm time, we can have several rows of buttons on the side of the alarm clock. Each
button would have the number written in braille as well as in colored text for a non-blind person.
Since we’re using buttons to set the time, we don’t need to use audio playback.

Hour 12 1 2 3 4 5 6 7 8 9 10 11

Minute

0 1 2 3 4 5 6 7 8 9 AM

0 1 2 3 4 5 6 7 8 9 PM

The AM / PM button would be pushed in to indicate it’s set to AM, otherwise it would be set to
PM. This might be far more buttons than we really need. Does anyone really need their alarm

54

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

clock to be capable of setting their alarm for exactly 9:23 a.m.? Probably not. Setting the alarm to
9:25 or 9:20 would work just fine.

Step 7: Wrap it up

Depending on the priority we place on not disturbing the user’s partner, we may or may not use
audio feedback. You could mention to your interviewer the sales benefit of marketing the alarm
clock as non-intrusive to bed partners.

The first design uses audio feedback to basically replicate what the physical display does. We still
have a visual display, in case non-blind people use the device, but we announce any alarm or
other changes out loud. The only non-obvious change here is suppressing the time-change
announcement when we hold down the “up” or “down” arrows.

If we’re concerned about waking up the user’s partner, we can remove the audio feedback and
replace it with vibration (to indicate the alarm status) and physical buttons (to set the time).

In either case, we will keep the physical display to ensure the device is usable by non-blind people
as well.

Question: Design a new iPad app for Google Spreadsheet.

CANDIDATE: Sure, I can design a new iPad app for Google Spreadsheet. Before I take a moment
to brainstorm some ideas, can I ask a few clarifying questions?

INTERVIEWER: Sure.

CANDIDATE: What is the goal of creating this iPad app?

INTERVIEWER: What do you think might be our motivation to create one?

CANDIDATE: Well, the PC’s share of total computing time is decreasing. More people are spending
their time on mobile devices. Google would want to promote spreadsheet usage across those
mobile devices. It would be a missed opportunity if Google allowed another competitor to
become the leader in mobile spreadsheet applications.

INTERVIEWER: That’s right. This is a defensive play.

CANDIDATE: Before I brainstorm ideas, do we have any constraints?

INTERVIEWER: We typically release new Google products on Android before iOS. However, in this
case, we are short on Android developers, so we’re going to do iOS first. And don’t worry about
time constraints. We’d rather get the product right than rush something to market.

CANDIDATE: Can I recruit Android developers to help?

INTERVIEWER: You can try, but let’s just say that’s a moot point for now.

CANDIDATE: I’m not familiar with the Google Spreadsheet customer base. Can you describe who
they are, what they’re trying to achieve, and what they would like to see in an iPad app?

INTERVIEWER: Why don’t you tell me what you think it is?

55

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

CANDIDATE: I feel the Google Spreadsheet user likes Google Spreadsheet for a few reasons. First,
all the documents are stored in the cloud, which means they can be revised on any machine
without worrying about version control.

Second, it’s very easy to collaborate. Multiple spreadsheet users can view and edit documents at
the same time, meaning no more emailing attachments to each team member, asking them to
make changes, and then compiling all the changes.

Lastly, it’s free for anyone with a Google Account.

Google Spreadsheet doesn’t have all the features of Microsoft Excel, so the spreadsheet is more
likely to appeal to casual rather than expert users.

When it comes to casual users, many of them are spending more time with their mobile devices.
They want to access their spreadsheets, which they can achieve through Google Drive or the
browser. However, manipulating a spreadsheet is very cumbersome. Tapping cells to edit, typing
formulas and selecting multiple cells—it’s just not fun to do on a mobile device.

There might be some other use cases, such as the need for more powerful charting functionality
and running sophisticated analyses like regression. But, it’s not likely something our casual user
base would care about. It also seems out of scope with the original question.

INTERVIEWER: You’re correct. We want to build an iPad app that addresses the deficiencies of
editing a spreadsheet in a mobile device. That’s our top problem.

CANDIDATE: Okay, give me a moment to jot some ideas.

I have three suggestions to address the problem of editing a spreadsheet on a mobile device.

The first idea is to have a voice input option that would allow the user to add the information or
compute formulas by speaking.

The pros: voice input is fast and easy. The cons: voice recognition is not perfect and can be error-
prone. It can also be embarrassing to use in a public place.

The second idea is gesture input. That is, create mobile device-friendly gestures to more easily
manipulate a spreadsheet. For example, to add a column, the user can select the beginning and
end of the column with his or her forefinger and thumb with a pinch motion. After the pinch
motion, he or she can immediately draw a plus to indicate an addition operation, or draw an “X”
to indicate a multiplication operation.

The advantage is that this uses multi-finger touch screen technology for spreadsheet editing. The
disadvantage is that the gestures are not intuitive or obvious. Significant training would have to
occur, possibly burdening the user with sitting through a tutorial.

The third idea is to use an “edit later on PC” feature. On the mobile device, a user would be able
to view spreadsheets and annotate changes to the spreadsheet. A simple one-click button sends

56

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

a notification to the user the next time he or she is on the PC to make the changes. The pros:
changes are made in an environment where the user is comfortable and likely efficient. The cons:
changes won’t be made immediately. It adds an extra step to the process, leading to the likelihood
that the user forgets or doesn’t have time to make the changes the next time he or she uses a PC.
This is my least favorite idea.

INTERVIEWER: Okay, I like those ideas. So which one do you think we should pursue first?

CANDIDATE: I would rule out the third idea, to begin with. It just doesn’t sound effective and is
not immediate. I use Siri a lot, so voice input is intriguing. However, voice recognition might not
be practical for manipulating a spreadsheet. Besides, I use Siri because I want to use my phone
hands-free, like when I’m driving. I doubt anyone would or should edit a spreadsheet while driving
a car.

That leaves us with a new gesture-based spreadsheet. I like that one. I think we can come up with
some creative ways to build a spreadsheet that leverages the multi-gesture approach possible on
touch screen devices.

INTERVIEWER: Thanks for the recommendation.

Type 2: Improving a Product

How will you improve LinkedIn’s home page?

CANDIDATE: Before I make suggestions, I want to review the web page, make sure I understand
all the features, and ask clarifying questions.

INTERVIEWER: Sure.

CANDIDATE: My first thought: who is this page for? It’s targeted to the LinkedIn user. The second
thought: what does it do?

I believe the primary goal is to show an evolving news feed. Here we see general news, but we
could also get updates on our social network — whether people got a new job, connected to other
people or received new skill endorsements.

The secondary goal is to allow the user to post updates and discover new connections.

The third goal is to search for people and identify new jobs.

Just to summarize, there are four things that one could do on this page:

1. Read updates for the work world, including general news and social updates.

2. Write updates.

3. Discover new contacts.

4. Identify new job opportunities.

57

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Now that I’m more familiar with the home page, let’s dive into the question.

My first thought: what’s the purpose of the redesign? I’m specifically thinking of business
objectives such as increasing ad revenue,

LinkedIn connections, or 30-day activities. I define 30-day activities as the number of LinkedIn
users that have visited the site in the last 30-days.

I’d propose looking at increasing 30-day activities, but I’m open to other suggestions too.

INTERVIEWER: No. Let’s focus on engagement.

CANDIDATE: Next, I want to have a customer in mind for the redesign. Here are four personas
that come to my mind:

● The newsreader. Officially, this persona is discovering new content to help them with their
job. Unofficially, they are wasting time or procrastinating.
● The social networker. This person is keeping tabs on business relationships, hoping that
it’ll help him climb up the career ladder.
● The job seeker. Some job seekers will be employed. Others will be unemployed, but
looking for better opportunities.
● The salesperson. Salespeople love LinkedIn because it is a treasure trove of contact
information, interests, and needs. It’s a great way to find new opportunities.

CANDIDATE: I’d like to focus on the salesperson.

INTERVIEWER: Go on.

CANDIDATE: What are the top use cases for salespeople? Here are the first three I thought of:

1. They’re looking for new customer opportunities.

2. They want to strengthen relationships with existing customers. Most sales people strengthen

relationships by offering value: solving customer problems, providing information that moves
their clients’ business forward and yes, even providing perks like tickets to the customer’s favorite
sports team.

3. They want to get inspired. Selling products is tough. They get rejected all the time. Inspirational
stories and words of encouragement help them soldier on.

In my opinion, the most important use case is finding new customer opportunities.

So let me share with you a mockup of what I envision.

58

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

CANDIDATE: It’s your standard LinkedIn newsfeed: logo, navigation, search box, news feed area,
and display ad. Here’s what’s different: the “new sales opportunities” section in the upper right.
As the title describes, this feature proactively identifies new sales opportunities for the customer.

For example, there are two new opportunities for Twitter and Tableau Software, a company that
helps users display data publicly in attractive ways. Sales opportunities with Twitter and Tableau
have a 22 percent and 11 percent chance to close, respectively. These predictions come from our
algorithm, which analyzes sales opportunities based on a news event, similarity with existing
customers, and the number of close connections to that company.

Clicking on “new sales opportunities,” you’ll see the next page, the “sales opportunity
dashboard.”

59

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

CANDIDATE: On the sales opportunity dashboard, the sales person can get more details including:

● Trigger event. For the Twitter sales opportunity, the dashboard indicates that it’s in the
same “target segment” as one of our existing customers. Competitive companies usually
have similar needs. They’re also easier to close because of the powerful sales intro, “I’m
calling because one of your competitors uses our software…”
● Headline. This is the news that triggered the event.
● Contact. Identifies a potential contact person, with the ability to click to see their LinkedIn
profile.
● Action. User can bookmark a sales opportunity and export to their customer relationship
management (CRM) software. As a future addition, perhaps LinkedIn can become the
forum through which contact is made. For instance, there could be cold call email
templates, progress tracking, automatic follow-up and reporting.

Question: How would you improve Google Maps?

CANDIDATE: Here’s how I’d think about improving Google Maps. First, I’d understand our goals
& constraints. Then, I’d explore the user and their use cases. I’d then prioritize the use cases and
focus on brainstorming solutions for the most urgent use case.

60

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

We can consider different goals such as increasing revenue, new user acquisition, engagement
and virality. I have engagement in mind, but is there another goal you prefer?

INTERVIEWER: Let’s go forward with an engagement goal.

CANDIDATE: For users, there are a lot of audiences we can target: college students, young
professionals, busy moms and retirees. I’d like to focus on brick-and-mortar shoppers. Is that okay
with you?

INTERVIEWER: Sure.

CANDIDATE: For a shopper, one of the most frustrating things is when you can’t find what you
need. Warehouse stores, such as Home Depot, Costco and IKEA, are examples where this is a
problem. You can imagine that there’s a dollar cost to this frustration. Just imagine the time away
from the job site, means billable hours lost.

There are a couple use cases we need to keep in mind.

First, the user should be able to specify what product they’re looking for.

Second, they should confirm that it is the right product.

Third, they should receive a detailed map location, including directions and shelf location.

Lastly, it should indicate the number of units available.

Let’s brainstorm solutions on how this might work. In the interest of time, let’s just pick one area
to focus on. Normally, we’d consider business value, customer impact and tech difficulty when
prioritizing. In this case, let’s just pick the most unique use case: the shelf locator.

INTERVIEWER: Okay.

CANDIDATE: The reason shelf locators are interesting is that a big box warehouse store – the
aisles are several feet high and a mile deep – which makes this a big problem.

I have a few solutions in mind for a shelf locator feature.

First idea is a picture that shows where a product is located. The picture will include surrounding
products, to help the person locate it more easily, with the specific product highlighted in red.

The pro is that visual context helps to find things. The con is that it’s not precise; the product could
still be anywhere in the aisle. Also, it will be labor intensive to get all the pictures.

Second, we can label each product area. E. g. “Aisle 6, Section 2, Shelf 4.” This is precise and easy-
to find. It does put a big burden on the store owner to geo-label each product area.

The third idea is to tag each product with RFID chips. RFID chips emit a radio frequency to allow
electronic devices to identify and track particular products. While RFID chips aren’t cost
prohibitive, not all shoppers have RFID readers on their phones. Attaching RFID chips to each
product would require some tooling overhead.

61

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

I’m attracted to the RFID idea because it’s cool, though honestly isn’t the most pragmatic. In this
case, the most pragmatic solution with respect to time, ease and cost of implementation is to
simply label each product area.

Type 3: Metrics Questions

What metrics will you look at to evaluate success of a product?

INTERVIEWER: LinkedIn is testing a new feature: asking a new user to upload their profile photo
during the signup phase. Currently, a new user is asked to upload a profile photo after the sign-
up process. What metrics will you look at to evaluate success of a product?

CANDIDATE: Give me a moment to collect my thoughts.

CANDIDATE: Here are the metrics that I would consider: acquisition, activation, engagement,
retention, and monetization.

For acquisition, we might track the number of lazy registrations.

For activation, we might look at the number of people that complete the entire signup process.

For engagement, we might look at a variety of metrics including profile completeness, the
number of users that posted, the number of posts they make, and the number of invites sent.

For retention, I’d look at how often they return to our site within single-, 7- and 30-day windows.

Lastly, for monetization, I’d consider a few different metrics including:

● Average revenue per user (ARPU)

● Average revenue per paying user (ARPPU)
● Average revenue per daily active user (ARPDAU)
● Lifetime value (LTV)
● Conversion rate

62

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

INTERVIEWER: Okay, let’s continue to the next step: you run the upload profile feature in signup
flow feature for a two-week A/B test. Here are the results:

Tell me what’s going on.

CANDIDATE: On the one hand, engagement has gone up significantly. Increases in the number of
posts and invites sent are remarkable. The number of users posting and 7 day retention is not bad
either. However, there’s a double-digit drop in the number of users completing signup.

This feels a bit one step forward and one step back.

However, I think we’re getting a good trade off here. That is, what’s the point of getting more
users (that complete the signup) if they’re not going to use the product down the road? Having
more engaged users is worth it in my opinion.

INTERVIEWER: One last question: why did we show the p-value column?

CANDIDATE: P-value is a measure of whether our results are statistically valid. That is, with a
sufficient p-value, we know that our results can’t be explained by something other than the
feature changed specified in the A/B test. Typically, I am comfortable accept data with a p-value
less than .05.

Tip: Making Decisions from A/B Test Results

Ideally, we would run A/B test experiments for win-win situations. For example, it would be
amazing if a new feature was found to increase revenue by 5 percent and seven-day logins by 3
percent.

63

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

However, the real world often requires us to make win-lose trade-offs. For instance, a particular
feature could increase revenue 5 percent, but 30-day logins decrease by 3 percent. What should
a product manager do?

When faced with this dilemma, make the decision that’s in-line with the corporate strategic goal.
Ask the interviewer. What’s the current objective? If the owner is desperate to meet a quarterly
profit goal, choose the feature that maximizes revenue at the expense of engagement. If
engagement’s the goal, choose that over maximizing revenue.

Type 4: New Product Entry Questions

Question: You are having tea with Mr. Ratan Tata. He has just returned from Germany where he
saw third party car service stations which were doing very well. So, he is thinking of opening a
chain of such stations in India. You need to give him your thoughts and make a pitch from our side
for helping him with the project.

Candidate: I’m not very sure of what you mean by third party service stations. Can you explain a
little?

Interviewer: To service a car there are service stations. They can be authorized stations like the
chain that Maruti has or they can be local garages. The third type, which is currently missing in
India, is an independent chain of service stations which will service any brand. These are third
party service stations

Candidate: Ok. This is a new business that Tata would want to enter. I’d like to look at a few things
while considering the new venture:

•Tata’s final aim - do they have a target profit /market share/return on assets as their target from

the venture

•Market scenario – growth & size, competition

•Tata’s capabilities – financial capability, expertise in area, synergies with other businesses

Interviewer: Tata is a big & profitable company, they want as high profits as possible from the
venture. Also, they have no constraints with regards to finances. They build automobiles as you
know and have authorized service stations for their automobiles.

Candidate: So, the aim of Tata is high profits and they have sufficient finances and expertise in
the automobile area. I’ll go on to look at the automobile maintenance market. Currently in India
there are 2 kinds of garages – the local ones and authorized service stations. So, when we enter
the market, would we be servicing all kinds of brands and providing a full range of services?

Interviewer: Yes. All brands and full range of services.

Candidate: We would need to differentiate ourselves from the 2 kinds of competitors that we
have in order to get customers.

Interviewer: Ok. How would you do that?

64

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Candidate: I’ll look at why a customer goes to a service station and why he chooses a particular
station to go to. A car would be taken to a service station for regular check-ups/services, in case
of an accident and maintenance when it breaks down.

Interviewer: Ok

Candidate: Now when an owner chooses a service station he would want:

• Quality – In terms of genuine parts if replacements are done, trained mechanics, the car
being
• treated properly, delivery on time
• Cost – He would want the service to be as cheap as possible
• Convenience – The service station should be close or should have a pick & drop service.

There would be a segment of customers who would lay a lot of emphasis on cost while another

segment would lay emphasis on quality. In case of an accident or break-down convenience would
play a big role. Local garages will have low quality and low cost while authorized service stations
will have high quality and high cost. Also, local garages are generally more in number so would be
more convenient to reach in most cases.

Interviewer: Ok. Now I want you to make a grid of the dimensions that you’ve mentioned and
figure out where our competitors lie and where we should go.

Let us club convenience with quality.

Candidate: We’ll just analyze the situation based on 2 parameters. Now, Tata wants to start a
third party chain of service stations which will serve all brands. If Tata targets low quality, local
garages will beat them since these garages can service all brands and charge very low unbeatable
prices. Also, they would be built at strategic locations which Tata may not be able to acquire,
coming late into the market.

On comparing Tata stations with authorized service stations, Tata could stand a chance. They
could ensure quality by sourcing parts from manufacturing companies and employing well trained
mechanics. Since such a service station will service all brands, it will be a convenient place to come
to for high quality services. However, the price charged will be high.

Interviewer: Do you think anyone will come to such a service station when they can go to a Maruti
or Hyundai authorized service station?

Candidate: In India a majority of cars are Maruti and Maruti has a very good chain of service
stations which are convenient to reach and high quality. Hence, Maruti cars will definitely not
come to Tata’s stations. Other brands like Hyundai would come since their service stations are
few and far apart. If Tata offers the same quality at the same price; it might be cheaper and more
convenient for consumers if Tata’s chain has numerous stations at strategic locations.
Interviewer: Maruti has almost 50% of India’s car market share. Now do you think it is beneficial to
set up Tata’s third-party service chain?

65

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Candidate: Owners of other brands will prefer to go to their authorized service stations as they would
be more trusted. And given such a lopsided market in favor of Maruti, it will be difficult for us to
compete with Maruti directly. So, the number of cars coming to Tata’s stations might be too low for
the venture to be viable. But, if there are expectations that many new brands will enter India as some
already have, then Tata’s venture could be viable given that these firms would not want to open a
service chain of their own due to small numbers and newer vehicles could mean that the local garages
might not be well-equipped to deal with all kinds of problems with the vehicle.

Interviewer: What would your final recommendation be?

Candidate: My final recommendation would be to not start such a venture currently since Tata would
not be able to beat competitors on any dimension - cost or quality. However, in the near future this
could turn sustainable so an eye should be kept on this market.

Tip:

66

Downloaded by alex sharma ([email protected])

 lOMoARcPSD|40030935

Useful Resources
➢ Frameworks - Must Read

The Ultimate Guide to Product Management Prioritization Frameworks

List of Frameworks for Structured Product Thinking - by My PM Interview

➢ Must Watch

Cracking the Product Manager interview AMA with Prashant Kumar Sonu, Sr. Product Manager
(Microsoft)

➢ Must Read

Fake It Till You Make It: Fitting into a PM role for interviews and the job

➢ Must Follow

Occam's Razor by Avinash Kaushik - Digital Marketing and Analytics Blog

➢ Learn PM With Me

➢ https://www.notion.so/The-Ultimate-PM-Resources-Repo-
2461013856a6411ea32a529d51e601b7

➢ Stratechery by Ben Thompson – On the business, strategy, and impact of technology.

➢ https://growth.design/

➢ https://www.sachinrekhi.com/

➢ https://www.notion.so/PM-Interview-Preparation-Guide-
e95959c5d5eb47e583306b5d32221455

➢ Flipkart Product Manager Mock Interview: Root Cause Analysis (Razorpay PM)
➢ Mock Product Manager Interview (LinkedIn PM): Improve Spotify's Social Features
➢ Amazon Product Manager Mock Interview: Solving Pain Points
➢ Mock Product Manager Interview: Improve WhatsApp (Google PM)
➢ Facebook Product Manager Mock Interview: Facebook Movies
➢ Google Maps Product Manager Mock Interview ft. Microsoft PM

67

Downloaded by alex sharma ([email protected])