www.workingbackwards.

com

Discovering Your Input Metrics

To effectively manage and improve each of their many businesses, Amazon puts significant time
and energy into developing, measuring, observing, and managing customer-facing, controllable
input metrics. Input metrics are the drivers that, when managed well, can lead to the right
outputs; profitable growth in revenue and customers. The right choice of input metrics will
deliver clear, actionable guidance. A poor choice will result in a statement of the obvious, a
non-specific presentation of everything your company is doing. Donald Wheeler, in his book,
explains:

“Before you can improve any system…you must understand how the inputs affect the outputs of
the system. You must be able to change the inputs (and possibly the system) in order to achieve
the desired results. This will require a sustained effort, constancy of purpose, and an
environment where continual improvement is the operating philosophy.”1

Amazon takes this philosophy to heart, focusing on leading indicators (we call these
“controllable, customer-facing input metrics”) rather than lagging indicators (“output metrics”).

Controllable, customer-facing input metrics are just as the name describes:

a) Controllable- these are factors that your company can control through your
team’s actions and initiatives. In the case of Amazon, these activities/initiatives
include adding more items to Amazon’s store, reducing costs so prices can be
lowered, or positioning inventory to facilitate faster delivery to customers. Output
metrics—things like orders, revenue, and profit—are important, but generally
can’t be directly manipulated in a sustainable manner over the long term.
b) Customer Facing- the inputs are the factors that positively improve the customer
experience. Each sustained improvement in the customer experience is creating
long-term value for customers. When Amazon improves and adjusts their
operations to reduce order to delivery time from 3 days to 1 day, customers win.

While the majority of input metrics are customer-facing, some big ones are not. Examples
include inventory record accuracy, days payable for AP, workplace safety, marketing spend,
vendor allowances, etc. Many metrics like these indirectly improve the customer experience, but
not all of your input metrics will be customer-facing.

Inputs and input metrics measure things that, when done right, improve the customer
experience, create value and bring about the desired results in your output metrics like sales,
profit and share price.

1
Donald J. Wheeler, Understanding Variation: The Key to Managing Chaos (Knoxville, TN: SPC Press,
2000), 13.

1
 www.workingbackwards.com

So, how do you figure out the input metrics for your company? It sounds easy but in fact,
discovering (really uncovering) your input metrics is an iterative process that requires months
and years. Here is how to get started.

Ideally, the first step is to articulate the inputs that drive the flywheel for your company as
described by Jim Collins in his book Good To Great. In short these are the activities or elements
of your business that form the intersection of a) your passion, b) where your company can be
best in the world, and c) what drives your economic engine.

From the early days at Amazon, Jeff identified a vision of enabling customers to find, discover
and buy anything online. Later he broke this down into discrete inputs; vast selection, low
prices, great customer experience (e.g. find what you want and get it shipped to your home
fast). Selection, price and customer experience are the drivers of Amazon’s flywheel and each
happens to be a controllable, customer-facing input.

Once you have articulated the inputs to your flywheel, you need to consider the factors that
affect and drive each of them.

Think of your metrics like an upside down tree with branches or nodes spreading outwards and
downwards from a starting point (Metric A in Diagram 1) which is one of the flywheel inputs to
your business. Each flywheel input is affected by other inputs (Metrics B and C in Diagram 1),
which are in turn affected to another layer of metrics (D through I). Discovering your inputs is the
process of mapping out the nodes/branches that affect each input. Keep going until you
bottom-out.

2
 www.workingbackwards.com

Diagram 1

Let’s apply this approach to Amazon’s business. The primary inputs that drive the flywheel for
Amazon’s retail business are selection, price, and customer experience. These inputs drive the
outputs of units ordered, Prime subscribers, revenue, and profit.

Let’s map the inputs to Price (low retail prices) as an example. You can’t offer low prices and
make a profit unless you have low fixed and variable costs. Four of the biggest cost drivers in
e-commerce are the cost of goods sold, fulfillment costs, inventory holding costs, and
transportation (delivery) costs (see diagram 2). Each of these costs have many inputs that affect
them.

Looking at the example of Fulfillment Cost, we list four possible input metrics in Diagram 2;
receive cost, pick cost, pack cost, stow & convey cost. For each of these nodes, there are many
other nodes affecting each of these four. For example, looking at ‘receive cost’ we can continue
outward from this node and add additional inputs (or input metrics) such as Tp90 time to unload
and receive per shipment, and accuracy of labor planning, etc.

3
 www.workingbackwards.com

Diagram 2

The process of identifying the nodes that affect each input and developing the right metrics to
measure each is a continuous and interactive process. This can and will result in a list of
hundreds of metrics.

How do you know when to stop adding metrics? The answer is that you stop when the
metric/activity cannot be meaningfully decomposed any further - e.g. "reduce search page
latency"

So how do you prioritize and manage it all? We would argue that defining the correct input
metrics is one of the top responsibilities of the executive team. The exec team is asking
everyone else in the company to organize around and perform activities that drive those input
metrics. If the exec team fails to identify the input metrics or picks the wrong ones, then the
responsibility for that likely outcome of missing your output metrics falls squarely on the exec
team. After all, the rest of the company is only doing what you asked them to do.

Once you have identified the right input metrics, exec leaders need to ensure that the company
is organized in the right way so that there are capable leaders and owners of each branch of the
tree. In this regard, it is useful to think about overlaying your org chart onto the upside-down
tree. This can help guide your thinking about where you have the right leaders and teams in
place and where you don’t.

4
 www.workingbackwards.com

Map the Customer Experience to Uncover Input Metrics

One of the best ways to identify your inputs and their drivers is to map out the end-to-end
customer experience for your product or service and then identify potential inputs and
corresponding metrics. Simply break down the customer experience into steps, capture each
step in a diagram or a spreadsheet, then brainstorm potential inputs and metrics for each step.
It helps to do this as a team and a group where you can brainstorm together.

At this stage, it's important to identify what metrics you want to track rather than what metrics
are available today. As you embark on the process to discover your input metrics, you likely will
find systems or processes that do not record the data you need (we call this instrumentation).
There may very well be some instrumentation work the engineering team needs to build in order
to collect this data in a useful manner.

Here is a spreadsheet that you can use to flesh out and brainstorm your input (and output). It is
pre-populated with a set of example metrics that are relevant for e-commerce (but may not be
relevant to your business).

Ask each line manager, P&L, or business owner, to fill out their section of the customer
experience journey and add potential input metrics to the list of prospective metrics. Your
customer support organization and marketing organization should add their metrics as well.
Once everyone has contributed their proposed metrics, have a meeting that includes each
leader or metrics owner, your C-Level leaders, and members of the BI and/or FP&A team to
review and discuss the list.

Another mode of thinking for deriving your inputs is from the perspective of the drivers of your
outputs (customer growth, revenue, gross margin, OPEX, etc). Consider what activities, and
metrics explain the changes in your output metrics. Hopefully, many and most of these are
controllable, customer-facing inputs. If these are metrics that you don't directly control and/or
don't relate to the customer experience, consider shifting the organization’s focus to controllable
customer-facing input metrics.

Tips and Watch-outs

1. At the outset, try not to limit your thinking to what you can and cannot measure at this
point, just focus on the customer journey. Once you have filled out the spreadsheet,
meet to review, debate, and discuss the various metrics, make modifications to the list,
and (finally) assign a priority to each. This prioritization will drive the work that follows to
gather the data, calculate and visualize each metric in a WBR report.

2. Try not to select and predict which of the potential input metrics is the most important.
This is largely unknowable at the start of the process. It is difficult to predict the value of
a metric until you establish a WBR using the 4-blocker visualization method, and have

5
 www.workingbackwards.com

been able to observe changes in the metric over time and how those changes
correspond with other metrics.

3. Don't omit the output metrics. We started every WBR at Amazon with a review of our
progress against our financial plan for the quarter. Adding input metrics to your
management dashboard doesn’t mean removing your outputs. In fact, it is imperative
that you review them together in order to determine which inputs and corresponding
metrics make an impact. If you move the right input metrics in the right direction, the
result will be positive growth in one or more output metrics.

4. Err on the side of too many metrics rather than too few at the outset. You need to throw
some spaghetti on the wall at this phase and then see what sticks. The list of metrics you
review in your WBR is ever-changing -- you are continuously subtracting, adding, and
modifying the metrics. At the outset, use your best judgment to prioritize the list of
metrics and the corresponding work your engineering and BI teams will need to do to
capture the appropriate data and calculate each metric.

5. Make sure you allocate sufficient resources from your BI or data analytics team. There
likely will be additional work required from these teams and the WBR cannot be
successfully implemented with that work being done.

FAQ
Q: It seems like this exercise will result in our executive team tracking hundreds of metrics and,
across the company, tracking thousands or tens of thousands of different metrics. How will we
know what to focus on? Wouldn’t we be better off focusing on “One Metric That Matters” as our
north star and pointing all efforts towards it?
A: In our experience, the “One Metric That Matters” philosophy is fundamentally flawed. It
assumes that your business is one-dimensional and thereby glosses over the multi-dimensional
complexity that is inherent in every business. A detailed understanding of how your business
works is vital to effective decision-making.

We are frequently asked what research Amazon conducted to decide what products to build and
actions to take? Our answer is that leaders at Amazon spent their time swimming in a sea of
input metrics. By being in touch with every end-point and every aspect of the customer
experience, we were able to see how the component parts were interconnected and understand
our customers, our processes to spot opportunities and defects at a level of depth that few
companies achieve. This detailed understanding produced new ideas for what to invent on
behalf of our customers and better decision-making at every level of the company.

6
 www.workingbackwards.com

As discussed earlier, this doesn't mean that all metrics are created equal. The work of the
management team is to identify the most important input metrics to drive, and to understand the
drivers of each. At the same time, it is advised to track a constellation of input metrics because if
one of those metrics starts to shift out of an acceptable range, it can be a signal of a customer
facing or process defect. You cannot anticipate the cascading effects of every action, and
swimming in the data is a way to manage this challenge.

For more on the topic of One Metric That Matters (and input metrics), we recommend reading
this article on the topic from Reforge:
https://www.reforge.com/blog/north-star-metric-growth

Q: Are advertising, social media, and other marketing and customer acquisition activities and
their corresponding metrics inputs or outputs?
A: New and active customers are important inputs to driving sales and revenue outputs.
Reviewing marketing spend by channel, and the cost of customer acquisition is an important
component of a WBR. How much, how, and where you are investing your marketing dollars as
well as measuring the return (LTV/CAC, and/or $ spend/incremental revenue) are all important
decisions that were continuously monitored and adjusted at Amazon.

Q Can a metric be both an input metric and an output metric? And should I be worried about
that?
A: A metric can be both an input metric that drives another metric. It can also have a set of input
metrics that drive it. So, the answer to the first question is, “Yes.” In this process, don’t get too
hung up on whether a metric is an input, output, or both. But you should avoid cases where you
have identified an important metric, it’s one you cannot directly control (i.e. an output metric),
and you have not identified any input metrics that drive it.

Q: Do all controllable input metrics need to be customer facing?

A: No, but most of them probably will be. However you may find that there are some key
controllable input metrics that are not customer facing but have a big impact on your output
metrics. One example is inventory record accuracy. If you don’t know exactly where each item is
stored across your fulfillment network, it will be difficult to ensure that you keep your delivery
promises you make to customers. A second example is Days Payable. If you strive to be a low
cost provider where every penny counts, then measuring, analyzing, and controlling your
company’s Days Payable portfolio frees up precious capital that can be deployed elsewhere.

7
 www.workingbackwards.com

Appendix A -- Input Metrics and WBR Process

Implementation Milestones
Summary

1. Intake for Bill & Colin of your current practices

2. Seminar/Discussion on Input Metrics and WBR with the Senior Leadership team
3. Identify team/org to pilot Input Metrics and WBR Process (or roll out company-wide)
4. Identify and assign a project manager for the project
5. Build a master spreadsheet to capture a complete list of output and input metrics
6. Meetings with each team leader to review/discuss their list of metrics
7. Prioritize the list of metrics
8. Build the metrics and the WBR report
9. Schedule weekly WBR meeting(s)
10. WBR Meeting Seminar
11. Launch WBR Meeting(s)

Descriptions

1. Intake for Colin & Bill of your current practices. Help us understand how you currently
operate by sharing current metrics reports.

2. Seminar/Discussion on Input Metrics and WBR with the Senior Leadership team. Bill or
Colin lead this session with an overview of the Input Metrics and WBR Process followed
by a Q&A and a discussion regarding implementation. The goal is to ensure that the
goals and objectives are clear, to enable skeptics to ask tough questions and gain
alignment and support from all members of the exec team.

3. Identify team/org to pilot Input Metrics and WBR Process (or roll out company-wide).
Note -- it is important that the Exec team leader of this team/org is fully aligned (willing to
champion the process change) and that there are periodic meetings (later) to review the
status of the implementation plan.

4. Identify and assign a project manager for the project. This is at least a few days a week
of work for two or three months for an individual to manage and coordinate the
implementation of metrics and WBR. This project manager can be from the pilot org but
is typically a member of the FP&A or Business Intelligence org. The work includes;
scheduling training sessions, own master spreadsheet capturing a complete list of output
and input metrics, working with various business unit and functional leaders to gather
prospective metrics, coordinating with relevant teams (e.g. data eng, BI) to ensure that

8
 www.workingbackwards.com

metrics are captured accurately in a data warehouse, lead the effort with relevant teams
(BI) to establish automated WBR report creation, establish a 30-minute weekly project
check-in meeting with relevant stakeholders and Colin or Bill. Note -- in addition to this
product lead this project will be time-consuming for the FP&A, BI, Data engineering, and
the operating teams too. Make sure that they will have the bandwidth to take on this
added work.

5. Build a master spreadsheet to capture a complete list of output and input metrics. Here
is a template that you can use as a starting point. Make sure to create a section (column
a) for each relevant P&L and function and/or stage of the customer experience flow. It is
critical that the source data is accurate and that each metric is defined and calculated
precisely. This list should include output metrics, input metrics, metrics that you currently
track, and metrics that don’t exist yet. Do not limit yourself when building this list. Bill or
Colin will review the list and provide feedback.

6. Meetings with each team leader to review/discuss their list of metrics. After each leader
and their team members have filled in their list of metrics it is advised for the project
leader along with Colin or Bill to have a brief meeting with each team to review and
discuss their list of metrics. This is a venue to answer questions and make suggestions.

7. Prioritize the list of metrics. Once you lock down the master list you will need to prioritize
(0, 1, 2) each metric in order of importance. Your priority 0 metrics define the minimum
set of metrics required before you are ready to launch the WBR meeting. This
prioritization is necessary as significant work is required to capture, calculate and add it
to the WBR report in an automated manner.

8. Build the metrics and the WBR report. Develop and plan for how all of the metrics in the
master sheet will be a) captured and calculated in your data warehouse and b), added to
the master WBR report. For part ‘A’ this requires ensuring that each metric is accurately
defined in terms of the underlying data source and how the metric is calculated. Part B is
building the WBR report(s) using the 4-blocker format and a plan for how to
automatically build the WBR report each Monday. We have resources/templates that we
can share with you for WBR 4-blocker report creation in Tableau, Looker, and Excel. It is
likely that you won’t have every metric available in the report immediately and that it will
require several weeks of work to add them all to the new report(s).

One approach here is to use a subset of the P0 metrics list to build a proof of concept for
the underlying data model and BI layer. Take care to ensure that the data model is built
in a way that it can handle future changes in metrics and the data cube.

It is recommended to add each metric to the report whether you have the source data or
not. For those without data add a label “WIP [date] and an owner” where the [date] is
when the metric will be added to the report. Create a timeline for completion.

9
 www.workingbackwards.com

9. Schedule weekly WBR meeting(s). The weekly cadence for this process is that the
report(s) is generated on Monday AM, Each team or business unit meets on Tuesday for
their department level WBR where they review the company-wide report as well as a
deeper list of metrics that they track for their team. They use this time to understand
what happened in their part of the business in the prior week and to prep for the
company-wide WBR the following day. The company-wide WBR occurs each
Wednesday.

10. WBR meeting seminar. Opportunity to train and teach all WBR participants hosted by Bill
or Colin.

11. Launch WBR Meeting(s). Time and practice are required to conduct a high-quality WBR
meeting. The team needs to get aligned on which metrics to cover, how to discuss them,
how deep to go, how to ask probing questions of their peers. Initially, much of the
discussion will be about the metrics themselves which is healthy. Determining which
metrics to review in your WBR is a continuous process.

10
 www.workingbackwards.com

Appendix B - CSAT/NPS Case Study

Here’s an example from a grocery delivery business. They wanted to increase their output
metric, Net Promoter Score (NPS), over time. After some initial analysis, the team identified a
metric called Perfect Order Score (calculated as the number Perfect Orders / the number of
Total Orders) as the most significant controllable input metric that would contribute to a higher
NPS score. A Perfect Order was measured as an order that had the following characteristics:
● The order was delivered within the promised 2 hour time window
● The order contained every item the customer originally ordered (no substitutes and no
missing items)
● There were no CS contacts or refunds within 24 hours of delivery

A typical grocery order contained about 20 unique items. Orders typically contained a mix of
perishable or fragile items such as fruits, vegetables, eggs, and ice cream. The shelf life of
some items could be measured in days. Once an order was loaded on a delivery vehicle, it was
difficult to reroute. In short, while a Perfect Order sounds easy, it was operationally complex to
achieve and relied on a number of different teams (vendor management, forecasting, quality
control, inventory record accuracy, picking, and delivery) to execute according to plan.

As the WBR started, the team realized there was not a strong link between Perfect Order Score
and NPS. They suspected that Perfect Order Score was a composite of too many individual
metrics that it masked one or more links between controllable inputs and outputs. So they broke
out Perfect Order Score into three separate metrics: On-Time Orders, Complete Orders
(everything ordered was delivered), Orders with No Contacts or Refunds. After a few weeks, the
strongest correlation to NPS was with On-Time Orders. But the team realized that even the
On-Time Orders metric was not actionable enough. After some more investigation, they found
out that the strongest accuracy with NPS and delivery statistics was not a binary on-time metric,
rather the number of minutes the delivery was late. There was not a noticeable dip in NPS for
orders that were 1-15 minutes late. A cliff occurred at 15 minutes and 60 minutes. So the
On-Time Delivery metric was changed to the number of minutes outside of the delivery window,
then rolled up into a 1-15 minute bucket and a 60+ minute bucket.

This new delivery metric was controllable, actionable, and really mattered to customers as
evidenced by their NPS score. Using the WBR and the new metric, the delivery team was able
to identify all deliveries that were at least 60 minutes outside of the window, analyze the
situation, then implement fixes that addressed the root causes. NPS improved. The next bucket
to focus on and fix was the 15-60 minutes bucket.

11