Getting Started with Data-

Driven Decision Making: A

Workbook
JANUARY 2 13

An NTEN Workbook prepared by

www.nten.org | 1020 SW Taylor Street | Suite 800 | Portland, Oregon 97205 | p: 415.397.9000 | f: 415.814.4056
Introductio
n
Over the last few years, NTEN has been working with nonprofits to understand how to better wrangle all the
data theyre creating and thats available to them into improved resultsmore mission impact. In 2010, we
organized an online workshop we affectionately titled Taming the Data Monster. That workshop focused on
the data and story side of the equation: how to translate the data you have into a snapshot of the work you
are doing today (and yesterday). The workshop was designed for the Communications and IT Staff who have
the task of collecting and distributing data in effective and meaningful ways. But what about the steps an
organization has to take before collecting or analyzing the data? How do we determine which data matters
in the first place, and how that data will impact the direction of our work?
In 2012, NTEN was able to survey a sample of US nonprofits to find out how they were doing when it comes to
being data-driven: were they collecting data and tracking metrics? Were they finding that data useful for making
organizational decisions? With the financial support of Google and the research expertise of Idealware, we
published the 2012 State of Nonprofit Data report which indicated that nonprofits werent having success with
those strategic steps in the equation: almost all nonprofits were collecting and tracking some type of metrics,
but only a small fraction were finding that data useful for making decisions that impacted their programmatic
work. Clearly, there was a disconnect between the tactical work of collecting and tracking data and the strategic
work of defining and using the data for the organizations mission.
With this workbook, we hope to provide nonprofit leaders with a set of worksheets that will help guide
thinking and planning around more effective data strategies.
While these worksheets are part of the strategic process of defining the right questions, metrics, and plans for
effecting data-driven decisions, and this workbook is designed for nonprofit leadersexecutive directors, board
members, and other organizational leadership staff who develop strategies and are responsible for making
organizational decisionsit will be helpful to include input and discussion from various staff. For example,
worksheets 4 and 5, especially, could benefit from input from your program, communications, and data or IT
team members.
Overall, we acknowledge that its important that your entire team buys-in to the strategydata collection,
analysis, and reiteration is a cultural shift for many of us, and its something that cant be done once or
intermittently or inconsistently across your organization if were going to be more effective. All of this takes
investment of time and resources as well as vision. Thats why this workbook includes worksheets that will help
you think through the onboarding and budgeting processes as well.
You may want to go through this workbook separately for various projects or go through it again to re-define
or update your metrics. In any case, we hope youll find this workbook helpful for launching or improving data
strategies that will impact your organizations work.
Wed like to thank Google for providing financial support for this project, and Idealware for helping us produce
this workbook.

Annaliese Hoehling
Publications Director, NTEN

GETTING STARTED WITH DATA-DRIVEN DECISION-MAKING JANUARY 2013

Table of
Contents
Welcome................................................................................................................................................................. 2
Worksheets
1. Define Your Action Question..................................................................................................................... 3
2. What Does This Question Mean To Your Organization?................................................................................ 5
3. How Will You Know What the Right Metrics Are?.................................................................................... 6
4. What Could You Measure?............................................................................................................................ 7
5. What Data Do You Have?.............................................................................................................................. 9
6. Find The Metrics That Make Sense For You......................................................................................... 10
7. The Metric Creation Process............................................................................................................................... 11
8. Define a Process for Using Them to Make Decisions.................................................................................. 12
9. Create an Implementation Action Plan.........................................................................................................13
About NTEN and Idealware...................................................................................................................................15

PAGE 1 GETTING STARTED WITH DATA-DRIVEN DECISION-MAKING JANUARY 2013

Welcome

Could you use more help thinking through how to use data to help your organization make decisions? If so,
youre not alone. Our recent report on how nonprofits are using data showed that although some organizations
are relying heavily on data, a number were doing very little to measure their work.
Measuring may not be as difficult as you suspect. When we talked to experts, they agreed that simply
starting to track a few strategic metrics was a huge step toward a more data-based culture. Once your staff
has data that they can use to make decisions, they will often start to want more. A few, straightforward
metrics can start the snowball to a more broad-based program.
Getting started isnt a trivial process, however. What metrics will be useful and actionablebut not require a ton
of time to collect and understand? How do you define and communicate data in order for your organization to
make decisions?
This workbook will help you with those questions. If you have a vague idea of what youd like to measure,
you can start with the first worksheet and proceed in order through the workbook. It will walk you through
the process of choosing and refining initial metrics, defining how you will collect the data, and explaining
how to disseminate that data for decision making. If youre further down the path than that, feel free to pick
and choose the worksheets that make sense for your needs. You may just want to think about the process
for creating and using your metrics or narrow down a list of metrics that will have the most impact.
As youre thinking about data-based decision making for your organization, dont forget the other resources in
this research series. The 2012 State of Nonprofit Data report provides information about what nonprofits are
doing with data and the factors that contribute to (or impede) success. Weve also gathered ten case studies
outlining how ten different organizations are using data to make decisions. They are available online in NTENs
case study section..
Dont feel overwhelmed by data. The journey to data-based decision making, like any, begins with the first
step. In this case, you can begin with the first worksheet. Once you start defining the metrics that will help your
organization, you may find that its easier than you thought.
1. Define Your Action
Question
This workbook is designed to help you hone some of POSSIBLE ORGANIZATIONAL AREAS
FOR EXPLORATION
the metrics that will be practical and meaningful for
you in a particular area. To focus your efforts more Recruiting clients
Efficiency of a process
productively, start by identifying the general area you
Projecting future income
want to use this workbook to explore in more detail. A specific program
Choose an item from the box to the right that Annual campaign
resonates with you, or define your own based on a Facebook outreach
Email list-building
topic your organization has been talking about. Staff development
Whats an area you want to explore? Staff recruitment
... Or choose your own
1) Recruiting Clients

2) Staff Development

Now define a particular action question within that area. Its not WHAT MAKES A GOOD QUESTION?
going to be easyyour action question must be specific enough
to be measurable and to help you decide how to move forward, Stumped as to the type of question
but also important enough to really matter to your organization. were looking for? Try to think of
Lets start by brainstorming. something that will help your organi-
zation improve and that you can have
What are some tactical questions youd ideally like to be able to
an impact onbut that can also be
answer in this area that would help your organization improve?
tested and measured. For example,
How can we improve fundraising? is
1. How many clients are coming in with due to the 150 brochures that were passed out?
too broadits not easy to pin down
2. Are the prices of the products affordable and inviting to customers? what you would specifically do to
affect or measure it. Whats our email
3. Are the standards of our goods better than our competition? open rate? is too narrow and its not
clear how it relates to organization
goals. Look for a question thats both
4. Are advancements being originated to summon the notice of new consumers?
important and detailed enough to be
testable, such as Is our blog worth
5. Are the incentives appealing to the employees and customers? the time we spend on it? or How can
we improve program attendance?

PAGE 3 NONPROFIT DATA DECISION-MAKING WORKBOOK JANUARY 2013

Lets drill a little more deeply into those questions. Refer to the questions you defined above by number.

Will the answer Will your actions as Can numbers help Is the question testable?
to this question an organization you answer the Can you create a
help you affect the answer to question? hypothesis about a
improve this question? potential answer, and then
as an organization? test it?
Yes/ Why or Yes/ Why or Yes/ Why or Yes/
QUESTION No why No why No why No Why or why not?
Yes
not?
If the reaction is Yes
not?
Definitely yes because Yes
not?
Yes numbers can Yes Yes because we could create a
minimum, we should we are in control of the help us because we minimal and decent reply for each 50
1. change the location and different facets of how want to know the flyers that have been delivered
form of delivery when these flyers can be used total feedback of
distributing flyers what is being
analyzed

Yes Yes because the prices Yes Yes because we have to Yes Yes because we Yes Yes because the value of productivity
are totaled and formed draw in prospective research the can be minimized so that the prices
on immediate and customers to our number of new can be reduced and have feedback
2. incidental costs. The reasonable prices on customers that
company could our goods answer to our
enhance the area of prices
manufacture.

Yes Yes because our Yes Yes because we need to Yes Yes because the Yes Yes because the marketplace can
products identify our make sure that we have goods can be provide merchandise that can be
organization and they a refined quality of computed through reviewed to differentiate value in
3. impact how everyone sales and business quality assurance quality assurance
sees our realization as a
company

Yes Yes because its up to Yes Yes because if we are Yes Yes because we Yes Yes because we can develop a theory
the company to truthful and legitimate could take a count to see how many goods we can sell
generate advertising our advancements to of all the new with any special advancements that
4. that will boost our gain new consumers consumers that we promote
innovations and team will most likely result have accepted our
work and this will with remarkable results advancements
hopefully have a great
reaction
Yes Yes because we could Yes Yes because we can Yes Yes we can balance Yes Yes because their determination to
evaluate if our decide which incentives the quantity of have appealing incentives will be
incentives are work better for response from our compared with their results from
5. appealing, make employees and employees and production
judgments of how to customers to better our customers
continue to work to business production
make employees and and sales
customers happy

Think through the answers you gave above. For one or more of the questions, were you able to answer
Yes in all of the boxes? If so, pick one of those questions to explore with this workbook, or combine
multiple questions together to define one overarching question. (Note that if you combine multiple
questions into one, you might want to plug it into the table above to make sure it fits all the criteria.)
Write your question here:
Are our advancements, advertisements, brochures, standards, prices and incentives giving us a continuous circulation of new consumers
and happy employees?

Did you answer No to at least one of the criteria for all of your questions in the table above? If so,
those questions will be difficult to answer using metrics. Brainstorm some other questions that are
important but also measurable to get to a place where you can define an action question that will be
the framework for the remainder of this workbook.
2. What Does This Question Mean To Your
Organization?

What would different people in your organization want to know about this question
in terms of how it affects their own jobs? How much does each care about the
information at all? Its likely that there are a number of different perspectives about
it. Not sure what people would like to know? Ask them...
In the mock organization chart below, do two things:
In the small box in the upper left, define how important the information would be to each type
of role in your organization (High, Medium, or Low).
In the larger space in each box, list some of the key pieces of information that a person in
that role in particular might be interested in seeing. Dont worry about whats possible or practical
just yet, simply brainstorm what theyd ideally like to see.
FOR EXAMPLE Board ED/CEO Other
High High High
High
Monthly financial gain due Transactions will increase Bankers/stakeholders will
Summary of to the demand of new on a monthly basis benefit from the clients
attendance numbers merchandise by clients Profit of the investment that will be drawn to this
High level over time if economy is business and have great
good (ROI) earnings
demographics

High Other Low Other Program High Fundraising/

Directors Marketing
Production should be a Directors
Common laborers will
concern for the line The marketing director
continue with their
manager especially to have needs to keep in mind the
customary obligations
a high volume of customers customers they currently
separately of how many
who will want the products have and plan to gain with
new customers are drawn in
their advertising strategies

Other High Line Staff Line Staff Other

If there is a high demand of

merchandise due to a
higher gain of clientele then
there will be more hours for
production line

Now that youve thought that through, do a reality check: Do other people care about this question
too? If youre the only person who really cares, is it an important question for your organization?
PAGE 5 NONPROFIT DATA DECISION-MAKING WORKBOOK JANUARY 2013
3. How Will You Know What the Right
Metrics are?
Soon well define metrics to help you answer your question. Metrics provide a
numerical yardstick to help you determine whether your efforts are making a
differenceand if so, in what direction. Before we go too far down that road, think
through how youll judge whether the metrics will actually help you define what you
really want to know.
There are different ways to think about this. Pick one of the questions below that
seems to make the most sense in the context of your overall action question and
write a brief description of what success will look like in this process.

1. What specific decisions do you want to be able to make based on the answer to your question?
1) The role of the production floor line should be associated with the price and cost of the merchandise

2) The elements that will be needed for the production floor line and the quality of the goods can be adjusted to refine the irrevocable
characteristics of the goods
3) The bulk of supplies used to create brochures, special offers, and how the merchandise will be circulated

2....OR what things will you need to understand in order to feel you have real knowledge to address
your question?

3....OR Will any answer to your question feel like success, or will you need to achieve a specific result
to feel successful?
3. How Will You Know What the Right
Metrics are?
4. What Could You
Measure?
There are a lot of different things you could measure for any given thing. Brainstorm
the different actions you could take that might have an impact on your overall
question and the metrics you could use to measure it. A metric is a numberoften
either a count or a percentagethat measures your success in an area.

1. WHAT ACTIONS COULD YOU TAKE THAT WOULD HAVE AN IMPACT ON YOUR ACTION QUESTION?
Do you have the If you were To what extent Now sum
ability to affect to measure would seeing a To what extent up the
What metric the measurement? this, how measurement would your orgs last
Action could you use actions quickly
Is it something many people for this help three columns
to measure you in your you improve result in a change to to create a
this? could change through organization your the measurement? Usefulness
your actions? would care? organization? Score
1= Very hard
For instance, # 1= Only slightly
1= Almost No to see change
participants, % 5=
Answer Yes or No. One 5= The 5= Actions visibly
satisfaction, # Completely
Whole change metric almost
units provided. transformed
Organization immediately
Refining value % of returned or No 2 2 3 7
ruined products
Better incentives # of clients who Yes 3 3 4 10
for customers requested the
incentives
Reducing prices % of the price that No 4 4 5 13
was reduced
Utilizing modern # of new Yes 4 3 5 12
and new customers for
brochures every 150
brochures passed
2. WHAT ACTIONS COULD YOUR CONSTITUENTS TAKE THAT WOULD AFFECT YOUR ACTION QUESTION?
Do you have the If you were To what extent Now sum
To what extent
ability to affect to measure would seeing a up the
What metric would your orgs
the measurement? this, how measurement last
Action could you use actions quickly
Is it something many people for this help three columns
to measure result in a change to
you in your you improve to create a
this? the measurement?
could change through organization your Usefulness
1= Very hard
For instance, # 1= Only slightly
1= Almost No to see change
participants, % Answer Yes or No. One 5= The 5= 5= Actions visibly
satisfaction, # Whole Completely change metric almost
units provided. Organization transformed immediately
Employ self- % of returned or Yes 2 2 3 7
reliant QC ruined products
analysts
Mediation with % discounts in the Yes 4 4 3 11
merchants fee of materials
Talk to our sales dept. % of discounts in Yes 4 2 1 7
manager/director to
merchandise to be
reduce costs
sold
Reliability when # of brochures Yes 2 2 5 9
distributing brochures distributed in person
 3. WHAT ACTIONS COULD OTHER PEOPLE TAKE THAT WOULD AFFECT YOUR ACTION QUESTION EVEN
IF THEY ARENT IMMEDIATELY ASSOCIATED WITH YOUR ORGANIZATION?
Do you have the If you were To what extent Now sum
ability to affect to measure would seeing a To what extent up the
What metric could the measurement? this, how measurement would your orgs last
Action you use to actions quickly
Is it something many people for this help three columns
measure this? you in your you improve result in a change to to create a
could change through organization your the measurement? Usefulness
your actions? would care? organization? Score
1= Very hard
For instance, # 1= Only slightly
1= Almost No to see change
participants, %
Answer Yes or No. One 5= The 5= 5= Actions visibly
satisfaction, # Whole Completely change metric almost
units provided. Organization transformed immediately
Ignore or throw # of brochures No 2 1 1 4
away thrown in the trash
Will disregard # of customers No 2 1 1 4
offers or incentives requesting offers or
incentives
Be drawn by TV # of customers Yes 2 1 1 4
ads requesting new
goods

Draw a line through anything in the table above that you said in the third column you didnt have
control over. Here you should be focused on being able to measure and improve your own actions, so
measurement of external factors is less relevant.
For the rest of the rows, based on which have the highest total score in the right hand column and your gut
reaction to how well the total score reflects reality, choose six that seem promising for exploring your action
question. For each, copy the metric, from the second column, and the overall sum for that row into the
table below.

Metric Sum (Usefulness Score)

# of new customers for every 150 brochures passed out 12

# of clients who requested the incentives 10

# of brochures distributed in person 9

% of discounts in the fee of materials 11

% of returned or ruined products 7

% of discounts in merchandise to be sold 7

5. What Data Do You
Have?
The next step in this process is to identify the data sources for the metrics youve
said youd like to track (if you cant collect the data, then you wont be able to
track that metric). For each of the six metrics you defined on the previous page, think
through the data you have that relates.
Metric (from What related What related data What additional What related data Looking across what youve
previous data is is automatically relevant data about could you get from written for each column,
worksheet). currently tracked by a system? actions, perceptions, other organizations or score the overall ease of
manually entered or processes could public sources? collecting data
into a system? your staf collect that to get this metric.
they arent currently
collecting? 1= Would require vast
new investment
10= We already have it
# of new Name of new client Address The brochure that The amount of 9
customers for Address Phone number they got brochures and
every 150 Phone number Name of client location of where they
brochures e-mail address were mailed to
passed

# of clients who Name of client Name of client Why they are None 8
requested the Incentives requested requesting an
incentives incentive

# of brochures None None Sign in sheet with None 6

distributed in signatures of clients
person who received a
brochure

% of discounts The actual fee for the The actual fee for the The discount fee The prices of other 9
of the fee for products needed products needed which is the product merchants that are not
materials of the mediation with in our contracts
merchants

% of returned or The amount of The amount of A description of the The percentage of 8

returned or ruined returned or ruined ruined products or ruined products in
ruined products
products products returned products other stores

% of discounts The discount that can The discount that can The discount amount The amount of their 7
in merchandise be offered to clients be offered to clients offered as a result of sales when they have
to be sold what needs to be sold discounts available
at a more convenient
price
 6. Find The Metrics That Make Sense For
You
Youve rated the usefulness of each metric (in Worksheet Four), and the difficulty of
getting the data (in Worksheet Five). Now plot each metric using those two scores.
15

14

13

12 x
11 x
10 x
USEFULNESS

9 x
8

7 x x
6

0 1 2 3 4 5 6 7 8 9 10
EASE OF COLLECTING DATA

HOW TO PLOT YOUR METRICS

It likely makes sense to start with the metric that is closest to the top If your metric had a sum of
and right of the plot. Choose one metric (or a small number) that will 6 according to worksheet
provide you the biggest bang for the buck. As you grow comfortable four, and a score of 5
with that metric, you may want to add more that also seem useful into according to worksheet five,
the mix. then youd plot that metric
What metric will you start with? like this example.
# of new customers for every 150 brochures passed 7

5 6

PAGE 10 NONPROFIT DATA DECISION-MAKING WORKBOOK JANUARY 2013

7. The Metric Creation
Process
Even if the data is readily at hand, the metrics wont create and distribute
themselves. Its important to map out the flow of how the data will become an
accurate metricboth to make sure youve thought it through, and to acknowledge
the actual work that will be required from your staff to ensure success. Think
through this process for your metric.

Who creates the

Data Sources How is this data Where is it stored?
metric from the
entered into a system?
Postal Service Department pc data?
By hand
Clientele
Sales Department
Manager/supervisor
Distribution
Department
Product Design
Department

Whats the incentive to Who distributes it?

ensure its entered?
Department manager
Sales Department can
Team leaders
earn commission
Supervisors
General Manager

Whos in charge of this whole process?

General Department Manager

How will you spot-check to ensure the metric accurately reflects reality?
Monitoring department floor sales with walk-ins or surprise visits and compare with the number of clients that were recorded

PAGE 11 NONPROFIT DATA DECISION-MAKING WORKBOOK JANUARY 2013

8. Define A Process for Using Them to Make
Decisions
You not only need a process for creating your metricsyou need a process to
actually use them. This is one of the most critical steps. If a metric measures in a
forest with no one around to hear it, it does not make a sound...

How frequently will you Who will they be In what meetings will they
create distributed to? be used?
the metrics?
Departmental meetings
and General Manager How frequently will
Monthly Sales Department
meetings you plan to adjust
Distribution your actions based
Department on what the metrics
say?
Product Design
Department As often as needed
depending on our
General Manger
What decisions shouldnt sales
be made without them?
Discount offers
Promotional offers

How frequently will you check in on whether the metrics themselves are an effective way to measure what youre
trying to measure?
Quarterly
9. Create an Implementation Action
Plan
Congratulations! Youve defined a strategy to create and use metrics to measure a
core question for your organization. But a strategy is one thing, and
implementation is
anotherto speed you on your way to a successful rollout of your metrics, think
through the implementation steps.

People Youll Need to Get Onboard

Who are the core individuals who will need to buy in? Think through both the official people who need to
be onboard and the other people who might become barriers if theyre not included. List them below.
Then think through the right way to include them in the process: Email? Presentation? Discussion?

Person or Group Method of Including Them

Production Design Department Meetings/e-mails

General Managers Department Meetings/e-mails

Department Sales Managers Meetings/e-mails

Production Design Staff Meetings/e-mails

Delivery Employees Meetings/e-mails

Postal Services Visit in person/e-mails

Processes Youll Need to Define

On the previous worksheets, did you define processes for collecting, analyzing, or distributing metrics that
need to be more fully fleshed out? If so, define what they are and how youll put more detail around them.
Maybe another meeting? More documentation? A larger project?

Process Method of Defining

Gather all pertinent and factual information punctually Meetings with manager, leaders supervisors

Effective methods of enhancement to be able to advance and prosper Departmental Managers

9. Create an Implementation Action
Plan
Keep track of all new clientele and the merchandise that they want to buy Meetings

Consider changes to enhance products or develop and create new goods Meetings/trainings/workshops/review surveys
Things Youll Need to Allocate a Budget For
Have you defined a process that involves things that you dont already havefor example, a survey tool, a
bar code scanner, a new staff member, or maybe a consultant to think it all through? Itemize anything new
that needs to be paid for, and what process youll need to go through to select and purchase the product.

Item to Buy Purchase Process

Modern and advanced technology for Product Design and Sales Department Compare prices and products needed

Reliable and dependable transportation for Distributing Department Compare different car vendors

Other Things Youll Need to Make Happen

Are there other things that need to happen before you can move forward? List them here along with useful
details.

Additional Step Details

Pinpoint areas of dispersal of brochures Review locations and select new areas

Identify new areas for potential customers of interest to look for customers

Those are your next stepsbut theyre probably not in the right order.
Go back through that list and decide what you should do first: Talk to some people? Define a process?
Put a number 1 next to that step. Determine whats next and put a number 2 next to it. Continue
through the whole list until you have an entire action plan in approximately the correct order.
And then go start with the first item on your list!
About NTEN
NTEN, a 501(c)(3) nonprofit, is the membership organization of nonprofit
technology professionals. As a community of nonprofit professionals, we
aspire to a world where nonprofit groups of all types and sizes use
technology
strategically and confidently to fulfill their missions. Together, the NTEN community helps members put
technology to work so they can bring about the change they want to see in the world.
NTEN facilitates the exchange of knowledge and information within our community. We connect our
members to each other, provide professional development opportunities, educate our constituency on
issues of technology use in nonprofits, and spearhead groundbreaking research, advocacy, and education on
technology issues affecting our entire community.

About Idealware
Idealware, a 501(c)(3) nonprofit, provides thoroughly researched,
impartial and accessible resources about software to help nonprofits
make
smart software decisions. By synthesizing vast amounts of original research into credible and
approachable information, Idealware helps nonprofits make the most of their time and financial resources.
Idealware specializes in combining traditional research techniques like interviews and surveys with
software- selection methodologieslike detailed ratings of software tools against a rubricto generate
important new knowledge on affordable budgets. We then package our findings into approachable
reports, articles, and trainings that help nonprofits make the on-the-ground decisions important to them.

PAGE 15 NONPROFIT DATA DECISION-MAKING WORKBOOK JANUARY 2013