T h e C ent er for H ig h I mp ac t Phi l a nt hro py

Sch ool of Soci al Pol i c y & P racti ce | Unive rsit y of Pe nnsylva nia In collaboration with alumni from the Wharton School

in partnership with

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

What Are We Talking About When We Talk About Impact?

W H Y T H I S PA P E R ? The word impact is everywhere these days, but not everyone uses or understands it in the same way. Why does this matter? A clear definition of impact is necessary to develop an effective and rewarding philanthropic strategy, since impact definitions drive decisions and ultimately move dollars. Differingbut unvoicedassumptions about the definition of impact can also create communication difficulties between donors, grantees, and beneficiaries. Donors have a particular obligation to seek clarity and consensus with regard to definitions of impact, as their definitions often carry disproportionate weight: programs can have more of an incentive to satisfy funders than beneficiaries. In particular, for donors focused on historically disempowered groups such as women and girls, it is critical to include those beneficiary voices in the impact definition process; otherwise, philanthropic efforts run the risk of recreating the same imbalanced power dynamics they are trying to counteract. To address the needs of donors navigating this complicated impact landscape, this paper explores current definitions of impact in philanthropy and international development, with a particular focus on the ways impact definition can affect work serving women and girls. Our hope is that it will be useful to donors developing their own definitions of impact; we also hope to clarify the many (sometimes conflicting) uses of the term impact, and the real-world repercussions of those uses. To that end, we present patterns of impact definition from three main perspectives: a) evaluation professionals; b) funders; and c) beneficiaries, particularly women and girls. We have also attempted to keep our gender and bias lenses on, highlighting where certain definitions may favor or discriminate against certain groups or situations.

A bout T he C ent er for H igh Impac t Ph ila nt h ro py

Founded in 2006, the Center for High Impact Philanthropy has emerged as a unique and trusted authority for donors seeking to maximize the social impact of their funds. We help move money to do more good through: independent analysis, advice, and thought-leadership. In areas as diverse as closing the achievement gap in the U.S., providing basic needs to the most economically vulnerable families, effective disaster relief after Haitis earthquake, and major global public health issues such as malaria and child mortality, the Center translates the best available information into actionable guidance for those looking to make the greatest difference in the lives of others. Put simply, success to us means enabling more money to do more good.

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

A bout Women M ovin g Mil lio n s

WMM is a diverse community of extraordinary individuals committed to funding the advancement of women and girls around the globe. Members are passionately committed to mobilizing unprecedented resources and donate at levels of $1 million or more, in the process sparking a larger movement of trailblazing philanthropists working to enhance the potential of half the worlds citizens.

Abou t t h e Au t h o rs
W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3
Katherine H. Hovde, MIA, is a senior analyst at the Center for High Impact Philanthropy, with expertise in education policy, program design, and evaluation. Since 1990, she has worked for the World Bank on a variety of education projects in Latin America, South Asia, and Eastern Europe. In the U.S., she has worked with the Consortium for Policy Research on Education at the University of Pennsylvania and other non-profit organizations focused on education and social issues. Carol A. McLaughlin, MD, MPH, is the research director at the Center for High Impact Philanthropy and teaches at the Perelman School of Medicine at the University of Pennsylvania. She is a primary care physician and public health specialist with experience in research, community engagement, and program implementation in the U.S. and the developing world. Katherina M. Rosqueta, MBA, is the founding executive director of the Center for High Impact Philanthropy, and adjunct faculty at the University of Pennsylvania School of Social Policy & Practice. Previously, she was a consultant at McKinsey & Company. Prior to joining McKinsey, she worked in community development, non-profit management, and corporate and venture philanthropy. Cecily W. Wallman-Stokes, MPA, is a social impact fellow at the Center for High Impact Philanthropy. Before joining the Center, she worked on international vaccine policy at Johns Hopkins University, with a particular focus on Nigeria. Prior to Johns Hopkins, she worked in U.S. reproductive health research at the Guttmacher Institute, and in international agricultural policy and development for the Bill & Melinda Gates Foundation.

To cite this manuscript: Wallman-Stokes, C., Hovde, K., McLaughlin, C. & Rosqueta, K. What we talk about when we talk about impact. Center for High Impact Philanthropy; Women Moving Millions. September 2013.

A P P R OAC H & M E T H O D O LO G Y
To meet our goal of providing smart, practical guidance to donors, we synthesize the best available information from three domains: research, informed opinion, and field experience. By considering evidence from these three sources, we seek to leverage the strengths while minimizing the limitations of each. We believe the most promising opportunities exist where the recommendations of these three domains overlap.

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

FIELD EXPERIENCE

FIELD EXPERIENCE  P ractitioner insights  Performance assessments  I n-depth case studies INFORMED OPINION  E xpert opinion  S takeholder input  Policy analyses RESEARCH  Randomized controlled trials and quasi-experimental studies  M odeled analyses (e.g., cost effectiveness)

MOST PROMISING INFORMED OPINION

RESEARCH

In the development of this paper, the Centers multidisciplinary team examined over 50 sources of information, including available academic research, policy briefs, and foundation materials. We drew from recognized experts in program evaluation, non-profit leaders, and websites and publications of the 10 largest U.S. foundations. We also reviewed literature on giving by individual donors and the literature on beneficiary voice in program development and evaluation. The goal was not to conduct an exhaustive review at this stage, but to provide an overview of the landscape of impact definition. While we touch on issues of impact measurement, a full discussion of measurement approaches is beyond the scope of this paper. Please see accompanying bibliography for additional information. The end result is the following report, with which we hope to provide a framework for considering impact definitions and a foundation for continuing discussion of this important issue. Going forward, this document will anchor our additional guidance on impact definition and its implications, both for donors and for the populations they aim to serve.

Table of Contents
I . I N T R O D U C T I O N : C L A R I F Y I N G AS S U M P T I O N S I N D E F I N I N G I M PAC T II.  T E C H N I CA L D E F I N I T I O N S : T H E P R O G R A M E VA L UAT I O N P E R S P E C T I V E I I I . C O L LO Q U I A L D E F I N I T I O N S : D O N O R A N D N O N - P R O F I T P E R S P E C T I V E S Foundations Individual Donors Impact Investors Non-profits I V. M I S S I N G D E F I N I T I O N S : T H E B E N E F I C I A RY P E R S P E C T I V E V. B R I N G I N G A G E N D E R L E N S TO I M PAC T D E F I N I T I O N S From attribution to contribution Measurability Time horizon bias Tendency to define impact as positive Who defines impact? V 1 . C O N C LU S I O N A N D R E C O M M E N DAT I O N S 1 2 2 3 3 4 5 6 7 7 7 8 8 9 9

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

Bibliography Acknowledgements

11 I N S I D E B AC K C OV E R

W H AT A R E W E TA L K I N G A B O U T W H E N W E TA L K A B O U T I M PAC T ?

IMPACT
Noun: a) the action of one object coming forcibly into contact with another; b) the effect of one person, thing or action on another. Verb: a) come into forcible contact with another object; b) (impact on) to have a strong effect on someone or something. From Latin impingere, to push against, drive infirst use early 17th Century.
Oxford English Dictionary
W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

S O M E E X A M P L E S O F U SAG E

As I have said, the first thing is to be honest with yourself. You can never have an impact on society if you have not changed yourself.Great peacemakers are all people of integrity, of honesty, but humility. Nelson Mandela, Statesman, Former President of South Africa If you think you are too small to have an impact, try going to bed with a mosquito. Anita Roddick, Founder, The Body Shop We test our assumptions and track achievements by measuring outcomes and impacts, and by understanding how and why we have succeeded or failed. Outcomes are the immediate, intended or achieved changes in technologies, systems, populations or behaviors; impacts are the ultimate, sustainable changes that we seek. The Bill and Melinda Gates Foundation Everyones got their own definition of impact and heres ours: impact is a change in the state of the world brought about by an intervention. Its the final result of behaviors (outcomes) that are generated by activities (outputs) that are driven by resources (inputs). Kevin Starr and Laura Hattendorf, Mulago Foundation (Impact) is the overall differences an organization, programme, or intervention makes. We use the terms impact and results interchangeably throughout this report. Dan Corry, New Philanthropy Capital, London Impact = ideas x implementation. Caroline Fiennes, Giving Evidence My life is not to be somebody elses impact you know what I mean? Alice Walker, novelist, author of The Color Purple

vi

T H E C E N T E R F O R H I G H I M PAC T P H I L A N T H R O P Y A N D W O M E N M OV I N G M I L L I O N S

I . I N T R O D U C T I O N : C L A R I F Y I N G AS S U M P T I O N S I N D E F I N I N G I M PAC T Looking across the quotations at the beginning of this paper, one sees the words effect, change, differences, and results. These quotes reflect the fact that for most people, impact implies a change brought about by some sort of action. Moreover, with the exception of the perspective represented by the Alice Walker quote, the effect or change is generally presumed to be positive. The accuracy of these assumptions is explored further in this paper, but our analysis indicates that they are widespread. The type of change and the relationship between action and change is explored further in the paper, but five key points have emerged from our analysis as areas where misunderstandings are common. To help frame the discussion going forward, we make the following observations: Actions can fail to produce change due to a host of factors. The absence of observed change does not necessarily indicate an ineffective action (see third observation below). Actions can produce unanticipated changes, including negative ones. Impact is not always positive, and negative impact is not always evidence of an ineffective intervention. Actions which prevent a particular change, even if they do not change the overall status quo, can still have an impact. Maintaining the status quo is an impact, if the alternate scenario is a worsening situation-- for instance, blowing up a meteor which otherwise would destroy a city. Change can occur and be observed independent of a particular action. Measuring outcomes and impacts can happen even without measuring a particular interventions contribution or lack thereof. The moon can be observed rising, whether or not a pack of wolves howls at it. Impact sought is subjective; it is defined by a person or group and for a person or group. Impact definitions are not abstract, objective truths. They are the product of decisions made by people and organizations, and they often aim to change behaviors or situations for those on the receiving end of an intervention. This is not always problematic, but it must be recognized in order to account for potential bias or disempowerment. It is also important to clarify some of the vocabulary around impact. Many discussions of impact explicitly or implicitly refer to what evaluation professionals call the impact, value, or results chain, representing actions and resources along with their expected effects. Those effects are described as outcomes, which lead to impacts. An example of a results chain for lowering malaria prevalence is presented below:

W H AT A R E W E TA L K I N G A B O U T W H E N W E TA L K A B O U T I M PAC T ?

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

While the example may appear straightforward, of change attributable to an intervention. Someachieving clarity about what is impact versus out- times there is also a temporal distinction made, come can become very difficultfor instance, if with impact being defined as longer-term, sustainyou raise the average income of women within able change, as opposed to shorter-term outcomes. the community, is that impact? Or is it an intermediate outcome, contributing to the larger impact of raising womens status within that commu- One must look beyond outcomes to ask whether nity? There is no single right answer; it is simply the strategy actually had impact. Although an necessary to define impact clearly at the start. organization and its funders may rightly take Relatedly, definitions can also shift depending on the value of their work depends on whether the where an organization or actor is located within a outcome would or would not otherwise have particular chain; many groups and interventions occurred. work toward the same goals, and each group may Paul Brest, The William and Flora define their outcomes and impact within a single Hewlett Foundation smaller portion of the broader impact chain. Finally, to add to the general confusion, the two terms are ofTo slightly oversimplify, [impact is] the results that ten confounded or used interchangeably in practice. 2 . T E C H N I CA L D E F I N I T I O N S : T H E P R O G R A M E VA LUAT I O N PERSPECTIVE While there is divergence amongst program evaluation experts with regard to the methodology of impact measurement, we found a modest but broad consensus with regard to definitions. For most program evaluation experts, the definition of impact is quite specific, and differs in important ways from colloquial uses common outside of the evaluation world. In particular, technical definitions address the issue of causality, or attribution of results to a specific intervention. For this group, the causal link between action and change must be tight, ideally measurable, and generally must include consideration of what might have happened in the absence of the action.4 The more definitively causality can be established, the more rigorous the evaluation of impact. Impact for this group is also differentiated from outcomes: it is an overall assessment
program or collective of programs, as determined by evaluations that are capable of factoring out (at a high level of statistical probability) other explanations for how these results came to be. pleasure in seeing their intended outcome occur,

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

can be directly attributed to the outcomes of a given

- Mario Morino, Venture Philanthropy Partners 3 . C O L LO Q U I A L D E F I N I T I O N S : DONOR AND NON-PROFIT PERSPECTIVES In reviewing colloquial definitions and uses of impact, we considered the stakeholders commonly involved in philanthropic donations or investments, such as foundations, individual donors, non-profits, and social impact investors (beneficiary perspectives are discussed in the following section). In contrast to the relative consensus found among program evaluation experts, these groups demonstrated considerable diversity regarding the use and implied definitions of the word impact.

Well known figures who either share or at least deeply understand and discuss the technical perspective include Paul Brest, Caroline Fiennes, David Hunter, Elizabeth King, Ruth Levine, Tris Lumley, Mario Morino, Martin Ravallion, Sean Stannard-Stockton, Kevin Starr, Tom Tierney, and Howard White.
4

T H E C E N T E R F O R H I G H I M PAC T P H I L A N T H R O P Y A N D W O M E N M OV I N G M I L L I O N S

Foundations: As a first step in our scan, we reviewed the websites and publicly available evaluation materials from the 10 largest foundations in the U.S. (by asset size). In the publicly available materials for these foundations, language around impact is common, though not universal, and there is variation in the ways that foundations apply the term to their work. When they exist, publicly available impact definitions are most often foundation-specific, and related to particular areas, indicators or outcomes that the foundation presumes to directly affect. Technical, conceptual definitions of impact (in the manner of evaluation professionals) are less common, though some impact-focused foundations are exceptions.5 There is a strong assumption that what the foundation is trying to achieve is a public good, and outcomes related to that good are evidence of the foundations impact. While in-depth discussions around mission definition, impact measurement, and related concepts may happen behind the scenes, public materials tend more towards the colloquial: something good, not necessarily a measurable, attributable change. When they do discuss their own impact, foundations often explicitly reference outcomes of grantees. Focusing at the outcome level allows foundations to draw clear causal linksthey provide dollar inputs, and their grantees return these outcomes. In some ways, this approach is also attributionfocused, but the limits of measuring attribution set the boundary of impact definition, rather than the definition driving the limits of measurement. This approach is not always entirely satisfying, as this quotation from the Ford Foundation suggests:

No single grant or grantee can, on its own, bring about the kinds of broad social change we seek, such as improving the transparency and effectiveness of government, preventing gender discrimination, or enabling families to move out of poverty. For this reason, we assess the effectiveness

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

of our work at multiple levels: strategic initiative, general approach, and individual grant.

The Ford Foundation In this case, the foundation distinguishes between the impact it seeks and the set of outcomes any individual organization is poised to deliver. Individual Donors: Individual donors are a very heterogeneous group. A number of studies over the past few years, however, have attempted to draw at least broad conclusions regarding high net worth donor characteristics and trends.678 Although donors may not use the word impact, they are clearly concerned with results:
Most donors, regardless of the vocabulary they use, want their donations to produce results. What characterizes results may be very different to various donors. Sometimes the desire to see results can lead donors to seek indicators, like low overhead expense ratios, that are actually bad proxies for results. Sometimes the result a donor seeks might simply be public recognition. But believing that donors do not seek results is akin to believing that they would be just as happy throwing their money in the trash.

Linda Raftree, Special Advisor to the Rockefeller Foundations Evaluation Office9

Note the definitions by the Gates and Mulago foundations in the beginning quotations section, for example. Katherina M. Rosqueta, Kathleen Noonan, and Miriam Shark. Im Not Rockefeller: Implications for Major Foundations Seeking to Engage Ultra- High-Net-Worth Donors, 2011. The Foundation Review: Vol. 3: Iss. 4, Article 9. 7 Hope Consulting. Money for Good II: Driving Dollars to the Highest-Performing Non-Profits , 2011. http://www.guidestar.org/ViewCmsFile.aspx?ContentID=4038 8 Bank of America and The Center on Philanthropy at Indiana University. The 2012 Bank of America Study of High Net Worth Philanthropy: Issues Driving Charitable Activities Among Wealthy Households, November 2012. http://newsroom.bankofamerica.com/files/ press_kit/additional/ 2012_BAC_Study_of_High_Net_Worth_Philanthropy_0.pdf. 9 Sean Stannard-Stockton Getting Results: Outputs, Outcomes & Impact. Tactical Philanthropy. June 29, 2010. http://www.tacticalphilanthropy.com/2010/06/outputs-outcomes-impact-oh-my/ Note also findings of 2012 Cygnus study: http://www.cygresearch.com/ files/free/Exec_Summary-The_Cygnus_Donor_Survey_2012-US.pdf
5 6

W H AT A R E W E TA L K I N G A B O U T W H E N W E TA L K A B O U T I M PAC T ?

In comparison with foundations which tend to be more explicitly mission-driven, a preponderance of donors are concerned with giving to nonprofits that speak to a personal interest. Results or impact for them may be tied not just to what that organization accomplishes in terms of outcomes, but also how involvement in that organization changes them, or makes them feel. Impact, therefore, may be a two-way street. That said, a 2012 study found that high net worth individuals gave more to those non-profits that they felt to be high impact (however they defined the term)10, and there are also indications that concerns about how to best identify those organizations and evaluate their worthiness are holding back investment.11
I think there is a lot of capital locked up right now because of exactly that problem that people just dont feel like they know how to evaluate.

Clearly, the latest buzz is about impact and who wouldnt like that? I mean, after all, who would launch a strategy with a focus on being ineffectual? Incompetent investing just doesnt have the same ring to it as impact investing. We want to do more than simply earn a financial return we want to actually change the world, and so for better and in some ways worse, the word of the day is: impact!

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

Sean Stannard Stockton Leaving aside the frustration, there are in fact lots of open questions regarding the definition of impact that impact investors are seeking. There is a prevailing assumption that it must be some kind of social good.
Impact investing is investing to achieve both financial return and a positive impact on social problems.

high net worth individual cited in: Im Not Rockefeller, Center for High Impact Philanthropy Impact Investors: Impact investors can be foundations, individual donors, or, depending on ones definition or impact, any organization that makes investments. There is a subset of thinkers for whom the term impact investing is an example of all that is wrong in the definition debates:
All organizations regardless of their aims and objectives have an impact, and so, for the noninitiated, impact isnt and cannot be something unique to social purpose organizations or their investors.

One of a number of diverse twitter responses to recent Calvert Foundation call for impact investing definitions

In some cases, this impact is seen as a bonus, with a distinction made from a traditional investment (which nonetheless may include social goods such as employment and income earning potential for certain groups).

Jeremy Nicholls, SROI Network

Bank of America and The Center on Philanthropy at Indiana University. The 2012 Bank of America Study of High Net Worth Philanthropy. 11 Money for Good II found that better communication around results could lead to significant additional capital moving towards highly effective nonprofits. http://www.guidestar.org/ViewCmsFile.aspx?ContentID=4038 12 Defining Impact Investing. Calvert Foundation Blog, August 13, 2013. http://www.calvertfoundation.org/blog/428-definingimpact-investing.
10

T H E C E N T E R F O R H I G H I M PAC T P H I L A N T H R O P Y A N D W O M E N M OV I N G M I L L I O N S

In others, a comparison is drawn to a traditional charitable donation, and the standards for anticipated impact are raised.13 There is also currently no consensus as to what kind of financial return an impact investment must yield to be considered as such. Non-profits: Non-profits generally depend at least partially on foundations and individual donors for support, so it should not be surprising that they are attentive to donor definitions of impact. Donor definitions and non-profit definitions are intimately related; the way that grantees define their own impact informs how funders can define theirs, and the way that a funder wants to define their impact can inform a non-profits impact definition and assessment approach. Things get sticky, however, when different donors push different definitions, or when the definitions most useful to the non-profit are different from those of a key donor. CHIP and the Wharton Social Impact Initiative recently interviewed four high-performing, wellresourced non-profits on the subject of impact assessment, performance management, and communication with funders around the distinctions, definitions, and best uses of each. Within this highperforming group, there is an understanding that their work supports big-picture, long-term impacts, but they felt that shorter-term, specific outcomes were more relevant to their needs, as they were immediately related and attributable to their work. The leaders of these non-profits felt that measuring outcomes provided the kind of feedback that they needed to manage their organizations performance, while measuring impact delivered results that were (a) less reliable, due to the methodological issues inherent in measuring further along the value chain; and (b) less useful, in terms of immediate relevancy to the organizations decision-making and activities.14

The same non-profits, however, described pressure from funders to measure and report all the way out to impact. While this tension is not solely a definitional issue, definitions are an underlying component; when funders and grantees are not aligned on exactly what they are trying to achieve and how they will monitor progress towards those achievements, it is difficult to align at other decision points throughout the engagement.

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

The Importance of Establishing Causality: Push the measurement or pull back the impact definition?15
Fo r s takeh o l ders w h o bel i eve th at unders tan di n g c au s al l i n kag es i s c ent ral to u n ders tan di n g i m pac t, th ere are at l eas t t w o di rec ti o n s to g o. On e i s to at tem pt to ex ten d th e c h ai n o f at t rib u ti o n -- o f ten at h i g h c os t, w i th i m pre c i s e o r u n rel i abl e resu l ts. Th e o th er is to s ay th at bec au s e at t ri bu ti o n i s s o c en t ral to i m pac t, an d bec au s e it is di f fi c u l t to m easu re, i m pac t defi n i tion an d/o r as s es s m en t s h o u l d be fo c u s ed c l os er to w h at c an ac tu al l y be at t ribu ted. Th e fi rs t appro ac h c arri es the ri s k o f i n ef fi c i en t res o u rc e al l o c ation, h i g h o ppo r tu n i t y c os ts, an d u n rel i a b le resu l ts; th e s ec o n d c arri es th e ri s k of m i s s i n g bi g pi c tu re ef fec ts (pos i ti ve or n eg ati ve), an d do es n o t c o n t ri bu te to th e ev i den c e bas e aro u n d i m pac t of a par ti c u l ar i n ter ven ti o n . Th es e do wns i des c an be es pec i al l y rel evan t for w o m en an d g i rl s, as i n ter ven ti o n s that are n o t c o n s i dered par t o f th e w o men an d g i rl s s pac e c an n ever th el es s have i m po r tan t i m pac ts o n w o m en an d girls as a g ro u p; th es e i m pac ts (pos i ti ve or n eg ati ve) w i l l n o t be c aptu red with o u t del i berate ef fo r ts to m easu re impac t fo r w o m en an d g i rl s s pec i fi c ally.

Acumen Fund Acumen Fund Concept Paper: The Best Available Charitable Option (BACO), 2009. http://acumen.org/wp-content/uploads/2007/01/BACO%20Concept%20Paper_01.24.071.pdf. 14 The Center for High Impact Philanthropy and Wharton Social Impact Initiative. A Donors Guide: Working with Nonprofits to Measure Performance and Manage to Impact. Forthcoming. 15 See also Jason Saul, The Dirty Little Secret About Measurement. Mission Measurement Blog, March 22, 2010. http://missionmeasurement.com//ideas/blog_entry/the-dirty-little-secret-about-measurement1
13

W H AT A R E W E TA L K I N G A B O U T W H E N W E TA L K A B O U T I M PAC T ?

As with the more technical group, attribution is central to the way these non-profits think about impact but the reasoning leads them to a different place. Rather than trying to extend the attribution chain all the way out to impact, the impact focus shifts to where the chain of attribution ends. Outputs and outcomes are directly affected by the organizations activities, and focusing measurement efforts on those components therefore returns data that can directly inform their organizational decisions. 4. MISSING DEFINITIONS: THE B E N E F I C I A RY P E R S P E C T I V E

impact, in any sense beyond the technical definition? The market dynamics of philanthropy encourage this top-down approach. Funderswhether individual or institutionalare in some sense the customers for the services of non-profits, even though they are not the ultimate consumers.

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

My life is not to be somebody elses impact

you know what I mean? Alice Walker, novelist, author of The Color Purple

[Beneficiaries]

arent

buying

your

service;

It has been noted that beneficiaries are often left out of the impact measurement process; this may be due in part to a growing focus on externally verifiable metrics, such as income, that can be more easily captured and assessed for statistical significance.16-18 While this is an important issue, the question of impact definition in some ways precedes it. Inclusion of beneficiaries in the definition of impact will naturally lead to an inclusion in the measurement approach. Conversely, leaving beneficiaries out of the definition process makes exclusion from measurement logicalonce your desired impact is defined, there is usually little benefit to measuring indicators that are not directly related. Some suggest that even the idea of impact inherently infers a power imbalance. Impact is active, implying a passive counterpartthe donor or organization is acting, while the beneficiary is being acted upon. It is important to note that even in this discussion, we are still relying on experts discussing beneficiaries, rather than direct reports from beneficiaries themselves.19 The scarcity of resources reporting directly from beneficiaries demonstrates the gap in opportunities for direct communication. This absence of voice, and the accompanying issues of power dynamics, are central to the gender lens discussion in the following section.

rather a third party is paying you to provide it to them. Hence the focus shifts more toward the requirements of who is paying versus the unmet needs and aspirations of those meant to benefit.

Daniel Stid, Bridgespan

Many social programs operate with a top-down approach; socioeconomic data indicate a need in a certain population, or a donor has a particular connection to a topic. Funders provide resources to non-profits, who implement a program designed to address the original need; they may also collect data to demonstrate how well their program is alleviating that needand that may lead to an understanding of that programs impact. But what if the target population would actually identify their need differently? Is it still
16 17

Twerksy, Buchanan, and Threlfall. Listening to Those Who Matter Most. Mario Morino, Leap of Reason: Managing to Outcomes In an Era of Scarcity. First Edition. Venture Philanthropy Partners, 2011. 18 Liz Allen, Opinion: Measure for Measurings Sake? Wellcome Trust Blog, January 31, 2013. http://blog.welcome. ac.uk/2013/01/31/opinion-measure-for-measurings-sake/. 19 Drawn from Fay Twersky, Phil Buchanan, Valerie Threlfall, Jeremy Nicholls, Daniel Stid, Rosalind Eyban, and others.

T H E C E N T E R F O R H I G H I M PAC T P H I L A N T H R O P Y A N D W O M E N M OV I N G M I L L I O N S

5 . B R I N G I N G A G E N D E R L E N S TO I M PAC T D E F I N I T I O N S The liveliness of the impact discussion is partly a function of increasing pressure for interventions to demonstrate value, often quantitativelygetting impact definitions right, in many cases, corresponds to getting dollars. Assumptions about impact therefore have direct implications for the women and girls space. The feminist evaluation literature suggests that traditional approaches to impact fall short in several key areas.20 In this section, we pick up five themes coursing through the previous discussion of impact definitions and examine them more closely through a gender lens. They are: 1) the issue of attribution, or causality; 2) measurability; 3) the tendency to define impact positively; 4) how the issue of time plays into impact definition; and 5) the question of who defines impact.

the hill, but no one person can know how much they contributedor even whether they were necessary to achieve the outcome.21 Organizations involved in this kind of effort may have difficulty communicating the value of their work to audiences more focused on identifying directly attributable outcomes. On the other hand, the notion of collective impact, and the idea of many parts contributing to something larger than the sum, may appeal to others.22

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

From attribution to contribution: As noted in earlier sections, concepts of attribution and causality are central to definitions of impact for many in the field of impact research. Much of the work in the women and girls space is about building collective impact to reach a critical mass. The end goal may in fact be measurable and quantifiable in some respect (for instance, the passage of a land rights bill) but the extent to which any one organization or program contributed to that outcome may be unknowable, or at least difficult to measure quantitatively.
Illustrating this principle as it relates to political advocacy, Paul Brest gives the example of a group of people pushing a boulder up a hill. Its easy to see when the group has succeeded in reaching the top of

Measurability: While much of the current debate around impact and measurement issues is beyond the scope of this paper, there are important ways in which measurement can inform impact definition, rather than an impact definition driving measurement choices. A bias toward quantification may bias definitions of impact toward more easily quantifiable goals: for instance, increasing income among female earners rather than increasing the status of women in society.23
Beyond the issues around measuring shared contribution and bias toward more easily quantifiable goals, there is mistrust around measurement in the gender in development literature because the impact of a wide variety of programs on women and girls has so frequently been overlooked. If beneficiary voices are almost never present in impact definitions, women and girls experiences often never even make it into impact assessment. All results for beneficiaries are assumed to be the same, and women and girls are effectively invisible. There is in fact ample evidence that program effects for women and girls are not the same as they are for men and boys. For example, in many coun-

20 This section draws from the literature on gender in development (Srilatha Batliwala, Alexandra Pittman, Aruna Rao, David Kellaher, and others) as well as materials from foundations and groups serving women and girls specifically (Global Fund for Women, International Network of Womens Funds, Womens Funding Network, etc.) 21 Paul Brest, Risky Business. Stanford Social Innovation Review, Summer 2012. http://www.ssireview.org/articles/entry/risky_business_2. 22 For examples of current work around collective impact measurement, see Faye Hanleybrown, Shared Measurement for Collective Impact. FSG Social Impact Blog, September 14, 2011. http://www.fsg.org/KnowledgeExchange/Blogs/SocialImpact/PostID/166. aspx. and resources collected in this piece from John Kania and Mark Kramer. Collective Impact. Stanford Social Innovation Review, Winter 2011, http://www.ssireview.org/articles/entry/collective_impact. 23 The RobinHood foundation approach to grant-making, for example, equates impact across different kinds of programs to the expected income and earning gains of program beneficiaries.

W H AT A R E W E TA L K I N G A B O U T W H E N W E TA L K A B O U T I M PAC T ?

tries, girls and women bear primary responsibility for obtaining water. Installing an improved water and sanitation system may improve primary health indicators for all, but it also affects women and girls time (usually positively) and freedom (sometimes negatively, as the time spent travelling to the water source is often spent with friends and peers, away from the directives of other members of the family). The point is you dont know if you dont measure. While one response to these past omissions is to be wary of measurement and quantification, the other is to embrace it, insisting that any effort to define and assess impact across all sectors must include a gender lens. For advocates of the latter approach, this is not just a women and girls issue, but is rather the way that development needs to think about interventions in order to do more good than harm.

Tendency to define impact as positive: As seen in the previous discussion, in many if not most definitions of impact, positive change is assumed as the ultimate goal; language around progress and gaining is common. An observed change is the foundation of attributionthe change is what is attributed--and the status quo (lack of change) is the default. Advocates for gender-lensed evaluation point out that
When you work for womens interests, its two steps forward if youre really smart and very lucky! and at least one step back. In fact, its often two or three steps back! And those steps back are, ironically, often evidence of your effectiveness; because they represent the threat you have posed to the power structure and its attempt to push you back. Sometimes, even your success stories are nothing more than ways the power structure is trying to accommodate and contain the threat of more fundamental change by making small concessions.

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

Time horizon bias: Impact definition and assessment in the women and girls arena is also frequently subject to a temporal bias. Evaluations are most often grant-fundedwhether as part of a program budget or as a distinct initiativeand as such are subject to the same limitations of many other grants. In particular, the vast majority of grants are time-limited; for evaluation, this means that investigators have a certain number of years to collect data, and those data will ideally demonstrate the extent to which a given program has achieved impact. However, important initiatives for women and girls often work towards structural shifts in society; these are big, amorphous goals that are difficult to capture in a typical 2-, 3- or 5-year evaluation cycle. To take the example of the suffragettes, gaining the vote was a decades-long process; had there been a typical 5- or even 10-year impact evaluation conducted in the midst of that movement, it likely would not have captured a great deal of progress. Again, setting interim outcome goals without losing track of ones ultimate objective (impact sought) may be one way of compensating for temporal bias.
24

Sheela Patel, SPARC India25 this assumption can be problematic for the women and girls sector, where positive impact may simply be holding the line.24 For instance, keeping a womens health clinic open despite organized opposition may be evidence of effective work, but a positive changefocused approach would not capture that benefit. Impact can even be negative; when work is challenging power structures, there may be a backlash in the face of progress. This would traditionally be taken as evidence that an intervention isnt working, but may in fact be the opposite: This problem can be addressed in part by choosing an appropriate counterfactual, taking the overall landscape into account. If the counterfactual is a movement away from the desired outcome, an intervention that holds the line will demon-

Srilatha Batliwala and Alexandra Pittman. Capturing Change in Womens Realities: A Critical Overview of Current Monitoring & Evaluation Frameworks. AWID Womens Rights , June 2010. 25 Batliwala, Pittman. Capturing Change in Womens Realities.

T H E C E N T E R F O R H I G H I M PAC T P H I L A N T H R O P Y A N D W O M E N M OV I N G M I L L I O N S

strate value. A gender-lensed approach to evaluation will assess the context around an intervention before assuming that impact will be an observable change and assigning a counterfactual accordingly.

impact has given rise to additional common misconceptions, and these misconceptions contribute to much of the gender bias in impact thinking. First, there is a perception that impact is always changeand secondly, that the change is always positive. Neither of these ideas hold true in every circumstance, particularly for interventions relating to women and girls, where holding the line (perhaps preventing the passage of a restrictive law) may represent an enormous accomplishment. Defining impact as positive change will undervalue the contributions of many effective organizations working on hard problems or under difficult conditions. A third common misconception is that there are some impacts that simply cant be measured; this stems in part from a fourth misconception, which is that impact must be attributed to a particular actor or action. In reality, some impactssuch as a change in attitudes towards womenare very difficult or perhaps impossible to attribute to specific causes, but the impact itself can still be measured, even without a clear causal attribution. This is especially relevant for the transformative social change that many in the women and girls space seek; these large-scale, systemic shifts can rarely be attributed to a single group or intervention, but they are visible and measurable nonetheless. Defining impact as attributable change can exclude work done as a component of collective efforts, perpetuating the idea that those efforts are outside the realm of the impact-focused donor. With all of these considerations in mind, it is clear that there is no one right or common definition of impact for every person in every situation. Good definitions are inclusive, but above all they are useful in clarifying a path to action. In that spirit, we offer three questions for the impact-focused donor to ask throughout their philanthropic engagement:

Who defines impact? As noted earlier, beneficiaries are often left out of the impact conversation. This omission is particularly problematic for populations of women and girls who have historically had little opportunity to voice their needs. Perpetuating that dynamic may have negative effects apart from or alongside other, positive impact. Additionally, even positive impact may miss the mark if it is not defined with input from the target population. For example, Helen Gichohi, speaking about an intervention to support women entrepreneurs in Kenya, noted that the conversations during the planning process were all about supporting collectives of female entrepreneurs; nobody thought to consider whether or not women might want to go into business on their own, as single business owners.26 Ultimately, getting impact definitions right includes representing the voices of a group that has historically been voiceless: actively seeking out those voices will lead to more responsible and respectful work overall.
6 . C O N C LU S I O N A N D R E C O M M E N DAT I O N S As illustrated in the preceding pages, impact does not mean the same thing for every person or organization in philanthropy and international development. In other words, what we talk about when we talk about impact depends on who is talkingand who is listening! This confusion can be problematic: for example, in impact investing, uniform guidelines might release more capital, but the noise around impact definition makes it difficult to arrive at such a consensus; for non-profits, juggling conflicting donor definitions requires time and resources when both may be limited. In particular, the confusion around
26

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

1. What difference do I want to make? By asking this question, a donor can take the

Pioneers Post TV. Gazing Through the Gender Lens with Helen Gichohi, 2013. http://www.pioneerspost.com/pp-tv/20130809/gazing-through-the-gender-lens-helen-gichohi.

W H AT A R E W E TA L K I N G A B O U T W H E N W E TA L K A B O U T I M PAC T ?

first step towards defining their own desired impact clearly, openly, and deliberately.

2. Is that difference meaningful to the population I hope to serve? In other words, does my definition of impact align with others, particularly those I hope to help? This can be a difficult question to answer, and a full discussion of possible methods is beyond the scope of this paper. Personal engagement with beneficiaries is valuable for many reasons, including the opportunity to hear on-the-ground perspectives on impact.
For many donors, the bulk of their philanthropic dollars flow through intermediary organizations. Broadly speaking, these organizations can incorporate beneficiary perspectives via: (1) a specific focus on women and girls, (2) representation from beneficiary groups in their leadership structures, and (3) impact assessments that evaluate impact on women and girls specifically, ideally using an approach that allows for participation from the populations served. This list is by no means exhaustive, but may provide a starting point for the donor looking to bring a bottom-up approach to their impact definitions.

3. How will I know if Ior weare moving closer to making that difference? With this question, a donor can make deliberate decisions to guide their measurement approach in a way that reflects their desired impact: is attribution important? Are effects on women and girls addressed specifically, with enough flexibility to capture positive, negative, or neutral impacts? What is the expected timeframe?
These three questions offer concrete ways to incorporate impact thinking into philanthropic decisionmaking, giving donors more confidence to invest in interventions that benefit women and girls in a range of ways. It is our hope that by outlining different ways of thinking about impact, all actors in the spaceresearchers, donors, non-profits and beneficiariesmay have clearer and more rewarding conversations that, in turn, may lead to more money doing more good.

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

10

T H E C E N T E R F O R H I G H I M PAC T P H I L A N T H R O P Y A N D W O M E N M OV I N G M I L L I O N S

BIBLIOGRAPHY
About the Foundation: FAQs. The David and Lucille Packard Foundation, 2013. http://www. packard.org/about-the-foundation/foundation-faq. About the Getty Foundation: Mission. The Getty Foundation, 2013. http://www.getty.edu/ foundation/about/. Acumen Fund. Acumen Fund Concept Paper: The Best Available Charitable Option (BACO), 2009. http://acumen.org/wpcontent/uploads/2007/01/ BACO%20Concept %20Paper_01.24.071.pdf. Allen, Liz. Opinion: Measure for Measurings Sake? Wellcome Trust Blog, January 31, 2013. http://blog.wellcome.ac.uk/2013/01/31/opinion-measure-for-measurings-sake/. Association for Womens Rights in Development. Strengthening Monitoring and Evaluation for Womens Rights: Twelve Insights for Donors. Edited by Srilatha Batliwala. Toronto: AWID, 2011. Approach. Skoll Foundation, 2013. http://www.skollfoundation.org/approach/. Bank of America and the Center on Philanthropy at Indiana University. The 2012 Bank of America Study of High Net Worth Philanthropy: Issues Driving Charitable Activities Among Wealthy Households, November 2012. http://newsroom.bankofamerica.com/files/press_kit/ additional/2012_BAC_Study_of_High_Net_Worth_Philanthropy_0.pdf. Batliwala, Srilatha, and Alexandra Pittman. Capturing Change in Womens Realities: A Critical Overview of Current Monitoring & Evaluation Frameworks. AWID Womens Rights, June 2010. Brest, Paul. Risky Business. Stanford Social Innovation Review, Summer 2012. http://www.ssireview.org/articles/entry/risky_business_2. Brest, Paul, Hal Harvey, and Kelvin Low. Calculated Impact. Stanford Social Innovation Review, Winter 2009. http://www.ssireview.org/articles/ entry/calculated_impact. Buchanan, Phil. Toward a Common Language: Listening to Foundation CEOs and Other Experts Talk About Performance Measurement in Philanthropy. Center for Effective Philanthrop , 2002. http://www.effectivephilanthropy.org/assets/pdfs/CEP_TowardsACommonLanguage.pdf. Burk, Penelope. The Cygnus Donor Survey: Where Philanthropy Is Headed in 2012. Chicago / Toronto / York, UK: Cygnus Applied Research Inc., June 2012. http://www.cygresearch.com/ files/free/Exec_Summary-The_Cygnus_Donor_Survey_2012-US.pdf. The Center for High Impact Philanthropy and Wharton Social Impact Initiative. A Donors Guide: Working with Nonprofits to Measure Performance and Manage to Impact. Forthcoming. Colby, Susan, Nan Stone, and Paul Carttar. Zeroing in on Impact. Stanford Social Innovation Review. Fall 2004. http://www.ssireview.org/articles/ entry/zeroing_in_on_impact. Davis, Patrick. Defining Impact Investing. Calvert Foundation Blog, August 13, 2013. http://www.calvertfoundation.org/blog/428-definingimpact-investing. Emerson, Jed. Jed Emerson on Impact Investing. Tactical Philanthropy, August 1, 2011. http://www.tacticalphilanthropy.com/2011/08/jed-emerson-on-impact-investing/. Evaluating Our Strategies. MacArthur Foundation, 2013. http://www.macfound.org/learning/ strategy/. Fiennes, Caroline. Most Charities Shouldnt Evaluate Their Work: Part One. Stanford Social Innovation Review Blog. May 29, 2013. http://www. ssireview.org/blog/ entry/most_charities_shouldnt_evaluate_their_work. Fiennes, Caroline. Most Charities Shouldnt Evaluate Their Work: Part Two. Stanford Social Innovation Review Blog. May 30, 2013. http://www. ssireview.org/blog/entry/most_charities _shouldnt_evaluate_their_work1. Fleischman, Joel, and Tierney Thomas. Give Smart: Philanthropy That Gets Results. Public Affairs, 2011. http://givesmart.org/success/definingsuccess.aspx. GEO, and Council on Foundations. Evaluation in Philanthropy: Perspectives from the Field, 2007. http://www.hfcm.org/CMS/images/GEO.Evaluation%20in%20Philanthropy.pdf. Fleischman, Joel, and Thomas Tierney. What Is Impact Investing and Why Should or Shouldnt Philanthropists Consider It? The Bridgespan Group, 2011. http://philanthropy.bridgespan.org/ getattachment/ccc20975-c3fa-4223-b732-b4cdcf7a018a/What-Is-Impact-Investing_Should-You-Consider-It.aspx. Guiding Principles. The William and Flora Hewlett Foundation, n.d. http://www.hewlett.org/about-us/values-policies/guiding-principles. Hanleybrown, Fay. Shared Measurement for Collective Impact. FSG Social Impact Blog, September 14, 2011. http://www.fsg.org/KnowledgeExchange/Blogs/SocialImpact/ PostID/166.aspx. Hedley, Sarah, et al. Talking About Results. New Philanthropy Capital, September 22, 2010.

W H AT A R E W E TA L K I N G A B O U T W H E N W E TA L K A B O U T I M PAC T ?

11

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

Hope Consulting. Money for Good II: Driving Dollars to the Highest-Performing Non-Profits, 2011. http://www.guidestar.org/ViewCmsFile. aspx?ContentID=4038. How We Develop Strategy. Bill and Melinda Gates Foundation, 2013. http://www.gatesfoundation.org/How-We-Work/General-Information/HowWe-Develop-Strategy. How We Measure Success. Bill and Melinda Gates Foundation, 2013. http://www.gatesfoundation.org/How-We-Work/General-Information/HowWe-Measure-Success. How We Think About Impact. The Mulago Foundation. n.d. http://mulagofoundation.org/ ?q=ideas/r/how-we-think-about-impact.

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

Impact: Our Approach to Impact Assessment and Learning. Ford Foundation, 2013. http://www.fordfoundation.org/impact/approach. Impact. Ford Foundation. 2013. http://www.fordfoundation.org/impact. Investing For Good. Social Investment Jargon Buster. Pioneers Post, May 14, 2013. http://www.pioneerspost.com/dealspace/20130514/social-investment-jargon-buster. Kania, John and Mark Kramer. Collective Impact. Stanford Social Innovation Review. Winter 2011, http://www.ssireview.org/articles/entry/collective_impact. Keystone Accountability. Impact Planning, Assessment and Learning an Overview (IPAL Guide 1) , n.d. http://www.world-federation.org/NR/ rdonlyres/42961DA8-CD5F-4C37-A869-163C28E62C70/0/NGO_8p1_KAImpactPlanning.pdf. More Detail and Examples - What You Want to Achieve. Jargonbuster, 2013. http://www. jargonbusters.org.uk/more-detail-and-examples/. King, Elizabeth M., and Jere R. Behrman. Timing and Duration of Exposure in Evaluations of Social Programs. The World Bank Research Observer 24, no. 1 (21, 2009): 5582. Lilly Endowment, Inc. Annual Report - 2001, 2001. http://www.lillyendowment.org/ annualreports/LEAR2001_pdf/2001-Cover.pdf. Lumley, Tris. Raising the Bar on Nonprofit Impact Measurement. Stanford Social Innovation Review Blog, July 10, 2013. http://www.ssireview.org/ blog/entry/raising_the_bar_on_nonprofits _impact_measurement. McCreless, Michael, and Brian Trelstad. A GPS for Social Impact. Stanford Social Innovation Review. Fall 2012. http://www.ssireview.org/articles/ entry/a_gps_for_social_impact. Morino, Mario. Comment: High Performance Vs. High Impact Nonprofits. Tactical Philanthropy, July 31, 2009. http://www.tacticalphilanthropy. com/2009/07/high-performance-vs-high-impact-nonprofits/. Morino, Mario. Leap of Reason: Managing to Outcomes In an Era of Scarcity. First Edition. Venture Philanthropy Partners, 2011. N gin, Eibhln, Tris Lumley, and David Pritchard. Making an Impact: Impact Measurement Among Charities and Social Enterprises in the UK. New Philanthropy Capital, 2012. Our Impact - Mission Driven Investing. W. K. Kellogg Foundation. http://mdi.wkkf.org/our-impact/overview.aspx. Ozumba, Michele. Want Real Disruption? 3 Reasons to Invest in Women. Huffington Post Impact Blog. April 29, 2013. http://www.huffingtonpost. com/michele-ozumba/want-real-disruption-3-re_b_3166394.html. Pioneers Post TV. Gazing Through the Gender Lens with Helen Gichohi, 2013. http://www.pioneerspost.com/pp-tv/20130809/gazing-through-thegender-lens-helen-gichohi. Robert Wood Johnson Foundation. 2011 Assessment Report. 2011. http://www.rwjf.org/en/ research-publications/find-rwjf-research/2011/11/ robert-wood-johnson-foundation-2011-assessment-report.html. Rosqueta, Katherine M, Noonan, Kathleen, and Miriam Shark. Im Not Rockefeller: Implications for Major Foundations Seeking to Engage UltraHigh-Net-Worth Donors, 2011. The Foundation Review: Vol. 3: Iss. 4, Article 9. Saul, Jason. The Dirty Little Secret About Measurement. Mission Measurement Blog, March 22, 2010. http://missionmeasurement.com//ideas/ blog_entry/the-dirty-little-secret-about-measurement1. Savedoff, William, Ruth Levine, and Nancy Birdsall. When Will We Ever Learn? Improving Lives Through Impact Evaluation. Washington D.C.: Center for Global Development, 2006. http://www.cgdev.org/files/7973_file_WillWeEverLearn.pdf. Seddon, Fred. Thinking You Do Good Work Is Not Enough. Pioneers Post, March 12, 2013. http://www.pioneerspost.com/dealspace/20130312/ thinking-you-do-good-work-not-enough. Stannard-Stockton, Sean. A Robust Definition of High Performance. Tactical Philanthropy, August 3, 2009. http://www.tacticalphilanthropy. com/2009/08/a-robust-definition-of-high-performance/.

12

T H E C E N T E R F O R H I G H I M PAC T P H I L A N T H R O P Y A N D W O M E N M OV I N G M I L L I O N S

Stannard-Stockton, Sean. Getting Results: Outputs, Outcomes & Impact. Tactical Philanthropy. June 29, 2010. http://www.tacticalphilanthropy. com/2010/06/outputs-outcomes-impact-oh-my/. Starr, Kevin, and Martin Fischer. A Brief Guide to High-impact Philanthropy. The Mulago Foundation, n.d. http://mulagofoundation.org/?q=ideas/ pwp/brief-guide-high-impact-philanthropy. Subramanian , Karti. Humans: Theres No App for That. Stanford Social Innovation Review: Blog, June 12, 2013. http://www.ssireview.org/blog/ entry/humans_theres_no_app_for_that. Twerksy, Faye, Phil Buchanan, and Valerie Threlfall. Listening to Those Who Matter Most, the Beneficiaries . Stanford Social Innovation Review. Spring 2012. http://www.ssireview.org/articles/entry/listening_to_those_who_matter_most_the_beneficiaries. Values & Policies: The William and Flora Hewlett Foundation Code of Ethics. The William and Flora Hewlett Foundation, 2013. http://www. hewlett.org/values. The William and Flora Hewlett Foundation. Outcome Focused Grantmaking: A Hard-Headed Approach to Soft-Headed Goals, March 2011. http:// www.hewlett.org/uploads/documents/ Outcome_Focused_Grantmaking_March_2012.pdf. White, Howard. Some Reflections on Current Debates in Impact Evaluation. International Initiative for Impact Evaluation, April 2009. http:// www.3ieimpact.org/media/filer/ 2012/05/07/Working_Paper_1.pdf. W.K. Kellogg Foundation. Logic Model Development Guide, January 2004. Womens Funding Network. Making the Case Overview. Webinar, January 2012. http://womensfundingnetwork.org/sites/wfnet.org/files/mtc1/ Webinar%20PowerPoint%20Handout.pdf.

W H AT A R E W E TA L K I N G A B O U T W H E N W E TA L K A B O U T I M PAC T ?

13

W O R K I N G PA P E R P R E S E N T E D AT W M M S U M M I T - S E P T E M B E R 2 0 T H , 2 0 1 3

AC K N O W L E D G M E N T S

This work was supported in part by a grant from The Jacquelyn and Gregory Zehner Foundation. We thank Katey Peck, Paul Marvucic and Lauren Kobylarz for their assistance on this project. This work also greatly benefitted from conversations with Jacquelyn Zehner and Jessica Houssian of the Women Moving Millions organization.

PREVIEW VERSION - WMM SUMMIT - SEPTEMBER 20TH, 2013

Please send comments about this guide to the Center for High Impact Philanthropy at impact@ sp2.upenn.edu. As the publisher of this report, we encourage the widespread circulation of our work and provide access to our content electronically without charge. You are welcome to excerpt, copy, quote, redistribute, or otherwise make our materials available to others provided that you acknowledge the Center for High Impact Philanthropys authorship. Should you have questions about this policy, please contact us at [email protected]. Copyright 2013 Center for High Impact Philanthropy

PREVIEW VERSION - WMM SUMMIT - SEPTEMBER 20TH, 2013

T h e C ent er for H ig h I mp ac t Phi l a nt hro py

Sch ool of Soci al Pol i c y & P racti ce | Unive rsit y of Pe nnsylva nia In collaboration with alumni from the Wharton School

3815 Walnut Street Philadelphia, PA 19104

WEBSITE www.impact.upenn.edu EMAIL PHONE

[email protected] (215) 573-7266

79 Fifth Avenue New York, NY 10003

WEBSITE www.womenmovingmillions.org