What Is Population Health Intervention Research?

Penelope Hawe, PhD,1 Louise Potvin, PhD2

ABSTRACT
Population-level health interventions are policies or programs that shift the distribution of health risk by addressing the underlying social, economic and
environmental conditions. These interventions might be programs or policies designed and developed in the health sector, but they are more likely to
be in sectors elsewhere, such as education, housing or employment. Population health intervention research attempts to capture the value and
differential effect of these interventions, the processes by which they bring about change and the contexts within which they work best. In health
research, unhelpful distinctions maintained in the past between research and evaluation have retarded the development of knowledge and led to patchy
evidence about policies and programs. Myths about what can and cannot be achieved within community-level intervention research have similarly held
the field back. The pathway forward integrates systematic inquiry approaches from a variety of disciplines.
Key words: Evaluation; population health intervention research; evidence-based practice; intervention research; population health
La traduction du résumé se trouve à la fin de l’article. Can J Public Health 2009;100(1):I8-I14.

T
here is an increasing move worldwide to shift the emphasis that operate within or outside of the health sector and have the poten-
of population health research away from purely descriptive tial to impact health at the population level.3
and analytic studies and towards the study of interventions
to reduce health problems and reduce health inequities.1 This We use the term “population health” in the way it is used in
requires an appreciation of the best that we have learned from the Canada to refer to the science underpinning the practice of public
diverse settings in which both health and social scientists have health and understandings about health that come only from an
been working. As far as possible we call for an integration of that appreciation of how health is generated in populations. However,
learning to assist in the development of the relatively new over- we recognize that in many countries the term “population health”
arching field of population health intervention research. is less used, and hence here “population health research” and “pub-
This paper outlines the practice of intervention research in pop- lic health research” can be taken to mean the same thing.
ulation health. We draw attention to a number of features that The definition refers to the use of scientific methods that have
mark problematic and unnecessary distinctions between interven- informed the development of many disciplines. In the case of pub-
tion research and evaluation research, arguing that these fields lic health, the original critical scientific developments were about
comprise very similar research practice and orientation. We discuss social statistics and virology.4,5 In public health the tradition of
the skill sets involved in this type of research and end by describ- intervention research is closely linked to that of experimental med-
ing and debugging common myths with regard to intervention icine, which goes back to the pioneer work of Claude Bernard. The
research. principles of experimental medicine as proposed by Bernard are to
systematically examine and, if possible, isolate the physiological
Defining intervention research consequences of actions undertaken in response to ill health and to
The definition of intervention draws from its Latin roots, venire, try to reproduce those consequences under various conditions. For
meaning to come and inter, meaning between, drawing attention Bernard, as for most scientists of his time (1870s), causality and sci-
from the outset that to intervene literally means to come in entific laws are only possible through the decomposition of the
between, to disturb the “natural” order of things or a foreseeable mechanism, understood as the sequence of events that produces
sequence of events. If we characterize descriptive or analytic research an effect. Altering the outcome of a sequence in a predicted direc-
in population health as seeking to understand phenomena, then tion constitutes evidence of the truth or validity of the scientific
intervention research is about testing those understandings by the proposition. Today, scientific methods from a variety of disciplines,
act of intervention in the causal mechanisms under investigation. including the social sciences, are included in the evaluator’s tool-
It is also about learning from the actions implemented to address box.
those phenomena in order to improve our practice. The iconic fig- The PHIRIC definition also points to interventions both inside
ure of John Snow removing the handle of the Southwark and and outside the health sector and is neutral on the intentionality
Vauxhall Company water pumps that he suspected were responsible Author Affiliations
for the London cholera outbreak2 is a dramatic example. 1. Population Health Intervention Research Centre, University of Calgary
2. Léa-Roback Research Centre on Social Health Inequality, University of Montreal
The Population Health Intervention Research Initiative for Correspondence and reprint requests: P. Hawe, Population Health Intervention
Canada (PHIRIC) defines population health intervention research Research Centre, University of Calgary, G012, 3330 Hospital Drive NW, Calgary, AB
T2N 4N1, Tel: 403-210-9383, Fax: 403-220-7272, E-mail: [email protected]
thus: Acknowledgements: Many thanks to Adria Rose and Erica Di Ruggiero for helpful
comments on an earlier draft. Penelope Hawe is a Health Scientist of the Alberta
Heritage Foundation for Medical Research and the Markin Chair in Health and Society.
Population health intervention research involves the use of scientific Louise Potvin is the holder of the CHSRF-CIHR Chair on Community Approaches and
methods to produce knowledge about policy and program interventions Health Inequality.

I-8 REVUE CANADIENNE DE SANTÉ PUBLIQUE • VOL. 100, NO. 1

of those events. If we confined ourselves to actions within the data-gathering around it than they are perhaps used to. Dual per-
health sector intended to improve health we would miss a great spectives are important and not incongruent.
deal. The definition reflects an interest in the social determinants We take the view that all systematic inquiry and learning from
of health – economic policy, education policy and environment observing an intervention’s process or implementation, impact or
policy. Actions in sectors outside health, designed for purposes outcome is encompassed in the term “intervention research”.
other than health, are often studied by people within health as Terms may always be used differently in different fields, but better
“natural experiments”, e.g., road construction, factory closures, research overall will derive from understanding the contributions
food market openings. Studies of the impact of such events are that have come from different vantage points. In this sense, rather
included in the PHIRIC definition of population health interven- than insisting on any particular language, it is better to pin down
tion research along with ongoing practices and policies in sectors the purpose of the research (e.g., testing effectiveness, elucidating
other than health that might affect population health. Evidence the process of action, documenting variations in implementation,
about population health impact has been successful so far in chang- tracking reach into populations with highest needs) and to ensure
ing practices in the motor vehicle construction industry,6 the food that the methods appropriately match that purpose.
and beverage industry7 and the petroleum industry.8
The final aspect of the definition is “impact at the population Is intervention research the same as evaluation
level”. The Canadian Institutes of Health Research, Institute of research?
Population and Public Health, reminds us that this does not simply Evaluation involves making judgements about the worth or value
mean improving health or reducing health risks but, rather, of something.18 Evaluation research is about the use of scientific
involves interventions intended to change the conditions of risk methods for that purpose.19 The focus of enquiry is to interpret an
in order to alter the distribution of health risk9 in keeping with the action and make a pronouncement about it, according to values or
ideas of Geoffrey Rose.10 To be truly effective, a population health standards that are pre-set (and usually enshrined in the goals and/or
intervention should be reducing risk exposure in successive cohorts objectives of the action). Evaluation is usually broken down into
of people within the setting(s) under investigation. components named variously but usually encompassing process
evaluation (how well an intervention is delivered, whether it reach-
Is intervention research the same as implementation es the intended target group), impact evaluation (immediate
research? effects) and outcome evaluation (subsequent or longer-term
Systematic observation built up around the roll-out of programs effects). The goals or objectives are enshrined within the sample
and policies as they are implemented in order to appreciate reach, size calculation that is required for quantitative studies in impact
context-level adaptation and effects has a strong tradition in the and outcome evaluation. The amount of desired important change
field of public administration, where it tends to be called “imple- is pre-specified.
mentation research”.11-13 But what is more associated with the Evaluation research and population health intervention research
phrase “intervention research” in the health field is the notion that encompass many of the same activities and methods. All evalua-
its primary purpose is to test a hypothesis or causal pathway. Hence, tion research in population health is population health interven-
attribution of effect to that intervention is a primary driver of the tion research, but not all population health intervention research
study design in intervention research, as its origin in the 19th cen- is evaluation research. This is because some population health inter-
tury underlines. Note that some health researchers have reserved vention research assesses the health impact of policies and pro-
the term “implementation research” for a phase of work that fol- grams in sectors outside of health. Because these policies and
lows the demonstration of a program’s or policy’s effects.14 This programs were not designed with a health outcome or objective in
implementation research phase is designed to elucidate more mind they do not conform to the definition of evaluation research
understanding about the process of a program that has already in the sense that the criteria for interpreting the health impact are
shown its effectiveness in a demonstration trial.14 This idea is often not preset. However, differences between the two fields of work
seen in clinical settings.15 However, others have argued that sys- have emerged in practice, some of which are listed in Table 1.
tematic observation of and improvement in process and imple- These differences mark a division in the culture of the two areas,
mentation should precede the measurement of effects. Indeed to in an everyday sense, that has some worrying features. They repre-
not do so might diminish the chance of a new intervention achiev- sent issues we need to address in Canada if we are to gain fully from
ing its effects.16 all the work being undertaken to understand ways to improve
The stepped-wedge cluster-randomized trial design has evolved health at a population level.
quite recently for situations in which there is high demand for a First, there is a tendency to see evaluation as “not research”. In
policy or program of unknown effectiveness and unlikely harm but many health regions in Canada, evaluation projects are not sent
insufficient resources for the program or policy to be uniformly for ethical review and therefore the investigation methods pro-
provided initially.17 Policy-makers are often more likely to be per- posed are not scrutinized externally. This may compromise quali-
suaded to adopt this staged, randomized roll-out design than a tra- ty.
ditional cluster-randomized trial. The stepped-wedge design allows Second, important questions about intervention effectiveness
effectiveness to be assessed optimally while local demand is served. may be being pursued under the guise of evaluation research and
From a traditional researcher’s perspective the stepped-wedge hence commissioned with insufficient resources to pursue answers
design remains an effectiveness trial and a classic case of interven- adequately. This contributes to a poverty of evidence on important
tion research. For policy-makers it is perhaps seen more as a pro- issues. For example, the US Task Force on Community Preventive
gressive introduction of a policy or program, with more Services has recently lamented that in 50% of the interventions

CANADIAN JOURNAL OF PUBLIC HEALTH • JANUARY/FEBRUARY 2009 I-9

Table 1. Common Differences That Have Arisen between Intervention Research and Evaluation Research
Intervention Research Evaluation Research
Intervention is often initiated by the researcher, although it may be Intervention under investigation is usually designed by practitioners or
designed in collaboration with practitioners. agencies.
Funded by a research grant. Funded by resources within the commissioning agency.
Budgeted according to the information required and the Funded as percentage of the cost of the program, e.g., at an arbitrary
cost to produce that information. level of 10%.
Results are destined for the public domain, e.g., in peer-reviewed journals. Results may be restricted to an internal report by contract agreements.
Usually focused on assessment of intervention outcomes, assisted Smaller budgets frequently limit enquiry to secondary data sources in
by a large budget for primary data collection. May also include relation to outcome or restrict the evaluation questions to matters of
assessment of process and mechanism of action. intervention process, reach or consumer satisfaction.
Requires ethics approval. Ethics approval not routinely sought.

reviewed there was insufficient research evidence to make any prac- The skill set of an intervention researcher
tice recommendations.20 This is probably not because a wide vari- As the task of intervention research is laid out, it becomes apparent
ety of practices have not been investigated; it is more likely that that the skill set to accomplish the task is complex. Technical com-
what has been investigated has not produced evidence that the Task petence in empirical enquiry is vital and, given the breadth of tasks
Force considers worthwhile. – e.g., study design, questionnaire design, interview design, data
A typical scenario comes from the World Health Organization’s management, data extraction, statistical analysis, qualitative data
(WHO) Safe Communities project. There are more than 80 such analysis, economic evaluation and economic modelling – may
projects worldwide, which are designed to mobilize and involve require the resources of a multidisciplinary team. However, any or
communities in reducing injuries. However, only seven of those all of these skills are part of the regular repertoire of any population
projects have undertaken controlled evaluations using objective health researcher. Because population health interventions are
sources of injury data, and only two have been shown to be effec- designed to address social conditions that determine risk, a good
tive.21 The dearth of evidence most probably arises because evalu- intervention researcher must have additional skills, including those
ations of Safe Communities projects typically are commissioned by that allow him or her to play a strategic role in the development
local agencies, with budgets insufficient to employ more than one and uptake of high-quality interventions (assuming here that the
person full time. They therefore tend to address questions about intervention research is real time and not historical, using second-
who is involved in the project, what people think about it, what ary data sets).
activities have been conducted and whether inter-sectoral collabo- In the first instance, researchers must be able to theorize about
ration increased, as described in a case study funded by a state change dynamics. Intervention research is about transformation
health department in Australia, for example.22 processes. Thus a researcher might need to look for more things,
Finally, the divide between evaluation and intervention research different things or different things in different ways, than if he/she
has meant that a different body of knowledge has evolved to serve were doing a descriptive or an analytical study. Investigators who
each professional field, and important opportunities for cross- have had to deal with the ramifications of interventions (side
development have been lost or delayed. For example, journals such effects, unintended effects)26 and the possibility that interventions
as Evaluation, Evaluation Review, New Directions for Evaluation, could cause harm have been led to theorize at multiple levels.27
Evaluation and Program Planning and Evaluation and the Health McLeroy reminds us that the “theory of the problem” and the “the-
Professions have published numerous studies about implementa- ory of the solution” are not the same.28 Some of the modest or neg-
tion assessment, the importance of context assessment and theories ative findings in population health intervention research might be
of change processes for a decade or more. Yet it has only been com- attributable to investigators, frustrated with their work in docu-
paratively recently that these notions have been given prominence menting the problems, trying their hand at intervention design
in the field of evidence generation in public health.23 One reason and intervention research without a thorough appreciation of the
for PHIRIC to bring these two fields closer together is to ensure that demands of intervention theory and practice. Unfortunately, as a
such misadventure does not persist. Some of the “great failures” in consequence, it may be hard to get policy-makers to reinvest in
population health intervention research24 can be attributed to issues interventions and intervention research in areas where previous
that experienced evaluation researchers would have detected earli- investigation has failed. Put crudely, the baby easily gets thrown
er. These include issues like inadequate intervention implementa- out with the bathwater.
tion, failure to stage the design of the research to the stage of the Skills in communication, policy and social analysis are vital.29
intervention’s development and/or inadequate program theory. Research has to be meaningful and convenient to the people and
These are domains that health promotion evaluators have long organizations with whom the researcher is working. Intervention
been encouraged to examine systematically at the outset of study research is about contributing directly to the implementation of
design during the process of evaluability assessment.16 We note, for actions to improve the population’s health. Yet, too often
example, that the failure of the Stanford Heart Disease Prevention researchers have been accused of designing and testing interven-
Program was predicted at the start by those who argued in journals tions that no one would be able to implement in real life, ignoring
published at the time that its community-based change logic policy-makers’ needs.30 The field of utilization-focused evaluation
(theory) was weak.25 A formal evaluability assessment of the inter- is helpful here in increasing researchers’ sensitivity to stakeholder
vention might have held the investigators’ decisions around this or end-user needs.31 Additionally, researchers need to gain the sup-
up to greater scrutiny and debate. port of practitioners. This can be difficult. Not only does research

I-10 REVUE CANADIENNE DE SANTÉ PUBLIQUE • VOL. 100, NO. 1

Table 2. Examples of the Diversity of Intervention Studies to Address Various Evaluation Questions
Evaluation Question of Interest Authors Examples
RELEVANCE: How relevant is the Bisset et al., 200438 Examines how decisions about the goal and mission of a community
program to targets of change? diabetes prevention program were informed by prevalence studies
conducted in the community.
Baker et al., 200739 Shows how a community prevention program to reduce childhood obesity
was designed on the basis of community asset mapping and led to
community engagement in the program.
COHERENCE: How does the Hawe & Stickney, 199740 Describes how, despite good will, an intersectoral food policy
theory of change underlying the committee was lacking a mechanism to successfully pursue its goals.
41
program relate to the theory Levesque et al., 2005 Examines the correspondence between the activities implemented in
of the problem? a community diabetes prevention program and the principles of the socio-
ecological approach to health promotion.
RESPONSIVENESS: How is Ho et al., 200642 Examines how the local conditions prevailing in remote communities
program implementation were related to changes in the implementation of a First-Nations
responsive to local conditions? Diabetes Prevention Program that had been successfully evaluated.
Corrigan et al., 200643 Describes how the use of qualitative methods helped improve the fit
between implementation variations in a randomized trial of secondary
prevention and local needs and conditions
ACHIEVEMENTS: What did Wickizer et al., 199844 Identified the critical factors for successful implementation of a
program activities and services community health promotion initiative in 11 communities randomly
achieve? assigned to receive program grants.
Cooke et al., 200745 Examines the changes in aggressive and related behaviors as well as in
discipline referrals following the successful implementation of a violence
prevention program in six schools.
RESULTS/IMPACT: With which O’Loughlin et al., 199946 Quasi-experimental study showing that although several
changes in local conditions was implementation indicators revealed a high level of program
the program associated? penetration in the community, there was no improvement in health and
behavioural indicators.
Wagenaar et al., 200647 Quasi-experimental study showing positive trends in many indicators in the
10 US States where the Reducing Underage Drinking through Coalition
Project funded coalitions designed to change policy and normative
environments.

take up practitioners’ time but it can also attract resources that Myth 1. Intervention Research Is Just About Intervention Effect
would have otherwise been spent on the intervention itself. Careful We have defined intervention research in a way that emphasizes
navigation is required when researcher and practitioner interests its role in understanding causal mechanisms, but showing that
do not coincide at the outset. something is effective is only part of the task. Intervention research
There may be covert as well as overt reasons for programs and is about all parts of the process of designing and testing solutions
services, and insensitivity on the part of the researcher can fail to to problems and about getting solutions into place. It can involve
recognize this. One example comes from DARE, Drug Abuse process evaluation of interventions (assessing reach, implementa-
Resistance Education in North America. This is a school-based tion, satisfaction of participants, quality). It can involve assessment
substance abuse prevention program that has been delivered to of how interventions adjust to different contexts.35 It can extend to
more than 33 million school children at an annual cost of $0.75 examinations of how interventions are sustained over time or
billion. Repeated evaluations have shown that it does not prevent become embedded in the host institutions.36 It includes diffusion
substance abuse.32 In 2001, a $13.7 million program renovation research or understanding of how interventions are spread to new
was undertaken, but there has still been no evidence that the new sites.37 The WHO Task Force on Health Promotion Evaluation pro-
DARE is effective.33 However, a recent qualitative analysis has sug- posed five questions as points of entry for enquiring about an inter-
gested that past researchers have possibly missed the point of the vention with these multiple aspects in mind.37 Table 2 presents
program. Its primary benefit is perceptual – principals and teach- examples of intervention studies conducted in relation to each of
ers like having police in schools making contact with children those five questions.
and youth.34 On this basis, schools may wish the program to con-
tinue in spite of its failure to prevent drug use. From a popula- Myth 2. Interventions Designed and Implemented With Communities
tion health perspective, there may be cheaper ways of building Should Not Be Called Intervention Research
school collaborations with the police that could allow the bulk The field of population and public health is interdisciplinary, eclec-
of DARE costs to be diverted to more effective programs against tic and contested. In community-based intervention research some
drug abuse. The point is that more astuteness on the policy analy- unhelpful schisms have grown up between studies primarily
sis side might have anticipated this finding many years previous- designed and controlled by researchers and those that are driven by
ly. communities. We believe that both types of interventions must be
accountable for their logic, values and outcomes.
Common myths in intervention research The whole spectrum of intervention research should be sup-
In practice, intervention research has often tended to be associat- ported, from those interventions driven by hypotheses formulated
ed with investigator-driven studies, and evaluation research has in academia to those in which interventions are designed and
been associated with studies commissioned by or conducted with implemented by local actors.48 However, we stop short of the sug-
the research users, consumers or decision-makers. This means that gestion that, with respect to communities, the term “intervention”
a particular profile, or image, of intervention research has arisen research be dropped in favour of terms like “community develop-
that needs to be interrogated. ment” or “community-based action”. The latter terms frame a tra-

CANADIAN JOURNAL OF PUBLIC HEALTH • JANUARY/FEBRUARY 2009 I-11

dition that is highly respected and characterized by particular ways researchers who use analogies from drug trials to explain the effi-
of working.49 We are not suggesting that these terms be replaced, cacy of community health interventions.14,58 In these analogies, the
but we suggest that it may be appropriate to use the term “inter- most efficacious interventions are framed as those designed by uni-
vention research” in conjunction with them when data are being versities or expert authorities of some type. The effectiveness of
collected by researchers, because the intervention terminology interventions is then considered to be progressively diluted by the
enshrines a dynamic that is important to remember: that of dis- transfer of these technologies from the academy into the hands of
turbing the regular order of things. A focus on disturbing, inter- local community practitioners.14,58 The alternative view defines effi-
rupting or changing an expected sequence of events draws cacy as starting with interventions designed or shaped by commu-
attention to the ethical issues involved, the relationship between nities and practitioners. These may intersect with universities to
the researcher and the researched and the duty of care enshrined in the extent that such relationships may be required to strengthen
the relationship.50 These are neglected issues in population health intervention theory and to gather convincing evidence that such
research, which we believe could become even more neglected if interventions work. These alternative views recognize the agency of
inadvertently hidden by language that disguises the duties and the practitioners and the capacity of communities to foresee and
responsibility of the researcher. When researchers become actors in shape the types of intervention that might work best.
local events, as opposed to being merely observers, many of us find So, by being well theorized and facilitated, it is entirely possible
ourselves untrained and unprepared. Preserving a language that for community-based, context-adapted, flexible interventions to
alerts us to the special nature of this role is precautionary and vital. be evaluated usefully, even in randomized trials. This is a point that
has been argued in theory56 and recently demonstrated in prac-
Myth 3. Intervention Research Is Only About Controlled Trials tice.59
A lot of intervention research has been about controlled trials in
schools, worksites and communities, but it does not have to be. Myth 5. This Is Just Health Promotion Research With a Different Name
Many different types of study design can be used to build acceptable There is an extraordinary legacy of work in health promotion
evidence in public health, although some scenarios are more desir- research that informs the way in which we should conceive of and
able than others when it comes to making causal inferences.51 measure the process and impact of interventions in population
Important work is advancing in the use of time series designs to illu- health.60 However, population health intervention research is wider.
minate the impact of policy52,53 and in the use of observational meth- The difference between health promotion research and inter-
ods to investigate the relationship between program implementation vention research in population health, as PHIRIC has defined it, is
conditions and impact. There has also been an expansion of the intentionality of the intervention. Health promotion research is
community-based participatory research, which has been critical for focused on interventions designed to improve health. Intervention
addressing social determinants of health in communities.54,55 research in population health is the umbrella term that also
includes explorations of the health effects of interventions in sec-
Myth 4. Intervention Research Is About Controlled Interventions tors outside of health designed for other purposes, such as increased
There is no reason for all interventions to be as tightly controlled transport usage. This is commonly known as a health impact assess-
as many investigators have imagined.56 Indeed it has been observed ment. An advantage in bringing a closer alliance of the two
that the reason so many interventions in schools, worksites and domains is that methods from one can inform the other. For exam-
communities have failed may be because investigators have over- ple, mathematical modelling with large secondary data sets is a
controlled the form of their interventions in the mistaken belief common means to explore the health effects of economic policy.61
that this is a design requirement of randomized controlled trials.56 Such methods are less well known in mainstream health promo-
An alternative way of thinking about standardization has been tion journals but could be better used. By contrast, exposure meas-
proposed that can liberate the randomized controlled design and urement with regard to an intervention, and all the subtleties
aid its use in more contexts.56 The key issue is that interventions enshrined in the notion of intensity of “preventive dose”, has been
have to be well theorized and recognizable, so that the evaluation well developed in the health promotion literature.62 However, it
is valid and so that another person could replicate the intervention appears to be less well captured in fields outside of health promo-
in another place. The essence of an intervention might be a process tion, where exposure may only be defined dichotomously (i.e., pro-
or set of functions.56 This type of intervention follows recognizable gram deemed to be present or not).
principles (a standard function, like organizational development
or community development) but necessarily takes a different form Myth 6. Intervention Research Is Too Expensive
from place to place and in that sense owes its effectiveness to how It is common to deplore the high costs of intervention trials,
it is tailored to context.56 Alternatively, the intervention could be demonstration projects or participatory research and to plea for
fixed or standard in form, like a leaflet based on the health belief resources to be spent in other ways, but the truth is that we do not
model, which draws its benefit from being sufficiently effective know whether intervention research is any more expensive than
overall, even though it is largely immune to local context and not descriptive or analytic research in population health. Certainly,
effective in every place.57 The point is that interventions standard- when an intervention study fails to record a reduction in a health
ized by form or standardized by function can be evaluated mean- problem, there always seems to be attention drawn to how much
ingfully in randomized controlled trials.56 Theorizing this at the it cost to find this out. But it is unclear whether, if one counted up
outset is part of trial design. the costs of all the cross-sectional studies and cohort studies that
The myth that interventions have to be tightly controlled in have been chasing various risk factors over the years, those results
terms of form unfortunately continues to be promulgated by are any less costly or of any more value. Overall, we do not have a

I-12 REVUE CANADIENNE DE SANTÉ PUBLIQUE • VOL. 100, NO. 1

13. Noble CH. The eclectic roots of strategy implementation research. J Business
system for monitoring the added value of any particular study or Res 1999;45(2):119-34.
field of study to insights in population health. Perhaps it is time 14. Nutbeam D, Bauman A. Evaluation in a Nutshell. Sydney: McGraw Hill, 2006.
15. Foy R, Eccles M, Grimshaw J. Why does primary care need more implemen-
that some metric was devised. tation research? Fam Pract 2001;18(4):353-55.
16. Hawe P, Degeling D, Hall J. Evaluating Health Promotion. A Health Workers’
Guide. Sydney: Maclennan and Petty, 1990.
The promise ahead
17. Brown CA, Milford RJ. The stepped wedge trial design: A systematic review.
This paper began with a reference to John Snow in the 1850s as an BMC Med Res Methodol 2006;6:54.
exemplar of a population health intervention researcher testing a 18. Suchman EA. Evaluative Research. New York: Russel Sage, 1967.
19. Weiss CH. Evaluation, 2nd ed. Upper Saddle River, NJ: Prentice Hall, 1998.
theory. As it happened, his colleagues at the time thought that his 20. Zaza S, Briss PA, Harris KW. The Guide to Community Preventive Services: What
theory of the source of the cholera was too exclusive, and it took Works to Promote Health? New York, NY: Oxford University Press, 2005.
21. Spinks A, Turner C, Nixon J, McLure R. ‘WHO Safe Communities’ model for
another decade for more evidence to be gathered and for actions to the prevention of injury in whole populations. Cochrane Database Syst Rev
be taken to control contaminated water sources.63 We followed up Issue 2, Art. No. CD004445.
with a reference to causal reasoning as it involved the decomposi- 22. Sefton C. The NSW Safe Communities pilot projects – evaluation methodol-
ogy. N S W Public Health Bull 2002;13(4):76-77.
tion of phenomena to mechanisms and sequences of events. That 23. Jackson N, Waters E and the Guidelines for Systematic Reviews in Health
was an illustration of reasoning primarily attributed to the 16th cen- Promotion and Public Health Taskforce. The challenges of systematically
reviewing public health interventions. J Public Health Med 2004;26:303-7.
tury contributions of Renee Descartes. But by the mid-20th centu- 24. Susser M. The tribulations of trials. Interventions in communities. Am J Public
ry, a new way of thinking called “systems thinking” emerged, Health 1995;85:156-60.
emphasising the view of living organisms as integrated wholes 25. Leventhal H, Safer MA, Cleary PD, Gutman M. Cardiovascular risk reduction
by community based programs for lifestyle change: Comments on the
whose properties cannot be reduced to those of the smaller parts.64 Stanford study. J Consult Clin Psychol 1980;48:150-58.
Our point is that intervention research has to extend itself to 26. Patton MQ. Qualitative Research Methods, 2nd ed. Newbury Park, CA: Sage,
1990.
understanding and accelerating the uptake of new practices and 27. Stokols D. Translating social ecological theory into guidelines for communi-
recognizing that the reasoning processes we use in science are ty health action. Am J Health Promotion 1996;10(4):282-98.
28. McLeroy K, Steckler A, Simons-Morton B, Goodman RM. Social science theory
under constant interrogation. In intervention research, investiga-
in health education: Time for a new model? Health Educ Res 1993;8(3):305-12.
tors are adopting system-thinking approaches,65 and there are 29. Ingle MD, Klaus R. Competency based program evaluation: A contingency
investigators putting “realist” views claiming that “theory-based” approach. Evaluation and Program Planning 1981;3:277-87.
30. Petticrew M, Whitehead M, Macintyre S, Graham H, Egan M. Evidence for
views can be contrasted with “scientific method”, which is pre- public health policy on inequalities: 1. The reality according to policymakers.
sumed to not be based on theory sufficiently. Agent-based ways of J Epidemiol Community Health 2004;58:811-16.
31. Patton MQ. Utilisation-Focused Evaluation: The New Century Text. Newbury
conceiving interventions are being contrasted with expert models. Park: Sage, 1997.
Views are hotly contested.66 32. Perry CL, Komro KA, Veblen-Mortenson S, Bosma LM, Farbakhsh K, Munson
In the meantime, the ever-growing burden of disease demands KA, et al. A randomised controlled trial of the middle and high school DARE
and DARE plus programmes. Arch Pediatr Adolesc Med 2003;157:178-84.
that we design effective interventions and put them into practice. 33. Lord M. Truth or dare. A new drug course. US News World Rep 2001;130(8):30.
Our plea is for systems to be created in Canada that attract the best 34. Birkeland S, Murphy-Graham E, Weiss C. Good reasons for ignoring good
evaluation: The case of the drug abuse resistance education (DARE) program.
minds and the best energies in the country to solving these issues. Evaluation and Program Planning 2005;28(3):247-56.
There has never been a more stimulating or crucial time to act. 35. Steckler A, Linnan L (Eds.). Process Evaluation for Public Health Interventions
There is no point in changing thinking in population health, if we and Research. San Francisco, CA: Jossey Bass, 2002.
36. Goodman RM, Steckler AB. A model of institutionalisation of health promo-
cannot change history with it. tion programs. Fam Community Health 1987;11:63-78.
37. Potvin L, Haddad S, Frohlich KL. Beyond process evaluation. In: Rootman I,
REFERENCES Goodstadt M, Hyndman B, McQueen DV, Potvin L, Springett J, et al. (Eds.),
Evaluation in Health Promotion. Principles and Perspectives. Copenhagen: WHO
1. Hawe P, Shiell A. Using evidence to expose the unequal distribution of prob- Regional Publications, European Series, 2001; No 92:45-62.
lems and the unequal distribution of solutions. Eur J Public Health 38. Bisset S, Cargo M, Delormier T, Macaulay AC, Potvin L. Legitimizing diabetes
2007;17(5):413. as a community health issue: A case analysis of an Aboriginal community in
2. MacMahon B, Pugh TF. Epidemiology: Principles and Methods. Boston, MA: Little Canada. Health Promotion Int 2004;19:317-26.
Brown, 1970. 39. Baker IR, Dennison BA, Boyer PS, Sellers KF, Russo TJ, Sherwood NA. An asset-
3. Institute of Population and Public Health, Canadian Institutes of Health based community initiative to reduce television viewing in New York State.
Research. Population Health Intervention Research Initiative for Canada Prev Med 2007;44:437-41.
(“PHIRIC”) Workshop Report. Ottawa, ON: CIHR. Available online at: 40. Hawe P, Stickney EK. Developing the effectiveness of an intersectoral food pol-
www.cihr-irsc.gc.ca/e/33515.html (Accessed December 2008). icy coalition through formative evaluation. Health Educ Res 1997;12:213-25.
4. Porter D. Health, Civilisation and the State. A History of Public Health from 41. Levesque L, Guilbault G, Delormier T, Potvin L. Unpacking the black box: A
Ancient to Modern Times. London, UK: Routledge, 1999. deconstruction of the programming approach and physical activity inter-
5. Fassin D. L’espace politique de la santé. Essai de généalogie. Paris: Presses vention implemented in the Kahnawake Schools Diabetes Prevention Project.
Universitaires de France, 1996. Health Promot Pract 2005;6:64-71.
6. Wagenaar AC, Webster DW. Preventing injuries to children through com- 42. Ho LS, Gittelsohn J, Harris SB, Ford E. Development of an integrated preven-
pulsory automobile safety seat use. Pediatrics 1986;78:662-72. tion program with First Nations in Canada. Health Promot Int 2006;21:88-97.
7. Mills JL, Signore C. Neural tube defects rates before and after food fortifica- 43. Corrigan M, Cupples ME, Smith SM, Byrne M, Leathem CS, Clerkin P, et al.
tion with folic acid. Birth Defects Res 2004;70(11):8445-55. The contribution of qualitative research to designing a complex intervention
8. Mathee A, Rollin H, von Schirnding Y, Levin J, Naik I. Reductions in blood for secondary prevention of coronary heart disease in two different health
lead levels among school children following the introduction of unleaded care systems. BMC Health Serv Res 2006;6:90.
petrol in South Africa. Environ Res 2006;100(3):319-22. 44. Wickizer TM, Wagner E, Cheadle A, Pearson D, Berry W, Maeser J, et al.
9. Institute of Population and Public Health, Canadian Institutes of Health Implementation of the Henry J. Kaiser Family Foundation’s Community
Research. Mapping and Tapping the Wellsprings of Health. Strategic Plan Health Promotion Grant Program: A process evaluation. Milbank Q
2002-2007. Ottawa: CIHR. 1998;77:121-47.
10. Rose G. The Strategy of Preventive Medicine. Oxford, UK: Oxford University 45. Cooke MB, Ford J, Levine J, Bourke C, Newell L, Lapidus G. The effects of city-
Press, 1992. wide implementation of “Second Step” on elementary school students’ proso-
11. Mischen PA, Sinclair TAP. Making implementation more democratic through cial and aggressive behaviors. J Primary Prev 2007;28:93-115.
action implementation research. J Public Admin Res Theory 2009;19:145-64. 46. O’Loughlin JL, Paradis G, Gray-Donald K, Renaud L. The impact of a com-
12. O’Toole LJ. The theory-practice issue in policy implementation research. munity-based heart disease prevention program in a low-income inner-city
Public Admin 2004;82(2):309-29. neighbourhood. Am J Public Health 1999;89:1819-26.

CANADIAN JOURNAL OF PUBLIC HEALTH • JANUARY/FEBRUARY 2009 I-13

47. Wagenaar AC, Erickson DJ, Harwood EM, O’Malley PM. Effects of state coali- 66. Cook TD. The false choice between theory-based evaluation and experimen-
tions to reduce underage drinking: A national evaluation. Am J Prev Med tation. In: Rogers PJ, Hasci TA, Petrosino A, Huebner TA (Eds.), Program
2006;31:307-15. Theory in Evaluation: Challenges and Opportunities. New Directions for
48. Potvin L, Goldberg C. Deux rôles joués par l'évaluation dans la transforma- Evaluation 2000;87:27-34.
tion de la pratique en promotion de la santé. In : O'Neill M, Dupéré S,
Pederson A, Rootman I (Eds.), La promotion de la santé au Canada et au Québec :
Perspectives critiques. Québec: Presses de l'Université Laval, 2006;457-73. RÉSUMÉ
49. Minkler M. Community Organising and Community Building for Health. Rutgers
University Press, 2004. Les interventions populationnelles de santé comprennent l’ensemble des
50. Riley T, Hawe P, Shiell A. Contested ground: How should qualitative evidence
actions qui visent à modifier la distribution des risques à la santé en
inform the conduct of a community intervention trial? J Health Serv Res Policy
2005;10(2):103-10. ciblant les conditions sociales, économiques et environnementales qui
51. Cook TD, Campbell DT. Quasi Experimentation: Design and Analysis Issues for façonnent la distribution des risques. Sous forme de programmes et
Field Settings. Chicago, IL: Rand McNally, 1979. politiques, ces interventions peuvent provenir du secteur de la santé mais
52. Biglan A, Ary D, Wagenaar AC. The value of interrupted time series experi- sont aussi souvent pilotées par d’autres secteurs comme l’éducation, le
ments for community intervention research. Prev Sci 2000;1(1):31-49. logement ou l’emploi. La recherche sur les interventions de santé des
53. Zerger SL, Irizarry R, Peng RD. On time series analysis of public health and
biomedical data. Annu Rev Public Health 2006;27:57-79.
populations poursuivent l’objectif de documenter la valeur et les effets de
54. Cargo M, Mercer SL. The value and challenge of participatory research: ces interventions, les processus par lesquels les changements opèrent et
Strengthening its practice. Annu Rev Public Health 2008;29:325-53. les conditions qui favorisent les effets. Dans le domaine de la recherche
55. Minkler M, Wallerstein N. Community Based Participatory Research for Health. en santé, des distinctions inutiles entre la recherche et l’évaluation ont
San Francisco: Jossey Bass, 2003. retardé le développement des connaissances sur l’intervention de santé
56. Hawe P, Shiell A, Riley T. Complex interventions: How far ‘out of control’
des populations et mené à une mauvaise intégration des données de
should a randomised controlled trial be? Br Med J 2004;328:1561-63.
57. Hawe P, McKenzie N, Scurry R. Randomised controlled trial of the use of mod- recherche pour soutenir la pratique et les décisions concernant les
ified postal reminder card on the uptake of measles vaccination. Arch Dis programmes et politiques de santé des populations. Cet article
Childhood 1998;79:136-40. déboulonne donc certains mythes pernicieux concernant la recherche sur
58. Flay BR. Efficacy and effectiveness trials (and other phases of research) in the les interventions, notamment relativement aux coûts associés, à ses visées
development of health promotion programs. Prev Med 1986;15(5):451-74. et à la croyance en un rôle nécessairement marginal des communautés
59. Patton GC, Bond L, Carlin JB, Thomas L, Butler H, Glover S, et al. Promoting
concernées pour développer des interventions efficaces. Cet article
social inclusion in schools: A group-randomized trial of effects on student
health risk behavior and well-being. Am J Public Health 2006;96(9):1582-87. retourne aussi comme arbitraire et injustifiée la distinction traditionnelle
60. Green LW, Kreuter MW. Health Promotion Planning: An Educational and entre la recherche sur les interventions et la recherche évaluative. En fait
Ecological Approach, 3rd ed. Mountain View, CA: Mayfield, 1999. cet article montre que la recherche sur les interventions a tout à gagner
61. Morrell S, Taylor R, Quine S, Kerr C. Suicide and unemployment in Australia d’un rapprochement avec la recherche évaluative et d’une intégration
1907-1990. Soc Sci Med 1993;36(6):749-56. des méthodes de recherche appliquée provenant d’une diversité de
62. Bartholomew LK, Parcel GS, Kok G, Gottlieb NH. Planning Health Promotion
Programs. An Intervention Mapping Approach. San Francisco: Jossey Bass, 2006.
disciplines.
63. Harrison M. Disease and the Modern World: 1500 to the Present Day. Cambridge: Mots clés : Évaluation; intervention pour la santé des populations;
Polity, 2004.
64. Miller JH, Page SE. Complex Adaptive Systems. New Jersey: Princeton University
pratique fondée sur des données probantes; recherche sur les
Press, 2007. interventions; santé des populations
65. Best A, Moor G, Holmes B, Clark PI, Brice T, Lieschow S, et al. Health pro-
motion dissemination and system thinking: Towards an integrative model.
Am J Health Behav 2003;27(Suppl):S206-S216.

I-14 REVUE CANADIENNE DE SANTÉ PUBLIQUE • VOL. 100, NO. 1