See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/313172778

The functions of program theory

Article  in  New Directions for Program Evaluation · January 1987

CITATIONS READS

145 3,603

1 author:

Leonard Bickman
Vanderbilt University
255 PUBLICATIONS   13,257 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

feedback View project

Fort Bragg Evaluation Project View project

All content following this page was uploaded by Leonard Bickman on 01 March 2021.

The user has requested enhancement of the downloaded file.

The role of theory in program evaluation is an important
but neglected area in evaluation practice and theory.
Program theory has saleral important functions that can
improve our ability to generalize from particular evaluations,
contribute to social science theory, and achieve consensus in
evaluation planning.

The Functions of
Program Theoy
Leonard Bickman

A Definition of Program Theory

Program theory, as defined in this chapter, is the construction of a

plausible and sensible model of how a progam is supposed to work. Typ-
ically these models are developed for a particular program and do not
represent "off-the-shelf" use of a single established social science theory.
The other authors in this volume also provide definitions of program
theory. Conrad uses the term program philosophy to describe a system of
beliefs, values, and goals that define the structure, process, and outcomes
of a program. Scheirer indicates that program theory clarifies the set of
cause-and-effect relationships and thus provides the rationale for the treat-
ment. She further elaborates on the differences between program theory
and implementation theory. Program theory according to McClin tock
includes scientific explanations, implicit theories, models of inputs, pro-
cesses, and outputs, as well as policy statements. McClintock concentrates
in his chapter on understanding the role of implicit theories. Wholey
defines the functions of program theory as the identification of resources,
activities, and outcomes of a program and the causal assumptions that
connect these. Shadish divides program theory in to program macrotheory
and microtheory. Microtheory provides a description of the program being
L.Bickman (4. ).Program Theory in Emluation
Using
New Directions for Program Evaluation, no. 33. San Franasco: Josq-Bass, Spring 1987. 5
disappointing. However, the poor use of program theory in evaluation is
not exceptional in otherwise well-designed evaluations. Lipsey and others
found equally dismal findings when examining measurement, design,
and statistical power.
Not all evaluation theorists emphasize the positive use of program
theory. As Cook and Shadish (1986)point out, both Campbell and Scriven
emphasize “identifying manipulable soh tions” over “identifying general-
izable explanations” (p. 225). Campbell’s (1 969) experimenting society
and Scriven’s (1983) consumer model of evaluation represent this point of
view. Cook and Shadish (1986) note that this approach to evaluation,
which dominated the beginning of evaluation, is now declining but they
caution that much of value will be lost if this declining perspective is
completely dismissed

Summary

Program theory is an important but neglected aspect of program

evaluation. Although its importance was recognized early in the history of
evaluation, it has not received much attention in the literature until
recently, and in practice it has received even less attention. This is surpris-
ing given the positive functions program theory has in program evalua-
tion. In this chapter we have noted ten functions program theory can
have, as follows:
Contributing to social science knowledge
Assisting policymakers
Discriminating between theory failure and program failure
Identifying the problem and target group
Providing program implementation description
Uncovering unintended effects
Specifying intervening variables
Improving formative use of evaluation
Clarifying measurement issues
Improving consensus formation.
Many programs are not developed with a strong and coherent theory, but
instead the theory underlying the program must often be developed by the
program evaluator in an attempt to perform some of the measurement
and design tasks, Finally, program theory is used in a small proportion of
reported evaluations.

References

Anderson, C. A., and Sechler, E. S. “Effects Explanation and Counterexplanation

in the Development and Use of Social Theories.” Journul of PersonaZity and
S O C ~ Psychology,
U~ 1986, 50, 24-34.
 17

Bandura, A. Aggression: A Social Learning Analysis. Englewood Cliffs, N. J.: Pren-

tice-Hall, 1973.
Bickman, L. “Program Evaluation and Social Psychology: The Achievement of
Relevancy.” Personality and Social Psychology Bulletin, 1979,5, 483-490.
Bickman, L. “Evaluating Prevention Programs.” Journal of Social Issues, 1983,39,
181.
Bickman, L. “Improving Established Statewide Programs: A Component Theory
of Evaluation.’’ Evaluation Review, 1985a79, 189-208.
Bickman, L. “Randomized Field Experiments in Education: Implementation Les-
sons.” In R. Boruch and W.Wothke (eds.), Randomization and Field Experimen-
tation. New Directions for Program Evaluation, no. 28. San Francisco: Jossey-
Bass, 198515.
Bickman, L., and Rog, D. “The Use of Stakeholders in Planning Evaluations of
Early Intervention Programs.” In L, Bickman and D. Weatherford (eds.), EvaZu-
ating Early Intmention Program for Severely Handicapped Children and Their
Families. Austin, Tex.: Pro-ed Press, 1986.
Brenner, M. “Intensive Interviewing.” In M. Brenner, J. Brown, and D. Canter
(eds.), The Research Interview: Uses and Approaches. London: Academic Press,
1985.
Campbell, D. T. “Reforms as Experiments.” American Psychologist, 1969, 24,
409-428.
Canter, D., Brown, J., and Groat, L. “A Multiple Sorting Procedure for Studying
Conceptual Systems.’’ In M. Brenner, J. Brown, and D. Canter (eds.), The
Research Interview: Uses and Approaches. London: Academic Press, 1985.
Chen, H.-T., and Rossi, P. H. “The Multi-Goal, Theory-Driven Approach to Eval-
uation: A Model Linking Basic and Applied Social Science.” In H. E. Freeman
and M. A. Solomon (eds.), Evaluation Studies Review Annual. Vol. 6. Beverly
Hills, Calif.: Sage, 1981.
Chen, H.-T., and Rossi, P. H. “Evaluating with Sense: The Theory-Driven
Approach.” Evaluation Review, 1983, 7, 283-302.
Cook,T. D., and Campbell, D. T. Qwi-Experimentation Design: Design and Anal-
ysis Issues for Field Settings. Skokie, Ill.: Rand McNally, 1979.
Cook,T.D., Leviton, L. C., and Shadish, W. R.,Jr. “Program Evaluation.” In G.
Lindzey and E. Aronson (eds.), Handbook of Social Psychology. (3rd ed.) New
York: Random House, 1985.
Cook, T. D., and Shadish, W. R., Jr. “Program Evaluation: The Worldly Science.”
Annual Review of Psychology, 1986,37, 193-232.
Cronbach, L. J. Designing Evaluations of Education and Social Programs. San
Francisco: Jossey-Bass, 1982.
Edwards, J., Hotch, D., and Bickman, L. “Measuring Children’s Health-Related
Attitudes and Knowledge.” In L. Bickman (ed.),Applied Social Psychology
Annual. Vol. 3. Beverly Hills, Calif.: Sage, 1982.
Janis, I. L., and Mann, L. Decision Making: A Psychological Analysis on Conflict,
Choice, and Commitment. New York: Free Press, 1977.
Lipsey, M. W., Crosse, S., Dunkle, J., Pollard, J., and Stobart, G. “Evaluation:
The State of the Art and the Sorry State of Science.” In D. S. Cordray (en.),
Utilizing Prior Research in Evaluation Planning. New Directions for Program
Evaluation, no. 27. San Francisco: Jossey-Bass, 1985.
Lord, C. G., Ross, and Lepper, M. R. “Biased Assimilation and Attitude Polariza-
tion: The Effects of Prior Theories on Subsequently Considered Evidence.” JOUT-
nal of Personality and Social Psychology, 1979,37, 2098-2109.
Rog, D., and Bickman, L. “The Feedback Research Approach to Evaluation: A
 18

Method to Increase Evaluation Utility.” Evaluation and Program Planning, 1984,

7, 169-175.
Rutman, L. Planning Useful Evaluations: Evaluability Assessment. Beverly Hills,
Calif.: Sage, 1980.
Scheirer, M. A. Program Imfilementation: T h e Organizational Context. Beverly
Hills, Calif.: Sage, 1981.
Scriven, M . T h e Logic of Evaluation. Inverness, Calif.: Edgepress, 1980.
Scriven, M. “Evaluation Ideologies.” In G. E Madavs, M. Scriven, and D. L. Stuf-
felbeam (eds.), Evaluation Models: Viewpoints on Educational and Human Ser-
vices Evaluation. Boston, Mass.; Klower-Nijhoff, 1983.
Shaklee, H., and Fischoff, B. “Strategies of Information Search and Causal Analy-
sis.” Memory and Cognition, 1982, 10, 520-530.
Stake, R. E. “The Case Study Method in Social Inquiry.’’ Educational Research,
1978, 7, 5-8.
Suchman, E. A. Evaluative Research: Principles and Practice in Public Seruzce and
Social Action Programs. New York: Russell Sage Foundation, 1967.
Weick, K. E., and Bougon, M. G. “Organization as Cognitive Maps.” In H. P.
Simms, Jr., and D. A. Gioia (eds.),Social Cognition in Organizations. San Fran-
cisco: Jossey-Bass, 1986.
Weiss, C. H. Evaluation Research: Methods for Assessing Program Effectiveness.
Inglewood Cliffs, N . J.: Prentice-Hall, 1972.
Wholey, J. S. “Evaluability Assessmenty’In L. Rutman (ed.), Evaluative Research
Methods: A Basic Guide. Beverly Hills, Calif.: Sage, 1977.
Wholey, J. S. Evaluation: Promise and Performance. Washington, D.C.: Urban Insti-
tute, 1979.

Leonard Bickrnan is professor of psychology at Peabody

College, Vanderbilt University. He is director of the Program
Evaluation Laboratory at Peabody and director of the Center
for Mental Health Policy at the Vanderbdlt Institute for Public
Policy Studies.

View publication stats