Assessing the Effectiveness of Problem-Based Learning in Higher Education:

Lessons from the Literature

 
Claire H. Major, University of Alabama
Betsy Palmer, University of Iowa
 
Major is an assistant professor of higher education administration. <[email protected]>. Palmer is  an
assistant  professor  in the Counseling,   Rehabilitation    and Student Development department <[email protected]>.
 
Abstract
Problem-Based Learning (PBL) is an innovative educational approach that is gaining prominence in higher education. A review of
the literature of PBL outcomes summarizes, across multiple studies, the positive effects of problem-based learning. Since PBLbrings
with it unique challenges to traditional assessment, however, this study suggests alternative approaches. Alternative assessment may
provide additional insight into the effectiveness of PBL and other alternative pedagogies.
 
Introduction
Traditional instruction, such as the typical lecture-based session that developed before textbooks
were mass-produced, often involves delivering as much information as possible as quickly as
possible. The lecture method was one of the most effective and efficient ways to disseminate
information and has often been used for this end. Because many faculty members are poor lecturers,
and because students are often poor participants in the lecture, this type of instruction has often
allowed students to be passive in the classroom. Students, not knowing how to be active participants
in the lecture, have relied on transcription, memorization, and repetition for learning.
 
In recent decades, however, we have learned a great deal from cognitive science research about the
nature of learning. Students construct knowledge; they do not take it in as it is disseminated, but
rather they build on knowledge they have gained previously (Cross, 1998). They benefit from
working together, and they may learn best from teaching each other (Annis, 1983; McKeachie, et
al., 1986). Research also suggests that students learn best in the context of a compelling problem
(Ewell, 1997); they learn through experience (Cross, 1999). In short, students learn through making
cognitive connections, social connections, and experiential connections (Cross, 1990). Because they
make these connections differently, students do not learn in the same way. This relatively new
information suggests that teaching is a complex activity, and it necessitates the emergence and
development of approaches to instruction that are consistent with what we know about the way that
learning happens (Ewell, 1997). This new understanding has given rise to the notion of a paradigm
shift in higher education, one from a focus on teaching to a focus on learning (Barr and Tagg 1995).
New "powerful pedagogies" emphasizing learning, such as project-based learning, inquiry-based
learning, case-based learning, research-based learning, situation-based learning, action learning, and
Problem-Based Learning (PBL) intimate that alternative pedagogies may be gaining in prominence
and may ultimately become the dominant classroom paradigm.
PBL is an educational approach in which complex problems serve as the context and the stimulus
for learning. In PBL classes, students work in teams to solve one or more complex and
compelling  "real world" problems. They develop skills in collecting, evaluating, and synthesizing
resources as they first define and then propose a solution to a multi-faceted problem. In
most PBL classes, students also summarize and present their solutions in a culminating experience.
The instructor in a PBL class facilitates the learning process by monitoring the progress of the
learners and asking questions to move students forward in the problem-solving process. Unlike
traditional classrooms, the faculty member is not the sole resource for content or process
information, but instead guides students as they search out appropriate resources.
 
The PBL approach had its start in the 1960s at McMaster Medical School as faculty
developed PBL out of the perceived need to produce graduates who were prepared to deal with the
information explosion, and who could think critically and solve complex problems. This institution
developed its entire curriculum around problem-based learning. Soon after medical schools around
the world began to adopt the McMaster model. In these cases, PBL is an approach to structuring the
curriculum that involves confronting students with problems from practice which provide a stimulus
for learning (Boud &Feletti, 1991). However, there are many possible forms that a curriculum and
process for teaching and learning might take and still be compatible with this definition
(Boud & Feletti, 1991). For example, educational and professional schools also began to feel many
of the same needs as medical schools, so they began to adopt the approach as well, although in
different forms, such as hybrid PBL and traditional curricula and course-by-course models; again
the approach spread to institutions around the world.
 
Finally, educators and employers alike began to call for change in undergraduate institutions (Jones,
1997). They also wanted students who could think critically, solve problems, and work in teams.
The 1998 Boyer Report, Reinventing undergraduate education: A blueprint for America's research
universities, for example, articulates these charges and recommends inquiry-based learning as a
vehicle for improvement (The Boyer Commission, 1998). And many undergraduate institutions
began to develop PBL programs and curricula. Aalborg has one of the most comprehensive
undergraduate PBLcurriculums, and Maastricht also has a developed PBL program of study. More
recently, in the U.S., the University of Delaware has turned attention toward PBL, as
has Samford University in Birmingham,Alabama. In addition to these more comprehensive efforts,
individual faculty members at more than 300 institutions are using PBL at the undergraduate level
(PBL Insight, p. 7).
 
In developing expectations about the outcomes that PBL can bring, we can learn from the research
that has been conducted in medical schools. Therefore, in this article, we review recent literature
on PBLfrom medical and professional schools that outlines PBL effectiveness compared to
traditional instruction. We also review recent literature from professional schools and undergraduate
programs, which suggests that traditional assessment may not be adequate to evaluate the
effectiveness of PBL as an instructional method. This research shows that PBL provides students
with the opportunity to gain theory and content knowledge and comprehension. In
addition, PBL helps students develop advanced cognitive abilities such as critical thinking, problem
solving, and communication skills. PBL also can improve students' attitudes toward learning. Thus,
as a pedagogical technique, problem-based learning promotes the kinds of active learning that many
educators advocate (Barr & Tagg, 1995).
 
 
 
Literature
Several studies focus on the change in knowledge and skill levels that occur with PBL instruction.
A few studies show slight decreases in knowledge of basic sciences (Albanese & Mitchell, 1993).
Other studies show that on tests of medical knowledge, students in traditional programs scored
higher than students in the PBL curriculum (Schmidt, Dauphinee, & Patel, 1987; de Vries, Schmidt,
& de Graaff, 1989). Overall, most studies show no significant difference between the knowledge
that PBL students and non-PBL students acquire about sciences (Albanese & Mitchell, 1993).
However, students who acquired knowledge in the context of solving problems have been shown to
be more likely to use it spontaneously to solve new problems than individuals who acquire the same
information under more traditional methods of learning facts and concepts through lectures
(Bransford, Franks, Vye, & Sherwood, 1989). In addition, students in the problem-based learning
environment have developed stronger clinical competencies although the differences were small
and non-significant (de Vries, Schmidt, & de Graaff, 1989). A study conducted in a nutrition and
dietetics course found that PBL students perceived that they developed stronger thinking and
problem-solving skills, effective communication skills, and sense of personal responsibility than did
students who received lectures (Lieux, 1996).
 
Much of the medical school research shows that student attitudes toward learning do change.
Students in PBL courses often report greater satisfaction with their experiences than
non-PBL students. For example, PBL medical students at Harvard reported their studies to be  more
engaging, difficult, and useful than did non-PBL students (Albanese & Mitchell, 1993). Additional
research studies document that students who experience PBL have substantially more positive
attitudes toward the instructional environment than do students in more traditional
programs. PBL students tend to give high ratings for their training whereas students in traditional
programs are more likely to describe their training as boring and irrelevant (de Vries, Schmidt, &
de Graaff, 1989; Schmidt, Dauphinee, & Patel, 1987). These changes in attitudes are marked by an
impact on student retention. In countries with high dropout rates among medical school students,
such as in Holland, students in the PBL medical program were much more likely to graduate and do
so in less time than students in the more traditional curriculum (de Vries, Schmidt, & de Graaff,
1989). In addition, attendance was significantly higher in the PBL class than in the lecture version
(Lieux, 1996).
 
Research also shows changes in student study behavior. Coles (1985) and Newble and Clark (1986)
report that students were more likely to use versatile and meaningful approaches to studying than
non-PBL students, who were likely to use reproduction. Nolte, et al., (1988) found that use of
reserve material went up. Blumberg and Michael (1992) found that PBL students were more likely
to use textbooks and other books and informal discussion with peers than did non-PBL students,
who were more likely to rely on lecture notes.
 
Research of PBL in medical schools, as seen in the reviews by Albanese and Mitchell
(1993), Vernon and Blake (1993), and others mentioned above has focused primarily on comparing
the outcomes ofPBL methods to more traditional pedagogical methods. Research on PBL as a
method for preparing professionals has followed in this tradition. These studies do provide insight
as to how PBL compares to traditional methods.
 
However, PBL presents some unique challenges for assessment. Because the focus of this pedagogy
is primarily on learning to learn and less on mastery of a particular body of knowledge, traditional
methods of course assessment such as examinations may not be very effective (Major, 1999). If
traditional assessment is a good measure of traditional pedagogy, it stands to reason that an
alternative assessment may be necessarily a better measure for an alternative pedagogy, such
as PBL. Using alternative assessment in the case of PBL can help bridge the gap between
instruction and assessment. Authentic assessment uses tasks developed from realistic activities in
the professional world (Nightingale, Te Wiata, Toohey, Ryan, Hughes & Magin 1996). Nightingale,
Te Wiata, et al., (1996) define authentic assessment tasks as "complex simulations, case studies, . . .
or multi-faceted projects . . . assessing a range of knowledge, skills and attitudes in the one
assessment task."
 
Some signs of a movement in this direction exist. Recently studies have begun to
investigate PBL outcomes, such as teamwork or presentation skills, that may not be associated with
traditional lecture methods. Cockrell, Caplow, & Donaldson (2000), for example, recently
conducted a study examining students' perspectives on their learning as members of collaborative
groups. The researchers, using interpretive methods, found that the collaborative groups fostered
students' sense of ownership of the knowledge that was created over the semester. The researchers
also suggested that within the groups, leadership moved from student to student as situations arose
and resolved. More studies like this one are needed to determine the effectiveness of PBL in higher
education.
 
Discussion
These recent studies show that it is time to think outside of the box on how we assess the
effectiveness of PBL and how we think about its outcomes. Alternative assessment which is
authentic to the learning environment can have a positive influence. An alternative assessment
measure might include constructed response items, essays, writing samples, oral presentations,
exhibitions, experiments, and/or portfolios (Ewing, 1998). In a PBL classroom, these measures
might be much more relevant and authentic to a problem-solving setting than a traditional
standardized multiple-choice test. Allowing students to engage in these kinds of measures can allow
us to assess important learning by examining and judging the students actual or simulated
performance on significant tasks (Worthen, 1993).
 
As with any assessment planning process, the principles of good assessment can help guide
practitioners. Instructors should begin by knowing what they want their students to achieve and how
they want students to get there. They should consider that learning is a multidimensional activity;
including knowledge and abilities as well as values, attitudes, and habits of the mind. In addition,
when formulating learning goals, faculty members should think of learning in the larger context of
the educational community. Assessment of whether goals have been attained and learning has
occurred should have a clearly stated purpose, one related to the learning. Thus, it should focus on
comparing educational goals and expectations with performance. In addition, assessment should be
ongoing, throughout a semester, rather than occurring only at the end (American Association for
Higher Education, 1992).
 
While each particular PBL instructional environment is unique, and therefore merits its own unique
assessment strategy, several alternative assessment techniques seem particularly appropriate for
the PBLlearning environment.
Outside Evaluation by Experts. Since many PBL experiences involve a culminating experience such as a
presentation, written projects, or portfolio, systematic evaluation of these capstone projects by a team of outside experts can
afford one means of assessing student performance.
Content Analysis of Projects. Because PBL uses ill-structured problems with many possible solutions, each
student group within a class may pursue a slightly different domain of knowledge. To assess the range of content
knowledge learned by students in the class, instructors may need to evaluate across assignments and groups to look for the
variety of resources students are collecting. Project analyses may also be useful in assessing skills such as researching,
critical analysis, or writing.
 
Focus Groups. One method that can prove useful for assessing outcomes such as teamwork or leadership is the use of
focus groups. Students can offer perspectives on their experience within the problem-solving group and may be able to
reflect on their own growth across the experience.
 
 Peer Evaluations. Many PBL instructors use some form of peer evaluation as a means of moderating individual
student behavior within the group setting. However, these evaluations can also be used to gather data about the level of
skill development across individuals in the class.
 
Journals or Activity Logs. Students in PBL classes often do the bulk of the work for a project outside the formal
classroom. The work completed outside the classroom can be difficult to assess, so many instructors require students to
keep a log or journal of the work they complete for the project. Mid-semester and/or end of semester evaluations of these
journals or logs can provide excellent evidence of student learning throughout the project.
 
Personal Reflections. For many students, the type of active learning that PBL requires is an unusual experience. One
method for assisting students in their metacognitive understanding of the PBL process is to ask them to reflect on the
experience of PBL at key points in the process. Qualitative analyses of these reflections can offer supportive evidence for
many process-type outcomes such as developing critical thinking or research skills.
 
These techniques focus on the contextual nature of PBL, requiring the students to produce an
authentic product that is related to the problem and to make judgments about their performances.
 
Conclusion
The academy and its constituents have called for change in the way that teaching and learning are
carried out in undergraduate education, and change is happening on many campuses and in many
classrooms. Assessment must keep pace with these trends. We must begin to look beyond
traditional measures toward new and different ways of knowing. PBL is just one of the many
pedagogies that illustrate this fact, but educators' experience with PBL can shed new light on what,
why, and how we are doing in the academy.
 
References
Albanese, M.A. (1993). Problem-based learning: A review of literature on its outcomes and implementation
issues. Academic medicine: Journal of the Association of American Medical Colleges, 68 (1), 52-81.
American Association for Higher Education (1992). Nine principles of good practices for assessing student
learning. Washington, D.C.: American Association for Higher Education.
Annis, L. F. (1983). The process and effects of peer tutoring. Human Learning, 2, 39-47.
Barr, R., & Tagg, J. (1995). From teaching to learning: A new paradigm for undergraduate education. Change, 27 (6),
12-25.
Barrows, H. S. (1996). Problem-based learning in medicine and beyond: A brief overview. New Directions for Teaching
and Learning, 68, 3-12.
Blumberg, P. & Michael, J. A. (1992). Development of self-directed learning behaviors in a partially teacher-directed
problem-based learning curriculum. Teaching and Learning in Medicine, 4 (1), 3-8.
Boud, D. & Feletti, G. I. (1991). The challenge of problem-based learning. London, Kogan.
The Boyer Commission on Educating Undergraduates in the Research University. (1998). Reinventing undergraduate
education: A blueprint for America's research universities. New York: The Carnegie Foundation for the Advancement of Teaching.
Bransford, J. D., Franks, J. J., Vye, N. J., & Sherwood, R. D. (1989). New Approaches to Instruction: Because Wisdom
Can't Be Told. In S. Vosiadou & A. Ortony (Eds.), Similarity and Analogical Reasoning (pp. 470-297). New
York: CambridgeUniversity Press.
Cockrell, K. S., Caplow, J. A. H., & Donaldson, J. F. (2000). A Context for Learning: Collaborative groups in the
problem-based learning environment. Review of Higher Education 23 (3), 347-363.
Coles, C. R. (1985). Differences between conventional and problem-based curricula in their students' approaches to
studying. Medical Education, 19 (4), 308-309.
Cross, K. P. (1998). Opening windows on learning: The cross papers number 2. Mission Viejo, CA: League for Innovation
in the Community College and Educational Testing Service.
Cross, K. P. (1999). Learning is about making connections: the cross papers number 3. Mission Viejo, CA: League for
Innovation in the Community College and Educational Testing Service.
de Vries, M., Schmidt, H. G. & de Graaf, E. (1989). Dutch comparisons: Cognitive and motivational effects of
problem-based learning on medical students. In Schmidt, H. G., Lipkin, M., de Vries, M. W. & Greep, J. M. (eds), New Directions
for Medical Education: Problem-based learning and community oriented medical education. (pp. 230-240) . New York:
Springer-Verlag.
Ewell, P. T. (1997). Organizing for learning: A new imperative. AAHE Bulletin, 50 (4), 3-6.
Ewing, S. C. (1998). "Alternative assessment: Popularity, pitfalls, and potential." Assessment Update: Progress, Trends,
and Practices in Higher Education 10, (1) 1, 2, 11, 12.
 Jones, E. A. (1997). Identifying college graduates' essential skills in reading and problem solving: Perspectives of faculty,
employers, and policy makers. University Park, PA: National Center on Postsecondary Teaching, Learning, and Assessment.
Lieux, E. M. (1996). A comparative study of learning in lecture versus problem-based format. About Teaching, 50, 25-27.
Major, C. (1999). Connecting what we know and what we do through problem-based learning. AAHE Bulletin, 51 (1), 7-9.
McKeachie, W. J., Pintrich, P. R. Lin, Y.-G, & Smith, D. A. F. (1986). Teaching and Learning in the College Classroom:
A Review of the Research Literature. Ann Arbor: University of Michigan, National Center for Research to Improve Postsecondary
Teaching and Learning.
National Education Goals Panel. (1992) . The national education goals report: Building a nation of
learners. Washington, DC: U. S. Government Printing Office.
Newble, D. I., & Clarke, R. M. (1986 ). The approaches to learning of students in a traditional and in an innovative
problem-based medical school. Medical Education, 20 (4), 267-273.
Nightingale, P., Te Wiata, I., Toohey, S., Ryan, G., Hughes, C., & Magin, D. (eds). (1996). Assessing learning in
universities. Sydney: UNSW Press.
Nolte, J., Eller, P. & Ringel, S. P. (1988). Shifting toward problem-based learning in a medical school neurobiology
course. Proceedings of the Annual Conference on Research in Medical Education, 27, 66-71.
Schmidt, H. G., Dauphnee, W. D., & Patel, V. L. (1987). Comparing the effects of problem-based and conventional
curricula in an international sample. Journal of Medical Education, 62 (4), 305-315.
Vernon, D. A., & Blake, R. L. (1993). Does problem-based learning work? A meta-analysis of evaluative
research. Academic Medicine, 68 (7), 550-563.
Worthen, B. (1993). Critical issues that will determine the future of alternative assessment. Phi Delta Kappan, 74, (6),
444-448, 450-454.