An Overview of Physics Education

Research on Problem Solving

David P. Maloney

Indiana University Purdue University Fort Wayne

Fort Wayne, IN 46805

Abstract:

An overview of the research on solving tasks commonly used as

“problems” in introductory physics courses is presented as an introduction
to this domain of physics education research. This overview, which is not
intended to be exhaustive, describes many aspects of the complex topic of
investigating human problem solving to help identify issues of potential
interest to researchers and instructors. The article identifies links between
research in physics education and more general research on problem
solving, and provides useful references. The article ends with a dozen
open questions which the author believes deserve answers.

Getting Started in Physics Education Research 1

Maloney Research in Problem Solving

1. Introduction
Investigating humans engaged in problem solving is a very diverse and
complex endeavor, and narrowing the focus to investigating students
engaged in problem solving in introductory physics only reduces the scope
a little. Early general problem-solving research established several
aspects/facets of problem solving that apply to all domains. However, how
these general aspects play out within a particular domain depends on the
domain, and problem solving within a specific domain has features that
are particular to that domain. Because there are many interactions between
the problem-solving processes people use and their domain knowledge,
presenting research on problem solving cannot be done in a nice, clean,
sequential manner. Consequently, deciding how to structure this
introductory overview presented the author with a rather challenging
problem.

The result, after much thought and struggle, is the following organization:
(A) General background on research on problem solving; (B) Aspects of
how individuals solve physics problems, which consists of three parts—
problem-solving steps, problem-solving strategies and problem-solving
states of mind (this framework is borrowed from the ThinkFun® Game
Club website: www.thinkfungameclub.com.); (C) Aspects of “teaching
problem solving”; and (D) Miscellaneous aspects. Finally, we take stock
of where we are in understanding problem solving in physics and identify
some possible future research questions.

It is important to point out two things here at the beginning. First, even
though the focus of this presentation is problem solving in physics, a
number of the studies that are described do not involve physics.
Nonetheless these studies, because problem solving is a concern for many
fields and shares characteristics in those fields, do tell us useful things that
can be applied to problem solving in physics.

The second thing to be aware of is that this presentation is not intended to

be in any way exhaustive. If one is interested in pursuing research in this
area it would be useful to consult the review by Maloney1 and the resource
letter by Hsu, Brewe, Foster, and Harper2 to get a better perspective on
what work has been done in physics. In addition, consulting problem-
solving reviews derived from other perspectives3-5 is critical for placing
research on physics problem solving in the broader context of problem-
solving research.

Getting Started in Physics Education Research 2

Maloney Research in Problem Solving

2. General Background
2.1 The Difficulty with Defining “Problem”

As stated above, problem solving involves a very diverse and complex set
of processes, and one of the difficulties with interpreting the problem-
solving literature is comparing and contrasting the activities associated
with people working the spectrum of tasks that are called problems. As
Adams points out, “The study of problem solving is almost impossible if
you try to look at it as one thing that a person does. It has many facets and
to study these it’s useful to isolate and identify the individual facets.”6

There are many definitions for problem in the literature and while they
share a number of elements, they are not identical. In many studies the
authors never provide a definition of what they are calling a problem,
rather it is taken for granted that the reader understands and agrees with
the researchers that the tasks presented to the subjects were problems for
the subjects. This approach is “problematic.”

One useful general definition of a problem is given by Hayes: whenever

there is a gap between where you are now and where you want to be, and
you don’t know how to find a way to cross that gap, you have a problem.7
It is useful to notice that this definition implies that tasks or situations are
not in themselves problems. The problem arises when an individual
interacts with the task or situation. Different people interacting with the
same task/situation might not all find it to be a problem. The skills and
knowledge an individual brings to a situation play a major role in whether
that individual thinks of a situation as a problem.

While the Hayes description provides a general definition of a problem, it

is actually of limited usefulness for comparing studies that have been
done. The reason for this is that anything from an ill-defined, complex,
multi-step task to a one-step knowledge recall true/false question can be
considered a problem under this definition. However, while some of the
processes employed by an individual responding to these two tasks might
be the same, there will also be large differences in the reasoning used to
deal with them, or with conceptual tasks versus computational tasks. This
means that identifying clearly and explicitly what tasks/situations are
being used to produce the interactions that qualify as problems for the
subjects in a study is very important.

Getting Started in Physics Education Research 3

Maloney Research in Problem Solving

As an example of the variation in the nature of the reasoning that two

different types of problems require, consider the following two physics
tasks. In task A the mass of an object and the magnitudes and directions of
two forces acting on the object are given, and the goal is to find the
acceleration of the object. In task B the mass of an airplane, its landing
speed and the magnitude of the net (braking) force it can have are given
along with the length of the runway, and a question is posed: Can the
plane land safely? These two tasks share a number of characteristics, but a
specific difference is how the goal of the task is identified. For the first
task the goal is to find a specific numerical value. The simplest way to
achieve this goal is to use the equation which involves the given values
and the unknown. No real analysis of the physical system and its behavior
is needed. In contrast, in the second task the solver has to first determine
how to use the given information to find something that will enable
him/her to answer the question. In this case someone would need to think
about what physical principles/relationships are relevant and which one(s)
will lead to being able to answer the question. Other task formats (problem
types), such as context rich problems,8 Jeopardy problems,9 or ranking
tasks10 require other and/or additional reasoning processes.

The wide range of tasks that qualify as problems for novices, who by
definition have little domain specific knowledge for a field of study, is
part of what makes investigating problem solving a complex domain of
study. Adding to the complexity is the fact that problems can be classified
in a variety of ways, such as conceptual or quantitative, or well-defined
versus ill-defined. For the latter contrast, there is actually a spectrum along
which problems can be placed.11

Using the definition of problem given above, there are three components
of a problem: the initial state, the goal state, and procedures to eliminate
the gap between them. A very well-defined problem would have all three
aspects explicitly identified. For example, end-of-chapter numerical
exercises fit this description of well-defined since the initial state (the
given values), the final state (the quantity to be found) and the procedures
to be used are all roughly specified. The procedures to be used are not
exactly specified, but these tasks are often labeled with the section to
which they relate and doing that specifies the procedures indirectly.
(Actually one might argue that such tasks do not qualify as problems
under the Hayes definition.) Problems that have one, or more, of these
three features not explicitly identified then fall further along the spectrum
toward ill-defined. The ill-defined end of the spectrum has situations
where the problem solver may not even be sure there is a problem; such

Getting Started in Physics Education Research 4

Maloney Research in Problem Solving

tasks require explicit identification and definition of the problem, i.e., the
initial state, goal state, and nature of the gap, before an attempt can be
made to solve them. The vast majority of academic problems are well-
defined while real-world problems tend to be ill-defined to various
extents.

In light of these aspects of the problem-solving domain, anyone studying

the literature in this domain should take care to identify what
tasks/situations the authors are calling problems and whether such
identification is reasonable for the subjects involved in the study.

2.2 Types of Problems

Since the nature of the tasks/situations that can be problems for people is
so broad, it is useful, at least at times, to have ways to classify and
distinguish among those that share characteristics. Not surprisingly,
considering the previously mentioned broad range of tasks that can be
problems, there are various ways to classify problems. Perhaps the most
obvious way is on the basis of the knowledge domain involved, i.e.,
physics or chemistry problems. However, there are other ways to classify
problems that can be useful if we are interested in gaining insight into
students’ solving approaches and abilities.

One general dichotomous classification scheme has insight problems as

one category and what might be called non-insight, or systematic,
problems as the other category.12 Systematic problems are those that are
amenable to solution by persistent application of known procedures. An
example would be the popular Sudoku puzzles for which even the most
challenging versions can be solved by systematic application of the guess
and test heuristic. (A heuristic is a general problem-solving procedure that
one typically employs when he/she doesn’t have specific actions available.
Much more will be said about heuristics below.) In contrast, insight
problems require, somewhere along the way, a breakthrough shift in
thinking—the Aha moment—to solve.13 The identification of these two
types of problems immediately raises a number of questions such as: Does
experience with systematic problems promote skill at solving insight
problems in a domain? Does solving one insight problem facilitate
solution of others? We will return to these questions later.

We have already mentioned above the classification of problems along a

spectrum from well-defined to ill-defined. Another classification scheme
which has some relation to the well-defined/ill-defined spectrum was

Getting Started in Physics Education Research 5

Maloney Research in Problem Solving

developed by Johnstone.14 This scheme for problem types is based on

status for the solver of three components: data given, method to be used,
and goal. These clearly correspond to the three parts, initial state,
procedures for reducing the gap and goal state identified in the Hayes
definition. The eight resulting problem types are:

Type Data Methods Goals/outcomes

1 Given Familiar Given
2 Given Unfamiliar Given
3 Incomplete Familiar Given
4 Incomplete Unfamiliar Given
5 Given Familiar Open
6 Given Unfamiliar Open
7 Incomplete Familiar Open
8 Incomplete Unfamiliar Open

Looking at this scheme it is pretty clear that the vast majority of tasks that
students encounter in academic situations are types 1 to 3, but real world
problems are more commonly types 4 to 8. How do we help students
develop the skills they need to tackle those types of problems?

Jonassen has developed a general typology of problems that includes 11

groups. The different types are: (1) logic problems, (2) algorithms, (3)
story problems, e.g., the typical end-of-chapter word problems found in
science, technology, engineering and mathematics (STEM) textbooks, (4)
rule-using problems, (5) decision making problems (which usually require
that problem solvers select maximal solutions from a set of alternative
solutions based on a number of selection criteria), (6) troubleshooting, (7)
diagnosis-solution problems, (8) strategic performance, (9) policy
analysis, (10) design, and (11) dilemmas (problems which involve social
and/or ethical conflicts).4 He contends that his scheme “represents a
developmental theory of problem solving” and that “How discrete each
kind of problem is, and whether additional kinds of problems exist, is not
certain.”11 Clearly not all of these categories are involved in scientific
problem solving, but several—e.g., algorithms, story problems and rule-
using problems—just as clearly are part of science.

Jonassen also argues that problems can be encountered as either discrete

items or as aggregates. He contends that the latter are more common in
real work contexts.11 To what extent experience with one type of
problem—troubleshooting—provides useful learning for dealing with a
different type—story problems—is very much an open question.

Getting Started in Physics Education Research 6

Maloney Research in Problem Solving

2.3 The Difficulty with “Problem-solving Skills”

The difficulties described above about defining problems carries over into
discussions about “problem-solving skills”. Teaching “problem-solving
skills” is fairly commonly cited as a major goal of physics, or mathematics
or chemistry, instruction. However, determining what this means, and
whether it can actually be done, is another matter. Even a rather quick
exploration of the problem-solving literature brings to light the fact that
there is a difficulty with even identifying what qualifies as a problem-
solving skill. While many researchers, not to mention teachers, strongly
believe that there are general problem-solving skills, those skills are
always applied to, and with, knowledge from some specific domain.
Consequently, identifying skills that operate in the same way in all
domains is very difficult.

On one hand few people would argue with identifying such skills as being
able to identify given information, being able to specify what the goal
state is (stating the unknown), determining what concepts, principles, and
relations are needed, and reviewing one’s work as general problem-
solving skills. However, these processes are so general that explicitly
stating, and trying to teach, them is of limited value, unless we are talking
about working with very novice subjects who lack both knowledge and
general reasoning skills. In contrast, skills such as drawing appropriate
representations (free-body diagrams, equipotential diagrams, or ray
diagrams), translating qualitative situations into quantitative relations, or
making assumptions in order to constrain a situation to one for which
quantitative relations can actually be developed are more specific to
context.

What is the value of identifying potential general problem-solving skills

like planning, using heuristics and evaluating the solution, when the actual
application of each varies widely depending on each situation? How useful
is it for an individual to be able to plan, but not have the specific domain
knowledge needed to accomplish the plan? On a related aspect, how do we
effectively teach such general problem-solving skills, assuming it is
worthwhile to do so? What has been called the dilemma of teaching
problem solving nicely describes the difficulty of teaching problem-
solving skills: “If a student is presented with problems which he/she can
solve with his/her current problem-solving procedures they will see no
need to change their approach. However, if they are presented with
problems on which their current methods fail they are likely to simply give
up and conclude that the problem cannot be solved.”8

Getting Started in Physics Education Research 7

Maloney Research in Problem Solving

2.4 Student Epistemology and Problem Solving

Another aspect of investigating students’ problem-solving approaches is

the matter of student epistemology. As Hammer found, students tend to
have one of two perspectives on learning physics: they either focus on
getting the answer, or they work to have the physics make sense.15 A
student who is only interested in getting an answer will approach a typical
end-of-chapter problem in general physics one way, i.e., find the equation
to plug the given numbers into and solve. In contrast, a student who is
trying to make sense of the physics will approach it a different way, e.g.,
by actually thinking about the physical system involved and how it is
behaving.

In addition to the issue of how students frame what it means to “learn

physics,” there is also the issue of how they frame what “problem solving”
means. We could think of the two perspectives on learning leading to
different representations of what “solving a problem” means. For many
students, in large part because of previous academic experience, solving
problems means applying “plug and chug” to some given information to
find a numerical answer. As we will see, plug and chug is a version of a
common heuristic called means-ends analysis, but limiting one’s
perspective of what problem solving is to just this process constrains the
individual’s thinking entirely too much. Getting students who have such a
perspective of problem solving to deal effectively with other
tasks/situations that are also problems for them is another part of the
dilemma of teaching problem solving.

How do we characterize the problem-solving skills of students who have

these different representations of what solving a physics problem means?
Is it reasonable to characterize their problem-solving skills on the basis of
how they do physics tasks when they might actually construct the opposite
representation of what solving the problem means if the task was an
economic one?

There have also been investigations of how student epistemology affects

the development of expertise, in contrast to just acquiring more
experience. For example, Bereiter and Scardamalia have argued that
sense-making and incorporating the understanding gained from doing so
when solving a new problem leads to expertise. In contrast, focusing only
on getting an answer leads to a restricted set of procedural skills.16

2.5 Methodology

Getting Started in Physics Education Research 8

Maloney Research in Problem Solving

In considering methods used in problem-solving research, the two basic

categories of techniques—quantitative or qualitative—employed are the
same as those used in other areas of educational research. Chief among the
qualitative approaches is the think-aloud protocol, which has been a part
of a number of studies. In think-aloud protocols the subject is asked to do
a task while verbalizing everything they are thinking about as they do the
task. Chi points out that there are two types of think-aloud protocols,
“protocol analysis” where the researcher interferes minimally and “verbal
analysis” where the researcher probes for self explanations.17 Quantitative
techniques often use paper and pencil tasks, such as open-ended problems,
with relatively large samples. What tends to be different in the problem-
solving work is the subject pool and the types of tasks used. The following
sections describe four common approaches used to study problem solving
in physics.

2.5.1 Expert-Novice Studies

This “technique” actually includes a variety of different investigations

which share the fact that the performance of two groups of subjects is
compared. One group, the novices, usually consists of students taking a
general physics course. The other group, the experts, ranges from
advanced undergraduate physics majors through graduate students, to
experienced college physics professors.

Contrasting the think-aloud protocols of experts and novices working end-

of-chapter tasks from general physics textbooks was one of the early
approaches to investigating problem solving in physics.18 The subjects
were audio, or video, taped as they worked the task and described their
thinking. The researcher’s job in such investigations is to take notes and
prompt, in a non-directive manner, the subjects if they are clearly thinking
but not vocalizing. The researcher strives to be unobtrusive and non-
directional. However, he/she can take note of situations where clarification
or amplification is needed and develop questions to ask when the subject
has completed the task.

An issue with those early expert-novice studies is the nature of the tasks
the subjects were working on. For example, Larkin, et al. used end-of-the-
chapter tasks from an introductory calculus-based physics text as their
problems. These tasks could legitimately be called problems for the
novices in their study. However, for the experts those tasks hardly
qualified as problems because of the experts’ experience and knowledge
base. Consequently, what was actually being investigated was how large

Getting Started in Physics Education Research 9

Maloney Research in Problem Solving

amounts of experience and an organized knowledge base alter the

approach to what can be called basic tasks. And what was found was that
the experts’ experience and knowledge base—hierarchically organized
knowledge of physics principles—enabled them to use a working-forward
approach. This working-forward approach was guided by a qualitative
analysis and specific physics representations. By contrast, the novices,
since they lacked these resources, were left to make use of whatever
heuristics they knew. What novices know in this situation is one version of
a general heuristic called means-ends analysis, which is typically
described as “plug and chug.”18

In contrast to those early studies, Singh conducted an expert-novice study

where the task could legitimately be called a problem for both groups.
With one exception, none of the subjects in either group was able to solve
the task in the time allowed. Consequently, this study tells us about the
more generic, i.e., not specific physics, resources the two groups
employed in their efforts to solve the problem. What was found was that in
this regard the experts still had more resources available in the form of
general heuristics, such as making simplifying assumptions, and specific
heuristics, such as thinking in terms of conservation relations. 19

2.5.2 Card-Sorting Studies

One technique that is unique to the problem-solving research was first

used in the classic study by Chi, Feltovich, and Glaser.20 The technique is
sorting cards, each of which has a single “problem” on it, into categories
on the basis of how the individual would solve the problem. These
researchers chose 25 problems from the ends of the chapters of the
mechanics section of a well-known general physics textbook and copied
each one onto a separate index card. They then asked their subjects to
classify the problems into as many groups as they thought reasonable
based on similarity in how they would solve them and to name or describe
each category. The researchers also compared how experts—graduate
students—and novices—students enrolled in a general physics course—
differed in doing the task. They found that the experts constructed
categories based on the physical concepts and principles that would be
used to solve the problems. In contrast the novices’ categories tended to be
based on “surface” characteristics of the problems such as objects on a
ramp or pulleys.

2.5.3 Isomorphic Problem Studies

Getting Started in Physics Education Research 10

Maloney Research in Problem Solving

Another approach is to have subjects work problems that are isomorphic,

i.e., share the same solution process, but which vary in “surface structure.”
By systematically varying the features between problems one can
investigate how those factors influence subjects’ techniques. For example,
Singh used this approach to investigate how factors, such as quantitative
versus conceptual task, context, and students’ alternative ideas about
friction, affected students’ ability to transfer problem-solving skills from
one task to another. 21 Subjects in the study solved more qualitative
problems when they were coupled with an isomorphic quantitative
problem, but the pairing did not promote more success with quantitative
problems. Another major finding was that students’ alternate ideas about
friction strongly interfered with their ability to benefit from isomorphic
problems that do not involve friction.

2.5.4 Instructional Studies

Another approach that has been used, especially with computer tutors, is
to attempt to teach subjects how to solve problems and carefully monitor
what they do. One of the early investigations in this area was the study by
Heller and Reif. 22 They taught a “prescriptive model” of problem solving,
which focused on formulating a theoretical description, i.e., a physical
representation, to a group of subjects. The subjects using the prescriptive
model performed significantly better than a control group, and also better
than a group using a modified version of the model. The prescription these
researchers developed was very specific and restricted to problems
involving the application of Newton’s second law.

Researchers at the University of Minnesota have developed an

instructional program involving innovative problem tasks, students
working in cooperative groups and a structured problem-solving
strategy.8,23 The innovative problem tasks are called “context rich
problems” and while these tasks vary in several ways they do share some
specific characteristics. (Context rich problems can legitimately be
classified as ill-defined, although they are usually not radically so.)
Among the shared features are: there is a “cover story” which explicitly
places the problem solver into the story—this is done to provide
motivation; there are no diagrams provided; the problems cannot be solved
in a single step; and the goal is not explicitly identified, but rather has to
be determined as part of the solution process.

The second part of the Minnesota program is a structured problem-solving

strategy modeled on the Polya four step framework—understand, plan,

Getting Started in Physics Education Research 11

Maloney Research in Problem Solving

carry out, and look back.24 The elements of the Minnesota strategy are 1)
focus the problem, 2) describe the physics, 3) plan the solution, 4) execute
the solution, and 5) evaluate the answer. The strategy is explicitly used by
instructors when they solve example problems for the students and the
students are required to use it when they submit work. The third element
in this program was the use of collaborative grouping, with assigned roles
for the students, for working on the problems in discussion sessions.

Leonard, Dufresne, and Mestre reported on an instructional program

which required students to write explicit strategies for how they would
solve problems. The researchers established guidelines for what
constituted a good strategy and gave students practice in writing these
strategies. The results of the study demonstrated that students who wrote
good strategies were better able to identify relevant principles and
remember important ideas at a later testing.25

Section 2 Summary: Problem solving—even the restricted problem

solving of students in introductory physics courses—is a very diverse and
complex domain. What situations are considered to be problems depends
on both characteristics of the task and characteristics of the human
interacting with the task. Problems can be classified on the basis of a
variety of features, but two important classifications are insight versus non
insight and the spectrum of ill-defined to well-defined. While a general
consensus exists that “problem-solving skills” exist, specifically
identifying such skills is difficult since all problem solving involves
specific domain knowledge. Student beliefs about the nature of learning
physics and the role of problem solving in learning physics (i.e., their
epistemology) affects how they approach problem-solving tasks. There are
four general approaches—expert-novice, card sorting, isomorphic
problems, and instructional studies—that have been used to investigate
physics problem solving.

3. Aspects of Individuals Solving Physics Problems

This section will focus on what has been learned about how individuals
function when solving problems. The author has chosen to use a
framework from the ThinkFun® Game Club website. The ThinkFun®
company makes a range of puzzles and games. Within the company’s
website there is a section devoted to exploring how games can teach
problem solving where they state: “If a child can explain to you: the Steps
s/he used to evaluate a problem, the Strategy s/he used to solve the

Getting Started in Physics Education Research 12

Maloney Research in Problem Solving

problem, his/her State of Mind when working through the problem, then
we believe s/he has learned the fundamentals of problem solving.” The
author believes this applies to our students also and provides a useful
guide for research on individual differences in problem solving and for
what we need to help students learn in order to be good problem solvers.

3.1 Problem-Solving Steps

In the ThinkFun® framework these are identified as: (1) understand, (2)
choose, (3) do, and (4) inspect. Obviously these are quite similar to the
four step Polya framework (see above). One can think of these aspects as
the overall model of the task and the general strategy one adopts toward
the problem. Several features of these steps, such as the fact that students
tend to jump right into trying to solve a problem, have been investigated in
physics.18 The common failure of people to inspect or review their
solutions is also known. 26 However, the focus will only be on one
aspect—the development and use of representations to understand the
problem—in this section. The reason for focusing on only this one aspect
is to keep this review from getting any longer and to highlight the critical
importance of representations.

With regard to the first step mentioned above, understand, a critical part of
understanding a problem is the representation the solver constructs, i.e.,
the internal mental model of the problem situation that the solver forms.
There are at least two types of representations associated with a problem
and problem solving. One is the internal mental model of the situation
which can lead to the type of stumbling block described in the next
paragraph. However, there is also the representation of what the process of
solving a problem involves, e.g., is the solver only trying to get an answer,
or is there also some learning and understanding supposed to come from
the process? More will be said about this later in the states of mind
section.

The process of forming a representation, i.e., constructing a mental model,

has been found to be a critical part of the problem-solving process. One of
the common examples of a problem where this feature is critical is the
Nine Dots problem. This task has nine dots laid out in a 3 x 3 grid and the
goal is to connect all nine dots with four straight lines without lifting the
pen or pencil from the paper. Most people find this task very challenging,
because the representation they construct has the constraint that the four
lines have to fit within the grid defined by the arrangement of the dots.
Actually this constraint, which is not part of the task, makes the task

Getting Started in Physics Education Research 13

Maloney Research in Problem Solving

impossible, so someone trying the task cannot make progress until they re-
represent the task.

One of the first studies of student’s approaches to problem solving in

physics was that of Chi and her colleagues mentioned in the methodology
section above. The subjects in their study were asked to classify a set of
mechanics problems based on how they would go about solving them.
Recall that the two groups produced different classification schemes. The
experts classified the problems on the basis of the physical principle(s)
needed to solve them. In contrast, the novices tended to classify the
problems on the basis of the objects and their behavior, i.e., on the basis of
the surface structure.20

Here we see one effect of the problem representation on novices’ attempts

to solve physics problems. Since the novices lack extensive domain
knowledge, they are forced to construct representations that employ ideas
and features, i.e., everyday or surface ones, with which they are familiar.
In contrast, the physics experts’ representations incorporate important
physics concepts, principles and relations.

One of the reasons representations are so important in the problem-solving

process is because it is actually the representation that the problem solver
works on. If the solver constructs an effective representation he/she is on
the way to solving the problem, but if the solver constructs an
inappropriate representation he/she will not be able to make any real
progress until they re-represent the problem accurately.12 One implication
of this feature of problem solving is that effectively representing and re-
representing problems is an important problem-solving skill. And this skill
includes being aware of the nature of representations and the possible need
to re-represent a problem, especially if one is stuck and cannot identify the
nature of the sticking point.26

Larkin also investigated the use of representations in physics problem

solving by investigating how six graduate students approached a virtual
work problem in classical mechanics. She found that several of the
students started with one representation, e.g., conservation of energy or
conservation of angular momentum, with which they analyzed the
situation. These students found their initial representations unproductive
so they changed one, or more, times until they got to the virtual work
representation. Once they developed the virtual work representation they
were able to write an equation and solve the problem.27

Getting Started in Physics Education Research 14

Maloney Research in Problem Solving

The term representation is actually used two ways when talking about
problems. As mentioned above the issue of the internal model of the
problem situation that a solver forms, i.e., the representation, is an
important aspect of problem solving. However, the term “representation”
can also be used to describe the format of the task, e.g., verbal, graphical,
diagrammatic, etc. This aspect of physics problems has been the focus of
some recent research. Meltzer investigated how students in an algebra-
based physics class performed on quizzes which had essentially
isomorphic problems in different formats (verbal, equation, graph, or
diagram). He found that student performance varied with format and
topic.28

Kohl and Finkelstein investigated how students handle different types of

physics task formats such as verbal descriptions, equations, graphs, and
diagrams. They found that experts and novices both used multiple external
representations on the tasks involved in their study. However, subjects
differed in how they used the external representations. Experts and
successful novices spent time using the external representations to make
sense of the physics in the task, while unsuccessful novices seemed to
draw pictures and free-body diagrams out of a sense of requirement.29 It is
important to realize that, while there is a relation between these external
representations such as graphs, free-body diagrams, ray diagrams, etc, and
the mental model the problem solver forms, it is the latter that is usually
meant when the term representation is used in discussions of problem
solving.

3.2 Problem-Solving Strategies

One of the earliest works on problem solving was mathematician George

Polya’s How to Solve It, published in 1945.24 In this volume Polya tried to
teach a method for solving mathematics problems. His focus was general
problem-solving approaches or procedures—heuristics—and he brought
their importance in mathematical problem solving into focus. Polya
actually presented two different levels of heuristics by identifying a broad
four step—1) understand the problem, 2) make a plan, 3) carry out the
plan, and 4) look back—approach, as well as more specific heuristics,
such as working backwards or exploiting symmetry.

Polya presented a prescription for an effective problem-solving strategy.

Few students actually employ such prescribed strategies. Rather they have
their own natural strategies, which are commonly much less well
organized and structured.

Getting Started in Physics Education Research 15

Maloney Research in Problem Solving

The problem-solving strategies identified in the ThinkFun® framework

are more specific heuristics such as subdividing, working backwards and
guess and check. The process of searching for a way to close the problem
gap is where heuristics such as those just mentioned come into play most
directly. In a study using the Chinese ring puzzle—a set of six interlocking
rings on a oval loop—Kotovsky and Simon found that all of the subjects
who successfully solved the puzzle went through a basically similar
process of exploration. The subjects initially randomly made progress
toward the solution and then backtracked, i.e., made moves that took them
away from the solution, then forward again, and more backtracking.
Subjects repeated this process until they figured out the structure of the
puzzle, at which point they then proceeded directly to the solution. In
essence they were exploring the task domain using various heuristics such
as trial and error or means-ends analysis until they understood the task,
which meant in this case determining the appropriate algorithm. Once that
was identified, they applied the algorithm to solve the problem.30

In terms of general strategies, one classic study found a definite difference

between experts and novices on the tasks usually assigned in general
physics courses. Larkin and her colleagues conducted think-aloud
protocols with experts and novices who were engaged in solving problems
drawn from general physics textbooks. The researchers then developed
computer programs that could basically reproduce the performance of the
two groups. The two groups differed in the basic approaches they
employed, with the novices using a working-backward strategy while the
experts used a working-forward process.18 What the researchers called the
working-backward strategy—identifying the unknown, searching for an
equations containing the unknown, seeing if all other quantities in the
equation are known, etc. (By the way, this working-backward strategy
should not be confused with the working-backwards heuristic.)—is
familiar to anyone who has taught physics for even a short period as “plug
and chug.” “Plug and chug” is actually a specific example of a general
heuristic process called means-ends-analysis. In contrast the experts (who
were working on tasks better described as exercises for them) started with
the given information, developed qualitative representations and worked
toward the mathematical relation needed to solve the problem. The
differences in the performance of the two groups are consistent with the
knowledge, and knowledge organization, the two groups brought to the
task.

Sweller and colleagues conducted a series of studies which led them to

argue that student use of means-ends analysis on typical numerical goal

Getting Started in Physics Education Research 16

Maloney Research in Problem Solving

problems was counterproductive in helping students develop a better

conceptual understanding.31-33 The main thesis of these studies was that
the use of means-ends analysis—in the form of plug and chug—requires
such a large fraction of an individual’s cognitive resources that they have
few resources left to devote to thinking about the concepts, principles, and
relations involved in the task situation.

Heuristic use has not been studied much in investigations of problem

solving in physics. An exception is one of the studies by Singh mentioned
above. One of the differences she found when she gave experts and
novices a task that was a problem for both groups was that the experts had
better heuristic skills. The experts thought about behavior in extreme cases
and had domain-specific general principles, such as conservation of
energy to help them. The novices were often at a loss when their plug and
chug approaches were inapplicable.19

3.3 Problem-Solving States of Mind

In physics education research this aspect of individual problem-solving

efforts has received relatively little attention until recent work on student
epistemology. In the ThinkFun® framework, believing that one is a good
problem solver, being confident, persevering, being willing to take some
risks and being confident are listed in this category. While there have been
some studies of how well students persevere (e.g., Schoenfeld, who found
that many students thought that if a problem could not be solved within
ten minutes then it was essentially impossible), there is very little work on
the other aspects mentioned.

However, Schoenfeld conducted several studies in which he explored the

effect of having students engage in metacognition about their problem-
solving efforts. From observation of many hours of video tapes of students
solving problems in pairs or groups Schoenfeld identified the importance
of control episodes, e.g., what planning solvers engaged in and whether
they monitored and reviewed their work as they went. He found that
students’ problem-solving abilities could improve if they were explicitly
taught about heuristics and how to use them.34

Schoenfeld also analyzed student problem-solving attempts using a system

that identified four facets of the process: (1) resources, which is the body
of knowledge an individual can bring to bear on the task, (2) heuristics,
which are the rules of thumb for developing an understanding of the task,
(3) control processes, which deal with issues of resource management

Getting Started in Physics Education Research 17

Maloney Research in Problem Solving

during problem solving, and (4) beliefs, which are one’s world view. The
beliefs component of this framework obviously relates to students’
epistemologies. As an example of how students’ beliefs affect their
problem-solving efforts, Schoenfeld found subjects took an empirical
approach to a mathematical construction problem that is better addressed
using deductive geometry, even though the students were subsequently
found to be capable of making deductive geometric arguments.34

Tuminaro and Redish looked into student attempts to solve problems from
the perspective of what they call epistemic games. They define an
epistemic game as: “a coherent activity that uses particular kinds of
knowledge and process associated with that knowledge to create
knowledge or solve a problem”. They found that students use a limited set
of these games which “…appear to provide the students with guidance as
to what knowledge and procedures to access and what to ignore.”35

Section 3 Summary: When an individual works on a problem, there are

three facets of their thinking that are critical: the steps they take, the
strategy they use as a guide for the overall process and their state of mind
about problem solving and their abilities to solve problems. The first, and
arguably most important, step to solving a problem is forming a mental
representation of the task. An inappropriate representation is an actual
obstacle which must be corrected before effective work can occur. Novice
strategies can not only be unproductive, they can actually be counter-
productive. Students’ beliefs about what constitutes a problem, how they
are solved, and their abilities to solve problems strongly affect what they
can accomplish.

4. Aspects of “Teaching Problem Solving” in Physics

4.1 Teaching “Problem Solving”

As Docktor and Heller point out: “Problem solving is one of the primary
goals, teaching tools, and evaluation techniques of physics courses.”36 In
this section we will look at some additional efforts to teach problem
solving beside those described above. All such efforts involve some form
of what can be called a global heuristic, having such steps as define the
problem or draw a diagram.

Gaigher et al. report on a study of teaching a “structured problem-solving

strategy” and its effect on improving students’ problem-solving skills and

Getting Started in Physics Education Research 18

Maloney Research in Problem Solving

conceptual understanding.37 Their structured strategy, which they based on

an “extended semantic model” of Greeno, has definite similarities to other
frameworks, going all the way back to Polya. Their strategy has seven
steps: 1) draw a simple diagram to represent the system, 2) indicate the
data on the diagram, 3) identify the unknown variable, 4) analyze the
problem in terms of physics principles, 5) write down the relevant
equation(s), 6) substitute and solve, and 7) interpret the numerical answer.
The authors argue that the results of the study “found that students who
had been exposed to the structured problem-solving strategy demonstrated
better conceptual understanding of physics and tended to adopt a
conceptual approach to problem solving.”

Cooper et al. describe a study where general chemistry students worked in

small collaborative groups on qualitative analysis chemical problems.
(Qualitative analysis here means the process of using physical and
chemical characteristics to identify unknown compounds.) They report
that student problem-solving ability improves as a result of the
collaborative work. These researchers also looked at two characteristics of
the students to see how they interacted with the problem-solving
improvement and found that female students who were classified as “pre-
formal in terms of logical thinking” and grouped with “concrete” students
showed the largest improvements.38

Ogilvie has recently reported on the effect of instruction that employed

context-rich problems on student heuristic usage. He found only a small
reduction in the frequency of students’ usage of means-ends analysis in
the form of plug and chug. However, there was also a large increase in
students using diagrams and thinking about concepts before searching out
an equation.39 The results of this study could provide useful insights into
how to transform novices into experts.

4.2 Computer Tutors

There has been little research on the use of computer tutors to teach
problem solving within PER until recently. Hsu and Heller have reported
on preliminary work to develop computer tutors as “personal assistants for
learning”40 which is based on the idea of cognitive apprenticeship. The
researchers are developing three types of tutors—(1) computer as coach
for the student, (2) student as tutor for the computer, and (3) computer as
assistant for the student who works more independently—and are
currently in the process of evaluating the usefulness of the tutors.41

Getting Started in Physics Education Research 19

Maloney Research in Problem Solving

More work on this approach has been done within the computer
science/AI community. VanLehn et al. report on using computer tutors in
a “minimally intrusive” way to help students learn to solve physics
problems.42 The ANDES tutor developed by these researchers is a very
different entity than the tutor described in the previous paragraph. ANDES
incorporates an artificial intelligence system designed to try to ascertain
the student’s mental and skill level. As such this system is not something
an instructor can modify to use with his/her course and students.

Lee and colleagues report on a “Web-based homework tutor

MASTERINGPHYSICS” which they argue uses a “Socratic tutoring
style.”43 While not exactly a computer tutor is the sense of ANDES, this
system is designed to give assistance to students engaged in problem
solving “when needed.” The authors found that students who used tutoring
between an incorrect first submission and a second answer submission
improved in performance. In contrast, students who did not avail
themselves of tutoring actually showed a decline.

4.3 Learning from Worked Examples

Putting this topic here rather than in section B may seem strange, but this
goes back to the intertwined nature of exploring problem solving. The
reason for putting it here is that this aspect doesn’t deal directly with
students solving specific problems, for the most part. There are two classic
studies that reported important differences in how strong and weak
students studied and used worked examples.44-45 Both groups of
researchers found that the stronger students would spend time when
studying worked examples figuring out aspects of the example that they
did not initially understand. In contrast, the weaker students tended to take
essentially everything for granted and did very little deep processing, i.e.,
trying to make sense of what was presented in the example. Not
surprisingly when the weaker students were engaged in solving problems
for themselves they would go back and look for a worked example that
they could use essentially as a “template” for the current problem. In
contrast, the stronger students would only go back to worked examples
when stuck on a specific step in their solution of a new problem, and they
would go in search of a specific idea or process, not a complete map.

Chi subsequently developed a model of learning from worked examples

that involves two processes: generating inferences and self-repairing. She
contends that these processes come into play when the solution omits
some information and the students are forced to explain to themselves

Getting Started in Physics Education Research 20

Maloney Research in Problem Solving

what is missing. The more important of these is the self-repair where the
student has a defect in his/her mental model that produces a conflict with
the scientific model conveyed in the worked example. Students who do
not recognize and address the conflict will obviously be limited in what
they learn from the worked example.46

Following Chi’s lead, Singh, Yerushalmi and colleagues have investigated

how students do with self-diagnosis tasks.47-49 Among their findings were
that the more explicit the instructions the students get, the better their self-
diagnosis; self-diagnosis performance did not correlate with performance
on a transfer problem; and that students seldom engaged in self-repair
during self-diagnosis.

Section 4 Summary: The results of the relatively few studies of teaching

problem solving have been mixed, at best, so far. Work with computer
tutors is really in its infancy, but likely to grow. Investigations of how
students use worked examples indicate that such examples are useful for
strong students who engage in deep processing while studying them.
However, such examples are of much more limited value for weaker
students.

5. Miscellaneous Aspects
5.1 Assessing students’ problem-solving skills.

There have been several recent attempts to development an instrument to

assess students’ problem-solving skill or ability. These efforts have taken
different directions.

Docktor and Heller have been developing a rubric to assess “written

solutions to physics” problems. This rubric looks at five aspects of the
solving process which the researchers argue are at least roughly
independent of each other. The five aspects are: useful description, physics
approach, application of physics, mathematical procedures, and logical
progression, which the researchers also describe as “overall
communication of an organized reasoning pattern”. There are six
numerical scores, 0 to 5, as well as two “not applicable” designations on
the rubric. The rubric is still in the development phase but the developers
contend: “The rubric provides more meaningful information than standard
grading by indicating areas of student difficulty that can be used to focus
coaching and improve problem writing.”36

Getting Started in Physics Education Research 21

Maloney Research in Problem Solving

In contrast to the Docktor and Heller approach described above, Adams

examined how subjects worked a complex problem in an everyday
context. In other words, instead of developing a way of scoring students
work on various physics problems, Adams had subjects work a specific
task framed in an “every day” context and carefully examined their work.
She identified a variety of skills and processes that the subjects used.6

5.2 Faculty Attitudes and Approaches to Assessing Problem

Assignments

Henderson et al. have explored how faculty evaluate students’ written

problem solutions. One of the interesting things they found was that
faculty often have difficulty following through on assigning low scores to
students’ solutions that do not contain explicit communication of
component processes such as a diagram if the “answer” is correct. In
addition, instructors have a tendency to assume, if the answer is present,
that students have used appropriate reasoning or processes even if the
students do not communicate the processes and reasoning they
employed.50

Yerushalmi et al. investigated how instructors’ beliefs and values

influenced what problems they assigned in introductory physics courses.
The researchers found that instructors’ goals and beliefs were consistent
with the findings of physics education research on how to develop
competent problem solvers. However, many instructors failed to actually
use such beneficial problem features because those features conflicted
with other values held by the instructors, specifically clarity of
presentation and reduction of student stress.51

5.3 Relation of Mathematics to Problem Solving in Physics

Problem solving in physics commonly involves the application of various

mathematical procedures, so there are a number of issues related to the
math-physics interaction worthy of investigation. One issue is how well
students understand the mathematics on a conceptual basis and how that
affects how they can use mathematics in working physics problems.
Another issue is what Bing and Redish call the epistemic role of
mathematics. As they state: “Mathematics thus fills many different
epistemic roles for a physicist. It reflects physical relations, provides a
calculation framework, forms a web of interconnected ideas, and provides
a packaging system for encoding rules and previous results.”52

Getting Started in Physics Education Research 22

Maloney Research in Problem Solving

Investigation into these issues is relatively new so there is much yet to

learn.

Sherin has investigated how students understand mathematical equations

when doing physics problems. He wanted to know what it means for a
student to understand a physics equation. He argues that: “successful
students learn to understand what equations say in a fundamental sense.”
He cites an example of two students who write a relation “F up = F down ” to
describe an object on the Earth falling with a terminal velocity and
contends that the students have developed this relation from using a
balance analogy. Sherin argues that people acquire knowledge elements he
calls “symbolic forms” that connect conceptual understanding with
equations.53

Bing and Redish investigated how students use mathematics when solving
physics problems. They contend students use a process they call
epistemological framing in which students decide the kind of knowledge,
(e.g., do they need to do a calculation, or do they need to map the
mathematics to the physical situation, or do they invoke authority, etc.), to
employ at various points in the problem-solving process. They argue that
students can get stuck when solving problems because they employ a
particular epistemological framing rather than a more appropriate one
which they also have as a resource. Bing and Redish contend that students
have identifiable warrants (reasons) that guide the framing decisions for
how to use mathematics when solving physics problems.52

5.4 Solving Ill-defined Problems

Researchers have started investigating how students approach ill-defined

problems rather than just the typical well-defined textbook style problems.
Shin, Jonassen & McGee studied how 9th grade students worked open-
ended well-defined and ill-defined astronomy tasks. They found two
useful predictors—domain knowledge and justification skill—for success
on the well-defined problems. And while these same two predictors
worked for the ill-defined problems, there were also two additional
aspects—science attitudes and regulation of cognition—involved in
making successful predictions.54 For these authors regulation of cognition
includes planning and monitoring skills.

Also recently, Fortus studied subjects, who all had at least a BA in

physics, working on one ill-defined and three well-defined problems. The
four problems all involved the same topic material. The author reports two

Getting Started in Physics Education Research 23

Maloney Research in Problem Solving

main findings: (1) for all subjects the most difficult aspect of solving the
ill-defined problem was the process of making constraining assumptions
to convert the problem to a well-defined one, and (2) only subjects who
had prior experience making such assumptions had much success.55

5.5 Problem Solving and Conceptual Understanding/Knowledge

Organization

One of the common ideas among physics instructors (and mathematics and
chemistry instructors) is that students “learn the physics” by solving
problems. However, the research has shown that if the “problems”
involved are the traditional end-of-chapter tasks this contention is not
valid. Kim and Pak found that even students who had solved more than
1000 traditional tasks in preparation for college entrance exams still had
most of the common alternate conceptions about how physical systems
behaved.56 Sherin explored the role of “intuitive knowledge” in physics
problem solving. He found that subjects did not always follow formal
physics rules, instead appealing to common sense as a guide for how to
proceed.57

Similar work has been done in chemistry. Nurrenbern & Pickering found
that students could solve numerical tasks, but struggle with essentially the
same task when presented in a qualitative manner.58 Sawrey and
Cracolice, Deming & Ehlert followed up with further investigations of the
relations and differences between effective problem solving and
conceptual understanding.59-60

There have also been studies investigating the relation between knowledge
organization and problem solving. Eylon and Reif explored how two
different knowledge organizations—hierarchical versus single-level—
affected recall and ability to use the knowledge in problem solving.
Subjects working with the hierarchical knowledge organization were
significantly better at using the knowledge on complex tasks.61

These studies, and others, make it clear that the relations between problem
solving and conceptual understanding or knowledge organization are
complex ones that we are only starting to understand.

Section 5 Summary: Several physics education researchers have turned

their attention to investigating ways to assess student problem-solving
skills, but only two approaches have been reported at this time. Studies of
faculty attitudes to assessing problem assignments has revealed conflicts

Getting Started in Physics Education Research 24

Maloney Research in Problem Solving

between what instructors believe is productive problem solving and the

tasks they assign, as well as how they grade those assignments. Another
recent topic of research is the relation between mathematics and physics
problem solving. These early studies are revealing something about how
complex this relationship is. Work has also started on how students
approach ill-defined problems, but has already strongly indicated that
additional skills, over and above those needed for typical well-defined
physics problems, are needed. Another complex area that needs much
more work is that of the relations between/among conceptual
understanding, knowledge organization and problem solving.

6. Taking Stock
So where are we at this point in time? We have learned a good bit about
what students do with ‘traditional’ introductory physics tasks and some
things about other aspects of students’ efforts to solve the tasks presented
to them. We are also starting to learn things about teaching problem
solving and instructors’ approaches and values for this task. In presenting
the rough summary below it is important to acknowledge that there are
other aspects of problem solving that this review has not discussed, but
which are worthy of research. Having said that, what can we say in
summary at this time?

We know students use surface features to classify problems and choose

solution approaches. We know that many students have an epistemology
of “answer making” and consequently, represent the problem-solving
process as a matter of finding the right equation to plug the given numbers
into. We know good worked examples point out the connections between
concepts, principles, relations, qualitative representations, and the
problem-solving process. We also know that good students can use
worked examples effectively by engaging in deep processing that
produces learning about the connections among conceptual knowledge and
mathematical representations, among other things. Along the same lines,
we know that poor students tend not to benefit from studying worked
examples, and so are left trying to “pattern match” assigned problems to
worked examples when necessary. We know that problems that are
isomorphic from a physics principle perspective will appear different to
students based on their knowledge.

We know that students’ knowledge bases are a major source of the

difficulty they have for several reasons. First, they lack domain

Getting Started in Physics Education Research 25

Maloney Research in Problem Solving

knowledge, which means they are essentially forced to employ primarily

“weak” methods such as general problem-solving heuristics. Second, what
little domain knowledge they do have is not organized effectively to foster
its use in either recognizing or working on the problems. Third, alternate
conceptions (ideas such as “motion requires a force” or “acceleration
means a change in speed”) can prevent students from using appropriate
domain knowledge because they believe in the conflicting idea more
strongly than the proper idea.

In addition to having poor knowledge of physics, students also have little

knowledge of problem-solving heuristics. Trial and error, working
backwards, or plug and chug may be the only approaches many students
know. Even when they are using these techniques, students are likely to be
using them with little, if any, conscious planning.

Because of these student characteristics we know that standard end-of-

chapter numerical exercises are indeed problems for them. Much of the
work that has been done has employed this type of task as the problems
for the subjects and much has been learned. It is also clear there is still
more that can be learned about students’ interactions with these tasks.
However, it also seems that common use of these types of tasks in
instruction is one of the reasons for many of the student difficulties found.
Use of these ‘problems’ has led students to a representation of the
problem-solving process as one of “plug and chug” and has done little to
help students learn the relevant concepts, principles and relations.

We know that problem representations are critical to problem solving. It is

also clear that formation of effective representations, especially for more
challenging, or ill-defined, problems requires good domain knowledge.
Representation, in the sense of problem format, has also been shown to
affect students’ work.

It is clear from the problem-solving work done to date, in all domains, that
problem-solving expertise is domain-specific. That is, someone who is an
expert at solving chess problems will not automatically also be expert at
solving physics problems. Nonetheless there is evidence for general
problem-solving skills, perhaps at several “levels.” We can envision at
least three levels, the most general level being reasoning processes at the
level of metacognition, which is domain independent. The next more
specific level is that of employing heuristics, such as guess and test or
identifying and applying constraints, which can be applied in all domains,
but which application requires some domain knowledge. And finally there

Getting Started in Physics Education Research 26

Maloney Research in Problem Solving

are domain specific skills such as drawing free-body diagrams or ray

diagrams and using a free-body diagram to write the Newton’s second law
relation for an object, or when analyzing a DC circuit distinguishing
between global aspects—those that apply to the circuit as a whole, e.g., the
battery potential—and local aspects—those that are particular to a specific
circuit element, e.g., the potential difference across a single resistor in a
series sequence.

Real progress has been made in understanding how students deal with
various aspects of solving physics problems. However, as mentioned at the
outset, the domain is so complex and diverse that we have actually just
scratched the surface. The author would argue that there are a couple of
things that should be made explicit in all investigations of problem
solving. First, the specific nature of the tasks which are qualifying as
problems in the investigation needs to be specified carefully. It should be
clear whether the tasks are well-defined, or ill-defined, and if the latter,
how. In a similar way, whether the tasks are primarily quantitative or
qualitative needs to be explicitly identified. Second, there is a definite
need when talking about teaching problem-solving skills to explicitly
identify what those skills are rather than taking, as most physics
instructors do, students’ ability to solve typical numerical tasks as the
operational definition of problem-solving skill.

7. Some Open Questions/Areas for Future Research

There are many open questions about problem solving in physics that
someone could investigate at this point; given below is a quick dozen.
Some would be straightforward to tackle, but a number would require a
fair amount of ingenuity to research productively. The following list is just
a small set of the possibilities.

 What, if anything, do students learn from doing typical end-of-

chapter numerical exercises? If students learn little or nothing from
doing these tasks, which are nonetheless still challenging for them,
why take the time to have students work them?
 What are reasonable alternative tasks to traditional end-of-chapter
numerical tasks that can promote conceptual understanding?

 Does having students solve different types of tasks/problems about

the same concept help the students learn the concept? If yes, what
types and number of problems are needed? If no, why not?

Getting Started in Physics Education Research 27

Maloney Research in Problem Solving

 What mathematical understanding is critical for solving what

physics problems? For example, does a student have to have a
conceptual understanding of divergence to be able to figure out
whether a Gaussian surface encloses a charge?

 What is the value and role of worked examples? What types of

worked examples are more effective for promoting conceptual
understanding? What about promoting specific problem-solving
skills?

 What is the value, if any, of explicitly teaching heuristics, and what

heuristics are worth teaching?

 How do, if they do, “scientific reasoning abilities” relate to how

well students can solve physics problems?

 What are the advantages/disadvantages of having students work

problems in collaborative groups versus having them work the
problems individually?

 To what extent, if any, does doing such well-defined problems as

end-of-chapter numerical tasks actually build a foundation to
improve ability with ill-defined problems?

 To what extent are heuristics important for dealing with ill-defined

problems compared to well-defined problems? Which heuristics
are important?

 Since problem solving serves three purposes in instructional

contexts (a primary goal, a critical teaching tool, and an important
evaluation technique) how well do instructors differentiate among
these, and how well do students? In addition, to what extent, and
how, is it necessary to do so?

 How do novices transition to expertise? What are the patterns in

how peoples’ domain knowledge, problem-solving skills,
epistemology, etc. morph from novice status to expert status?

Acknowledgements

Getting Started in Physics Education Research 28

Maloney Research in Problem Solving

I am very appreciative of the tremendous patience, which I tested

thoroughly, and assistance of the editors, and of the productive comments
of the reviewers.

References
1
D. P. Maloney, “Research on Problem Solving: Physics” in Handbook of
Research on Science Teaching and Learning, D. L. Gabel, ed.
(MacMillan, New York, NY, 1994).
2
L. Hsu, E. Brewe, T. M. Foster, and K. A. Harper, “Resource Letter on
Problem Solving Research in Physics Education,” Am. J. Phys. 72,
1147-1156 (2004).
3
R. E. Mayer and M. C. Wittrock, “Problem Solving Transfer” in
Handbook of Education Psychology, D. C. Berliner and R. C. Calfee,
eds. (Macmillan, New York, NY, 1996).
4
D. H. Jonassen, “Toward a Design Theory of Problem Solving,” Ed.
Tech. Res. Devel. 48 (4), 63-85 (2000).
5
J. E. Davidson and R. J. Sternberg, eds., The Psychology of Problem
Solving (Cambridge University Press, New York, NY, 2003).
6
W. K. Adams, “Development of a Problem Solving Evaluation
Instrument: Untangling of Specific Problem Solving Skills,”
Unpublished doctoral dissertation, University of Colorado (2007).
7
J. R. Hayes, The Complete Problem Solver (Franklin Institute Press,
Philadelphia, PA, 1981).
8
P. Heller and M. Hollabaugh, “Teaching Problem Solving Through
Cooperative Grouping. Part 2: Designing Problems and Structuring
Groups,” Am. J. Phys. 60 (7), 637-645 (1992).
9
A. Van Heuvelen and D. P. Maloney, “Playing Physics Jeopardy,” Am. J.
Phys. 67, 252-256 (1999).
10
D. P. Maloney, “Ranking Tasks: A New Type of Test Item,” J. Coll. Sci.
Teach. 25, 205-210 (1987).
11
D H. Jonassen, “What Makes Scientific Problems Difficult?” in
Learning to Solve Complex Scientific Problems, D. H. Jonassen, ed.
(Lawrence Erlbaum Associates, New York, N Y, 2007).
12
Y. Chu and J. N. MacGregor, “Human Performance on Insight Problem
Solving: A Review,” J. Prob. Solv. 3, 119-150 (2011).
13
D. Perkins, Archimedes’ Bathtub (W. W. Norton and Company, New
York, NY, 2000).

Getting Started in Physics Education Research 29

Maloney Research in Problem Solving

14
A. H. Johnstone, “Introduction” in Creative Problem Solving in
Chemistry, C. Wood and R. Sleet, eds. (The Royal Society of Chemistry,
London, England, 1993).
15
D. Hammer, “Two Approaches to Learning Physics,” Phys. Teach. 27,
664-670 (1989).
16
C. Bereiter and M. Scardamalia, Surpassing Ourselves: An Inquiry into
the Nature and Implications of Expertise (Open Court Publishing,
Chicago, IL, 1993).
17
M. T. H. Chi, “Quantifying Qualitative Analyses of Verbal Data: A
Practical Guide,” J. Learn. Sci. 6, 271-315 (1997).
18
J. H. Larkin, J. McDermott, D. P. Simon, and H. A. Simon, “Expert and
Novice Performance in Solving Physics Problems,” Science 208, 1335-
1342 (1980).
19
C. Singh, “When Physical Intuition Fails,” Am. J. Phys. 70, 1103-1109
(2002).
20
M. T. H. Chi, P. S. Feltovich, and R. Glaser, “Categorization and
Representation of Physics Problems by Experts and Novices,” Cog. Sci.
5, 121-152 (1981).
21
C. Singh, “Assessing Student Expertise in Introductory Physics with
Isomorphic Problems II: Effect of Some Potential Factors on Problem
Solving and Transfer,” Phys Rev. ST-PER 4, 010105 (2008).
22
J. I. Heller and F. Reif, “Describing Effective Human Problem Solving
Processes: Problem Description in Physics,” Cog. Instr. 1, 177-216
(1984).
23
P. Heller, R. Keith, and S. Anderson, “Teaching Problem Solving
Through Cooperative Grouping. Part 1: Group Versus Individual
Problem Solving,” Am. J. Phys. 60 (7), 627-636 (1992).
24
G. Polya, How to Solve It (Princeton University Press, Princeton, N J,
1945).
25
W. J. Leonard, R. J. Dufresne, and J. P. Mestre, “Using Qualitative
Problem-Solving Strategies to Highlight the Role of Conceptual
Knowledge in Solving Problems,” Am. J. Phys. 64, 1495-1503 (1996).
26
J. Mason, L. Burton, & K. Stacey, Thinking Mathematically (Addison-
Wesley, NY, 2010).
27
J. H. Larkin, “The Role of Problem Representation in Physics” in Mental
Models, D. Gentner and A. L. Stevens, eds. (Lawrence Erlbaum,
Hillsdale, NJ, 1983).
28
D. E. Meltzer, “Relation Between Students’ Problem-solving
Performance and Representational Format,” Am. J. Phys. 73, 463-478
(2005).

Getting Started in Physics Education Research 30

Maloney Research in Problem Solving

29
P. B. Kohl and N. D. Finkelstein, “Patterns of Multiple Representation
Use by Experts and Novices During Physics Problem Solving,” Phys
Rev. ST-PER 4, 010111 (2008).
30
K. Kotovsky and H. Simon, “What Makes Some Problems Really Hard:
Explorations in the Problem Space of Difficulty,” Cog. Psych. 22, 143-
183 (1990).
31
J. Sweller, “Cognitive Load During Problem Solving: Effects on
Learning,” Cog. Sci. 21, 257-285 (1988).
32
J. Sweller and M. Levine, “Effects of Goal Specificity on Means-ends
Analysis and Learning,” J. of Exp. Psych: Learn., Mem. Cog. 8 (5), 463-
474 (1982).
33
J. Sweller, R. Mawer, and M. Ward, “Consequences of History: Cued
and Means-end Strategies in Problem Solving,” Am. J. Psych. 95 (3),
435-483 (1982).
34
A. H. Schoenfeld, Mathematical Problem Solving (Academic Press, New
York, NY, 1985).
35
J. Tuminaro and E. F. Redish, “Elements of a Cognitive Model of
Physics Problem Solving: Epistemic Games,” Phys Rev. ST-PER 3,
020101 (2007).
36
J. Docktor and K. Heller, “Robust Assessment Instrument for Student
Problem Solving,” Proc. NARST 2009 Ann. Meet. (2009).
37
E. Gaigher, J. M. Rogan, and W. W. H. Braun, “Exploring the
Development of Conceptual Understanding Through Structured
Problem-solving in Physics,” Int. J. Sci. Ed. 29, (9), 1089-1110 (2007).
38
M. M. Cooper, C. T. Cox, M. Nammouz, and E. Case, “An Assessment
of the Effect of Collaborative Groups on Students’ Problem-solving
Strategies and Abilities,” J. Chem. Ed. 85, 866-872 (2008).
39
C. A. Ogilvie, “Change in Students’ Problem-solving Strategies in a
Course That Includes Context-rich Multifaceted Problems,” Phys Rev.
ST-PER 5, 020102 (2009).
40
F. Reif and L. A. Scott, “Teaching Scientific Thinking Skills: Students
and Computers Coaching Each Other,” Am. J. Phys. 67, 819-831 (1999).
41
L. Hsu and K. Heller, “Computer Problem-solving Coaches,” Proc.
NARST 2009 Ann. Meet. (2009).
42
K. VanLehn, C. Lynch, L. Taylor, A. Weinstein, R. Shelby, K. Schulze,
D. Treacy, and M. Wintersgill, “Minimally Invasive Tutoring of
Complex Physics Problem Solving” in Intelligent Tutoring Systems,
Cerri, Gouarderes, & Paraguacu, eds. (Springer, New York, NY, 2002).
43
Y-J.Lee, D. J. Palazzo, R. Warnakulasooriya, and D. E. Pritichard,
(2008) “Measuring Student Learning with Item Response Theory,” Phys
Rev. ST-PER 4, 010102 (2008).

Getting Started in Physics Education Research 31

Maloney Research in Problem Solving

44
M. T. H. Chi, M. Bassock, M. W. Lewis, P. Reimann, and R. Glaser,
(1989) “Self-explanations: How Students Study and Use Examples in
Learning to Solve Problems,” Cog. Sci. 5, 121-152 (1989).
45
M. G. M. Ferguson-Hessler and T. deJong, “Studying Physics Texts:
Differences in Study Processes Between Good and Poor Performers,”
Cog. Instr. 7, 41-54 (1990).
46
M. T. H. Chi, “Self-explaining Expository Tests: The Dual Processes of
Generating Inferences and Repairing Mental Models” in Advances in
Instructional Psychology, R. Glaser, ed. (Lawrence Erlbaum Associates,
Mahwah, NJ, 2000).
47
E. Cohen, A. Mason, C. Singh, and E. Yerushalmi, “Identifying
Differences in Diagnostic Skills of Physics Students: Students Self-
diagnostic Performance Given Alternative Scaffolding,” Proc. 2008
Phys. Ed. Res. Conf., AIP Conference Proceedings 1064, 99-102 (2008).
48
E. Yerushalmi, E. Cohen, A. Mason, and C. Singh, “From Diagnostic
Skills to Success in the Physics Classroom,” Proc. 2008 Phys. Ed. Res.
Conf., AIP Conference Proceedings 1064, 147-150 (2008).
49
R. Safadi and E. Yerushalmi, “Students’ Perceptions of a Self-diagnostic
Task” Proc. 2009 Phys. Ed. Res. Conf., AIP Conference Proceedings
1179, 257-260 (2009).
50
C. Henderson, E. Yerushalmi, V. Kuo, P. Heller, and K. Heller,
“Grading Student Problem Solutions: The Challenge of Sending a
Consistent Message” Am. J. Phys. 72 (2), 164-169 (2004).
51
E. Yerushalmi, E. Cohen, K. Heller, P. Heller, and C. Henderson,
“Instructors’ Reasons for Choosing Problem Features in a Calculus-
based Introductory Physics Course,” Phys Rev. ST-PER 6, 020108
(2010).
52
T. J. Bing and E. F. Redish, “Analyzing Problem Solving Using Math in
Physics: Epistemological Framing via Warrants,” Phys Rev. ST-PER 5,
020108 (2009).
53
B. Sherin, (2001) “How Students Understand Physics Equations,” Cog.
Instr. 19 (4), 479-541 (2001).
54
N. Shin, D. H. Jonassen, and S. McGee, (2003) “Predictors of Well-
structured and Ill-structured Problem Solving in an Astronomy
Simulation,” J. Res. Sci. Teach. 40, 6-33 (2003).
55
D. Fortus, (2009) “The Importance of Learning to Make Assumptions,”
Sci. Ed. 93 (1), 86-108 (2009).
56
E. Kim and S-J. Pak, (2002) “Students Do Not Overcome Conceptual
Difficulties After Solving 1000 Traditional Problems,” Am. J. Phys. 70,
759-765 (2002).

Getting Started in Physics Education Research 32

Maloney Research in Problem Solving

57
B. Sherin, “Common Sense Clarified: The Role of Intuitive Knowledge
in Physics Problem Solving,” J. Res. Sci. Teach. 43, 535-555 (2006).
58
S. C. Nurrenbern and M. Pickering, “Concept Learning versus Problem
Solving: Is There a Difference,” J. Chem. Ed. 64, 508-510 (1987).
59
B.A. Sawrey, “Concept Learning versus Problem Solving: Revisited,” J.
Chem. Ed. 67, 253-254 (1990).
60
M. S. Cracolice, J. C. Deming, and B. Ehlert, “Concept Learning versus
Problem Solving: A Cognitive Difference,” J. Chem. Ed. 85, 873-878
(2008).
61
B. Eylon and F. Reif, “Effects of Knowledge Organization on Task
Performance” Cog. Instr. 1, 5-44 (1984).
62
A number of other sources were useful in constructing this paper,
including, but not limited to, the following: E. Bagno and B. Eylon,
“From Problem Solving to a Knowledge Structure: An Example from
the Domain of Electromagnetism,” Am. J. Phys. 65, 726-736 (1997);
Duncker, “On Problem Solving,” Psych. Monog. 58, 5 (1945); D. H.
Jonassen, ed. Learning to Solve Complex Scientific Problems (Lawrence
Erlbaum Associates, New York, N Y, 2007); P. B. Kohl and N. D.
Finkelstein, “The Effect of Instructional Environment on Physics
Students Representational Skills,” Phys Rev. ST-PER 2, 010102 (2006);
A. E. Lawson, “The Development and Validation of a Classroom Test of
Formal Reasoning,” J. Res. Sci. Teach. 15, 11-24 (1978); H. A. Simon
and W. G. Chase, “Skill in Chess,” Am. Sci. 61, 394-403 (1973); R.
Taconis, M. G. M. Ferguson-Hessler, and H. Broekkamp, (2001)
“Teaching Science Problem Solving: An Overview of Experimental
Work,” J. Res. Sci. Teach. 38, 442-468 (2001); M. Ward and J. Sweller,
“Structuring Effective Worked Examples,” Cog. Instr. 7, 1-39 (1990).

Getting Started in Physics Education Research 33