Journal for Research in Mathematics Education

2019, Vol. 50, No. 2, 114–120

Editorial

Posing Significant Research Questions

Jinfa Cai, Anne Morris, Charles Hohensee, Stephen Hwang, Victoria Robison,
Michelle Cirillo, Steven L. Kramer, and James Hiebert
University of Delaware

In 2002, the National Research Council (NRC) released Scientific Research in

Education, a report that proposed six principles to serve as guidelines for all
This material may not be copied or distributed electronically or in other formats without written permission from NCTM.

scientific inquiry in education. The first of these principles was to “pose signifi-
Copyright © 2019 by the National Council of Teachers of Mathematics, Inc. www.nctm.org. All rights reserved.

cant questions that can be investigated empirically” (p. 3). The report argued that
the significance of a question could be established on a foundation of existing
theoretical, methodological, and empirical work. However, it is not always clear
what counts as a significant question in educational research or where such ques-
tions come from. Moreover, our analysis of the reviews for manuscripts submitted
to JRME1 suggests that some practical, specific guidance could help researchers
develop a significant question or make the case for the significance of a research
question when preparing reports of research for publication.
Building on the JRME archive of nearly 50 years of research articles, this issue
marks the beginning of a series of editorials aimed at discussing how to conduct
and report high-quality research in mathematics education. In this first editorial
in the series, we discuss what counts as a significant research question in math-
ematics education research, where significant research questions come from, and
how researchers can develop their manuscripts to make the case for the signifi-
cance of their research questions. Although we are beginning a new series of
editorials, we will continue to draw on the ideas from our editorials over the past
2 years (e.g., Cai et al., 2018; Cai et al., 2017). In particular, we consider what
significant research questions might look like in the aspirational future world of
research that we have described in those editorials—a world in which mathematics
education research is carried out by widespread, stable partnerships of teachers
and researchers and in which research both takes root in and shapes the everyday
practices of mathematics teaching and learning.

Significant Research Questions

It is difficult, if not impossible, to judge the significance of a research question
just by reading the question. Certainly, significant research in mathematics educa-
tion should advance the field’s knowledge and understanding of the teaching and
learning of mathematics (Heid, 2010; Simon, 2004). We believe this implies that
the characteristics that make a research question significant are dependent on

1 We analyzed the reviews for every manuscript that underwent full review and received a decision
in 2017. For those manuscripts that were ultimately rejected, not a single reviewer stated that the
research questions were particularly relevant or insightful. In contrast, for those manuscripts that
ultimately received a revise and resubmit decision or were accepted (pending revisions), only one
reviewer raised the concern that the research questions would not make a contribution to the field.
Cai, Morris, Hohensee, Hwang, Robison, Cirillo, Kramer, and Hiebert 115

context and specifically on assumptions about what kind of knowledge is useful.

Research can advance our understanding of teaching and learning mathematics
in ways that are more distant from the classroom or more connected to practice.
Although we acknowledge the value of research that is more distant and might
eventually have an effect on classroom teaching or learning, we have developed
the argument in our previous editorials that significant research in mathematics
education can, and perhaps should, be much closer to the classroom and aim to
directly impact practice. From this perspective, we begin by asserting that a
research question that addresses teachers’ shared instructional problems and one
whose answer helps the field (students, teachers, policy makers, researchers)
understand why and how the answer is a solution to the problem is likely to be a
significant research question.

Addressing Instructional Problems

We focus on teachers’ instructional problems because they provide a strong
basis for connecting the work of research to the challenges of teaching and learning
mathematics. Confrey (2017) argued that mathematics education research is
grounded in a “practical wisdom” that reflects the challenge of operating in
complex decision-making environments like classrooms and schools. Because of
this, significant research questions can and do arise directly or indirectly from
teachers’ problems of practice.2 Within the idealized portrait of a future world of
mathematics education research described in our previous editorials (Cai et al.,
2017a, 2019), significant research questions arise from interactions between
researchers and teachers about challenges that teachers face in establishing and
helping students achieve well-defined learning goals. Grounding a research ques-
tion in instructional problems that are experienced across multiple teachers’
classrooms helps to ensure that the answer to the question will be of sufficient
scope to be relevant and significant beyond the local context.
Significance is also drawn from the importance of the mathematics that is
investigated. Instructional problems that lead to significant research questions are
problems related to teaching and learning powerful mathematics—mathematics
that is valued by the mathematics education community (broadly conceived). Our
vision of important mathematics is inclusive. It includes mathematics content, the
nature and practices of mathematics as a discipline, beliefs about mathematics and
affective perceptions of mathematics as a powerful and useful tool, and the role
and use of mathematics in addressing inequities (Cai et al., 2017b).3
For an example, we revisit the instructional problem faced by the fourth-grade
teacher, Mr. Lovemath, described in our earlier editorials (Cai et al., 2017a, 2017b).
Mr. Lovemath intended for his students to explore multiple strategies for
completing a fraction comparison task, but the students were unable to make

2 This point also finds support in Scientific Research in Education (NRC, 2002) in its discussion
of Pasteur’s quadrant—the intersection of the quest for fundamental understanding and consid-
erations of use (Stokes, 1997).
3 Indeed, Confrey (2017) points out that the fourth of Flyvbjerg’s (2001) questions that
characterize research in social science—Who gains and loses from the intervention?—puts
questions of equity squarely in the sights of mathematics education researchers.
116 Posing Significant Research Questions

progress on the task and ended up employing a single procedure (using common
denominators) to perform all the comparisons. The students’ difficulties with the
task led Mr. Lovemath and Ms. Research, a mathematics education researcher, to
identify several relevant questions: Why did the students encounter difficulties?
Why did the intended opportunity to learn mathematics not materialize? What
prior knowledge do students need to take advantage of this learning opportunity?
These questions are grounded in an instructional problem that is likely shared by
many teachers who are trying to help their students achieve this learning goal.
Answering these questions would generate insight into students’ learning of
important mathematics and would also shed light on ways to make the learning
opportunities in the task available to all students.

Understanding How and Why

Research questions that focus on teachers’ instructional problems gain addi-
tional significance when they move from only finding answers to the problem to
also understanding how and why the answer is a solution to the instructional
problem. This distinguishes significant research from many other educational
activities (e.g., conducting a successful professional development). Understanding
how and why builds knowledge of a type that enables the solution to the initial
instructional problem to be adapted for a related problem or a different context.
Research questions that aim to understand often ask about the conditions under
which the solution to a problem will work rather than simply asking about the
nature of the solution. Years ago, Cronbach (1986) noted the value of these kinds
of questions, using the classic Brownell and Moser (1949) study4 as an example
of research that went beyond a simple comparison of treatments (a what-works-best
horse race) to examine how different treatments operated under different condi-
tions. More recently, Maxwell (2004) argued for the importance of causal explana-
tion in educational research, specifically highlighting the explanatory significance
of underlying causal processes and the importance of the context in shaping those
processes in particular situations. Studying the conditions under which a solution
works allows teachers and researchers to generate further hypotheses about the
changes in students’ learning that other instructional choices might produce.
For example, in the case of Mr. Lovemath’s instructional problem, Ms. Research
might ask what prior knowledge the students need to take advantage of the learning
opportunity in the fraction comparison task. An answer that reflects the potential
significance of this question would be more than just a list of prerequisite concepts.
Ms. Research’s answer should address the field’s theoretical understanding of the
role that earlier concepts play in new learning. Furthermore, the answer’s descrip-
tion of prior knowledge should, itself, attend to important mathematics—the
conceptual structures that underlie fraction comparison and how particular
concepts are needed at different points to engage productively with the task.

4 Brownell and Moser (1949) studied two approaches for teaching subtraction (meaningful and
mechanical) under two different conditions (using the regrouping algorithm for subtraction and
using the equal additions algorithm). By crossing the two instructional approaches with the two
different algorithms (regrouping and equal additions), Brownell and Moser found, among other
results, that the meaningful approach produced better outcomes than the mechanical approach for
the regrouping algorithm but not for the equal additions algorithm.
Cai, Morris, Hohensee, Hwang, Robison, Cirillo, Kramer, and Hiebert 117

Making these conditions explicit would allow Mr. Lovemath and Ms. Research to
make and test new predictions about how students would engage with the task
after making specific changes to the instruction leading up to the task. The
thinking behind these predictions could also then inform research and practice in
other classrooms (perhaps using different curricula) in which teachers encounter
a similar instructional problem.
Looking across the problems of practice discussed in our previous editorials,
we can identify additional types of significant questions that might arise from
problems of practice. These include questions about the resources (in addition to
the prior knowledge discussed above) that students bring with them that would
help or hinder them in taking advantage of a learning opportunity, questions about
the arrangements of learning goals and subgoals into learning trajectories, ques-
tions about the kinds of data that would usefully inform teaching, and questions
that focus on teacher–researcher partnerships and their work. Fundamentally, our
message is that significant research questions can be generated by addressing
problems of practice while striving to understand underlying mechanisms and
their interactions with the context. Although we have not described every signif-
icant research question that can be posed in mathematics education, we believe
that the kind of knowledge produced by answering research questions like these
is useful and likely to have an impact on practice.

Communicating the Significance of Research Questions

Perceiving and formulating a significant research question is both a science and
an art. The mathematician Jacques Hadamard (1945) wrote that “this delicate
choice is one of the most important things in research” (p. 126). Einstein and Infeld
(1938) claimed that “to raise new questions, new possibilities, to regard old prob-
lems from a new angle, requires creative imagination and marks real advance in
science” (p. 95). Indeed, Klamkin (1968) claimed that “among professional math-
ematicians, asking questions rates almost as high as answering them” (p. 132).
However, formulating a significant research question does not ensure that audi-
ences will perceive its significance. It is still necessary to communicate that ques-
tion in such a way that the field appreciates its significance. This communication,
often embedded in research papers, depends on clearly formulating the research
question for readers and making convincing arguments for its significance.
It can be a challenge to formulate a research question clearly in a research report.
As Heid and Blume (2011) observed regarding manuscripts submitted to JRME,
the statement of the research question is often an issue in submissions. Authors some-
times fail to specify their research question(s), and even when they do, they sometimes
report only a general research problem or area of interest rather than a specific research
question. (p. 106)

Our analysis of the reviews for the manuscripts submitted to JRME that received
a full-review decision in 2017 provides empirical data supporting this observation
by Heid and Blume. Fully 55% of the reviews for those manuscripts that were
rejected in 2017 included concerns about the research questions, including the lack
of a clear motivation for the research questions and a failure to appropriately
connect the research questions to other parts of the manuscript (e.g., situating the
118 Posing Significant Research Questions

questions with respect to the theoretical framework or describing methods appro-

priate for investigating the questions). Even for those manuscripts that were ulti-
mately accepted pending revisions or that received a decision of revise and
resubmit, 17% and 23% of the reviews, respectively, included concerns about the
research questions such as the need to make the statement of the research questions
somewhat clearer.
Thus, communicating the significance of a research question involves several
considerations. First, the question must be explicitly stated with specificity and
precision. It is neither sufficient nor fair to the reader to merely imply the question,
to phrase it only as a goal of the study, or to merely describe a general problem,
instructional or otherwise. A precisely stated research question should make clear
what kinds of data are needed to answer the question and what an answer would
look like. Precision in the statement of a research question can pay dividends in
terms of how well the data will help the researcher and, ultimately, the readers to
understand the phenomenon being studied.
A second consideration is that the research question must be clearly connected
to prior research to situate it in the larger field of mathematics education research.
The significance of a research question cannot be determined just by reading it.
Rather, its significance stands in relation to the knowledge of the field. The justi-
fication for the research question itself—why it is a significant question to inves-
tigate—must therefore be made clear through an explicit argument that ties the
research question to what is and is not already known. Indeed, nearly one quarter
of the JRME reviews that highlighted issues with the research questions in manu-
scripts rejected in 2017 specifically called for authors to make this kind of argu-
ment to motivate the research questions, whereas none of the manuscripts that
were ultimately accepted (pending revisions) received this kind of comment.
Thus, through a research question’s connections to prior research, it should be
clear how answering the question extends the field’s knowledge because it is based
on hypotheses suggested by previous research. The argument that there is a lack
of research in a particular area is not, on its own, a strong justification. To success-
fully make the case that a research question extends the field’s knowledge, the
question must be situated within a theoretical framework that helps readers under-
stand how answering the question informs the field and, consequently, practice or
policy. Although an appeal to an external source can be helpful to establish that
the field has an interest in the question, it is not a shortcut to making the case for
the significance of the question. That case relies on a chain of justification forged
from a theoretical framework that draws on the knowledge of the field. From the
perspective of the future world of mathematics education research that we
described in our previous editorials, that case can rest on whether the question
addresses instructional problems shared by teachers and how the question will
aim the investigation toward the conditions under which a solution to the instruc-
tional problem works (i.e., how answering the question will help the field under-
stand why and how the answer is a solution to the problem).
Finally, a clear and warranted question must be presented in such a way that it
can be empirically investigated and such that the methods for investigation make
sense and follow logically from the question. Indeed, the research question should
be coherent with the methods and data analysis so that, together, they make a
Cai, Morris, Hohensee, Hwang, Robison, Cirillo, Kramer, and Hiebert 119

strong argument. That argument should be tight but should also flow smoothly
like a convincing story or a winning argument in a debate. It should make it easy
for readers to be convinced, and readers should not need to fill in part of the argu-
ment. On the one hand, the argument for the significance of the research question
depends on a theoretical framework. The theoretical framework shapes the
researcher’s conception of the phenomenon of interest, provides insight into it, and
defines the kinds of questions that can be asked about it. On the other hand, there
are many possible theoretical frameworks. Choosing among them depends on how
productively they allow the researcher to engage with the research problem and
to formulate good questions. This mutual dependence means that formulating a
significant research question is an iterative process, one that successively moves
from a broad, general sense of an idea which is potentially fruitful to a well-
specified theoretical framework and a clearly stated research question.
Like formulating a significant research question, the choice or construction of
a theoretical framework is also something of an art.5 In the next editorial, we will
discuss in detail how a theoretical framework can be chosen or constructed to
justify and communicate the significance of research questions. In addition, we
will address how the coherence of the research question, design, data coding and
analyses, and presentation and discussion of the findings as a chain of arguments
depends on presenting a relevant theoretical framework.

References
Brownell, W. A., & Moser, H. E. (1949). Meaningful vs. mechanical learning: A study in Grade III
subtraction. Durham, NC: Duke University Press.
Cai, J., Morris, A., Hohensee, C., Hwang, S., Robison, V., & Hiebert, J. (2017a). A future vision
of mathematics education research: Blurring the boundaries of research and practice to
address teachers’ problems. Journal for Research in Mathematics Education, 48(5), 466–473.
doi:10.5951/jresematheduc.48.5.0466
Cai, J., Morris, A., Hohensee, C., Hwang, S., Robison, V., & Hiebert, J. (2017b). Clarifying the
impact of educational research on students’ learning. Journal for Research in Mathematics
Education, 48(2), 118–123. doi:10.5951/jresematheduc.48.2.0118
Cai, J., Morris, A., Hohensee, C., Hwang, S., Robison, V., & Hiebert, J. (2018). Reconceptualizing
the roles of researchers and teachers to bring research closer to teaching. Journal for Research in
Mathematics Education, 49(5), 514–520. doi:10.5951/jresematheduc.49.5.0514
Cai, J., Morris, A., Hohensee, C., Hwang, S., Robison, V., & Hiebert, J. (2019). Research pathways
that connect research and practice. Journal for Research in Mathematics Education, 50(1), 2–10.
doi:10.5951/jresematheduc.50.1.0002
Cai, J., Morris, A., Hwang, S., Hohensee, C., Robison, V., & Hiebert, J. (2017). Improving the impact
of educational research. Journal for Research in Mathematics Education, 48(1), 2–6. doi:10.5951/
jresematheduc.48.1.0002
Confrey, J. (2017). Research: To inform, deform, or reform? In J. Cai (Ed.), Compendium for research
in mathematics education (pp. 3–27). Reston, VA: National Council of Teachers of Mathematics.
Cronbach, L. J. (1986). Social inquiry by and for earthlings. In D. W. Fiske & R. A. Shweder (Eds.),
Metatheory in social science: Pluralisms and subjectivities (pp. 83–107). Chicago, IL: University
of Chicago Press.
Einstein, A., & Infeld, L. (1938). The evolution of physics: The growth of ideas from early concepts
to relativity and quanta. Cambridge, United Kingdom: Cambridge University Press.

5 We see a parallel to formulating good research questions in mathematics, as Hadamard (1945)

describes it: “The guide we must confide in is that sense of scientific beauty, that special esthetic
sensibility” (p. 127).
120 Posing Significant Research Questions

Flyvbjerg, B. (2001). Making social science matter: Why social enquiry fails and how it can succeed
again. Cambridge, United Kingdom: Cambridge University Press.
Hadamard, J. (1945). An essay on the psychology of invention in the mathematical field. Princeton,
NJ: Princeton University Press.
Heid, M. K. (2010). The task of research manuscripts—Advancing the field of mathematics
education. Journal for Research in Mathematics Education, 41(5), 434–437.
Heid, M. K., & Blume, G. W. (2011). Strengthening manuscript submissions. Journal for Research
in Mathematics Education, 42(2), 106–108. doi:10.5951/jresematheduc.42.2.0106
Klamkin, M. S. (1968). On the teaching of mathematics so as to be useful. Educational Studies in
Mathematics, 1(1–2), 126–160. doi:10.1007/BF00426240
Maxwell, J. A. (2004). Causal explanation, qualitative research, and scientific inquiry in education.
Educational Researcher, 33(2), 3–11. doi:10.3102/0013189X033002003
National Research Council. (2002). Scientific research in education. Washington, DC: National
Academies Press. doi:10.17226/10236
Simon, M. A. (2004). Raising issues of quality in mathematics education research. Journal for
Research in Mathematics Education, 35(3), 157–163. doi:10.2307/30034910
Stokes, D. E. (1997). Pasteur’s quadrant: Basic science and technological innovation. Washington,
DC: Brookings Institution Press.