Chapter 4, Guide to an Overall Critique of a Quantitative Research Report and Guide to

an Overall Critique of a Qualitative Research Report

Guide to an Overall Critique of a Quantitative Research Report

ASPECT OF CRITIQUING QUESTIONS DETAILED

THE REPORT CRITIQUING
GUIDELINESa
Title  Is the title a good one, succinctly suggesting key variables and the
study population?
Abstract  Does the abstract clearly and concisely summarize the main features
of the report (problem, methods, results, conclusions)?
Introduction Box 6.3, p. 111
Statement of the  Is the problem stated unambiguously, and is it easy to identify?
problem  Does the problem statement build a cogent and persuasive argument
for the new study?
 Does the problem have significance for nursing?
 Is there a good match between the research problem and the
paradigm and methods used? Is a quantitative approach appropriate?
Hypotheses or  Are research questions and/or hypotheses explicitly stated? If not, is Box 6.3, p. 111
research their absence justified?
questions  Are questions and hypotheses appropriately worded, with clear
specification of key variables and the study population?
 Are the questions/hypotheses consistent with the literature review
and the conceptual framework?
Literature  Is the literature review up-to-date and based mainly on primary Box 7.1, p. 127
review sources?
 Does the review provide a state-of-the-art synthesis of evidence on
the research problem?
 Does the literature review provide a solid basis for the new study?
Conceptual/theo  Are key concepts adequately defined conceptually? Box 8.1, p. 143
retical  Is there a conceptual/theoretical framework, rationale, and/or map,
framework and (if so) is it appropriate? If not, is the absence of one justified?
Method Box 5.2, p. 93
Protection of  Were appropriate procedures used to safeguard the rights of study
participants’ participants? Was the study subject to external review by an
rights institutional review board/ethics review board?
 Was the study designed to minimize risks and maximize benefits to
participants?
Research design  Was the most rigorous possible design used, given the purpose of the Box 9.1, p. 170
research?
 Were appropriate comparisons made to enhance interpretability of
the findings?
 Was the number of data collection points appropriate?
 Did the design minimize biases and threats to the internal, construct,
and external validity of the study (e.g., was blinding used, was attrition
minimized)?
Population and  Was the population identified and described? Was the sample Box 12.1, p. 243
sample described in sufficient detail?
 Was the best possible sampling design used to enhance the sample’s
representativeness? Were sample biases minimized?
 Was the sample size adequate? Was a power analysis used to
estimate sample size needs?
Data collection  Are the operational and conceptual definitions congruent? Box 13.1, p. 261;
and  Were key variables operationalized using the best possible method Box 14.1, p. 278
measurement (e.g., interviews, observations, and so on) and with adequate
justification?
 Are the specific instruments adequately described and were they
good choices, given the study purpose and study population?
 Does the report provide evidence that the data collection methods
yielded data that were high on reliability and validity?
Procedures  If there was an intervention, is it adequately described, and was it Box 9.1, p. 170
properly implemented? Did most participants allocated to the
intervention group actually receive it? Was there evidence of
intervention fidelity?
 Were data collected in a manner that minimized bias? Were the staff
who collected data appropriately trained?
Results Box 15.2, p. 295
Data analysis  Were analyses undertaken to address each research question or test
each hypothesis?
 Were appropriate statistical methods used, given the level of
measurement of the variables, number of groups being compared, and
so on?
 Was the most powerful analytic method used? (e.g., did the analysis
help to control for confounding variables)?
 Were Type I and Type II errors avoided or minimized?
Findings Was information about statistical significance presented? Was Box 15.2, p. 295
information about effect size and precision of estimates (confidence
intervals) presented?
 Are the findings adequately summarized, with good use of tables
and figures?
Are findings reported in a manner that facilitates a meta-analysis, and
with sufficient information needed for EBP?
Discussion Box 16.1, p. 303
Interpretation of  Are all major findings interpreted and discussed within the context
the findings of prior research and/or the study’s conceptual framework?
Were causal inferences, if any, justified?
 Are the interpretations consistent with the results and with the
study’s limitations?
 Does the report address the issue of the generalizability of the
findings?
Implications/  Do the researchers discuss the implications of the study for clinical Box 16.1, p. 303
recommendatio practice or further research—and are those implications reasonable and
ns complete?
Global Issues
Presentation  Is the report well written, well organized, and sufficiently detailed
for critical analysis?
In intervention studies, was a CONSORT flow chart provided to
show the flow of participants in the study?
 Was the report written in a manner that makes the findings
 accessible to practicing nurses?
Researcher  Do the researchers’ clinical, substantive, or methodologic
credibility qualifications and experience enhance confidence in the findings and
their interpretation?
Summary  Despite any identified limitations, do the study findings appear to be
assessment valid—do you have confidence in the truth value of the results?
 Does the study contribute any meaningful evidence that can be used
in nursing practice or that is useful to the nursing discipline?
a
Page numbers refer to the location of the box in Essentials of Nursing Research, 8e.

ASPECT OF CRITIQUING QUESTIONS DETAILED

THE REPORT CRITIQUING
GUIDELINESa
Title  Was the title a good one, suggesting the key phenomenon and the
group or community under study?
Abstract  Does the abstract clearly and concisely summarize the main
features of the report?
Introduction  Is the problem stated unambiguously and is it easy to identify? Box 6.3, p. 111
Statement of the  Does the problem statement build a cogent and persuasive argument
problem for the new study?
 Does the problem have significance for nursing?
 Is there a good match between the research problem on the one
hand and the paradigm, tradition, and methods on the other?
Research  Are research questions explicitly stated? If not, is their absence Box 6.3, p. 111
questions justified?
 Are the questions consistent with the study’s philosophical basis,
underlying tradition, conceptual framework, or ideological
orientation?
Literature  Does the report adequately summarize the existing body of Box 7.1, p. 127
review knowledge related to the problem or phenomenon of interest?
 Does the literature review provide a solid basis for the new study?
Conceptual  Are key concepts adequately defined conceptually? Box 8.1, p. 143
underpinnings  Is the philosophical basis, underlying tradition, conceptual
framework, or ideological orientation made explicit and is it
 appropriate for the problem?
Method Box 5.2, p. 93
Protection of  Were appropriate procedures used to safeguard the rights of study
participants’ participants? Was the study subject to external review by an
rights IRB/ethics review board?
 Was the study designed to minimize risks and maximize benefits to
participants?
Research design  Is the identified research tradition (if any) congruent with the Box 10.1, p. 183
and research methods used to collect and analyze data?
tradition  Was an adequate amount of time spent in the field or with study
participants?
 Did the design unfold in the field, giving researchers opportunities
to capitalize on early understandings?
 Was there evidence of reflexivity in the design?
 Was there an adequate number of contacts with study participants?
Sample and  Was the group or population of interest adequately described? Were Box 15.1, page
setting the setting and sample described in sufficient detail? 289
 Was the approach used to gain access to the site or to recruit
participants appropriate?
 Was the best possible method of sampling used to enhance
information richness and address the needs of the study?
 Was the sample size adequate? Was saturation achieved?
Data collection  Were the methods of gathering data appropriate? Were data Box 13.1, p. 261
gathered through two or more methods to achieve triangulation?
 Did the researcher ask the right questions or make the right
observations, and were they recorded in an appropriate fashion?
 Was a sufficient amount of data gathered? Were the data of
sufficient depth and richness?
Procedures  Were data collection and recording procedures adequately Box 13.1, p. 261
described and do they appear appropriate?
 Were data collected in a manner that minimized bias or behavioral
distortions? Were the staff who collected data appropriately trained?
Enhancement of  Did the researchers use strategies to enhance the Box 18.1, p. 350
trustworthiness trustworthiness/integrity of the study, and was the description of those
strategies adequate?
 Were the methods used to enhance trustworthiness appropriate and
sufficient?
 Did the researcher document research procedures and decision
processes sufficiently that findings are auditable and confirmable?
 Is there evidence of researcher reflexivity?
Results Box 16.2, page
Data analysis  Were the data management and data analysis methods sufficiently 316
described?
 Was the data analysis strategy compatible with the research
tradition and with the nature and type of data gathered?
 Did the analysis yield an appropriate “product” (e.g., a theory,
taxonomy, thematic pattern, etc.)?
 Did the analytic procedures suggest the possibility of biases?
Findings  Were the findings effectively summarized, with good use of Box 16.2, page
excerpts and supporting arguments? 316
 Do the themes adequately capture the meaning of the data? Does it
appear that the researcher satisfactorily conceptualized the themes or
patterns in the data?
 Did the analysis yield an insightful, provocative, authentic, and
meaningful picture of the phenomenon under investigation?
Theoretical  Are the themes or patterns logically connected to each other to form Box 8.1, p. 143;
integration a convincing and integrated whole? Box 16.2, page
 Were figures, maps, or models used effectively to summarize 316
conceptualizations?
 If a conceptual framework or ideological orientation guided the
study, are the themes or patterns linked to it in a cogent manner?
Discussion Box 18.1, p. 350
Interpretation of  Are the findings interpreted within an appropriate social or cultural
the findings context?
 Are major findings interpreted and discussed within the context of
prior studies?
  Are the interpretations consistent with the study’s limitations?
 Does the report support transferability of the findings?
Implications/rec  Do the researchers discuss the implications of the study for clinical Box 18.1, p. 350
ommendations practice or further inquiry—and are those implications reasonable and
complete?
Global Issues
Presentation  Was the report well written, well organized, and sufficiently
detailed for critical analysis?
 Was the description of the methods, findings, and interpretations
sufficiently rich and vivid?
Researcher  Do the researchers’ clinical, substantive, or methodologic
credibility qualifications and experience enhance confidence in the findings and
their interpretation?
Summary  Do the study findings appear to be trustworthy—do you have
assessment confidence in the truth value of the results?
 Does the study contribute any meaningful evidence that can be used
in nursing practice or that is useful to the nursing discipline?
Guide to an Overall Critique of a Qualitative Research Report
a
Page numbers refer to the location of the box in Essentials of Nursing Research, 8e.