HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

HST2122 Assessment 2
Evaluation Worksheet for Critiquing Research Study Report
Instructions: Use this template to record your evaluation of the article that you have critiqued for Assessment 2. See
separate instructions for detailed guidance on completing assessment.

NOTE: Hide or remove the rows in the summary table, and pages of the detailed critique, not relating to the article chosen
for critique. Your summary table should reflect the score and overall assessment given (and supported by evidence) for
each component, in the detailed tables to follow. The current row width can be expanded as high as necessary to provide a
complete base of evidence.

Critic’s Name Dianne Hawk

End-text citation of article critiqued:

Xu, F., Chepyator-Thomson, J., Liu, W., & Schmidlein, R. (2010). Association between social and environmental factors
and physical activity opportunities in middle schools. European Physical Education Review, 16(2), 183-194.

Summary Table

Component of evaluation Max Score Overall assessment of

score component

Part A: Evaluating the Front Matter in a Research Report 12 11 High Quality

Part B: Evaluating the Statement of the Problem in a 21 12 Adequate Quality

Research Report
Part C: Evaluating the Literature Review in a Research 21 5 Low Quality
Report
Part D: Evaluating a Purpose in a Research Report 21 13 Adequate Quality

Part E1: Evaluating the Research Design in a Quantitative 9 6 Adequate Quality

Research Report
Part F1: Evaluating the Participants and Data Collection in a 24
Quantitative Research Report
Part G1: Evaluating the Data Analysis and Results in a 21
Quantitative Research Report
Part H: Evaluating the Conclusion and Back Matter in a 21
Research Report
Maximum possible score / total score given
And Overall assessment
1 Quantitative OR 150
2 Qualitative 129

Summary evaluation and justification

1|Page Adapted from tables in HST2122 textbook Plano Clark and Creswell, 2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

Part A: Evaluating the Front Matter in a Research Report

Quality Criteria Quality Rating Your Evidence and/or Reasoning

3 = Excellent
2 = Good
0 = Poor

1 = Fair
The Key Elements
1. The study’s authors and journal 3 Xu, F., Chepyator-Thomson, J., Liu, W., & Schmidlein, R. are
are reputable. the authors. All have PhDs and are affiliated with different US
tertiary institutions which give them a research credential. (See
author information at end of references). The lead author, Xu,
holds a PhD in physical education and sports studies, giving her
credibility with regard to the study topic (physical activity or PA).
Chepyator-Thomson and Liu along with Xu, have multiple
publications in academic journals in their research areas related
to physical education and health. Robert Schmidlein can be
most recently be found at San Jose State University (Faculty,
Kinesiology) but his qualification and current affiliation cannot be
confirmed.
2. The title reflects the content 2 The title, Association between social and environmental factors
and focus of the study. and PA opportunities in middle schools, indicates that :
• The focus is related to an association between 2 sets of
variables which satisfies the topic;
• The setting is middle school but no indication of where in
world;
However, the title does not indicate who participants might be
and only by inference can one discern it may be a quantitative
study (because it looks at relationships between variables).
Journal reference information is present (see reference at top of
table).
3. The abstract concisely but 3 The abstract provides a good succinct description of the study
accurately summarises the purpose (relationship between middle school children’s PA
aim, methodology and main opportunities and social and environmental factors) by
findings of the report. surveying participants (middle school physical education
teachers). Key findings were reported: more PA opportunities for
middle school children needed and some social and
environmental factors (e.g. facilities availability, school location,
and family support) influence the availability of opportunities.
General Evaluation
4. The front matter accurately 3 The main elements of the front matter have been addressed
reflects the content of the enabling a first assessment of whether the article may be
report and allows an evaluation relevant to a person’s needs (their motivation for searching the
of relevance. literature).

Overall Quality Part A Total Part A My Overall Assessment of Part A =

Score =
0 - 6 = Low quality
7 - 9 = Adequate quality 11 High quality
10 - 12 = High quality

2|Page Adapted from tables in HST2122 textbook Plano Clark and Creswell, 2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

Part B: Evaluating the Statement of the Problem in a Research Report

Quality Criteria Quality Rating Your Evidence and/or Reasoning

3 = Excellent
2 = Good
0 = Poor

1 = Fair
The Key Elements
1. The topic is interesting. 1 The 1st paragraph indicates the topic is physical activity and
health which is very relevant to the health and medical fields and
of great interest in the public media sphere; however, the study
report does not make this appear to be a very interesting topic (no
‘hook’) and its low quality of writing makes it ‘work’ continue
reading.
2. There is a meaningful problem. 2 The problem affects many, has serious consequence and is of
concern for practitioners (PE teacher). The problem is: the
relatively low levels of physical activity that children currently
engage contribute to obesity and, ultimately to chronic disease in
adulthood.
3. The importance of the problem 1 The problem is justified by citing evidence from the academic
is justified. health and education literature about:
• how low levels of physical activity among children (only 8% get
recommended levels);
• cause obesity in children into adulthood (no data presented); &
• result in high levels of adult chronic disease and ultimately
high health care costs (no statistics offered).
While the problem and logic is stated in 1st paragraph the
relationships are not well explained (only 1 paragraph explaining
the problem) nor fully supported by statistics to give perspective.
4. There are deficiencies in the 3 1. A research study is needed to fill a gap in existing literature
knowledge about the problem. about the research problem: Xu et al. argue that little is known
about
• levels of PA, particularly among middle school-aged children;
• the relationship between physical and environmental factors of
the middle school setting and the opportunities for children’s
physical activity; and
• ‘best practices’ that could inform the development of effective
interventions in the middle school environ that increase
children’s PA.
2. A research study is needed to learn from people affected by the
problem whose voices may not have been heard: research is
needed to understand middle school PE teachers’ views on the
factors that influence children’s PA.
5. There are audiences who can 2 Further, such knowledge about the relationship between these
benefit from the missing factors could inform interventions aimed at increasing PA in
knowledge. middle schools.

General Evaluation
6. The passage clearly argues 2 The key elements are all present, albeit at times cryptically, so
that the study is warranted. that the reader needs to reread to find the whole story.

7. The passage is well written. 1 The key elements are present, albeit at times cryptically, but they
are not presented in clear language or order -- the report is
missing transitions to explain connections between elements of
the researchers reasoning.

Overall Quality Part B My Overall Assessment of Part B =

3|Page Adapted from tables in HST2122 textbook Plano Clark and Creswell, 2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

Total Part B
0 - 10 = Low quality Score =
11 - 16 = Adequate quality 12 Adequate quality
17 - 21 = High quality

Part C: Evaluating the Literature Review in a Research Report

Quality Criteria Quality Rating Your Evidence and/or Reasoning

3 = Excellent
2 = Good
0 = Poor

1 = Fair
The Key Elements
1. The review includes the 0 The literature review
relevant literature. • briefly provides justification for the research problem (could
be document better, see Table B3);
• partially documents what is and is not known about the topic
(see elaboration below);
• briefly identifies the theoretical framework behind a study
(Social Ecological Theory);
• does not offer other study models for methods & procedures
• does not provide similar study results to help interpret study
results
Some key knowledge areas are missing, notable, (1) literature
that defines and describes social and environmental factors as
used in other studies and (2) prior studies with a similar purpose
These are big omissions because what is and isn’t known about
a key set of variables is not articulated and similar studies have
not been reviewed for use when interpreting current study
results. Overall, the review is extremely brief (only 3
paragraphs) and lacks detail.

2. The review examines sources 1 The literature review is quite thin, citing ~19 primary sources
that are recent and of high across the total of 3 paragraphs. Sources are mostly from peer-
quality. reviewed academic journals and include several government
reports. Source quality appears reasonable but could be more
current (for a 2010 study report): Many were dated late-1990s
through 2002. More studies with 8-10 years (2002-2010) would
better reflect current state of knowledge.

3. The literature review is 3 The literature review provides correctly formatted in-text
appropriately documented. references and end-text references;

4. The literature is thoughtfully 0 The literature review is very short and not clearly organized
synthesized. according to theme. The transition from statement of problem to
statement of study purpose is choppy, with a section on
intervention studies that does not seem to contribute to the
subsequent study design or interpretation.
5. The literature is critically 0 There is little critical analysis across studies with respect to any
examined. theme – sources are primarily used to justify statements not to
compare and contrast the ‘dialogue’ within the literature on a
given point. Several prior intervention studies from the literature
were mentioned and said to provide ‘useful guidance’ but that
guidance was not specified – the design features or results of
those interventions were not explained and the authors’ purpose
in reviewing them was unclear (except to say that more studies
could inform intervention ‘best practices).

4|Page Adapted from tables in HST2122 textbook Plano Clark and Creswell, 2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

General Evaluation
6. The study has a strong 0 The brevity, lack of detail, thinness of sources and relative lack
foundation in the literature. of critical analysis does not provide a strong foundation for a
prospective study.
7. The use of the literature fits the 1 The literature review of a quantitative study serves to fully locate
study's overall research and bound the study within the literature. It must provide
approach. adequate guidance for narrowing the intent of the study,
providing the known landscape of what is known, selecting
methods and ultimately, contextualizing and interpreting the
study’s own results. Because of:
• It omitted key themes or knowledge areas (e.g. what are,
and is known, about social and environmental factors, other
studies with similar purpose) (see Table C1&2.);
• It lacks critical analysis showing (not just state) the known
and unknown landscape (e.g. where there’s agreement,
disagreement) (See Table C6.);
• It lacks clear thematic organization (See Table C4.); and
• Its ‘thinness’ of evidence (See Table C2.)
there is not enough information to justify/explain why this study
was designed as it was or to contextualize and interpret study
results in prior literature.

Overall Quality Part C Total Part C My Overall Assessment of Part C =

Score =
0 - 10 = Low quality
11 - 16 = Adequate quality 5 Low quality
17 - 21 = High quality

5|Page Adapted from tables in HST2122 textbook Plano Clark and Creswell, 2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

Part D: Evaluating a Purpose in a Research Report

Quality Criteria Quality Rating Your Evidence and/or Reasoning

3 = Excellent
2 = Good
0 = Poor

1 = Fair
The Key Elements
1. The study's purpose is clearly 2 Purpose has 5 elements and all are stated within the
specified. Introduction; but, overall, they are not always clearly stated,
explained or explicitly related to each other. See items 2-5
below for elements.
2. The focus of the study is 1 The study focus is on a set of variables related to (1) the status
appropriate. of physical activity opportunities and (2) social and
environmental factors (e.g., school policies) in the middle school
setting. The authors seek to describe and explain the
relationship between these variables. These variables are not
defined or explained in the introduction so it is hard to know
exactly what is being measured without reading ahead in the
study (variables are listed in the Methods-Instrumentation
section). This focus on specific variables and on explaining the
relationship between variables is suited to a quantitative
approach.

3. The overall intent of the study 2 The study intent is to (1) describe trends in opportunities for
is appropriate. children’s PA and (2) examine social and environmental factors
related to the presence of these opportunities; ultimately, the
study seeks to determine if there is a relationship between the
opportunity for student physical activity (dependent variable)
and social/environmental factors of the setting (independent
variables). Again, this intent is consistent with a quantitative
approach. While present, this information is difficult to discern
without reading several times.

4. The participants and sites are 2 The study participants are middle school physical education
appropriate. teachers and seem appropriate as they seem well-placed to
offer insights as to children’s school PA opportunities and
influencing factors.
The setting is middle schools in the Southwestern US in 2006
which seem appropriate as study could provide a regional
perspective on the problem.

5. The purpose is narrowed 1 The theoretical framework for the study is the Social Ecological
through appropriate research Model. It is used to explain the relationship between variables;
questions and/or hypotheses that is how different aspects of middle schools’ physical and
and theory. social environments may influence the physical activity
behaviours of middle school students.

The study research questions are not stated explicitly but

suggested through a study aim: “Specific aims of this study
were to investigate the status of PA [physical activity]
opportunities afforded to public middle school students and to
examine interactions between existing factors and PA
opportunities in the middle school setting” (p. 185).
General Evaluation
6. The purpose follows logically 2 The problem was how low physical activity of children contribute
from the statement of the to the development of obesity and chronic disease in later life.

6|Page Adapted from tables in HST2122 textbook Plano Clark and Creswell, 2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

problem and literature review. This study purpose follows by seeking to provide insight into the
extent of the problem in a new setting (middle school) and the
factors that may influence it; however, the extensive discussion
of interventions detracts (confuses) from a logical flow from the
main problem (PA/obesity/chronic disease) to study purpose.

7. The purpose is consistent with 3 This focus on specific variables and on explaining the
the study's overall approach. relationship between variables is suited to a quantitative
approach

Overall Quality Part D Total Part D My Overall Assessment of Part D =

Score =
0 - 10 = Low quality
11 - 16 = Adequate quality 13 Adequate quality
17 - 21 = High quality

7|Page Adapted from tables in HST2122 textbook Plano Clark and Creswell, 2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

Part E1: Evaluating the Research Design in a Quantitative Research Report

Quality Criteria Quality Rating Your Evidence and/or Reasoning

3 = Excellent
2 = Good
0 = Poor

1 = Fair
The Key Elements
1. The choice of the research 1 The methods section says that a quantitative design
design is appropriate and was used, but no formal design type was specified
justified. and no hypotheses or research questions were
stated making interpretation of the study purpose
and appropriate design difficult. The study aim (in
intro) is dual-purpose: (1) To identify PA
opportunities and (2) to identify factors that may
influence opportunities that middle school children
have for PA. Though it seeks to explain relationships
between variables, the types of variables (social and
environmental factors) would be virtually impossible
to manipulate making it most likely for this to be a
non-experimental design. Because it seeks to relate
variables but no intervention is described, this is a
non-experimental design. As purpose (1) queries
trends in PA opportunities utilising a survey
instrument, it seems that this may be a survey
research design; however, as participants were
sampled using non-random (convenience) sampling,
it cannot be a well-crafted survey research design.
Instead, looking further, study purpose (2) appears
to drive the design selection as a correlational
research study design. This research design
enables a researcher to explain how some variables
which corresponds to the Xu et al. study purpose –
to explain how social and environmental factors
influence opportunities for PA. Other study details
that confirm the correlational design type include
using of only 1 group of participants (292 middle
school teachers), data collected for all variables
(using a questionnaire), and statistical analysis used
for testing relationships (factor analysis). I conclude
that the study purpose justifies the correlational
design but he authors have not provided any such
arguments or justification.
General Evaluation
2. The study used a rigorous 2 Study purpose (to identify factors that may influence
research design. opportunities that middle school children have for
PA) seeks to identify relationship among variables
that cannon be manipulated – this is a purpose that
suits a correlational research study design; a range
of relevant variables were measured using a survey
with what appears to be closed-ended questions –
this is appears to be reasonable data collection
method for these variables; results of statistical
analyses of survey items were reported in tables
containing correlation statistics; and the conclusion
claims an influence between factors and PA

8|Page Adapted from tables in HST2122 textbook Plano Clark and Creswell, 2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

opportunities (not a causal one). All of these

elements seem to fit together.
3. The use of the quantitative 3 The dual study aim is to identify (1) PA opportunities
research design addressed the and (2) factors that may influence opportunities that
study’s purpose. middle school children have for PA. Quantitative
designs, and correlational research study designs
specifically, are suited to identifying relationships
between variables when an experimental design is
infeasible (the social and environmental variables
can’t be manipulated).

Overall Quality Part E Total Part E My Overall Assessment of Part E =

Score =
0- 4 = Low quality
5- 7 = Adequate quality 6 Adequate quality
8- 9 = High quality

9|Page Adapted from tables in HST2122 textbook Plano Clark and Creswell, 2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

Part F1 : Evaluating the Participants and Data Collection in a Quantitative Research Report
Quality Criteria Quality Rating Your Evidence and/or Reasoning

3 = Excellent
2 = Good
0 = Poor

1 = Fair
The Key Elements
1. The sampling strategy is
appropriate and justified.

2. The sample size is appropriate

and justified.

3. Good quantitative procedures

are used to select and assign
participants
4. High quality instruments are
used to gather data.

5. The data are gathered using

ethical quantitative procedures.

6. The data are gathered using

standardized quantitative
procedures.

General Evaluation:
7. The study has a high level of
internal validity.

8. The study has a high level of

external validity.

Overall Quality Part F Total Part F My Overall Assessment of Part F =

Score =
0 - 12 = Low quality
13 - 20 = Adequate quality
22 - 24 = High quality

10 | P a g e Adapted from tables in HST2122 textbook Plano Clark and Creswell,

2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

Part G1: Evaluating the Data Analysis and Results in a Quantitative Research Report
Quality Criteria Quality Rating Your Evidence and/or Reasoning

3 = Excellent
2 = Good
0 = Poor

1 = Fair
The Key Elements
1. The data were rigorously
scored and prepared.

2. Good descriptive analyses

were conducted.

3. Good hypothesis testing

procedures were used.

4. The results are comprehensive.

5. The results include sufficient

information.

General Evaluation:
6. The data analysis represents a
good quantitative process.

7. The results provide a good

explanation of the study’s
purpose.

Overall Quality Part G Total Part G My Overall Assessment of Part G =

Score =
0 - 10 = Low quality
11 - 16 = Adequate quality
17 - 21 = High quality

11 | P a g e Adapted from tables in HST2122 textbook Plano Clark and Creswell,

2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

Part E2: Evaluating the Research Design in a Qualitative Report

Quality Criteria Quality Rating Your Evidence and/or Reasoning

3 = Excellent
2 = Good
0 = Poor

1 = Fair
The Key Elements
1. A research design guides the
conduct of the qualitative study.

2. The choice of the research

design is appropriate and
justified.

General Evaluation:
3. The study used a rigorous
research design.

4. The use of the qualitative

research design addresses the
study’s purpose.

Overall Quality Part E Total Part E My Overall Assessment of Part E =

Score =
0 – 6 = Low quality
7 – 9 = Adequate quality
10 – 12 = High quality

12 | P a g e Adapted from tables in HST2122 textbook Plano Clark and Creswell,

2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

Part F2: Evaluating the Participants and Data Collection in a Qualitative Report
Quality Criteria Quality Rating Your Evidence and/or Reasoning

3 = Excellent
2 = Good
0 = Poor

1 = Fair
The Key Elements
1. The sampling strategy is
appropriate and justified.

2. The sample size is appropriate

and justified.

3. The data types are appropriate.

4. The data are gathered using

rigorous qualitative procedures.

5. Data collection issues are

handled ethically and
thoughtfully.

General Evaluation:
6. The selected participants are
information rich.

7. The database provides

extensive and credible
information about the central
phenomenon.

Overall Quality Part F Total Part F My Overall Assessment of Part F =

Score =
0 - 10 = Low quality
11 - 16 = Adequate quality
17 - 21 = High quality

13 | P a g e Adapted from tables in HST2122 textbook Plano Clark and Creswell,

2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

Part G2: Evaluating the Data Analysis and Findings in a Qualitative Report
Quality Criteria Quality Rating Your Evidence and/or Reasoning

3 = Excellent
2 = Good
0 = Poor

1 = Fair
The Key Elements
1. The analysis process used
rigorous qualitative procedures.

2. Strategies were used to

validate the findings.

3. The findings include description

of the people, places, or events
in the study.

4. The findings include

appropriate themes about the
central phenomenon.

5. The findings relate multiple

themes to each other.

General Evaluation:
6. The data analysis represents a
good qualitative process.

7. The findings provide a good

exploration of the central
phenomenon.

Overall Quality Part G Total Part G My Overall Assessment of Part G =

Score =
0 - 10 = Low quality
11 - 16 = Adequate quality
17 - 21 = High quality

14 | P a g e Adapted from tables in HST2122 textbook Plano Clark and Creswell,

2015.
HST2122_171 Assessment 2 Evaluation Worksheet for Critiquing Research Study Report

Part H: Evaluating the Conclusion and Back Matter in a Research Report

Quality Criteria Quality Rating Your Evidence and/or Reasoning

3 = Excellent
2 = Good
0 = Poor

1 = Fair
The Key Elements
1. The major results are identified
and summarized.

2. The results are thoughtfully

examined in relation to the
literature and personal
reflections.

3. Appropriate implications of the

results for practice are
identified and justified.

4. Thoughtful critiques of the

study’s limitations are provided
and appropriate for the
research approach.
5. Suitable implications of the
results for future research are
identified and justified.

General Evaluation:
6. The interpretations are
consistent with the study’s
results and limitations.

7. The back matter is appropriate

for the study report.

Overall Quality Part H Total Part H My Overall Assessment of Part H =

Score =
0 - 10 = Low quality
11 - 16 = Adequate quality
17 - 21 = High quality

15 | P a g e Adapted from tables in HST2122 textbook Plano Clark and Creswell,

2015.