Making Connections - Evaluation of STEM Professional Development

Journal of STEM Teacher Education
Volume 53 | Issue 1 Article 2
April 2018
Making Connections: Evaluation of a Professional

Development Program for Teachers Focused on
STEM Integration
Judy Lambert
University of Toledo, [email protected]
Carmen Cioc
Sorin Cioc
Dawn Sandt
Follow this and additional works at: https://ir.library.illinoisstate.edu/jste

Part of the Engineering Education Commons, Other Teacher Education and Professional
Development Commons, and the Science and Mathematics Education Commons
Recommended Citation
Lambert, Judy; Cioc, Carmen; Cioc, Sorin; and Sandt, Dawn (2018) "Making Connections: Evaluation of a Professional Development
Program for Teachers Focused on STEM Integration," Journal of STEM Teacher Education: Vol. 53 : Iss. 1 , Article 2.
DOI: doi.org/10.30707/JSTE53.1Lambert
Available at: https://ir.library.illinoisstate.edu/jste/vol53/iss1/2
This Article is brought to you for free and open access by ISU ReD: Research and eData. It has been accepted for inclusion in Journal of STEM Teacher
Education by an authorized editor of ISU ReD: Research and eData. For more information, please contact [email protected].
Making Connections: Evaluation of a Professional Development Program
for Teachers Focused on STEM Integration
Cover Page Footnote
Support for "Making Connections" was provided in 2014 and 2015 by a grant under the federally funded
Improving Teacher Quality Program, administered by the Ohio Department of Higher Education.
This article is available in Journal of STEM Teacher Education: https://ir.library.illinoisstate.edu/jste/vol53/iss1/2

Journal of STEM Teacher Education
2018, Vol. 53, No. 1, 3–25
Making Connections: Evaluation of a Professional Development Program for

Teachers Focused on STEM Integration
Judy Lambert, Dawn Sandt, Carmen Cioc, and Sorin Cioc

University of Toledo
Abstract
This article reports on a 2-year evaluation of a STEM integration professional development
(PD) program for 40 math, science, and special education teachers in Grades 5–9 from a
large Midwestern public school district. The National Research Council’s framework for
integrated STEM education (Honey, Pearson, & Schweingruber, 2014) was used to explain
the goals, outcomes, nature and scope, and implementation of the program. Teachers were
measured on their growth in STEM content knowledge, technology integration, teaching
confidence, pedagogical beliefs, and impact of PD. Increases resulted in all these areas with
statistically significant improvements in most of them, particularly in Year 2. A significant
increase in math and science scores were also found in 413 students before and after
participation in an integrated STEM lesson. According to teachers, the greatest strengths
of the program were the STEM connections that teachers began making; the changes in
teachers’ instructional practices; improved attitudes, beliefs, and confidence in teaching;
increased comfort with using technology; and the enthusiasm that students exhibited during
a STEM lesson. Quantitative data and teacher feedback both indicate that the program was
highly successful and had a positive impact on teachers and students.
Keywords: Engineering; Mathematics; Pedagogy; Professional development; Science;
STEM Integration; Technology; Technology integration
Over the last 2 decades, there have been many calls for improvements in the quality of education
in science, technology, engineering, and mathematics (STEM). To answer this call, advocates have
proposed STEM integration as a new approach, which they argue will provide STEM instruction in
a more connected manner. According to the International Technology and Engineering Educators
Association (2017),
Integrative STEM Education is operationally defined as “the application of technological/
engineering design based pedagogical approaches to intentionally teach content and
practices of science and mathematics education through the content and practices of
technology/engineering education . . .” (Wells & Ernst, 2012/2015). (para. 3)
As early as 1996, the National Research Council (NRC) understood the need for STEM
integration when they stressed in their professional development (PD) standards, the National
Science Education Standards, that science teachers must “be able to make conceptual connections
within and across science disciplines, as well as to mathematics, technology, and other school
subjects” (p. 59). Sixteen years later, an NRC committee articulated a vision of STEM integration
This research was supported by a grant under the federally funded Improving Teacher Quality Program
administered by the Ohio Department of Higher Education in 2014 and 2015.
3
Journal of STEM Teacher Education Vol. 53 No. 1, Spring 2018
in A Framework for K-12 Science Education (2012). The framework called for deeper connections
and broad learning goals within three dimensions: (1) “Scientific and Engineering Practices”
(e.g., “asking questions (for science) and defining problems (for engineering),” “developing and
using models,” and “planning and carrying out investigations”), (2) “Crosscutting Concepts”
(e.g., “patterns”; “cause and effect: Mechanism and explanation”; “scale, proportion, and quantity”;
and “systems and system models”), and (3) “Disciplinary Core Ideas” (i.e., “Physical Sciences,”
“Life Sciences,” “Earth and Space Sciences,” and “Engineering, Technology, and Applications of
Science”; p. 3). In support of this vision, the Next Generation Science Standards (NGSS) include
“practices and core disciplinary ideas from engineering alongside those for science, raising the
expectation that science teachers will be expected to teach science and engineering in an integrated
fashion” (Honey, Pearson, & Schweingruber, 2014, p. 1). According to framework, which these
standards are based on,
Engaging in the practices of science helps students understand how scientific knowledge
develops; such direct involvement gives them an appreciation of the wide range of
approaches that are used to investigate, model, and explain the world. Engaging in the
practices of engineering likewise helps students understand the work of engineers, as well
as the links between engineering and science. Participation in these practices also helps
students form an understanding of the crosscutting concepts and disciplinary ideas of
science and engineering; moreover, it makes students’ knowledge more meaningful and
embeds it more deeply into their worldview. (National Research Council [NRC], 2012,
p. 42).
Likewise, the Common Core State Standards for Mathematics (CCSSM) call for students to
use mathematics in applied contexts (National Governors Association Center for Best Practices
& Council of Chief State School Officers [NGA & CCSSO], 2010). The standards also identify
practices in mathematics that can link to those of science and engineering. For example, the
standards explain:
Mathematically proficient students can apply the mathematics they know to solve problems
arising in everyday life, society, and the workplace. In early grades, this might be as simple
as writing an addition equation to describe a situation. In middle grades, a student might
apply proportional reasoning to plan a school event or analyze a problem in the community.
By high school, a student might use geometry to solve a design problem or use a function to
describe how one quantity of interest depends on another. Mathematically proficient students
who can apply what they know are comfortable making assumptions and approximations
to simplify a complicated situation, realizing that these may need revision later. They are
able to identify important quantities in a practical situation and map their relationships
using such tools as diagrams, two-way tables, graphs, ﬂowcharts and formulas. They can
analyze those relationships mathematically to draw conclusions. They routinely interpret
their mathematical results in the context of the situation and reﬂect on whether the results
make sense, possibly improving the model if it has not served its purpose. (p. 7)
Furthermore, the International Technology Education Association (ITEA; 2007) established
standards for K–12 technology literacy that require students to use a variety of technologies and
knowledge of design processes to solve problems and to collect and analyze data to test the solutions
to those problems. In its PD standards, ITEA expects that teachers are provided with PD to achieve
technology literacy, understand the basic concepts of design, and “comprehend the integrative
nature that links technology with science, mathematics, engineering, and other disciplines”
(ITEA, 2003, p. 43).
4
https://ir.library.illinoisstate.edu/jste/vol53/iss1/2
Although the trend seems to be moving toward an integrated approach to STEM education, there
is currently little research to help us understand how best to make cross-disciplinary connections
in K–12 STEM education or research on whether more explicit connections across disciplines will
significantly improve student learning (Honey, Pearson, & Schweingruber, 2014). According to
the NRC, the existing research has shown to result in knowledge gains in the science, technology,
engineering, and mathematics, more so for science than for mathematics, and the impact depends
on the integration approach and the supportive strategies offered during instruction. The NRC also
examined research from cognitive psychology, the learning sciences, and educational psychology
to explain the potential and challenges of STEM integration. Based on this research, integration
may be effective cognitively because connected concepts, rather than disconnected concepts, can
be more effectively organized in the brain. Achieving highly sophisticated concept organization,
a defining characteristic of experts, promotes concept comprehension, concept retrieval for future
use, and concept transferability to novel problems. Also, being able to represent the same concept
within and across disciplines in multiple ways can enhance learning. Social and cultural experiences
such as teamwork, active discussion, joint decision making, and collaborative problem solving can
support integrated learning and help students be successful with challenging tasks. On the other
hand, integration of too many complex concepts
(1) places excessive demands on resource-limited cognitive processes such as attention
and working memory, or (2) attempts to make bridges between ideas that were not well
learned, or (3) obscures important differences in STEM disciplines about how knowledge
is constructed and revised. (Honey et al., 2014, p. 78)
Furthermore, the use of real-world problems can either highly engage or impede students’ learning.
The concrete situation can give students a more meaningful context for learning but can also
prevent them from transferring their experiences to other settings. Together, the research has three
key implication for STEM education: (1) connections within and across disciplines “should be
made explicit” for students, (2) “students’ knowledge in individual disciplines must be supported”
in order to connect ideas from different disciplines effectively, and (3) “more integration is not
necessarily better” (Honey et al., 2014, p. 5). The benefits and challenges of integration requires a
strategic approach that considers the trade-off in cognition and learning.
Because the current literature includes a variety of integrative approaches with a range of
experiences and different degrees of connections, the NRC (Honey et al., 2014) developed a
descriptive framework to help them identify, describe, and investigate specific integrated STEM
initiatives in the K–12 education system. This framework will be used to report on 2 years of the
PD project Making Connections: Preparing Teachers to Integrate STEM, hereafter referred to as
Making Connections. Teachers typically attend training in STEM subjects as separate disciplines.
However, Making Connections was unique in its strong focus on STEM integration. Because
integrated STEM education is a relatively recent phenomenon, little is known from research about
how best to support the development of educator expertise in this domain (Honey et al., 2014).
Research on programs such as Making Connections for STEM educators’ can provide insight
into challenges and opportunities for preparing teachers to teach integrated STEM. In this article,
authors provide a description of the project and research study and discuss conclusions and
implications for other researchers and individuals interested in the PD of teachers in STEM
integration.
5
A Framework for Integrated STEM Education

The NRC framework (Honey et al., 2014) consisted of four features: (1) goals of integrated
STEM education, (2) outcomes of integrated STEM education, (3) the nature and scope of integrated
STEM education, and (4) implementation of integrated STEM education.
Goals of Integrated STEM Education

According to the NRC (2012), educator goals should include: (a) increased STEM content
knowledge and (b) increased pedagogical content knowledge. In accord with the NRC, Making
Connections goals were to provide opportunities for:
1. Teachers to review learning standards for mathematics and science and understand grade
level scope, sequence, and progressions;
2. Teachers to learn and understand Thinking Mathematics principles and practices and
5E science inquiry methods;
3. Teachers to practice and become more comfortable with using technology;
4. Teachers to understand and value the concept of STEM integration and to plan and
implement integrated STEM lessons; and
5. Students to be exposed to integrated STEM learning experiences.
Outcomes of Integrated STEM Education

The NRC (2012) suggested two educator outcomes of STEM integration: (a) changes in
practice and (b) increased STEM content knowledge and pedagogical content. Based on these and
Making Connections goals, anticipated outcomes included:
1. Teachers’ increased knowledge and understanding of mathematics and science;
2. Teachers’ increased attitude, beliefs, and degree of confidence in content knowledge and
pedagogy;
3. Teachers’ increased level of comfort with using technology during instruction;
4. Teachers’ increased knowledge, understanding, and appreciation of STEM integration and
ability to implement; and
5. Students’ increased achievement scores in mathematics and science.
Nature and Scope of Integrated STEM Education

The NRC (2012) identified three important elements that determine the nature and scope of
integration: (1) type of STEM connections, (2) disciplinary emphasis, and (3) duration, size, and
complexity of initiative.
Type of STEM connections. As part of all PD in Making Connections, instructors made explicit
a variety of connections among all the STEM disciplines. Two days of the program are described
here to illustrate the nature of the PD and to provide examples of the kinds of connections made.
In a physical science inquiry lesson on the energy of force, motion, mass and friction (science),
groups of teachers collected data using a LabQuest motion sensor (technology) that measured
the position and speed of a plastic car going down a ramp (science inquiry). Teachers entered the
data from the trials and created position time graphs using Google Sheets (technology), allowing
them to examine the slope of the line (math) from which they were able to analyze the results.
Using Google Docs (technology), teachers then created lab reports to communicate their results,
6
providing them time to look at the evidence, make interpretations, and develop a claim based on
results (science inquiry). Time was provided for teachers to compare their results with other groups
and discuss differences. In the next lesson, teachers reviewed concepts of kinetic friction including
magnitude and direction (science) to examine how surface texture affects friction (science inquiry).
Sliding a block of wood across surfaces with different textures, teachers used LabQuest force
sensors (technology) this time to record the friction force required to slide the block across each
surface before, during, and after each pull. As in the last lesson, data were entered, graphed, and
analyzed (math) using a Google Sheet (technology). Google Docs (technology) was again used to
communicate results.
In a subsequent lesson, online simulations (technology) were used to extend teachers’ knowledge
of the concepts of kinetic and potential energy (science). After teachers themselves manipulated
the simulations, the instructor explained the differences between science inquiry and engineering
design (practices) and introduced the Mars Rover Design activity (engineering). In this activity,
teachers were asked to design a Mars exploration vehicle that could carry a 100-gram load and get
the most traction over a sandy planet surface with minimum force. Teachers were provided with
a small rectangular paper loaf pan, straws, round pieces of foam and lollipop sticks to design and
modify their models (engineering application of motion, weight, and force of friction). Granulated
sugar was provided to simulate the surface of Mars. Teachers used the LabQuest force sensor
to test and evaluate their prototypes by recording the weight of the vehicle and its pulling force
(weight) through the sugar when designed with the different materials for the wheels. As in previous
lessons, data were recorded and graphed, and teachers discussed the weaknesses and strengths of
their designs as part of the engineering design process.
Three other engineering design projects were introduced during the program: “let it fly,” “keep
it cool,” and the roller coaster. In the second design challenge, “let it fly,” the teams were asked to
use the engineering design strategies to brainstorm, develop, test, and improve a rocket made of
a water bottle and balloons that would travel the length of a classroom. This challenge was later
improved on by finding the fastest rocket and adding a load in the form of water to the rocket.
Since the project, this engineering design has been adopted into the science curriculum by several
teachers and was presented at a major STEM conference, STEMCON 2017 in Cleveland. The third
design challenge, “keep it cool,” made use of the science content related to matter and involved
the development of prototype insulation materials able to keep water cold as long as possible.
Temperature readings for the cold water were recorded and analyzed (math) to evaluate different
designs and proposed improvements (engineering design). Finally, the roller coaster engineering
design made use of the science content related to kinetic and potential energies by designing a
roller coaster using foam tubes as coaster tracks.
These kinds of experiences provided teachers with multiple types of integration. They were
bringing together concepts from more than one discipline: science (force and motion, friction,
materials, and kinetic and potential energy), technology use (to organize, analyze, and communicate
data and to learn difficult concepts through simulations), engineering (four design challenges
each requiring the teachers to brainstorm, evaluate, choose a solution, build a prototype, test the
prototype, and make improvements), and mathematics (data collection, analysis, and interpretation
and mathematical thinking). Teachers were also connecting a concept from one subject to a
practice of another (applying force and motion to engineering design and applying mathematical
practices to understand data collected from science inquiry). Finally, teachers were combining
7
practices from two disciplines (science inquiry, engineering design, and Thinking Mathematics
methods).
Disciplinary emphasis. The program was designed to give teachers equal amounts of
instruction in science, math, and engineering with technology infused where needed to conduct
investigations. The disciplinary emphasis in Year 1 included the following core ideas in physical
sciences NGSS (NGSS Lead States, 2013): “Forces and Motion” (PS2.A), “Types of Interactions”
(PS2.B), “Definitions of Energy” (PS3.A), “Conservation of Energy and Energy Transfer” (PS3.B),
and “Relationship Between Energy and Forces” (PS3.C). The focus of Year 2 included the same
physical sciences core ideas as in Year 1 with the addition of electromagnetism (PS4.B).
Additionally, emphasis was given to practices within all three disciplines so that teachers could
increase their pedagogical skills in science practices (inquiry), mathematical practices (Thinking
Mathematics), and engineering design. PD must endeavor not only to increase the content
knowledge of teachers, but also, the knowledge of how students think and learn about a specific
discipline and engage in the practices of that discipline. The eight practices deemed essential by
the NRC (2012, p. 42) and thus the NGSS (NGSS Lead States, 2013, p. 48) for all students to learn
integrate the STEM disciplines. The practices listed below, which are intended to overlap and
interconnect, were interwoven in the program:
1. Asking questions (for science) and defining problems (for engineering)
2. Developing and using models
3. Planning and carrying out investigations
4. Analyzing and interpreting data
5. Using mathematics and computational thinking
6. Constructing explanations (for science) and designing solutions (for engineering)
7. Engaging in argument from evidence
8. Obtaining, evaluating, and communicating information. (NRC, 2012, p. 42)
The emphasis in mathematics was derived directly from the Common Core State Standards for
Mathematics (NGA & CCSSO, 2010). The following standards, depending on grade level, were
emphasized in the PD.
1. Geometry: “Graph points on the coordinate plane to solve real-world and mathematical
problems” (p. 38).
2. Statistics and probability: “Develop understanding of statistical variability” (p. 45).
3. Statistics and probability: “Summarize and describe distributions” (p. 45).
4. Statistics and probability: “Investigate patterns of association in bivariate data” (p. 56).
5. Expressions and equations: “Represent and analyze quantitative relationships between
dependent and independent variables” (p. 44).
6. Expressions and equations: “Solve real-life and mathematical problems using numerical
and algebraic expressions and equations” (p. 49).
7. Expressions and equations: “Understand the connections between proportional relationships,
lines, and linear equations” (p. 54).
8. Ratios and proportional relationships: “Analyze proportional relationships and use them to
solve real-world and mathematical problems” (p. 48).
As in science and engineering, the emphasis in math was not only standards but also practices.
The Thinking Mathematics principles (American Federation of Teachers, 2003) and the CCSSM’s
Standards for Mathematical Practice (NGA & CCSSO, 2010) were central themes throughout
8
instruction, both of which support the practices of math and engineering. The Thinking Mathematics
approach to math instruction is based on scientific research about how children learn mathematics.
Participants discuss research findings and their implications for classrooms and use strategies
in their classrooms and reflect on the results. Participants discuss the real-time implications
of practices such as using concrete models, managing classroom discussions, and maintaining
students’ attention and involvement. The 10 principles of Thinking Mathematics and the Standards
for Mathematical Practice are linked and attention is paid to how concepts and skills develop.
Practices such as reasoning, constructing viable arguments, precision, and looking for mathematical
structure combine with principles of Thinking Mathematics such as helping students visualize
problems, requiring them to discuss and justify their mathematical thinking, using situational
problems to connect mathematics to life, and balancing conceptual and procedural knowledge to
develop such understanding. The Mathematical Practices (NGA & CCSSO, 2010) include:
1. “Make sense of problems and persevere in solving them”;
2. “Reason abstractly and quantitatively”;
3. “Construct viable arguments and critique the reasoning of others” (p. 6);
4. “Model with mathematics”;
5. “Use appropriate tools strategically”;
6. “Attend to precision” (p. 7);
7. “Look for and make use of structure”; and
8. “Look for and express regularity in repeated reasoning” (p. 8).
Duration, size, and complexity of initiative. Making Connections was offered as a year-
long PD program consisting of six afterschool sessions, a 10-day Summer Institute, six online
monthly discussions during the academic year, and classroom visits. Faculty members from a large
midwestern university provided a portion of the instruction during the program. These included a
member from the Department of Math and Sciences (math), three members from the College of
Education’s Department of Curriculum and Instruction (educational technology, math, and special
education), and two members from the College of Engineering (Departments of Engineering
Technology and Mechanical Engineering). The university partnered with a local school district,
and a district-level science teacher and a math teacher from that district, both already on special
assignment from the classroom, provided the majority of instruction in math, science, and
pedagogical practices. In Year 1, math, science, and special education teachers in Grades 5–9 were
recruited.
In Year 2, four faculty members, one from educational technology, one from special education,
and two from engineering worked on the project. Teams of teachers from Grades 7–8 were recruited
as participants from the same school to encourage collaboration and support while planning and
teaching an integrated STEM lesson. Two mentors, one science and one math teacher from the
district, were also recruited to assist in the program instruction, to support teachers during instruction,
and to scaffold online discussions during the academic year. Additionally, more explicit training
and time was given to integrating differentiated instructional strategies in the direct instruction,
inquiry-based activities, and lesson planning using the guidelines and principles of Universal
Design for Learning (UDL) framework (CAST, 2011). CAST explains how the UDL framework is
grounded in modern neuroscience research showing that our learning brains are comprised of three
different networks: recognition, strategic, and affective. UDL guidelines align the networks with
three principles: recognition to representation, strategic to action and expression, and affective
9
to engagement. UDL draws from other research in cognitive psychology and learning sciences;
therefore, it was deemed most suitable for inclusion in a program focusing on STEM integration
because it could help teachers learn to strategically plan for the success of every learner in their
STEM classrooms. Finally, in both years, teachers were provided with Chromebooks, notebooks
with instructional materials to take back to the classroom, cables to connect Chromebooks to video
projectors in the classroom, and books and other materials to use in their integrated STEM lessons.
The program was designed in large part to help teachers understand the value of STEM
integration, to recognize connections across the STEM disciplines, and to provide them with time
to design a STEM integrated lesson plan, which they would then teach in the formal classroom. All
program instruction and participant lessons were based on current state standards and practices in
each discipline. Therefore, only minor changes were needed in the classroom to implement lessons
such as when teams of teachers worked together to coteach or when one of the teachers needed to
change their schedule to teach a topic earlier or later in the school year.
Implementation of Integrated STEM Education

Making Connections consisted of a variety of instructional approaches. The mornings began
with 15-minute icebreaker or community-building activities that at times focused on a core idea in
one of the disciplines. The day typically began with direct instruction, the most common method used
to deliver content knowledge, which was often accompanied by independent readings, discussions,
hands-on activities, or online explorations. Direct instruction was followed in the afternoons by
science inquiry experiments or engineering design problems. Teachers were given time at the end
of each day to reflect on their learning and how they might incorporate their new knowledge into
an integrated STEM lesson. Direct instruction of UDL Principles was presented at the beginning
of the workshop and was supplemented with inquiry prompts throughout each day by faculty
members, science and math personnel, and mentors about how the UDL principles may be applied
within particular content areas or activities to increase student comprehension, engagement, or
expression of learning. During the 10-day summer institute, teachers were provided 2 days to write
design integrated STEM lessons (some combination of math, science, or engineering disciplines)
and student assessments, which they were required to implement in the fall of that year. Teachers
were also required to include the use of 5E inquiry and Thinking Mathematics methods, to integrate
differentiated instructional strategies, and to employ technologies during in their lessons. Educator
support was provided when needed by the district math and science personnel during lesson
implementation, and lessons were video recorded.
In Year 1, 237 students participated in the STEM lesson taught by Making Connections teachers,
and in Year 2, 190 students participated. The professional learning community that developed
among teachers during the summer institute was sustained through participation in six online
discussions during the school year. Discussion prompts were posted to encourage teachers to share
and reflect on their pedagogical strategies, challenges, and successes during the implementation
of integrated STEM lessons. Afterschool sessions were offered to either introduce new concepts,
review learned concepts, or offer time for sharing and discussion on how teachers were experiencing
their integrated STEM lessons. As part of participation in the program, teachers could earn district
continuing education contact hours (90 hours in Year 1 and 96 hours in Year 2) and gift cards to
make classroom purchases ($400 in Year 1 and $200 in Year 2).
10
Method
A mixed-method explanatory research design was used to evaluate the outcomes of the Making
Connections program. A large midwestern university partnered with an urban school district defined
as high need, serving over 22,000 children with about 77% of them from families identified as
economically disadvantaged and 59% minorities (Ohio Department of Education [ODE], 2016a).
The district’s 2015–2016 student proficiency data (ODE, 2016b) show a general pattern of low
performance in mathematics and science with the average number of proficient students for all
grade levels well below the state average, pointing towards a significant need for PD in these
subject areas. This program evaluation sought to answer the following research questions.
1. Will the PD increase teachers’ content knowledge in math and science?
2. Will the PD increase teachers’ attitudes, beliefs, and degree of confidence in content
knowledge, pedagogy, and use of technology?
3. Will the PD improve teachers’ appreciation and understanding of STEM integration and
ability to implement it?
4. Will the PD increase students’ achievement in mathematics and science?
Participants
A convenience sample of teachers included those individuals who registered for and committed
to the PD for a full year. In Year 1, 28 teachers (Grades 5–9 from 9 schools) participated in the
program who served 85% of high-need students (e.g., need special services, low income, below
grade level). In Year 2, 13 teachers (Grades 7–8 from 6 schools) participated in the program who
served 94% high need students. None of the teachers from Year 1 participated in Year 2. Also, a
convenience sample of students included those individuals assigned to each teacher at the beginning
of the year and who were either in a self-contained classroom or were in a teacher’s class period
during the integrated STEM lesson. See Table 1 for number of teachers by discipline and grade
level and number of students in each year.
See Tables 2 and 3 for participant demographics. In Year 1, there were seven males, and 21
females. Of those, there were one Hispanic, five Black, three Asian, and 19 White teachers. Eleven
of those teachers were certified in high school math, two were certified in high school science, five
were certified in K–12 special education (SPED), and 10 were certified in the middle grades (with
math or science being an area of concentration). In Year 2, all teachers were females. Of those,
there were two Black and 11 White teachers. Eleven of those teachers were certified in the middle
grades (with math or science being an area of concentration), and two were certified in K–12
special education.
Table 1
Participants by Content Areas
Year 1 Year 2 Special educationa Gifted & talenteda
Math Grade 9 teachers 12 2
Math Grades 5–8 teachers 12 7 2 1
Science Grade 9 teachers 1
Science Grades 5–8 teachers 3 6 2
Total teachers (n = 40) 28 13
Total students (n = 413) 237 176 53 7
a
Teachers or students included in total count.
11
Table 2
Year 1 Participant Demographics
Participant Gender Race/ethnicity Grade level taught Subject area taught Certification
1 Female Hispanic 9 Math HS math
2 Female Black 7, 8 SPED - Math K–12 SPED
3 Male White 9 Math HS math
4 Female White 9 Math HS math
6 Female Black 6 Math Middle grades
7 Female White 5, 7, 8 Math Middle grades
8 Female Black 5 SPED -Science K–12 SPED
9 Female White 5, 6 SPED - Math K–12 SPED
10 Male White 6, 7, 8 Math Middle grades
11 Female White 5 Math Middle grades
12 Male Black 8 Math Middle grades
13 Female White 6, 7, 8 Math Middle grades
14 Female Black 5, 6, 7, 8 Math Middle grades
16 Female White 6, 7, 8 SPED - Math K–12 SPED
17 Male White 7, 8 Science Middle grades
20 Female Asian 5 Math Middle grades
22 Female White 5, 6, 7 SPED - Math K–12 SPED
23 Female Asian 9 Math HS math
24 Female White 9 Science HS Science
26 Male White 5 Science HS Science
28 Female Asian 9 Math HS math
Table 3
Year 2 Participant Demographics
Participant Gender Race/ethnicity Grade level taught Subject area taught Certification
1 Female Black 7–8 Science Middle grades
2 Female White 7–8 Math & social studies Middle grades
3 Female White 6–8 Science Middle grades
4 Female White 7–8 Science & reading Middle grades
6 Female White 6–8 Self-contained K–12 SPED
All subjects
7 Female Black 7–8 Math Middle grades
9 Female White 6–7 Math Middle grades
10 Female White 7–8 Science, math intervention K–12 SPED
11 Female White 8 Science & social studies Middle grades
12 Female White 6–8 Math Middle grades
13 Female White 6–8 Science Middle grades
12
Instruments
Teacher content knowledge. In Years 1 and 2, teachers were given a math and science content
knowledge test, designed by project personnel, at the first afterschool session and again at the end
of the summer institute. The test for Year 1 was composed of 48 questions related to math, science,
engineering, and pedagogical strategies with a maximum score of 48. Revisions were made on the
test after Year 1 to take out questions related to Grade 5 learning standards and to include more
questions specifically related to standards in Grades 6–8. In Year 2, the test contained 50 questions
with a maximum score of 100.
Technology integration. The Teacher Technology Integration Survey (TTIS; Vannatta &
Banister, 2009) measured teachers’ attitude and level of comfort at integrating technology in the
classroom using a Likert-type scale from 1 (strongly disagree) to 4 (strongly agree). This survey
was given to teachers at the first afterschool session and again at the end of the year-long program.
Three of the 10 TTIS subscales, which measured three constructs, were used in this study:
Risk-taking Behaviors and Comfort with Technology (α = .8540), Perceived Benefits in using
Technology in the Classroom (α = .8490), and Beliefs and Behaviors about Classroom Technology
Use (α = .8790).
Teaching confidence, pedagogical beliefs, and impact of PD. The Ohio Department of Higher
Education’s preliminary (2016b) and follow-up (2016a) surveys were used to measure teachers’
confidence levels about teaching, beliefs about effective pedagogy in teaching mathematics
and science, quality of the PD, and impact of the project activities on participants’ students and
participants. This survey was given to teachers at the first afterschool session and again at the end
of the year-long program. See Appendix for descriptions of subsections.
Student content knowledge. In Year 1, teachers created their own assessments to measure
students’ content knowledge before and after they taught their integrated STEM lesson. Teachers
within a grade level collaborated to create the same student assessment for their specific grade level
in order to offer comparisons. However, several concerns regarding Year 1 student tests motivated
project personnel to change procedures in Year 2. These concerns include whether the requirement
for teachers to create common tests forced some of them into teaching a particular lesson or
teaching to the test and whether the test questions were entirely reliable based on content covered
in the PD. Therefore, in Year 2 teachers were provided a bank of test items, created by program
personnel based on content covered in the summer institute, from which they chose test items to
measure student learning on their integrated STEM lessons. In both years, program personnel
reviewed lesson plans and assessments to ensure they adequately measured the content knowledge
that would be taught by teachers and that corresponded with the content teachers learned in the PD.
Summer institute daily evaluations. At the end of each day during the 10-day summer
institute, teachers were asked in an online survey to rate the instruction provided, give suggestions
for improvement, and describe what they learned or could apply in different areas such as
differentiated instruction or STEM integration. Anecdotal evidence will follow from teachers’
answers to the question regarding knowledge of STEM integration.
Final questionnaire. Teachers were asked to respond to seven open-ended interview questions
at the conclusion of the program to provide some description of teachers’ final perceptions. Some
of the questions included: What has benefited you the most by participating in this project, can you
describe some ways that your teaching practice has changed or improved as a result of participating
13
in this project, what challenges have you encountered in trying to implement your lesson plans, and
what are a couple of things that you thought were the major strengths of this PD or that you liked
the most?
Results and Discussion

Teacher Content Knowledge
These results address Research Question 1: Will the PD increase teachers’ content knowledge
in math and science? A paired-samples t-test was calculated to compare pre- and post-test scores in
math and science before and after the summer institute. In Years 1 and 2, teachers scored higher on
math and science content knowledge after instruction (see Table 4). In Year 1, the difference was
a statistically significant increase, t(23) = 6.119, p < .0005, d = 1.25. In Year 2, the difference was
also a statistically significant increase, t(12) = 3.982, p < .002, d = 1.10 (see Table 5).
Technology Integration
These results help to answer Research Question 2: Will the PD increase teachers use of
technology? Based on the TTIS, Year 1 teachers had only slightly higher scores on technology
attitude after the program (M = 3.0567, SD = 0.599) than before (M = 3.030, SD = 0.625). However,
Year 2 teachers showed statistically significantly higher scores after the program (M = 3.692,
SD = 0.210) than before (M = 3.196, SD = 0.390), t(11) = 6.153, p < .0005, d = .168. However,
the TTIS more closely measures teachers’ comfort level in taking risks when using technology
(e.g., troubleshooting and learning new technologies) and perceiving the benefits of using
technology (e.g., technology is a priority for me and modeling effective use for students), all of
which require greater levels of comfort and continued practice that go beyond the initial use of
Table 4
Paired Samples Statistics for Teachers’ Content Knowledge in Years 1 and 2
N M SD
Year 1
Pre 24 15.458 6.199
Post 24 24.666 8.442
Year 2
Pre 13 65.538 15.387
Post 13 78.307 13.437
Note. Scores were only reported for teachers who completed both the pretests and posttests. Teachers who
did not complete the posttests either dropped the program or were not present on the day of testing.
Table 5
Paired Samples Test for Teachers’ Content Knowledge in Years 1 and 2
Paired differences
95% CI for mean
difference
Std. error Sig.
M SD mean Lower Upper t df (2-tailed)
Year 1: Pre–Post 9.208 7.3719 1.504 12.321 6.095 6.119 23 .000
Year 2: Pre–Post 12.769 11.562 3.206 19.756 5.782 3.982 12 .002
14
technology. In Year 2, more focused instruction was provided on using Google Classroom and
other classroom technologies through online tutorials, which may have resulted in higher scores
for these teachers.
Teaching Confidence, Pedagogical Beliefs, and Impact of PD

The results presented in this section (including Sections 1–6) answer Research Question 2:
Will the PD increase teachers’ attitudes, beliefs, and degree of confidence in content knowledge,
pedagogy, and use of technology? The following data describes the outcomes of the Ohio
Department of Higher Education’s preliminary (2016b) and follow-up (2016a) surveys.
Section 1: Teaching confidence. Year 1 teachers had slightly higher scores on confidence about
their teaching after the program (M = 1.773, SD = 0.706) than before (M = 2.000, SD = 0.724).
In Year 2, teachers had statistically significantly higher scores after the program (M = 1.369, SD =
0.350) than before (M = 1.991, SD = 0.215), t(11) = 6.153, p < .0005, d = .269).
Section 2: Teaching approach. With a continuum of 1 (strongly agree) to 5 (strongly
disagree), higher scores indicated that teachers use more student-centered or constructivist
approaches. Table 6 reports mean scores by teacher, and Table 7 reports mean scores by teaching
approach.
Overall, teachers tended to use a mix of traditional and constructivist teaching approaches
(middle score = 3) with equal numbers of teachers leaning slightly more toward one of the
approaches. Year 1 teachers became only slightly more constructivist in their teaching approaches
after the program (M = 3.086, SD = .320) than before (M = 2.920, SD = .616). In Year 2, teachers
were also only slightly more constructivist after the program (M = 3.170, SD = .247) than before
(M = 3.053, SD = .356), but the increase was not statistically significant.
Table 6
Mean Scores on Teaching Approaches by Teacher
Year 1 Year 2
Pre Post Pre Post
Teacher (n = 15) (n = 15) (n = 13) (n = 13)
1 3.13 3.00 3.00 3.63
2 3.25 3.75 3.50 3.38
3 2.75 3.25 3.00 3.50
4 2.75 3.00 2.50 2.75
5 2.88 3.13 3.75 3.13
6 3.13 3.13 2.75 3.13
7 3.25 3.00 3.38 3.25
8 2.88 3.00 2.75 3.13
9 3.13 3.00 2.75 3.13
10 2.88 2.50 3.13 3.13
11 2.88 3.38 3.13 3.00
12 2.38 2.88 3.00 2.88
13 2.63 3.63
14 3.00 2.88
15 2.88 2.75
Grand mean 2.92 3.09 3.05 3.17
15
Table 7
Mean Scores by Teaching Approach
Year 1 Year 2
Pre Post Pre Post
Teaching approach (traditional vs. constructivist) (n = 15) (n = 15) (n = 13) (n = 13)
Teacher-led lecture vs. dialogue 3.73 4.00 3.75 3.58
Cooperative group learning vs. independent work 2.53 2.47 2.17 2.17
Central ideas vs. broad coverage of topics 2.27 1.93 2.33 1.92
Students’ role is to receive facts vs. apply inquiry 3.47 3.80 3.75 4.33
Hands on approaches vs. reading and lectures 2.40 3.73 3.75 4.00
Encouraging vs. difficulty in encouraging 2.07 1.53 2.25 2.00
Conventional assessments vs. alternative methods 3.20 3.27 2.92 3.50
Use same instructional techniques vs. 3.67 3.93 3.50 3.83
differentiation
In Year 1, teachers tended to move toward student-centered, constructivist teaching approaches

by using more dialog and interaction, inquiry, hands-on learning, and differentiation. In Year 2,
teachers moved towards using more inquiry, alternative assessments, and differentiation.
Section 3: Impact of PD on teachers’ content knowledge. Based on mean scores, teachers
strongly agreed that the PD impacted their content knowledge with a majority of scores falling
between 1 and 2. Technology, inquiry and hands-on learning, and improvement of teaching were
three areas most impacted, closely followed by enthusiasm for teaching, math or science concepts
and standards, and instructional approaches. Student assessment and questioning techniques were
the least impacted but still received high scores of 1.93 and 2.01 respectively. See Table 8 for
means scores by teacher and Table 9 for mean scores by content area.
Section 4: Quality of PD. Table 10 depicts teachers’ mean scores on the quality of PD. All
scores were between 1 and 2, indicating teachers’ agreement that the PD was of very high quality.
Table 8
Mean Scores on Impact of PD on Content Knowledge by Teacher
Teacher Year 1 Year 2
1 1.90 1.25
2 1.40 1.75
3 1.40 1.38
4 2.00 2.13
5 2.00 1.00
6 2.50 1.75
7 1.60 2.13
8 1.00 1.63
9 1.90 1.00
10 1.00 2.75
11 1.30 1.75
12 2.00 1.13
13 1.00
14 1.00
15 1.30
Grand mean 1.55 1.64
16
Table 9
Mean Scores on Impact of PD by Content Area
Content Area Year 1 Year 2 Grand mean
Math/science concepts 1.47 1.67 1.57
Math/science standards 1.47 1.67 1.57
Ways to assess students 1.93 1.92 1.93
Questioning techniques 1.93 2.08 2.01
Technology 1.40 1.17 1.29
Differentiation 1.87 1.92 1.90
Instructional approaches 1.40 1.75 1.58
Inquiry, hands-on activities 1.27 1.33 1.30
Improvement of teaching 1.33 1.42 1.38
Increased enthusiasm for teaching 1.47 1.50 1.49
Table 10
Mean Scores on Quality of PD
Year 1 Year 2 Grand mean
Ample time 1.73 1.75 1.74
Adequate follow-up 1.80 1.58 1.69
Useful resources 1.67 1.33 1.50
High quality, sustained and intensive 1.67 1.50 1.59
Linked to state and national standards 1.47 1.33 1.40
Highest agreement was that the PD was linked to state and national standards, that it provided
useful resources, and that it was sustained and intensive. Lower scores might indicate that teachers
desired ample time and adequate follow-up as part of the PD, even though these areas were still
relatively high with scores of 1.74 and 1.69, respectively.
Section 5: Impact of PD on students. Table 11 depicts teachers’ mean scores on the impact
of PD on their students. With scores between 2 and 3, teachers agreed that students were impacted
positively by the PD, reporting increased attention, enthusiasm, and involvement in classroom
activities as well as improved quality in their school work.
Section 6: Impact of PD on teachers. Table 12 shows mean scores on the impact of PD on
teachers. Scores ranged between 1 and 2, indicating that the PD had a very positive impact on
teachers. All teachers would recommend this program to other teachers, established a professional
network among participants, maintained contact with other participants, and shared what they
learned with others.
Student Content Knowledge

These results will answer Research Question 4: Will the PD increase students’ achievement in
mathematics and science? Based on student assessments that were given before and after instruction
of an integrated STEM lesson, both Years 1 and 2 resulted in statistically significantly higher
scores after instruction than before, showing that student content knowledge greatly increased as a
result of the STEM lesson. It also indicates that the instruction of teachers while integrating STEM
produced positive results in students (see Table 13).
17
Table 11
Mean Scores on the Impact of PD on Students
My students are more attentive, enthusiastic and involved 2.13 2.17 2.15
in classroom activities
The quality of student work is noticeably improved 2.20 2.42 2.31
My students are participating in science and/or math 2.53 2.83 2.68
activities outside of the classroom to a greater degree
Table 12
Mean Scores on the Impact of PD on Teachers
Maintained contact with other participants 1.13 1.00 1.07
Maintained contact with college faculty 1.47 1.42 1.45
Established a professional network among participants 1.20 1.25 1.23
Attended a professional association conference 1.93 1.33 1.63
Have or would recommend this program to other teachers 1.00 1.00 1.00
Shared what I learned with colleagues through informal 1.07 1.00 1.04
interactions
Have shared what I learned with colleagues through 1.53 1.75 1.64
formal interactions
In Year 1, the difference was a statistically significant increase, t(236) = 23.856, p < .0005,
d = 1.59. In Year 2, the difference also was a statistically significant increase, t(170) = 15.756,
p < .0005, d = 1.20 (see Table 14).
Summer Institute Daily Evaluations

These results will partially address Research Question 3: Will the PD improve teachers’
appreciation and understanding of STEM integration? On the survey given at the end of each
day of the summer institute, teachers were asked, “What new knowledge did I learn today about
integrating math and science/engineering content, technology use, or teaching methods?” The
quotes that follow provide some anecdotal evidence.
• “Data collection in science offers many opportunities to use math with the data.”
• “I could work with the science teacher to use the numerical results from the electric circuit
experiment and have my students find which central means of tendency best represents the
data.”
• “Physics, Engineering and Math use a multitude of cross-curricular concepts. Graphing
and analyzing data is one way; following formulas is another; converting units is a third.”
• “The creation of table and graphs from the data collected during the electricity lab could be
integrated into a STEM math lesson.”
• “Engineer a solution to an insulation/heat transfer problem, using science background to
inform the engineering process. Then use technology to collect and graph data, and graph
and interpret the data collected using math knowledge.”
• “The science is directly related and an amazing connection to math that we cover in the 7th
grade.”
18
Table 13
Paired Samples Statistics for Students’ Content Knowledge in Years 1 and 2
N M SD
Year 1
Pre 237 27.722 20.583
Post 237 69.532 22.945
Year 2
Pre 171 41.374 17.389
Post 171 68.830 21.656
Table 14
Paired Samples Test for Students’ Content Knowledge in Years 1 and 2
Paired differences
95% CI for mean
difference
Std. error Sig.
M SD mean Lower Upper t df (2-tailed)
Year 1: Pre–Post 41.81 26.981 1.753 45.263 38.357 23.856 236 .000
Year 2: Pre–Post 27.456 22.788 1.742 30.896 24.016 15.756 170 .000
• “I could have students make rockets and use their distances as data to use to find the central
means of tendency.”
• “I use hot wheels cars for my force and motion activity, I would like to use the engineering
piece where they design and test their own car. I would give them ownership of the activity.”
• “Having the Mathematics embedded with the Science and engineering shows me how I can
adjust parts of my lesson plan to incorporate data and make the connections to math and the
real world connection to engineering.”
• “STEM is heavy in problem solving, kids needs to get better at this and making mistakes
and being okay with that.”
• “The lessons will be more meaningful to the students. Collecting real data to graph is so
much more personal and interesting than data someone else gathered.”
• “These concepts are so interconnected and by integrating students will see the connection
and might gravitate towards careers in one of the areas.”
• “Students will benefit in the long run, understanding that learning, math, science, problem
solving, designing goes on outside of the four walls of a classroom.”
Final Questionnaire
These results will also answer Research Question 3: Will the PD improve teachers’ appreciation
and understanding of STEM integration and ability to implement? Evidence of the ability to
implement STEM integration can best be found in the lesson plans and video recordings of actual
lessons as they were implemented. Lesson plans were completed at the end of the summer institute
and were examined for accuracy and fidelity to the concepts learned in summer. However, the
videos are in the process of being analyzed and will be reported in a future article examining
teachers’ ability to implement STEM integration. For now, comments on the final questionnaire
will give some evidence that teachers made concerted efforts to teach their integration STEM
lesson and discovered positive results for themselves and their students.
19
On one question, teachers were asked how they benefited most from this program. For many
teachers, the math and science instructional strategies were deemed most valuable. Thinking about
how to integrate mathematics and science was a new concept for some of the teachers, but they
shared how this project helped them to better understand how science and mathematics are related
and the how the two subjects can be integrated easily in a lesson. Most teachers mentioned a
better understanding of learning standards in their content area. They found the various resources,
particularly those available in the schools, to be helpful in supporting STEM integration. One
participant commented,
I have benefited the most from being forced to plan that math/science lesson. I had to
see how I could incorporate the math into the science and vice versa. I had to also utilize
technology. Now I think about these things and look for ways to incorporate as much as I
can into my lessons.
They felt the need to spend more time on a topic and add more exploration so that students could
further deepen their understanding. Another participant wrote,
Participating in this project has allowed me to open my eyes about combining classes that
don’t typically learn together. For example, I have combined my MD [moderately disabled]
class with a regular education class for labs at various times during the year so far.
Teachers also found the collaboration with colleagues in workshops and in online discussions
valuable because it gave them time to share ideas and get feedback and to explore instructional
strategies for teaching math and science. Several teachers commented on how much the technology
training impacted their teaching. One teacher shared,
Access, time, and instruction on how to use technology in my classroom. This has changed
my life and teaching. I feel more connected than ever to my students and the way they
think. I am easy for them to access and my class is almost 1/2 online now. I love keeping
connected with my students who have attendance issues as well.
Receiving a Chromebook as part of the project helped teachers to have immediate access to apps
and software as well as finding relevant resources for integrating mathematics and science. The
teachers also discussed how apps and software could promote student comprehension and student
expression of that learning, thereby, supporting UDL principles. Teachers mentioned having greater
comfort with using technology in the classroom and a newfound interest in looking for other ways
to use technology with their students.
Teachers were also asked how their teaching practices had changed or improved as a result of
the project. One teacher commented,
I am emphasizing more of the math that is present in science and see the connection much
more clearly. My students are making that connection as well and have started talking to me
about what they are learning in math class. I have always been an “out-of-the-box” thinker,
but having all of this knowledge has given me more tools to explore that thinking.
Other teachers shared that they were incorporating more hands-on experiences and labs that include
engineering and found that students responded really well to those activities. Another teacher said,
I have my students do activities using Chromebooks much more frequently now. I used
to only use them for testing. Now we enter data and make graphs, create slide show
presentations, go on math websites like prodigy to play games, etc.
Other teachers commented that they were including more UDL strategies to promote student access
20
to learning. Workshop faculty members observed such strategies being included, for example, using
graphic organizers to promote comprehension (UDL Principle 1), gradually releasing scaffolds to
increase independence (UDL Principle 2), and providing contextually relevant problems (UDL
Principle 3). Other teachers noted that they were incorporating more intentional connections
between math, science, and technology.
When asked about challenges, teachers mentioned the lack of resources to do engineering
design projects, available technology, and time needed to implement more student-centered, hands-
on approaches because these require more time “to prepare, set up, and put away equipment.”
Standardized testing was another issue that took away time from classroom instruction. When
asked what the major strengths of the project were, teachers responded: learning about technology,
connections of mathematics and science, Thinking Mathematics methods, LabQuest activities,
strengthened knowledge of standards, making the instruction and materials useful for the classroom,
and hands-on lessons. The following quotes offer a clearer view of teacher perceptions of strengths.
• “Actual lessons I can take back to my classroom.”
• “Google Chromebooks!!!!! Hands-on Labs Time to work!”
• “Getting time over the summer to actually create a unit plan. In the PDs, having access to
information presented so we can easily implement it in our classrooms.”
• “I liked the hands-on labs and engineering design challenges. I think I am much more likely
to do them in my classroom after doing them myself.”
• “I liked the incorporation of the Vernier technology into the labs. I thought the after-school
sessions were beneficial because they helped remind me throughout the year of what we
worked on over the summer.”
• “I really enjoyed the presentation of information and activities related to the STEM
instruction. I had no idea there were so many free resources online and ready to use. The
summer session was the most beneficial for me. I appreciated that we focused on different
topics each day and were shown firsthand how to integrate math, science, and engineering.”
Teachers were asked what improvements were needed in the program. Some teachers felt that
the physics instruction or engineering challenges were too advanced or did not relate well to their
math content, even though the content was chosen according to the grade group. Teachers would
have liked more resources to take back to the classroom to do hands-on labs and more training in
using the Chromebook. Finally, more time was needed to “practice some of the stuff.”
Summary and Conclusions

Improvements were implemented after Year 1 to make the program more successful in
Year 2. These included narrowing the ranges of grade levels served, adding mentors to assist with
instruction and scaffolding with online discussions, and focusing more intentionally on modeling
STEM integration during workshops. However, more improvements still need to be considered.
For example, although teachers who opt for Middle Childhood Licensure may have a stronger
background in mathematics and science as chosen subjects for specialization, it is likely that some
of them still did not have as much content knowledge or a deep understanding of these subjects.
Future programs might consider the range of content experience of participants either in the
proposal or planning stages, particularly when offering higher level content such as physics. On the
other hand, the authors believe that teachers should be prepared at a significantly deeper level of
understanding of the material covered in class to gain confidence and be able to convey and explain
21
the concepts correctly and to better understand misconceptions that their students may develop
when transferring their experiences to other settings. Program personnel might also want to focus
on fewer concepts during the PD and provide more depth of these topics with time to explore. The
program might also consider how to provide more resources for teachers to implement hands-on
activities in the classroom.
The greatest strengths of the program were the STEM connections that teachers learned about
and the changes in instructional practices that resulted. Nonetheless, future efforts might find it
rewarding to incorporate more follow-up or sustained efforts with teachers who have participated
in the program from year to year as a way to continue the momentum begun in STEM integration.
The introduction of various technologies and ways to use them was another strength of the
program. Learning to make STEM connections, having dedicated time to design integrated STEM
lessons, and providing technology were factors that most likely enabled teachers to be successful
in their STEM lessons. Teachers’ comments illustrated how the program experiences encouraged
them to begin thinking more about and incorporating other STEM connections in their classroom
instruction, which in turn enabled students to increase their understanding and engagement in
STEM.
This program evaluation clearly shows that all the program goals were met and that anticipated
outcomes were achieved. Teachers significantly increased their knowledge of math and science
and relevant state learning standards; improved in attitudes, beliefs, and degree of confidence in
teaching; developed more comfort with using technology; and learned to appreciate and understand
the practice of STEM integration. Student scores in STEM also increased after participating in
an integrated STEM lesson. However, this finding is tentative because it is difficult to ascertain
whether the students gained more than they would have in any other classroom situations. Teachers
did note students’ enthusiasm during these lessons, how they were learning to make their own
connections between subjects, and how responsive they were to hands-on and engineering activities.
Quantitative data and teacher feedback both indicate that the program was highly successful and
had a positive impact on teachers and students.
Although the program design might be replicated, one of the limitations of this research study
is that it examined a program that served particular grade levels. Programs may need to be designed
differently for lower elementary or even high school levels above ninth grade. Additionally,
participating schools only included teachers and students from an urban school district, so any
results should not be generalized to other populations.
References
American Federation of Teachers. (2003). AFT’s Thinking Mathematics program [Brochure].
Retrieved from http://files.eric.ed.gov/fulltext/ED497911.pdf
CAST. (2011). Universal Design for Learning Guidelines (Version 2.0). Wakefield, MA: Author.
Honey, M., Pearson, G., & Schweingruber, H. (Eds.). (2014). STEM integration in K–12 education:
Status, prospects, and an agenda for research. Washington, DC: National Academies Press.
doi:10.17226/18612
International Technology Education Association. (2003). Advancing excellence in technological
literacy: Student assessment, professional development, and program standards. Reston, VA:
Author.
22
International Technology Education Association. (2007). Standards for technological literacy

(3rd ed.). Reston, VA: Author.
International Technology and Engineering Educators Association. (2017). Integrative STEM
education. Retrieved from https://www.iteea.org/Resources1507/IntegrativeSTEMEducation.
aspx
National Governors Association Center for Best Practices & Council of Chief State School Officers.
(2010). Common core state standards for mathematics. Washington, DC: Author. Retrieved
from http://www.corestandards.org/wp-content/uploads/Math_Standards1.pdf
National Research Council. (1996). National science education standards. Washington, DC:
National Academy Press. doi:10.17226/4962
National Research Council. (2012). A framework for K-12 science education. Washington, DC:
National Academies Press. doi:10.17226/13165
NGSS Lead States. (2013). Next generation science standards. Washington, DC: National
Academies Press. doi:10.17226/18290
Ohio Department of Higher Education. (2016a). FY 2016 follow-up participant survey: For
assessing the effectiveness of ITQ professional development activities. Retrieved from
https://www.ohiohighered.org/sites/ohiohighered.org/files/uploads/itqp/FY_2016_PD_
Follow-Up_Participant_Survey.docx
Ohio Department of Higher Education. (2016b). FY 2016 preliminary participant survey. Retrieved
from https://www.ohiohighered.org/sites/ohiohighered.org/files/uploads/itqp/FY_2016_PD_
Preliminary_Participant_Survey_0.docx
Ohio Department of Education. (2016a). Typology of Ohio school districts [Data File]. Retrieved
from http://education.ohio.gov/Topics/Data/Frequently-Requested-Data/Typology-of-Ohio-
School-Districts
Ohio Department of Education. (2016b). Proficiency levels with student & test disagg (district)
[Data File]. Retrieved from http://bireports.education.ohio.gov/
Vannatta, R., & Banister, S. (2009). Validating a measure of teacher technology integration. In I.
Gibson, R. Weber, K. McFerrin, R. Carlsen, & D. Willis (Eds.), Proceedings of SITE 2009—
Society for Information Technology & Teacher Education International Conference (pp. 1134–
1140). Chesapeake, VA: Association for the Advancement of Computing in Education.
Author
Judy Lambert
Professor and Assistant Chair
Department of Curriculum and Instruction
Email: [email protected]
Dawn Sandt
Assistant Professor
Department of Early Childhood, Physical, and Special Education
23
Carmen Cioc
Assistant Professor and MET Program Director
Engineering Technology Department
Sorin Cioc
Assistant Professor
Department of Mechanical, Industrial, and Manufacturing Engineering
24
Appendix
Subsections of Teaching Confidence, Pedagogical Beliefs, and Impact of PD Instrument
Section 1: Teaching Confidence (Pre- and Post-Test)

Some of the questions in this section included: I have a good understanding of fundamental core content
in my discipline, I have a good understanding of Ohio’s New Learning Standards in Mathematics and/or
in Science, I have a good understanding of how to differentiate instruction in the classroom, I have a good
understanding of the methods necessary to teach math and/or science concepts effectively, and I believe I
am an effective teacher. A Likert-type scale from 1 (strongly agree) and 5 (strongly disagree) was used on
all sections except Section 2.
Section 2: Teaching Approach (Pre- and Post-Test)

Teachers described along a continuum (1 = traditional, 5 = student-centered or constructivist) their
teaching approach on the following eight items: teacher-led lecture versus dialogue among teacher and
students, cooperative group learning versus independent work, focused on central ideas versus broad
coverage of topics, students’ role is to receive facts vs apply inquiry, hands on approaches versus reading
and lectures, encouraging vs difficulty in encouraging, conventional assessments vs alternative methods,
and use same instructional techniques vs differentiation.
Section 3: Impact of PD on Teachers’ Content Knowledge (Posttest Only)

Teachers rated 10 items such as: I learned new content in Math and/or Science (concepts, facts and
definitions); I learned about learning standards in Mathematics and/or in Science; I learned inquiry-based,
hands-on activities; and I learned methods to differentiate instruction in my classroom.
Section 4: Quality of PD (Posttest Only)

Teachers rated the following five items regarding the quality of the PD: It provided ample time to
achieve the stated objectives, provided useful resources and/or materials to assist with my instruction in the
classroom, was high quality, was sustained and intensive, and was linked to state and national standards.
Section 5: Impact of PD on Students (Posttest Only)

Teachers rated the impact of the PD on their students on the following three items: My students are
more attentive, enthusiastic and involved in classroom activities; the quality of student work is noticeably
improved; and my students are participating in science and/or math activities outside of the classroom to a
greater degree.
Section 6: Impact of PD on Teachers (Posttest Only)

Teachers rated the impact of the PD on themselves on the following seven items: I have maintained
contact (or plan to maintain contact) with other participants, I have maintained contact (or plan to maintain
contact) with college/university faculty who provided the professional development, The program led
to the establishment of a professional network among participants, I attended a professional association
conference, I have or would recommend this program to other teachers, I have shared what I learned with
colleagues through informal interactions, and I have shared what I learned with colleagues through formal
interactions.
25

Making Connections - Evaluation of STEM Professional Development

Uploaded by

Copyright:

Available Formats

Making Connections - Evaluation of STEM Professional Development

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Making Connections - Evaluation of STEM Professional Development

Uploaded by

Copyright:

Available Formats

Journal of STEM Teacher Education

Volume 53 | Issue 1 Article 2

Making Connections: Evaluation of a Professional

Follow this and additional works at: https://ir.library.illinoisstate.edu/jste

This article is available in Journal of STEM Teacher Education: https://ir.library.illinoisstate.edu/jste/vol53/iss1/2

Making Connections: Evaluation of a Professional Development Program for

Judy Lambert, Dawn Sandt, Carmen Cioc, and Sorin Cioc

A Framework for Integrated STEM Education

Goals of Integrated STEM Education

Outcomes of Integrated STEM Education

Nature and Scope of Integrated STEM Education

Implementation of Integrated STEM Education

Results and Discussion

Teaching Confidence, Pedagogical Beliefs, and Impact of PD

In Year 1, teachers tended to move toward student-centered, constructivist teaching approaches

Student Content Knowledge

Summer Institute Daily Evaluations

Summary and Conclusions

International Technology Education Association. (2007). Standards for technological literacy

Subsections of Teaching Confidence, Pedagogical Beliefs, and Impact of PD Instrument

Section 1: Teaching Confidence (Pre- and Post-Test)

Section 2: Teaching Approach (Pre- and Post-Test)

Section 3: Impact of PD on Teachers’ Content Knowledge (Posttest Only)

Section 4: Quality of PD (Posttest Only)

Section 5: Impact of PD on Students (Posttest Only)

Section 6: Impact of PD on Teachers (Posttest Only)

You might also like