International Journal of Instruction April 2020 ● Vol.13, No.

2
e-ISSN: 1308-1470 ● www.e-iji.net p-ISSN: 1694-609X
pp. 393-406
Received: 28/02/2019
Revision: 13/11/2019
Accepted: 19/11/2019
OnlineFirst:22/01/2020

The Effectiveness of Computer-Based Problem Solving to Improve Higher

Order Thinking Skills on Prospective Teachers

Yurniwati
Jakarta State University, Indonesia, [email protected]
Dudung Amir Soleh
Jakarta State University, Indonesia, [email protected]

Computer-Based Problem Solving is a combination of problem-based learning and

computer-assisted learning. The method facilitates students to use computer
programs as cognitive tools to solve problems. The purpose of this study is to
determine the effect of Computer Based Problem Solving on High Level Thinking
Skills (HOTS) of prospective teachers. The study was conducted in the 5th
semester of the 2018/2019 academic year. This is a quasi-experiment with pre-test
and post-test control group design consist. In the process of learning, the
experimental group used computer-based problem solving as treatment whereas the
control gorup used traditional methods. HOTS data was collected by conducting
essay tests and the data was analyzed by using t-test and ANOVA. The findings of
this study revealed that using computer-based problem solving approach improved
HOTS of the prospective teachers as well as enhance their collaboration and
motivation. It was found out that there was a moderate improvement on HOTS
aspects in communication. The study also showed very slight improvement on the
prospective teacher’s problem solving abilities.
Keywords: computer-based problem solving, HOTS, thinking skills, prospective
teachers, computer-assisted learning
INTRODUCTION
In this global era, developing students’ 21st century skills has been the great concern of
many educators. The 21st century education includes problem solving, critical thinking
and higher-order thinking (Higgins, 2014; Kivunja, 2014; Soule & Warrick, 2015),
specifically in academia and the workplace (Lee & Choi, 2017). The effect of HOTS on
learning outcomes is the ability to obtain information through active learning such as
exploration, discovery and experimentation. Students who have HOTS are able to access
information to use new contexts and easily transfer knowledge into different situations
(Mainali, 2012). While the influence of HOTS on work is mainly to obtain information,

Citation: Yurniwati, & Soleh, D. A. (2020). The Effectiveness of Computer-Based Problem Solving to
Improve Higher Order Thinking Skills on Prospective Teachers. International Journal of Instruction,
13(2), 393-406. https://doi.org/10.29333/iji.2020.13227a
394 The Effectiveness of Computer-Based Problem Solving to …

however, the most important point is how to manage, analyze, criticize and change
acquired information into useful knowledge (van Laar at al., 2017). For example as a
teacher, it is intended to design appropriate, effective and efficient learning including
the learning methods, learning tools, and assessment. Teachers are required to be able to
analyze appropriate teaching methodologies to specifie content areas, identify learning
tools to support learning, develop student reasoning and problem solving abilities.
According to Yen & Halili (2005) HOTS is an indispensable skill for every individual in
any educational environment.
At present, many definitions have been proposed by experts on high-order thinking.
Since 1987, Resnick has argued that high-level thinking is characterized by the
following characteristics: not algorithmic, complex, multi-solution, involving decision
making and interpretation, application of various criteria, often uncertainty, self-
regulation, looking for flawed structures. HOTS is the ability to apply facts to new
situations (Moursund, 2003). Similar opinion was also conveyed by Vui (2007) that
high-level thinking skills will occur if someone associates new information with
information that has been stored in his memory and connects it or rearranges and
develops that information to achieve the goal. Zoller & Pushkin (2007) used High Level
Cognitive Skills (HLCS) instead of HOTS. They argue that the HLCS has enhanced
students’ problem solving, decision making and critical thinking skills development.
High-level thinking is compiling arguments, asking questions, making comparisons,
resolving non-routine problems (Zohar & Dori, 2009). Yang (2015) identifies creative
and critical thinking and problem solving as high-level thinking. Yen and Halili (2015)
also characterize high-level thinking: (a) not routine / not fully known beforehand, (b)
complex, (c) produce various solutions/points of view, (d) involving uncertainty, (e)
Involving the process of making meaning, (f) Need effort, requires mental work. All
HOTS definitions emphasize on non-routine problems, knowledge construction, requires
considerable effort.
In mathematics, Webb & Coxford as quoted in Sumarmo & Nishitani (2010) states that
in mathematics, the ability to understand ideas is implied; compile guesses, analogies
and generalizations; logical reasoning, solving problems; communicate mathematically;
associating mathematical ideas with other intellectual activities belong to the HOTS
aspect. In other words, HOTS in mathematics is reasoning, Mathematical connection,
mathematical communication and problem solving. NCTM (2000) identified problem
solving is students create new mathematical knowledge that arise in mathematics and
other contexts. Reasoning include apply various types of reasoning, construct
conjectures and mathematical arguments. Mathematical communication as tool to
express mathematical ideas, and analyze mathematical thinking of others. Mathematical
connection provides understanding the way mathematical ideas interrelate and build on
one another. If students are faced with problem solving, students will think of the
information in the problem and connect it with the concept which is the basis to answer
the problem. From these conditions, strategies are developed to solve problems and
communicate in the form of equations, drawings, charts, etc.

International Journal of Instruction, April 2020 ● Vol.13, No.2

Yurniwati & Soleh 395

On the other hand, according to Newman and Wehlage (2003) lower Order Thinking
Skills (LOTS) restate facts or apply rules and algorithms through routine problems. As
recipients of information, students are given knowledge from simple concepts to
complex. Students in this case repeat the knowledge that has been received to answer
memory questions. LOTS occurs because in the learning process, students do not get the
opportunity to develop thinking skills and actively acquire knowledge. In addition,
students are also not challenged with open questions and non-routine problems.
There are two categories of HOTS, namely high-order thinking in scientific disciplines
and general high-order thinking skills (Resnick, 1987). General high-order thinking
skills are special classes that emphasize that high-order thinking skills can be
transferred among subjects. High-order thinking in subjects leads to the integration of
high-order thinking skills and techniques in subjects.
Fisher (in Puchta, 2012) argues "thinking does not occur naturally and needs to be
developed". This means that it is necessary for student to undergo continuous practice to
be able to think at a high level. Rajendaran (2000) argues similarly that thinking is not
automatic, but can be learned. The Department for Education and Skill (DfES, 2005)
suggests that learning activities that can enhance high-level thinking skills are: (1)
students work collaboratively in groups, (2) resolve problems, (3) carry out activities
that can accommodate different learning styles, (4) develop different thinking skills
(such as analysis, synthesis, creative, inductive, deductive, divergent and convergent).
One learning approach that meets the learning criteria described above is problem-based
learning. Bay (in Shinn et al., 2003) explains problem-based learning in mathematics is
to teach knowledge and skills in the form of problem solving. Problems are presented as
contexts and stimuli for learning. According to Liu (2005) problem-based learning has
essential characteristics such as the following: (1) Student-cantered learning; (2) Forms
of authentic problems that direct the focus of learning; (3) New information is obtained
through self-directed learning; (4) Learning takes place in small groups, and (5) The
teacher acts as a facilitator.
Along with the development of technology rapidly, the way of teaching and learning are
changing accordingly. Technology, which has directly changed the way of interacting,
communicating, studying and investigating (Pescador, 2014). Also, the students mode of
learning change from passive learning becoming active learning, working in groups
interacting through computer technology to think critically to find solution of problems
(Henderson, 2017). Similar to Jonassen and Car (2000) who discussed learning by using
technology from a constructivist perspective. They advocate the use of computers as a
cognitive tool for constructing and facilitating the learning of high-level thinking skills.
Besides, the role of computers in learning include support "internal negotiation" and
understand meaning, concept representation construction and support logical thinking
(Bukar, Bello, & Ibi, 2016).
Consider the role of computer as logical thinking development, Ekstig (2004) suggested
to combine computer technology and problem solving namely Computer-Based Problem
Solving (CBPS). CBPS places students as learning centres using software tools to
construct, investigate, represent, implement during the problem solving process. Santos

International Journal of Instruction, April 2020 ● Vol.13, No.2

396 The Effectiveness of Computer-Based Problem Solving to …

at al. (2003) suggested students need to use different representational media to express
their ways of thinking while dealing with tasks or problems. The use of different tools
offers students the possibility of examining situations from perspectives that involve the
use of various concepts and resources. Thus, during this process, specially dynamic
constructions help students focus their attention on common properties that appear while
moving elements within the same configuration or representation. It provide the
opportunity to implement techniques and tools that are not obtainable in a paper-pencil
environment in the investigation of mathematical problems (Arcavi & Hadas, 2000;
Çekmez & Bülbül, 2018).
In relation with the learning process, Passey at al. (2000) divides it into three parts,
namely: internalization, internal cognition and externalization. Internalization is
explained through a process related to involvement and attention. Internal cognition is a
process relating to knowledge, understanding, analysis, synthesis, and evaluation.
Externalization involves expressing ideas through writing and oral. Integration of
technology into mathematics learning specifically reaches the three parts to the learning
process. Computers are excellent tools in presenting images or diagrams of the right size
that attracts students' attention. Visualization provided by computers helps students to
explore to build knowledge, find relationships, test, interpret information, design,
analyze, synthesize, elaborate, solve problems and take decisions. Finally, the computer
helps present what students know to others.
The process of thinking that occurs in internal cognition supported by computer
facilities includes high-level thinking. Therefore, technology in learning can be used as a
means to engage students actively in constructing knowledge, problem-solving and
critical thinking to improve higher-order thinking skills (Jonnasen & Car, 2000).
Furthermore, the role of technology in HOTS in mathematics are: (1) Improving
students multi-representation capabilities, (2) facilitating students to understand and
develop concepts, (3) improving mathematical reasoning capabilities, (4) engaging
students to solve problem; (5) Mathematical communication tools (Kimmins, 2005)
Not all computer technology can be used to enhance high-level student thinking. Glazer
(2001) distinguishes computer use in the classroom, namely, computers as a tool for
exploration and as a tutor. Computers as a tool of exploration or problem solving will
build high-level thinking skills. Instead, using a computer as a tutor, giving instructions
and feedback only emphasizes low-level thinking skills.
Some empirical studies related to computer-based learning, beginning with Roh & Park
(2010) conducted a meta-analysis of 27 studies on the effect of Computer-Based
Education on nursing students from 1990-2009. They found out that Computer-Based
Education has a positive influence on knowledge, attitudes and skills. Larwin & Larwin
(2011) conducted a meta-analysis of the effects of Computer-Assisted Instruction on
student statistics. There were studies conducted between 1970-2010. The conclusion of
his study was that CAI had a high influence, namely d = 0,566 on student learning
outcomes in the statistics department. Furthermore, Sung, Yang and Lee (2017)
conducted a meta-analysis of the effect of mobile-Computer-Supported Collaborative
Learning on group learning effectiveness in 48 journals between 2000-2015. Their

International Journal of Instruction, April 2020 ● Vol.13, No.2

Yurniwati & Soleh 397

findings are that mCSCL significantly increases student collaboration with an effect size
of 0.516. Sudibyo, Jatmiko, & Widodo (2016) examined the effectiveness of the
Computer-Based Learning (CBL) model to improve students' thinking skills in sports
science. The findings of their study were that CBL effectively improved the analytical
thinking skills of sports science students by N-gain = 0.78 and included in the high
category.
To put a higher level of previous research results, this study combines Computer-Based
Learning with Problem Solving called Computer-Based Problem Solving (CBPS). In
applying the CBPS model, learning uses the problem context, and GeoGebra acts as a
cognitive tool for solving mathematical problems. GeoGebra is an interactive geometry
software founded by Hohenwarter in 2001(Hohenwarter & Fuchs, 2004). The basic
objects in GeoGebra are points, vectors, segments, polygons, straight lines, all conic
sections and functions in x. GeoGebra as dynamic constructions can be altered
dynamically by dragging free objects. Furthermore, it is possible to enter coordinates of
points or vectors, equations of lines, conic sections or functions and numbers or angles
directly. Hohenwarter & Fuchs (2004) argued GeoGebra is powerful tool for
mathematics education cause: 1) GeoGebra support demonstration and visualisation, 2)
GeoGebra has all the abilities which are demanded construction graphics, 2 dimensional
and 3 dimensional shapes, etc., 3) GeoGebra contribute in discovering mathematics
knowledge and knowledge processing.
Research Question
The purpose of this study was to determine the effect of CBPS on HOTS of prospective
teachers. Specifically, this study aims to find out:
1. Are there a significant differences in the HOTS pre-test of prospective teachers
in the two research groups?
2. Is there a significant effect on the HOTS of prospective teachers in the two
research groups?
3. Is there a significant effect of HOTS aspect of prospective teachers in the two
research groups?
METHOD
The target population of the study are prospective teachers of primary school in the
department of teacher education, faculties of Education at Jakarta State University. The
participants are 32 of 160 prospective teachers enrolled in the 5 th semester in 2018-2019
Academic Year. In Indonesia, Elementary Teacher Education consist of 6 semesters
learning in campus, 2 semesters for field practice and research. They study content
knowledge, pedagogical knowledge and 5 subjects matter (Mathematics, Science, Social
Studies, Language, Moral Studies). One subject of content knowledge is Basic
Mathematics. In Basic Mathematics they learn basic concept of mathematics and
problem solving like logic, relation and function, linear equation and quadratic equation.
The research method is a quasi-experimental. The research design was the pre-test-post-
test design. Two classes were chosen as research samples, namely the experimental and
control groups. The research was conducted for 5 weeks. In this study involved linear

International Journal of Instruction, April 2020 ● Vol.13, No.2

398 The Effectiveness of Computer-Based Problem Solving to …

equation and function. The experimental group and the control class studied the same
material and were given the same test. The experimental group’s treatment was CBPS
while the control group was using expository learning.
Subject Delivery
Learning begins by giving questions to participants then, they work in groups and do
intense discussion to be able to answer the questions. As an illustration, the following
are examples of learning activities for linier equation.
1. Make a line through point A (0,3) and B (2,0), observe the equation of the line
(in the format )
2. Make a line through C (0,5) and D (4,0). Note the equation of the line that
appears.
3. Observe the two line equations and relate them to the intersection of the line with
the x-axis and -axis.
4. What is the equation if the line is through points E (0, a) and F (b, 0) 5. Your
conclusion is:
Participants work on steps 1 and 2 on GeoGebra . On the GeoGebra screen it will look
like Figure 1.

Figure 1
Graph of 3x + 2y = 6 and 5x + 4y = 20
In step 3, participants try to create a function equation based on the results of step 1 and
step 2. Participants observe the relationship between the coordinates of the point and the
equation of the line on the algebraic screen. In step 1, the line goes through A (0,3) and
B (20) and equals 3x + 2y = 6 . This means that the ordinate point A is the x coefficient
and the abscissa point B is the coefficient y. In step 2, the line through C (0,5) and D
(4,0). and the equivalent is 5x + 4y = 20 . Also obtained by the ordinate point C is the x
coefficient and the absent point D is the coefficient y. By paying attention to the
ordinate connection and the point abscissa with the line equation, participants can
determine the function of the line through E (0, a) and F (b, 0) is ax + by = ab.
When participants determine the function equation, participants involve deductive
reasoning to formulate function equations. Communicating the mathematical expression

International Journal of Instruction, April 2020 ● Vol.13, No.2

Yurniwati & Soleh 399

in of equations, graphs or tables is called mathematical communication. In step 5,

participants draw conclusions from the findings obtained based on the overall activity.
While on the control group, participants learn using paper and pencil. To learn the

function equation, the teacher shows a graph that intersects the x-axis and -axis. With
the question and answer strategies, participants are directed to observe the relationship
of the intersection of line and line equations. As an illustration, the teacher displays a
graph in Figure 2.

Figure 2
Graph of 2x + 5y = 10 and 4x + 3y = 12
Teacher ask questions to participants as follows:
a) Pay attention to the line through point A and B
b) note the function equation
c) Can you see the relationship between the function equation and the graph
intersection with the axis line?
d) Now look at the lines that pass point C and D
e) Observe the function equation. What is the relationship between the function
equation and the intersection point of the line?
f) What is your conclusion?
After the question and answer activities, participants did worksheets and closed by
discussing the answers.
The instrument refers to HOTS definition according to Webb & Coxford, as presented
in Table 1.

International Journal of Instruction, April 2020 ● Vol.13, No.2

400 The Effectiveness of Computer-Based Problem Solving to …

Table 1
Instruments
Domain HOTS
Reasoning Mathematical Mathematical Problem solving
Connection Communication
Linier Equation 1 1 1 1
Function 1 1 1 1
Total 2 2 2 2
The pre-test and post-test scores were processed according to the research questions.
The statistical test used is t-test, ANOVA and Normalized Gain.
FINDINGS
Table 2
The Pre-Test and Post-Test Scores of Research Groups
Test Groups N Mean StDev Minimum Median Maximum
Pre-test Exp. 18 20.222 10.050 7.000 16.500 46.000
Control 18 17.778 5.275 4.000 18.000 28.000
Pos-test Exp. 18 39.444 10.799 25.000 38.500 65.000
Control 18 27.333 7.631 8.000 29.000 40.000
Based on Table 2 it is known that there is little difference in the pre-test data between
the experimental and control groups. Whereas in the post-test data there was a
considerable difference between the experimental and control groups. Descriptively it
can be stated that there are differences in HOTS between the experimental class and the
control class. In this case the experimental group had HOTS higher than the control
group
Research Question 1:
Table 3
Analysis of the Normality on the Pre-Test Scores on the Experimental and Control
Groups
Group Anderson Darling -Value df P-value
Exp. 0.55 17 0.1380
Control 0.45 17 0.245
Table 3 shows the P-value of Experiment group = 0.1380> 0.05 and the P-value of
control group = 0.245> 0.05. This means the pre-test data of both groups is normal
distribution
Next, an independent t test was conducted to determine the differences between the two
research groups. The results can be seen in Table 3.
Table 4
Independent Samples T-Test from the Experimental and Control Groups
Group T Student -Value df P-value
Pre-test 0.914 34 0,367

International Journal of Instruction, April 2020 ● Vol.13, No.2

Yurniwati & Soleh 401

In table 4, the results of the Independent Samples T-Test is obtained for the pre-test with
P-value= 0.367, meaning that there is no statistically significant difference in the HOTS
initial ability in the experiment and control group. As a result, both groups are feasible
to use in the study to compare the two methods.
Research Question 2:
This section aims to determine whether there are significant HOTS differences between
the experimental and control groups after receiving treatment. Statistical tests were
carried out with ANOVA test. The normality test and homogeneity test were ppreviously
carried out as analysis requirements test.
Table 5
Analysis of the Normality on the Pre-Test Control Groups
Group Anderson Darling -Value df P-value
Exp. 0.51 17 0.1749
Control 0.35 17 0.4334
Table 5 shows the P-value of Experiment group = 0.1749> 0.05 and the P-value of
control group = 0.4334> 0.05 This means the post-test scores of both groups is in
Normal distribution. Next, Homogeneity is test carried out using Multiple Comparisons
and Levene’s.
Table 6
Homogeneity Test on Post-Test
Method Test Statistic df P-Value
Multiple comparisons 1.21 17 0.2715
Levene 1.13 17 0.2958
Table 6 shows the P-value of Multiple Comparison = 0.2715> 0.05 and the P-value of
Levene = 0.2958> 0.05 This means the post-test scores of both groups is homogenous.
Because the post-test data is normal and homogeneous, then ANOVA test.
Table 7
Analysis of Variance
Source DF Adj SS Adj MS F-Value P-Value
Factor 1 1320.11 1320.11 15.10 0.0004
Error 34 2972.44 87.42
Total 35 4292.56
Table 7 indicates that p value = 0.0004 <0.05 is statistically significant the difference of
HOTS between Experiment and control group. The post-test mean scores of the
experimental group was higher than the post-test mean of the control group. Therefore,
it can be concluded that the application of CBPS can increase HOTS of the prospective
teachers.
Research Question 3
This section discusses how improvements have occurred in the HOTS aspect after being
treated. As discussed earlier that the aspects of HOTS include Problem Solving,

International Journal of Instruction, April 2020 ● Vol.13, No.2

402 The Effectiveness of Computer-Based Problem Solving to …

Reasoning, connection, Communication. The improvement of HOTS aspect is

determined by normalized gain as shown in Table 7.
Table 7
Average Normalize Gain of HOTS Aspects
Aspect N Gain Level
Communication 0.53 Moderate
Reasoning 0.41 Moderate
connection 0.37 Moderate
Problem Solving 0.24 Low
HOTS 0.42 Moderate
Based on the table it is known that there is a moderate level increase in all aspects of
HOT except problem solving.
DISCUSSION
The results of the hypothesis test indicate that experimental group obtained a higher
HOTS score than the control group because prospective teachers treated by CBPS have
better HOTS abilities than conventional learning. Based on problem given, prospective
teacher use GeoGebra to explore, investigate causal effect of changing graphic to
function equation or vice versa. It was observed that the use of technology helped
teachers to develop ways of reasoning and forms of reflecting about the meaning and
connections among mathematical objects. For example, the dynamic software enabled
teacher to search for various forms of justifying a conjecture, in which the use and
integration of visual, empirical and deductive arguments were useful. They grasp
similarity and difference until the determinate formula or conclusion. This finding in
line with Kuzle (2017) found that Dynamic Software Geometry such as GeoGebra as a
cognitive tool that can enhance and reorganize the problem solving process. It allowed
and supported prospective teachers to discover new methods of thinking, and inhibited
the problem solving processes.
GeoGebra is highly beneficial because it has advantages such as visualization,
especially the construction of accurate images and Loci. This founding in line to Doruk
at al. (2013) that computer-assisted learning is very helpful in learning process.
Similarlly Moreover, technology as a cognitive tool was used to find information, make
models and present solutions to improve collaboration, share ideas, motivation and
social behaviour (Glazer, 2001). Moreover, Geogebra able perform various
mathematical communication in graphs, equation, cartesian coordinate system. It
improves prospective teachers understanding on concepts and interconnected between
concepts and finally develop conceptual knowledge.
While in conventional learning, prospective teachers gain knowledge about facts,
concepts and procedures such as rules and formulas from teachers and source books.
Then the knowledge is used to answer repetitive questions and application procedures to
routine problems. Doing routine problems continuously makes math work mechanical.
As a result, the prospective teacher thinking limited on memorizing, understanding and
application (Lower Order Thinking Skills).

International Journal of Instruction, April 2020 ● Vol.13, No.2

Yurniwati & Soleh 403

Unlike in conventional learning, the classroom atmosphere in CBPS is dynamic.

Prospective teachers are conditioned and engaged in group discussion in an effort to
solve problems. The student's deep engagement in activities, occurs because CBPS
learning emphasizes not only knowledge but also the skills needed in learning such as
problem solving, acquiring knowledge and working together in groups. When
prospective teachers work in groups, there seems to be a very good interaction between
prospective teachers. They collaborate help each other, appreciate the efforts or
opinions of friends and give awards in the form of praise for the success of friends in
finding solutions to problems. Jackson (2001) concluded that group learning improved
self-esteem, social behaviour and increased HOTS. In line with this opinion, Charles
(2001) found that collaborative learning and active learning play a role in improving
problem-solving skills and high-level thinking. According to the results of a meta-
analysis of small-group learning conducted by Springer, Stanne, & Donovan (2007),
learning in small groups affects the learning outcomes of group members. Furthermore,
studying in groups is also more effective than lecture-based instruction.
CBPS is extremely different from conventional learning where in each student works
individually, giving rise to a sense of competition between them. Instead of CBPS, a
sense of responsibility arises for the group as a result. Everyone wants to help other
group members, and each member contributes to the achievement of group goals.
Interpersonal skills are also developed because working in groups involved aspects of
communication, leadership, decision making and reflection of the progress that the
group has achieved (Tarricone & Luca, 2002).
Consider to HOTS aspects, prospective teachers have moderate improvement in each
aspect except problem solving. The power of problem solving is that gaining a
successful solution requires students to refine, relate, and transform knowledge they
have already learned (Lester & Cai, 2016). In this condition, prospective teachers
demand to keep practicing on problem solving skills continuously in order to get more
experience and challenge to design and modified problem solving strategies.
The results of this study proofs CPBS develop prospective teachers HOTS trough
problem solving process. Focusing on problem solving in the classroom not only
impacts the development of students’ higher-order thinking skills (Lester & Cai, 2016).
Regards of computer technology contribution in visualization, accurate image and Loci,
accelerating reasoning, mathematical connection, mathematical communication. Also
developing working in groups, provide opportunities to collaborate in conducive
environment with joyful and reduces learning anxiety.
CONCLUSION
Based on the research findings it can be concluded that there is a significant positive
effect of CBPS learning methods on the HOTS abilities of prospective teachers. CBPS
facilitates thinking skills and the ability in using technology to acquire knowledge and
solve problems. Positive influence occurs in all aspects of HOTS except problem
solving, which has a very low increase. The implication of the findings of this study is
that teachers should prepare a learning plan as well as consider the ability level of
prospective teacher students and the level of difficulty of the material. The success of

International Journal of Instruction, April 2020 ● Vol.13, No.2

404 The Effectiveness of Computer-Based Problem Solving to …

CBPS is greatly influenced by problems as a trigger for learning activities. Lecturers

need to be careful in determining problems and should pay attention to the difficulty
level of the problem. Easy problems will make prospective teachers find the solution
directly without exploration first. On the other hand, a difficult problem makes the
prospective teachers get cognitive conflict and frustration. The best strategy is to give
questions from easy categories to complex ones. Significantly, lecturers should also pay
attention to each student's progress.
Future studies are recommended to examine why problem solving abilities of the
prospective teachers does not improve like other HOT aspects. This study has
limitations in sample size and future studies recommended conduct in large sample size.
The focus of this study is only on mathematics subject and expected for next research is
to be able to carried out focusing on another subject as well.
REFERENCES
Arcavi, A., & Hadas, N. (2000). Computer mediated learning: an example of an
approach. International Journal of Computers for Mathematical Learning, 5(1), 25–45.
https://doi.org/10.1023/A:1009841817245.
Bukar, I. B., Bello, S., & Ibi, M. B. (2016). Role of computer in instruction, assessment
and administrative delivery of education goals in the university of Maiduguri,
Nigeria. Journal of Education and Practice, 7(20), 81-87.
Cañas, A. J., Reiska, P., & Möllits, A. (2017). Developing higher-order thinking skills
with concept mapping: A case of pedagogic frailty. Knowledge Management & E-
Learning, 9(3),348–365
Çekmez, E., & Bülbül, B. Ö. (2018). An example of the use of dynamic mathematics
software to create problem-solving environments that serve multiple purposes.
Interactive Learning Environments, 26(5), 654–663.
https://doi.org/10.1080/10494820.2017.1385029.
Department for Education and Skills (Dfes). (2005). Excellence and enjoyment: Social
and emotional aspects of learning: Guidance. London: HMSO
http://sealcommunity.org/files/resources/Primary%20SEAL%20Guidance%20English.pdf
Ekstig, K. (2004). Improved understanding in mathematics through computer based
problem solving. [Online]. Retrieved from www.vxu.se/msi/picme10/L5EK.pdf.
Glazer, E. (2001). InterMath1-professional and cognitive development through problem
solving with technology. Retrieved from
http://www.arches.uga.edu/~eglazer/nime2001.pdf
Higgins, S. (2014). Critical thinking for 21st century education: A cyber-tooth
curriculum? Prospects, 44(4), 559-574.
Henderson, P. (2017). Computer science, problem solving, and discrete mathematics,
(January), 333–342. https://doi.org/10.1090/dimacs/036/28
Hohenwarter, M., & Fuchs, K. (2004). Combination of dynamic geometry, algebra and
calculus in the software system GeoGebra. Combination of dynamic geometry, algebra
and calculus. Retrieved from http://www.geogebra.org/publications/pecs_2004.pdf.

International Journal of Instruction, April 2020 ● Vol.13, No.2

Yurniwati & Soleh 405

Jonassen, D. H., & Carr, C. S. (2000). Mindtools: Affording multiple knowledge

representations for learning. Mahwah, NJ: Lawrence Erlbaum
Kimmins, D. (1996). Technology in school mathematics: A course for prospective
secondary school mathematics teachers. [Online]. Retrieved from
http://archives.math.utk.edu/ICTCM/EP-8/C89/MSWord/paper.doc.
Kivunja, C. (2014). Do you want your students to be job-ready with 21st century skills?
Change pedagogies: A pedagogical paradigm shift from Vygotskyian social
constructivism to critical thinking, problem solving and Siemens’ digital
connectivism. International Journal of Higher Education, 3(3), 81-91.
Kuzle, A. (2017). Delving into the nature of problem solving processes in a dynamic
geometry environment: Different technological effects on cognitive processing.
Technology, Knowledge and Learning, 22(1), 37–64. https://doi.org/10.1007/s10758-
016-9284-x.
Larwin, K., & Larwin, D. (2011). A meta-analysis examining the impact of computer-
assisted instruction on postsecondary statistics education: 40 years of research. Journal
of Research on Technology in Education, 43(3), 253-278.
Liu, M. (2005). Motivating students trough problem-based learning. Retrieved from
http://library.sullivan.edu/QEC/Problem_Based_Learning/Liu_
NECCOS_handoutMinLiu_RP.pdf.
Lester, F. K., & Cai, J. (2016). Can mathematical problem solving be taught?
Preliminary answers from 30 years of research. In P. Felmer, E. Pehkonen, & J.
Klipatrick (Eds.), Posing and solving mathematical problems (pp.117–135). Springer.
https://doi.org/10.1007/978-3-319-28023-3_8.
National Council of Teachers of Mathematics. (2000). Principles and standards for
school mathematics. Reston, VA.
Newman, F. M., & Gary, G. W. (2003). Five standards of authentic instruction.
Educational Leadership, 50(7), 8-12.
Passey, D. (2000). Higher order thinking skill. Retrieved from
http://www.notherngrid.org/ngflwebsite/hots/HOTSintro.pdf.
Pescador, B. (2014). ¿Hacia una sociedad del conocimiento? Revista Med., 22(2), 6-7.
doi: 10.18359/rmed.1194.
Puchta, H. (2012). Developing thinking skills in the young learners’ classroom.
Retrieved from
http://www.herbertpuchta.com/wpcontent/files_mf/1337014114YL_Thinking_booklet.pdf.
Rajendran, N. (2002, June). Using constructivist approach to teach higher-order thinking
skills: Transforming teaching practice to facilitate mindful learning. Paper presented at
the 10th International Conference on Thinking. Retrieved from
http://www.nsrajendran.com/documents/articles/harrogate2002.pdf.
Ranee, S. (2002). Computer based simulations in enhancing higher-order thinking in
mathematics. Retrieved from http://math.ecnu.edu.cn/earcome3/sym3
/Earcome3_Selva%20Ranee_sym3.doc.

International Journal of Instruction, April 2020 ● Vol.13, No.2

406 The Effectiveness of Computer-Based Problem Solving to …

Resnick, L.B. (1987) Education and learning to think. Washington: National Academy.
Roh, K. H., & Park, H. (2010). A meta-analysis on the effectiveness of computer-based
education in nursing. Healthcare Informatics Research, 16(3), 149-157.
Santos, M., Aguero, E., Borbon, A., & Paez, C. (2003). Students’ use of technology in
mathematical problem solving: Transforming technological artifacts into mathematical
tools. In N. A. Pateman, B. J. Dougherty, & J. T. Zillox (Eds.), Proceedings of the join
meeting of the international group for the psychology of mathematics education and the
north American chapter (Vol. 4, pp. 119–126). Hawaii: The University of Hawaii
Soulé, H., & Warrick, T. (2015). Defining 21st century readiness for all students: What
we know and how to get there. Psyc. of Aesthetics, Creativity, and the Arts, 9(2), 178.
Sudibyo, E., Jatmiko, B., & Widodo, W. (2016). The effectiveness of CBL model to
improve analytical thinking skills the students of sport science. Int Edu Stu, 9(4), 195.
Sumarmo, U., & Nishitani, I. (2010). High level mathematical thinking. Kaname Osamu
Natural Science, 9(8). Retrieved from https://gair.media.gunma-
u.ac.jp/dspace/bitstream/10087/5130/1/03_Nishitani.pdf
Sung, Y. T., Yang, J. M., & Lee, H. Y. (2017). The effects of mobile-computer-
supported collaborative learning: meta-analysis and critical synthesis. Review of
Educational Research, 87(4), 768-805.
Tarricone, P., & Luca, J. (2002). Successful teamwork: A case study. Proceedings of the
25th HERDSA Annual Conference, Perth, Western Australia..
Van Laar, E., van Deursen, A. J., van Dijk, J. A., & de Haan, J. (2017). The relation
between 21st-century skills and digital skills: A systematic literature review. Computers
in Human Behavior, 72, 577-588.
Vui, T. (2007). Enhancing classroom communication to develop students' mathematical
thinking. Retrieved from http://74.125.155.132/
search?q=cache:CmaHbsBDf1kJ:www.criced.tsukuba.ac.jp/math/apec/apec2008/papers
/PDF/21.Tran_Vui_Vietnam.pdf.
Yang, Y. T. C. (2015). Virtual CEOs: A blended approach to digital gaming for
enhancing higher order thinking and academic achievement among vocational high
school students. Computers & Education, 81, 281-295.
Yen, T. S., & Halili, S. H. (2015). Effective teaching of higher order thinking (HOT) in
education. The Online Journal of Distance Education and e-Learning, 3(2), 41-47.
Zohar,A. dan Dori, Y.J. (2003). Higher Order Thinking Skill and Low Achieving
Students: Are There Mutually Exclusive? [Online]. Retrieved from
http://www.cc.gatech.edu/lst/jls/vol12no2.html#Article1
Zoller, U., & Pushkin, D. (2007). Matching higher-order cognitive skills (HOGS)
promotion goals with problem-based laboratory practice in a freshman organic
chemistry course. Chemistry Education Research and Practice, 8(2), 153–171.
https://doi.org/10.1039/B6RP90028C.

International Journal of Instruction, April 2020 ● Vol.13, No.2