Journal of Physics:

Conference Series

PAPER • OPEN ACCESS You may also like

- Effect of cortical bone thickness on shear
Applying CBT in physics learning to measure stress and force in orthodontic miniscrew-
bone interface – A finite element analysis
students’ higher order thinking skills Balamurali V, Varadaraju Magesh and
Pandurangan Harikrishnan

- A Cutting-Edge Sensor to Detect

To cite this article: E Istiyono et al 2020 J. Phys.: Conf. Ser. 1440 012061 Clenbuterol in Animal Biological Fluids
Nguyen Mau Thanh, Do Mai Nguyen, Anh
Quang Dao et al.

- Traceable Coulomb blockade thermometry

O Hahtela, E Mykkänen, A Kemppinen et
View the article online for updates and enhancements. al.

This content was downloaded from IP address 103.136.59.150 on 28/11/2024 at 14:17

The 5th International Seminar on Science Education IOP Publishing
Journal of Physics: Conference Series 1440 (2020) 012061 doi:10.1088/1742-6596/1440/1/012061

Applying CBT in physics learning to measure students’ higher

order thinking skills

E Istiyono1, W S B Dwandaru1, L Erfianti1 and W Astuti1

1
Physics Education, Universitas Negeri Yogyakarta, Sleman, Yogyakarta

Corresponding Author: [email protected]

Abstract. Development in science and technology that exist today can be optimized to be used
as media in learning process, one example is in assessment of learning using computer based
test (CBT). Higher order thinking skills (HOTS) are important aspects in physics learning.
CBT is used to measure the HOTS (PhysTeHOTS-CBT). The subjects of the study are 251
students of class XI science program. The students’ HOTS by using PhysTeHOTS-CBT show
4.78% of students have very low category; 13.1% have low category, 60.9% have medium
category, 9.16% have high category, and 11.9% have very high category. Based on the result
of the study, the students’ HOTS are dominated in the medium category.
Keywords: CBT, HOTS, PhysTeHOTS-CBT, reasoning multiple choices

1. Introduction
The curriculum is a component of the education system that contains lesson plan that is given over a
period of education. The use of curriculum is done to understand the competencies in the learning
process and to train students in higher-order thinking skills (HOTS) [1]. So, students become more
accustomed to HOTS on the learning process.
One of the success factors of the educational process is determined by scoring system that used.
Appropriate assessment of learning outcomes will provide accurate information about achievement of
competence of students and the quality of learning that is used by the teacher [2]. Assessment is also a
process for obtaining information on the performance of students [3]. Proper assessment system is
necessary given regarding to the importance of assessment in the learning process. Proper assessment
is provide accurate information, encourage students to learn, motivating teachers to teach, improve the
performance of the institution, and improve the quality of education [4]. In a preparing the assessment
must consider many aspects to comply with existing standards.
One effort in assessment is to use the test. Tests conducted to determine the cognitive obtained
students during the learning process. Tests increasingly varied forms of development along with the
development of science and technology [5]. Previous tests were only done on paper, now use
computers as media tests to address the weakness in paper based test [6]. In addition, using computer
based test is also able to eliminate shortage of tests on paper that is not environmentally friendly.
Tests that are used in high school to know the learning outcomes of physics so far is limited to
the assessment of low-level of thinking and have not yet been developed to measure HOTS [7], this
resulted in need to create a test to measure the ability of HOTS. Development of high-level test can be
done by varying multiple choice or use the constructed response questions and essays [8]. Multiple
choices reasoned have been chosen because the assessment process is more objective and more easily

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution
of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
Published under licence by IOP Publishing Ltd 1
The 5th International Seminar on Science Education IOP Publishing
Journal of Physics: Conference Series 1440 (2020) 012061 doi:10.1088/1742-6596/1440/1/012061

processed by computers. Computers have been used due to the development of science and technology
and also have many advantages [9]. Computer Based Test (CBT) have recently emerged as a viable
form of alternative assessment used around the world [10]. For that, a CBT test is developed to
provide accuracy and measure the ability of each student.
The development of a reasoned multiple choice format to cover multiple choice
shortcomings has been investigated in [11] and the study used it for scoring students’ answer and
reason. An assessment score can be seen in table 1.
Table 1. Assessment Score in PhysTeHOTS-CBT.
Score Criteria
4 Answer and Reason are right
3 Reason is right, but answer is wrong
2 Reason is wrong, but answer is right
1 Answer and Reason are wrong

2. Research method
This research was conducted at senior high schools class XI Science Program in Kota Yogyakarta,
Yogyakarta Province. The research subjects are 251 students selected by stratified random sampling
technique, namely by choosing high schools with high and low physics scores. The research subjects
are asked to carry out physics tests using CBT that are able to measure the HOTS, i.e.: PhysTeHOTS-
CBT.
PhysTeHOTS-CBT is developed in accordance with the indicators of physics class XI Curriculum
2013 and in accordance with indicators of HOTS. PhysTeHOTS-CBT consists of five subject matters
that are rigid body equilibrium, elasticity and spring, static fluid, dynamic fluid, and temperature and
heat. The questions consist of 4 packages with each package consists of 45 questions with 8 anchor
items.
The question test in PhysTeHOTS-CBT meets the requirements of a good test item. These
requirements are 1) content validation obtains V Aiken of 1.00, which means that all test items are
valid to assess HOTS; 2) based on empirical validation seen from the suitability of items to the model,
all items are in the range of 0.77 to 1.30, which means that all items match with the PCM model; 3)
the difficulty of items in the PhysTeHOTS-CBT are in the range of -0.54 to 1.00, so it can be said that
all items are in good condition; 4) the results of the reliability tests found that PhysTeHOTS-CBT is
suitable for measuring students with abilities of -2.2 to 2.0.
Scoring PhysTeHOTS-CBT used the partial credit model (PCM) which is a development of 1-PL.
The test is taken using PhysTeHOTS-CBT and the results are categorized in five levels of HOTS
adopted from the distribution method by Azwar [12]. The category of HOTS can be seen in Table 2.
Based in table 2, the category of HOTS can be ilustrated in table 3
Table 2. Interval equation and category of HOTS.
No. Interval Equation Category
1. Very high
2. High
3. Medium
4. Low
5. Very low

2
The 5th International Seminar on Science Education IOP Publishing
Journal of Physics: Conference Series 1440 (2020) 012061 doi:10.1088/1742-6596/1440/1/012061

Table 3. The result and category of HOTS.

No. Interval Equation Category
1. Very high
2. High
3. Medium
4. Low
5. Very low
Information: = the students’ HOTS; = ideal mean skills = ½ ( = ½ (
; and = ideal standard deviation = 1/6 ( = 1/6 ( .

3. Results and Discussion

The instrument of HOTS developed is 160 items with 8 are anchor items. The HOTS test that has been
developed is packaged into PhysTeHOTS-CBT. The students that are measured using PhysTeHOTS-
CBT represent schools with HOTS in low and high categories. The PhysTeHOTS-CBT home page
can be seen in figure 1.

Figure 1. Home page of PhysTeHOTS-CBT.

The results of students’ HOTS measurements can be seen in the statistics menu showing
calculations and categories of HOTS and conversion value from 1-100. The statistics menu display
can be seem in figure 2. PhysTeHOTS-CBT are conducted for high school grade XI students. The test
implemented is to measure the students’ HOTS using CBT. The ability (θ) of each student can be
categorized in the predicted ability from very high to very low based on the ideal score and standard
deviation [12] which is shown in table 4. The result of categorizing the abilities (θ) of students are
shown in completing the PhyTeHOTS using CBT in table 5

3
The 5th International Seminar on Science Education IOP Publishing
Journal of Physics: Conference Series 1440 (2020) 012061 doi:10.1088/1742-6596/1440/1/012061

Figure 2. Statistics menu

Table 4. Students’ ability interval.
No Ability Intervals Categories
1 Very high
2 High
3 Medium
4 Low
5 Very low

Table 5. Category of HOTS by using PhysTeHOTS-CBT

No Category Sum Percentage (%)
1 Very high 30 11,9
2 High 23 9,16
3 Medium 153 60,9
4 Low 33 13,1
5 Very low 12 4,78

Table 5 shows the distribution of HOTS category in the PhysTeHOTS-CBT. From the table , it can
be concluded that the majority of students have HOTS in the medium category. Figure 3 shows that
students’ HOTS in physics learning can be measured by CBT. Students with very high HOTS category
is only 11.9%. The use of CBT also has many advantages, e.g.: paperless and time efficient. The result
comes out as soon as the students finish the test. This is in line with the results of the research
conducted in [13] that CBT has the advantage of faster test distribution, time-efficient benefits, and
fast feedback. Also, in line with the results in [14], CBT is able to provide faster test results.

4
The 5th International Seminar on Science Education IOP Publishing
Journal of Physics: Conference Series 1440 (2020) 012061 doi:10.1088/1742-6596/1440/1/012061

180

160

140

120

100 Amount of students

80
Presentase
60 (%)

40

20

0
Very high high medium low

Figure 3. The distribution of HOTS in PhysTeHOTS-CBT.

4. Conclusions
Based on the results of the study, it is found that PhysTeHOTS developed based on the IRT algorithm
can measure the students’ HOTS. From 256 students of grade XI, there are 11.9 % that have very high
ability, 9.16% high ability , 60.9% medium ablity, 13.1% low ability, and 4.78% very low ability. It is
conclude that the majority of students have the ability in the medium category.

References
[1] Roza Y, Satria G and Siregar S N 2017 https://doi.org/10.1088/17426596/855/1/012038
[2] Kunandar 2014 Penilaian Autentik: (Penilaian hasil belajar peserta didik berdasarkan
Kurikulum 2013) Suatu Pendekatan Praktis disertai dengan Contoh (edisi revisi) (Depok.
PT. Raja Grafindo Persada)
[3] Miller M D, Linn R L and Gronlund N E 2009 Measurement and Assessment in Teaching
(Upper Saddle River: Pearson Education Inc)
[4] Mardapi, D 2012 Pengukuran, Penilaian dan Evaluasi Pendidikan (Yogyakarta: Nuha Medika)
[5] Trilling B and Fadel C 2009 21st Century Skills: Learning for Life in Our Times (San Francisco,
CA: John Wiley & Sons)
[6] Hambleton R K, Swaminathan, H., & Rogers, H.J. 1991. Fundamentals of item response theory.
Newburry Park, CA: Sage Publications, Inc
[7] Istiyono E, Mardapi D and Suparno 2013 Jurnal Penelitian dan Evaluasi Pendidikan 14 No.1,
p: 1-12
[8] Brookhart, S & M. 2010. How to assess higher order thinking skills in your classroom.
Alexandria: ASCD
[9] Embretson S E and Reise S P 2000 Item response theory for psychologist. Marwah, NJ:
Lawrence Elbraum Associates Publisher.
[10] Alakyleh A S 2018 International Journal of Assessment Tools in Education 5 176-86.
http://dx.doi10.21449/ijate370494
[11] Istiyono E 2018 IT-based HOTS assessment on physics learning as the 21st century demand at
senior high schools: Expectation and reality. In AIP Conference Proceedings (Vol. 2014)
[12] Azwar, S. 2010. Metode Penelitian. Yogyakarta: Pustaka Pelajar. Pp. 158
[13] Redecker C and Johannessen 2013 Changing Assessment – Towards a New Assessment

5
The 5th International Seminar on Science Education IOP Publishing
Journal of Physics: Conference Series 1440 (2020) 012061 doi:10.1088/1742-6596/1440/1/012061

Paradigm Using ICT. European Journal of Education, 48(1), 79-96

[14] Bodin M and Winberg M 2012 Physical Review Special Topics – Physics Education Research,
8(1)

Acknowledgement
This work was supported by the Ministry of Research, Technology and Higher Education (Indonesia)