JPII 8 (1) (2019) 39-51

Jurnal Pendidikan IPA Indonesia

http://journal.unnes.ac.id/index.php/jpii

STUDENT ACHIEVEMENT BASED ON THE USE OF SCIENTIFIC

METHOD IN THE NATURAL SCIENCE SUBJECT IN ELEMENTARY
SCHOOL

B. Subali1, Kumaidi2, N. S. Aminah3, B. Sumintono4

1
Biology Education Department, Faculty of Mathematics and Natural Sciences,
Universitas Negeri Yogyakarta, Indonesia
2
Universitas Muhammadiyah Surakarta, Indonesia
3
Universitas Sebelas Maret, Indonesia
4
University of Malaya, Malaysia

DOI: 10.15294/jpii.v8i1.16010

Accepted: November 6th, 2018. Approved: March 25th, 2019. Published: March 28th, 2019

ABSTRACT

This research aims at investigating elementary school student achievement based on the use ofscientific method
in teaching science from the test item types, as reflected by the item difficulty indexusing the classical test theory
(CTT) and modern test theory (IRT). The first stage in developing the test was preparing the learning continuum
of scientific method aspects by referring to the learning continuum of science process skill as developed by the
previous existing research. In this research, the learning continuum was validated by expert judgment. As the tests
were administered/carried out at the same time, four sets of tests were developed and administered to students
of Grade 1 to 6 in Yogyakarta and Sleman Regency in the 2016-2017 school year. Samples were taken from three
Technical Management Units (TMUs). Three TMUs were determined by observing the distribution of school
locations from the center to the suburbs. The items were analyzed using CTT and IRT.The results of the research
show that the student achievement reflected by item difficulty index based on CTT and IRT indicates the same
level of category except for several sub-aspects. Those items from certain testsindicate higher difficulty level for
Grade 4 to 6 students than for Grade 1 to 3 students. This case is not relevant to the expected learning outcomes.

© 2019 Science Education Study Program FMIPA UNNES Semarang

Keywords: scientific method, test types, CTT, IRT

INTRODUCTION involved in an inquiry. Science process skills are a

necessary tool to produce and use scientific infor-
An inquiry has become the heart of scien- mation, to perform scientific research, and solve
ce education reform. Despite its prevalence in problems. The integration of explicit, reflective
school and policy rhetoric, the term remains instruction about the nature of science (NOS)
quite ambiguous. Generally, an inquiry becomes and scientific inquiry (SI) in traditional science
conflated with the scientific method, taught as content is addressed as a means through which
a series of steps ranging from asking a question the development of scientific literacy is fostered
to drawing conclusions (Tang et al., 2010). Ac- (Lederman et al., 2013).
cording to Zeidan & Jayosi (2015), science edu- For science instruction to be productive,
cation is aimed at teaching students how to get teachers should consider physical and mental
skills to generate and test reliable knowledge and
*Correspondence Address
E-mail: [email protected] generalization. In learning science, the process
40 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51

skills include observing, measuring, classifying, asurement is not associated with the learning pro-
inferring, predicting, hypothesizing, identifying cess which has just taken place. In this case, for
variables, experimenting and interpreting data. the Natural Science subject at elementary school,
Generally, these are processes carried out by the tests must measure student achievement onthe
scientists during investigations.The aim of this scientific product (scientific knowledge) and pro-
approach is for pupils to learn and obtain an un- cess skills (scientific methods aspects). If this is
derstanding through the development of their implemented, the result of the assessment can be
own ideas. Therefore, the teaching method that used to improve learning. This is relevant with
should be adopted for the teaching of science Black et al. (2004) who suggest that an assess-
must be carefully considered and should encou- ment for learning should be interpreted as the te-
rage the development of science process skills by achers’ effortsin improving students’ learning by
the learners. The attention of many science edu- utilizing different assessment results, i.e. assess-
cators has continued to be directed at searching ments which are carried out from time to time,
for such appropriate methods of science teaching from meeting to meeting, and from day to day.
(Nweke et al., 2014). In Biology learning, high- Therefore, it is called a mechanism of assessment
school biology students meet authentic science for learning because the results of measurements
regarding their participation in a science outreach on the achievement of scientific knowledge are
programme (Tsybulsky et al., 2019). used as an input for learning.
Science Learning depends on the kinds of The cognitive aspects developed in the
strategies used by teachers. For example, project- 2013 Curriculum refer to the principles of
based learning and demonstration strategies of Bloom’s taxonomy as proposed by Anderson &
teaching are potent in increasing student achieve- Krathwohl (2001). These principles explain that
ment. Through project-based learning, students scientific knowledge consists of facts, concepts,
can be motivated to explore, negotiate, interpret, and procedures (factual, conceptual, and pro-
and create in collecting data using the scientific cedural knowledge). In science, the achievement
method and formulating the concepts (Olatoye of procedural knowledge is essential because it
& Adekoya, 2010).There are many factors which enables students to practice scientific procedures.
influence student achievement. Those factors, ac- The scientific method is the process by
cording to Beyessa (2014), include parental invol- which scientists conduct investigations to pro-
vement, peer pressure, schools support and other duce scientific products (Carin & Sund, 1989).
stakeholders’ commitment to improving students’ LeBoffe & Wisehart (1989) state that the scien-
science education. Moreover, the lack of labora- tific method is a science process skill which is
tory chemicals, rooms, apparatuses, technicians systematically arranged to solve a problem. In
and well-organized laboratory manuals negative- regards to the scientific process and scientific at-
ly affect the effective implementation of science titude, scientists discover scientific products such
education and students’ academic achievement. as facts, concepts, and new principles in scien-
The implementation of the scientificmet- ce (Carin & Sund, 1989). Scientific procedures
hod in teaching science may become a research consist of stages of (a) identifying problems; (b)
method. For example, Çaparlar & Dönmez gathering facts and information; (c) designing in-
(2016) implemented scientific research forthe vestigations; (d) reporting the results; (e) repea-
purpose of contribution towards sciencethrough ting investigations/ experiments; (f) checking the
systematic collection, interpretation,and evalua- results; and (g) drawing conclusions (Belardo &
tion of data in a planned manner. Therefore, stu- Samia, 1999). The science teaching process to 15
dents should be taught scientific methods so that years-old students must encourage them to design
they can master research methods. and carry out investigations, create data, and in-
One of the main issues in classroom terpret the data (Watts et al., 1989). Therefore, te-
assessment is the application for diagnostic and aching scientific procedures to the students must
formative purposes. In diagnosed student lear- be performed in stages.
ning, misconceptions are one of the big issues. Bryce et al. (1995) state that scientific
Misconceptions may occur because students try methods consist of a number of basic and pro-
to comprehend their previous experiences based cess skills that will produce investigative skills
on their interactions with their environment when they are arranged into systematic stages.
(Cañada et al., 2017). Thus, misconceptions on According to Rezba et al. (2007), science process
scientific knowledge can be identified if the me- skills are basic skills. When they are integrated,
 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51
41

they will generate integrative skills in the form In Indonesia, the scientific method in the
of scientific methods. According to Bryce et al. teaching of science should be introduced to lear-
(1995), the aspects of basic skills consist of some ners as early as possible.This has been stated in
sub-aspects skills,namely (a) observing using sen- the 2013 Curriculum for Natural Science subject
ses; (b) recording data/ information; (c) follo- inthe elementary school level.Since it consists of
wing instructions; (d) classifying; (e) measuring; a series of complex scientific process, its teaching
(f) manipulating movements; (g) implementing must be carried out little by little, aspect by as-
procedures and using equipment; and (h) predic- pect, and even sub-aspect by sub-aspect. Thus, te-
ting. Meanwhile, process skills include sub-aspect aching the scientific method to elementary school
skills of (a) inferencing and (b) selecting procedu- students means teaching the students to acquire
res. However, Subali (2009) has indicated that the every aspect and sub-aspect of science process
sub-aspect of “predicting” should be classified skills.
in the aspect of process skills. Moreover, Subali The tests for measuring the student achie-
(2009) adds that skills of conducting investigati- vement based on the scientific method teaching
on include the sub-aspect skills of (a) planning may take form as a performance test in the form
investigation; (b) conducting investigations; and of work sample tests and paper-pencil/written
(c) reporting the findings either in the written or test. A paper-pencil test can measure the achieve-
spoken form. ment of students’ cognitive skills in the scientific
According to Wenning (2010), there are method. This is one aspect that must be measu-
six levelsof inquiry: (a) discovery learning (de- red to examine student achievement based on the
veloping concepts on the basis of first-hand ex- use of scientific method in teaching science. This
periences, introducing terms); (b) interactive de- study intends to investigate the status of elemen-
monstration (eliciting, identifying, confronting, tary school student achievement based on scien-
and resolving alternative conceptions); (c) inquiry tific method, in relation to the test item types as
lesson (identify scientific principles and/or rela- reflected by item difficulty indexes based on the
tionships); (d) inquiry labs (establishing empirical classical test theory (CTT) and modern test the-
laws based on measurement of variables; (e) real- ory (IRT). The first stage in developing test was
world applications (applying prior knowledge to preparing the learning continuum of scientific
authentic problems); and (f) hypothetical inquiry method aspects by referring to the learning con-
(deriving explanations for observed phenomena). tinuum.
The skills were developed by discovery learning A good test development ideally refers
as the lowest level inquiry.They belong to rudi- to a learning continuum (Northwest Evalua-
mentary skills which includeobserving, formu- tion Association, 2001) so that it can measure
latingconcepts, estimating, drawing conclusions, the students’ skills. The learning continuum is
communicating results, and classifying results. also called a learning trajectory. It refers to stu-
Arlianty et al. (2017) state that the scien- dent ability which ranges from the lowest to the
tific method was firstly introduced in the United highest ability related to the complexity and dif-
States in educational science in the 19th century ficulty of content knowledge. A number of stu-
by emphasizing the laboratory formalistic met- dies have discussed the topic of how to develop
hods which direct scientific facts. The scientific a written test,such as those by Millard (2012),
method has the characteristic of “hands-on scien- Miller (2008), Popham (2005), Gronlund & Linn
ce.” This method enables teachers and curricu- (1990), Gronlund (1998), as well as Roid & Ha-
lum developers to improve the learning process. ladyna (1982). Meanwhile, research on test types
According to McInerney (1986), The Biological has been performed by Luo and Zhang (2011). In
Science Curriculum Study was established in addition, Shete et al. (2015) conducted research
1958 as a non-profit corporation which is respon- on item analysis for evaluating multiple choice
sible to restructure the Biology Curriculum. The questions in a physiology examination.
results were initially introduced in 1970 entitled In constructing a test, one must consider
“The Antiquated Content of the Biology Courses the use of language. Sumantri & Satriani (2016)
in High School” with more experimental, quanti- find that most teachers agree that communicati-
tative content. Moreover, the designed materials on in the process of teaching and learning is very
promoted the teaching of science as a process of important to improve students’ understanding.
investigation and inquiry, rather than in a simple Effective communication increases students’ un-
and holistic manner as a body of knowledge. derstanding of the topics being taught, which
42 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51

enables them to overcome high-level problems By using IRT, the children’s ability level
correctly. Effective language is important in the and item difficulty index can be plotted in a
construction of test items so that students can un- single line using a logit scale. Thus, the difficulty
derstand the questions easily. Non-standard test level of the items can be compared to the abili-
items are more difficult for students to answer ty of the testee. Meanwhile, CTT item difficulty
correctly than the standard test items, as it provi- level cannot be compared to the student’s ability
des no enhanced ability to discriminate between (Wright, 1999; Wright & Masters, 1982).
higher and lower-performing students, resulting Research on the empirical characteristics
in poorer student performance. With regard to of items that attempts to compare CTT and IRT
this, item-writing guidelines should be taken into approaches has been widely performed. For
account during test construction (Caldwell & instance, Stage (2003) conducted a research on
Pate, 2013). the experience of using CTT and IRT among
In some studies compiled by Ulu (2017), Swedish examinees. Petrillo et al. (2015) carried
the procedural knowledge may be insufficient in out research on utilizing the measurementtheo-
first-time situations, and contextual knowledge is ries of CTT, IRT, and Rasch to evaluate the re-
needed for such situations. With regard to this, sults of patience measurement. Thorpe & Favia
when the measurement of the achievement of the (2012) conducted research on the data analysis
scientific method is taken, its success may rely on using IRT methodology for selected programs
the testee’s experience in applying the scientific and applications. Zoghi & Valipour (2014) con-
method and context encountered when they utili- ducted a study on the comparison of CTT and
ze a scientific method. IRT to predict an item test parameter in linguis-
The quality of the test must meet the tics tests. The research investigating the charac-
theoretical validity viewed from the aspects of teristics of items viewed from the characteristics
construct validity, content validity, and empirical of classical and modern tests theory in relation
validity. In this case, validity can be viewed from to scientific method or science process skills
classical test theory (CTT) and modern test theo- have been performed by Pada et al. (2016).
ry/Item Response Theory (IRT). The application The achievement scientific method can
of modern test theory for analyzing test items has be measured by performance testas well as by
been carried out by Le (2013). paper-pencil or written performance test. The
According to Le (2013) whoreferred to Os- problem in developing an achievement test on
teen, IRT is considered as the standard validity scientific methods is the extent to which the
test. A lot of testing programs still refers to CTT written performance test of scientific achieve-
in their design and assessment of test results. This ment methods influences the characteristics of
is due to some advantages of CTT over IRT. For the test type based on CTT and IRT. In this case,
example, CTT explains the relationship between the problems are (a) whether the multiple choice
the true score and observed score in a linear fashi- test with two options has a different difficulty
on which makes the CTT model easy to under- index from those with three options based on
stand and is applicable for a lot of researchers. CTT and IRT; (b) whether the difficulty index
CTT also offers smaller sample sizes which are of the analysis performed using CTT is the same
smaller than IRT. Compared to IRT, CTT’s mat- as that using IRT; and c) whether the items of a
hematical procedures are much simpler. In CTT, true-false test type have different characteristics-
parameter estimation is conceptually straightfor- based on CTT and IRT when the answers are
ward and requires minimum assumptions,making opposite (if a statement is categorized as true in
the model useful and widely applicable. CTT ana- a true-false test of A model, the statement in B
lyses do not need strict goodness of fit studies like model is changed into the false category). The-
that of IRT. However, CTT has somemain weak- refore, the aim of this study was to explore stu-
nesses. One of them is that the test scores rely on dentachievement based on the use of scientific
the testees. It means that the examinees can get method in teaching science, as reflected on the
a better score on easier tests and bad scores on item difficulty indices and the test item types,
difficult tests. Thus, there is no real score which without investigating the learning performan-
can be extracted because there is no information ce in the classroom. In that regard, this study
about the examinees’ abilities. This does not al- attempts to analyze the problems in the use of
low the test items to matchwith ability levels. scientific method in the teaching and learning
 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51
43

process using to improve its quality in future te- Students in each class were to answer 8
aching and learning process. test sets that had been previously prepared. Stu-
dents who sat side by side were to answer questi-
METHODS ons with different codes. With regard to this, the
results were accountable for the Rasch model. In
The test was designed by developing the order to compare the results among students of
learning continuum of the scientific aspects Grade 1-6, two pairs of the test were provided
method based on the learning continuum of with an item anchor, for example, code test 1-3
science process skill developed by Subali & Ma- for two options and code test 2-4 for three op-
riyam (2013) and validated by eight (8) experts tions. Therefore, a score would be gained within
on Natural Sciences Education from Universitas a single measurement scale. The number of the
Sebelas Maret Surakarta and Universitas Negeri anchorwas 15% of the total items. The item ana-
Yogyakarta. lysis was performed using the Quest Program
The population of this study was ele- (Adams & Kho, 1996) to get the results based on
mentary school students in Yogyakarta and the CTT and IRT models.
Sleman Regency in the 2016-2017 school year. Since the test was administered simulta-
The sample consisted of two Technical Mana- neously, different tests were developed in order to
gement Units (TMUs) in Yogyakarta and one prevent the examinees who sat side by side to have
TMU in the Sleman Regency, i.e. TMU of Ka- the same test. The tests were developed in two
lasan sub-district. Three TMUs werechosen by different models namely a true-false and multiple
observing the distribution of school locations choice. The multiple choice test type consisted of
from the center to the suburbs. They were nort- test items with two options and those with three
hern Yogyakarta TMU in the center, eastern options. For the true-false test, there were two
Yogyakarta TMUextendingto the suburb, and models of which the key answers are opposite. If
Kalasan district TMU which was partly located a statement of the true-false test model coded A
in the countryside. Among those three TMUs, was declared true, the same statement was chan-
13 elementary schools were taken from each as ged into a false statement on the test coded B. The
samples which were categorized intolow, me- test item development considers the aspects of
dium, and high level of performance based on substance, development, and language.
the assessment of the supervisors. Then, one Relevant to the characteristics of natural
class from Grade 1 to Grade 6 was taken from sciences, each sub-aspect/ indicator of the scien-
each school. If a school had a parallel class, all tific method can be related to both living and
the parallel classes in the school were taken as non-living objects. Each test set consisted of 35
samples. Thus, each TMU consisted of 78 clas- items in which 20 items were related to living ob-
ses. Therefore, the total sample was 234 classes, jects and 15 items were interrelated to non-living
in which the number of students for each class objects. The sub-aspects of the scientific method
ranged from 20 to 30 students. This sample was were divided into four sub-aspects, namely sub-
expected to represent the hypothetical popula- aspects I, II, III, and IV which covered basic and
tion of the students which may be relevant to process skills. Some test sets contained sub-as-
another province with the same characteris- pects I and III, and others contained sub-aspects
tics. This number of sample was also expected of II and IV. Groups with sub-aspects I and II
to meet the requirement for testing, which was were related to living objects, and those with sub-
analyzed using the Graded Model in which the aspects III and IV were related to non-living ob-
minimum number of examinee required for re- jects. They are presented in Table 1. Moreover,
search is 250 examinees (Muraki & Bock, 1998). Table 1 presents the result of test analysis using a
To get a description of the student achie- quest program to get information about test vali-
vement of Grade 1-6 in elementary school, each dity and reliability. Test validity is achieved using
test set type was tested to all students.Therefore, IRT information in which the test is declared to
in every class, each student was tested using one be “valid” based on classical theory if each item
model of the scientific method achievement test fit the Parameter Logistic (PL) model in IRT. In
in the form of a paper-and-pencil performance other words, the test is “valid” based on classical
test according to Gronlund’sterminology (1998). test theory (Wright & Master 1982). In addition,
Due to the many aspects and sub-aspects of the the table presents the amount of reliability index
scientific method, the tests were limited to basic which is expressed in the form of error of measu-
and process skills. rement error and internal consistency.
44 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51

Table 1. Types of the Test Set on Student Achievement Based on Scientific Method Teaching along
with the Specification of Questions, Sub-Aspects, Scientific Methods, and Natural Objects

Types Types of Sub-Aspect Specification of Items Related to Items Fit Reliability

of the question Groups of the Objects With the (Error of
test set Scientific Rasch Measure-
Method Model ment/Inter-
nal Consis-
tency)

Group of 20 items are related to living thing

Multiple choice Code I objects (item number 1 to 20) All items
Test set 1 type with two 0.72/0.72
Group of 15 items are related to non-living fit
answer choices
Code III thing objects (item number 21 to 35)
Group of 20 items are related to living thing
Multiple choice Code I objects (item number 1 to 20) All items
Test set 2 type with three 0.70/0.70
Group of 15 items are related to non-living fit
answer choices
Code III thing objects (item number 21 to 35)
Group of 20 items are related to living thing
A true-false type Code I objects (item number 1 to 20) All items
Test set 3 0.51/0.49
of A model Group of 15 items are related to non-living fit
Code III thing objects (item number 21 to 35)
A true-false type Group of 20 items are related to living thing
of B model (the Code I objects (item number 1 to 20)
All items
Test set 4 key answer is op- 0.59/0.57
Group of 15 items are related to non-living fit
posite to that of
A model) Code III thing objects (item number 21 to 35)

Group of 20 items are related to living thing

Multiple choice Code II objects (item number 1 to 20) All items
Test set 5 type with two 0.70/0.72
Group of 15 items are related to non-living fit
answer choices
Code IV thing objects (item number 21 to 35)
Group of 20 items are related to living thing
Multiple choice Code II objects (item number 1 to 20) All items-
Test set 6 type with three 0.71/0.71
Group of 15 items are related to non-living fit
answer choices
Code IV thing objects (item number 21 to 35)
Group of 20 items are related to living thing
A true-false type Code II objects (item number 1 to 20) All items
Test set 7 0.54/0.51
of A model Group of 15 items are related to non-living fit
Code IV thing objects (item number 21 to 35)
A true-false type Group of 20 items are related to living thing
of B model (the Code II objects (item number 1 to 20)
All items
Test set 8 key answer is op- 0.55/0.53
Group of 15 items are related to non-living fit
posite to that of
A model) Code IV thing objects (item number 21 to 35)

Note: Based on a quest program, an item fitsthe model if it is in between the InfitMNSQ range of 0.7-1.3.

RESULTS AND DISCUSSION difficulty index on the aspects and sub-aspects of

scientific methods based on the testing results on
The following is the result of data analysis a group of examinees of Grade 1 to 3 and Grade
using the classical theory to determine the item 4 to 6.
 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51
45

Table 2. The Test Difficulty Index Related to the Aspects and Sub-Aspects of Scientific Method Using
CTT Analysis for Sub-Aspects with Indicators I and III

DI of DI of
DI of MC DI of TF
Grade of MC TF
Code Indicators of Scientific Processes
Students Two Three A B
Options Options Model Model
1.1 1–3 0.7 0.57 0.61 0.64
Observing skills (4 items)
4–6 0.8 0.66 0.54 0.73
1.2 Recording data skills using sense of sight/ 1–3 0.53 0.4 0.71 0.42
hearing/smell/taste/touch), (4 items) 4–6 0.72 0.59 0.71 0.42
1.3 1–3 0.37 0.19 0.7 0.48
Following instruction skills (1 item)
4–6 0.45 0.17 0.7 0.48
1.4 1–3 0.38 0.63 0.6 0.84
Classifying skills (2 items)
4–6 0.65 0.58 0.6 0.84
1.5 1–3 0.49 0.36 0.62 0.45
Measuring skills (6 items)
4–6 0.61 0.47 0.62 0.45
1.6 1–3 0.61 0.4 0.43 0.76
Manipulating movement skills (5 items)
4–6 0.74 0.54 0.43 0.76
1.7 Implementing procedures/techniques/ 1–3 0.57 0.33 0.47 0.57
equipment usage skills (4 items) 4–6 0.65 0.44 0.47 0.57
 2.1 1–3 0.65 0.61 0.81 0.6
Inferencing skills (5 items)
4–6 0.83 0.82 0.82 0.61
 2.2 1–3 0.73 0.5 0.62 0.89
 Predicting skills (3 items)
4–6 0.88 0.58 0.62 0.89
1–3 0.58 0.67 0.63 0.9
 2.3 Selecting procedure skills (2 items)
4–6 0.66 0.91 0.63 0.9

Note: DI: Difficulty Index; MC: Multiple choice; TF: True-false

Based on the CTT theory, the lowest the I-III, observing skill is more difficult for the Gra-
difficulty index is, the easier the item will be. Nor- de 4 to 6 students than for Grade 1 to 3 students
mally, a certain skill is easier for the students of when being tested using a true-false test type A.
Grade 4 to 6 than for those of Grade 1 to 3. This In addition, recording data/information and
is because Grade 4 to 6 students have more lear- classifying skills are more difficult for the Grade
ning experience and apply more scientific met- 4 to 6 students than for the Grade 1 to 3 students
hod aspects than Grade 1 to 3 students. Based on when being tested using a multiple-choice test
the results of analysis using CTT for indicators with three options.

Table 3. The Test Difficulty Index Related to The Aspects and Sub-Aspects of Scientific Method Us-
ing a CTT Analysis for Sub-Aspects with Indicators II and IV

DI of DI of DI of
DI of TF
Grade of MC MC TF
Code Indicators
Students Two Three A B
Options Options Model Model
1–3 0.65 0.65 0.58 0.5
1.1 Observing skills (4 items)
4–6 0.76 0.83 0.72 0.57
1–3 0.48 0.4 0.5 0.53
1.2 Recording data/information skills (5 items)
4–6 0.58 0.42 0.56 0.53
46 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51

1–3 0.39 0.38 0.66 0.37

1.3 Following instruction skills (2 items)
4–6 0.45 0.63 0.64 0.48
1–3 0.47 0.37 0.73 0.35
1.4 Classifying skills (2 items)
4–6 0.72 0.49 0.8 0.51
1–3 0.69 0.44 0.55 0.52
1.5 Measuring skills (4 items)
4–6 0.81 0.59 0.69 0.64
1–3 0.73 0.35 0.66 0.52
1.6 Manipulating movement skills (2 items)
4–6 0.89 0.43 0.64 0.65
Implementing procedures/techniques/ 1–3 0.55 0.43 0.56 0.48
1.7
equipment usage skills (7 items) 4–6 0.71 0.56 0.6 0.56
1–3 0.64 0.52 0.48 0.74
2.1 Inferencing skills (5 items)
4–6 0.8 0.63 0.61 0.8
1–3 0.56 0.6 0.87 0.86
2.2  Predicting skills (1 item)
4–6 0.57 0.83 0.95 0.93
1–3 0.56 0.49 0.43 0.49
2.3 Selecting procedure skills (3 items)
4–6 0.69 0.59 0.49 0.57

Note: DI: Difficulty Index; MC: Multiple choice; TF: True-false

Based on the CTT theory, the higher the manipulating movement skill, which shows an
difficulty index is, the easier the item will be. opposite condition in which it is more difficult
The results of the analysis using the CTT for the students of Grade 4 to 6 than for those of
approach for the indicators II-IV presented in Grade 1 to 3 when being tested using a true-false
Table 3 show that there is only one aspect i.e. test type A.

Table 4. The Test Difficulty Index Related to the Aspects and Sub-Aspects of Scientific Method Us-
ing An IRT Analysis for Sub-Aspects with Indicators I and III

DI of DI of DI of
DI of TF
Grade of MC MC TF
Code Indicators
Students Two Three A B
Options Options Model Model
1–3 -0.78 -0.7 0.19 -0.49
1.1 Observing skills (4 items)
4–6 -0.8 -0.76 0.19 -0.49
1–3 0.33 0.33 -0.78 1.16
1.2 Recording data/information skills (4 items)
4–6 0.22 0.12 -0.78 1.16
1–3 1.01 1.37 -0.3 0.86
1.3 Following instruction skills (1 item)
4–6 1.51 2.22 -0.3 0.86
1–3 0.98 -0.74 0.17 -0.93
1.4 Classifying skills (1 item)
4–6 0.6 0.21 0.17 -0.93
1–3 0.49 0.52 0.13 0.97
1.5 Measuring skills (6 items)
4–6 0.7 0.72 0.05 0.97
1–3 -0.14 0.28 0.94 -0.68
1.6 Manipulating movement skills (5 items)
4–6 -0.24 0.36 0.94 -0.68
Implementing procedures/techniques/ 1–3 0.13 0.59 0.72 0.49
1.7
equipment usage skills (4 items) 4–6 0.52 0.84 0.72 0.49
1–3 -0.2 -0.71 -0.96 0.28
2.1 Inferencing skills (5 items)
4–6 -0.46 -1.206 -0.96 0.28
 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51
47

1–3 -0.69 -0.16 -0.24 -1.74

2.2  Predicting skills (3 items)
4–6 -1.16 0.2 -0.24 -1.74
1–3 0.095 -0.99 0.07 -1.57
2.3 Selecting procedure skills (2 items)
4–6 0.45 -1.95 0.07 -1.57

Note: DI: Difficulty Index; MC: Multiple choice; TF: True-false

Based on the IRT theory, the higher the dif- options. Those indicators for multiple-choice
ficulty index is, the more difficult the item will be. tests with two options include following instruc-
The results of the analysis using IRT for tions, measuring, and implementing procedures/
indicators I-III as presented in Table 4 show that techniques/equipment usage skills. Moreover, for
some indicators are more difficult for the students multiple-choice tests with three options, the indi-
of Grade 4 to 6 than for those of Grade 1 to 3 cators include those 3 aforementioned skills and
when being tested with multiple-choice tests of the skills of classifying and manipulating move-
two options and multiple-choice tests of three ment.

Table 5. The Test Difficulty Index Related to the Aspects and Sub-Aspects of Scientific Method Us-
ing an Analysis of Modern Test Theory for Sub-Aspects with Indicators II And IV

DI of DI of DI of DI of
Grade of MC MC TF TF
Code Indicators
students two three A B
options options model model
1.1 1–3 -0.34 -0.93 -0.1 0.18
Observing skills (4 items)
4–6 -0.21 -1.49 -0.34 0.26
1–3 0.43 0.29 0.27 0.04
1.2 Recording data/information skills (5 items)
4–6 0.74 0.89 0.4 0.43
1–3 0.86 0.42 -0.43 0.72
1.3 Following instruction skills (2 items)
4–6 1.39 -0.1 0.05 0.67
1–3 0.55 0.53 -0.74 0.85
1.4 Classifying skills (2 items)
4–6 0.09 0.59 -0.74 0.57
1–3 -0.55 0.14 0.07 0.07
1.5 Measuring skills (4 items)
4–6 -0.58 -0.04 -0.16 -0.11
1–3 -0.69 0.56 -0.45 0.12
1.6 Manipulating movement skills (2 items)
4–6 -1 0.89 0.02 -0.1
Implementing procedures/techniques/ 1–3 0.16 0.14 0.05 0.25
1.7
equipment usage skills (7 items) 4–6 0.11 0.25 0.19 0.29
1–3 -0.3 -0.28 0.32 -1
2.1 Inferencing skills (5 items)
4–6 -0.6 -0.3 0.04 -1.09
1–3 0.11 -0.57 -1.66 -1.7
2.2  Predicting skills (1 item)
4–6 0.87 -1.2 -2.34 -2.1
1–3 0.08 -0.12 0.58 0.15
2.3 Selecting procedure skills (3 items)
4–6 -0.05 -0.06 0.68 0.18
Note: DI: Difficulty Index; MC: Multiple-choice; TF: True-false

Based on the IRT theory, the lowest the indicators II-IV presented in Table 5 show that
difficulty index is, the more difficult the item some indicators are more difficult for the Grade
will be. The results of the analysis using IRT for 4 to 6 students than for the Grade 1 to 3 students
48 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51

when being tested with a multiple-choice test of king the national and state test in Elementary and
two options, multiple-choice test of three options, Secondary Education Act program or No Child
true-false test type A, and true-false test type B. Left Behind (NCLB) in 2001. Criticisms about
Those indicators include the indicators for mul- these tests are described in the articles written by
tiple-choice tests with two options such as ob- Garrison (2009). Almost all states, schools, and
serving, recording data/information, following teachers focus on pursuing a learner’s success in
instructions, and predicting skills; the indicators dealing with the state tests. Many praxes of lear-
for multiple-choice tests with three options such ning focus on the students’ better achievement in
as recording data/information, classifying, and the test, or it is often called teaching for the test.
selecting procedure skills; the indicators for the The third finding also indicates that the
true-false test type A such as recording data/in- implementation of the 2013 Curriculum does
formation, manipulating movement, implemen- not give an impact on the student achievement
ting procedures/techniques/equipment usage, in skills, including their achievement in scientific
and selecting procedure skills; and the indicators method aspects in the teaching of Natural Scien-
for the true-false test type B such as observing, ce subjects at elementary schools. This is because
recording data/information, implementing pro- the teachers, supervisors, and even the Principal
cedures/techniques/equipment usage, and selec- of TMU keep focusing on the student achieve-
ting procedure skills. ment in the National Examination even though
The results show that indicators I-III and this examination does not measure the student
II-IV signify a different index of difficulty regar- achievement in skills which are parts of the scien-
ding the student achievement in the scientific tific method aspects. Therefore, it is necessary to
method aspects both in basic and process skills conduct research on how the scientific method is
when being tested with a different test type. Alt- actualized in learning Natural Science subjects in
hough the indicator is the same, the difficulty elementary schools.
index will be different when being tested with a The importance of conducting research on
different test type. In addition, a multiple-choice the actualization of scientific learning performed
test with threeoptions is not always more difficult in an inquiry model is based on the literature of
than that with two options. Similarly, a multiple- science literacy which encourages teachers to
choice test with threeoptions is not always more employ inquiry as a regular part of teaching prac-
difficult than a true-false test type. This implies tice (e.g., National Science Education Standards,
that designing a test by changing the test types is Science for All Americans: Project 2061). Unfor-
not reliable. However, further research is needed. tunately, this does not always happen. In summa-
Although an IRT is employed, it will not be af- ry, the success of those 6 inquiry levels depends on
fected by who the examinee is (Le, 2013). the teacher’s ability (Wenning, 2010). The low ca-
The second finding is that the difficulty pability of teachers in teaching scientific method
index for Grade 4 to 6 students becomes higher is due to their low understanding of the Nature
than that of Grade 1 to 3 students. This indicates of Scientific Inquiry (NOSI). It happens because
that learners who have more learning experiences the teachers do not understand how to do it when
do not always have better achievement in science they were taking the study. Miller et al. (2010) sta-
process skills. This occurs because the skills of the te that pre-service and in-service training enables
scientific process as parts of the scientific method teachers to possess informed conceptions about
are not taught optimally to the students. Some of NOSI. With these informed conceptions, teach-
the supervisors involved in community service ers may internalize the instructional importan-
activities repeatedly state that the main focus of ce of NOSI which, in turn, may help avoid the
the teaching is the student achievement in the na- lack of attention to NOSI currently evidenced in
tional examination. Thus, the national examina- teachers’ instructional decisions. This might re-
tion is regarded as a high-stakes test. When a test sult in teachers’ orientation shifting towards an
is viewed as a high-stakes test, the teachers are explicit inquiry-based approach from that of an
possibly focusing on the students’ success to deal implicit science process and discovery approach.
with it and may lead to teaching for the test. Criti- In addition, Hairida & Junanto (2018) say that
cisms about the implementation of the test, both the low science literacy skills of students in Indo-
national tests and state tests, addressed to the go- nesia may be influenced by several factors, such
vernment are also found in developed countries as the instructional model applied by teachers
like the United States (US). Both tests are also and the teaching materials used by the students.
considered as a high-risk test. This is due to the Moreover, Dudu (2014) says that students’ scores
many failures experienced by students when ta- on sections that address the six aspects of Nature
 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51
49

of Science (NOS) were significantly different in Wesley Longman, Inc.

most cases, showing notably uninformed views Arlianty, W.N, Febriana, B.W, & Diniaty, A. (2017).
of the distinctions between scientific theories and An Analysis of Learning Process Based on Sci-
laws. Evidence-based insight into students’ NOS entific Approach in Physicalchemistry Experi-
ment. AIP Conference Proceedings 1823, 020084
views can aid in reforming undergraduate science
(2017).
courses and will add to faculty and researcher un- Belardo, P.S. & Samia, E.R. (1999). Integrated Science &
derstanding of the impressions of science held by Technology I: Laboratory Manual. Quezon City:
undergraduates, helping educators improve the FNB Educational, INC.
scientific literacy of future scientists and diverse Beyessa, F. (2014). Major Factors that Affect Grade 10
college graduates. Based on the discussion above, Students’ Academic Achievement in Science
the test results can be used as an input for teachers Education at Ilu Ababora General Secondary
to improve the quality of learning which focuses of Oromia Regional State, Ethiopia. Interna-
on the learning which introduces the components tional Letters of Social and Humanistic Sciences,
32, 118-134.
of the scientific method to train the students to
Black, P., Harrison, Ch., Lee, Cl., Marshall, B., &
find new facts and concepts. Dylan, W. D. (2004). Assessment for Learning:
Putting it into practice. New York: Open Univer-
CONCLUSION sity Press.
Bryce, T.G.K., McCall, J., MacGregor, J., Robertson,
The conclusion of this research is that I.J., & Weston, R.A.J. (1995). Techniques for
different types of tests show a different index of assessing process skills in practical science: Teach-
difficulty when measuring student achievement er’s guide. Oxford: Heinemann Instructional
in scientific method aspects, including basic skills Books.
Caldwell, D.J. & Adam N. Pate, A.N. (2013). Effects of
and process skills. In addition, there are some as-
Question Formats on Student and Item Perfor-
pects of the scientific method, both basic and pro- mance. Am J Pharm Educ, 77(4), 71.
cess skills, that are even more difficult for Grade Cañada, Fl.C., González-Gómez, D., Airado-Rodrí-
4 to 6 students than for Grade 1 to 3 students. guez, D., Niño, L.V.M., & Acedo, M.A.D.
The student achievement based on the scientific (2017). Change in Elementary School Stu-
method reflected by item difficulty index based dents’ Misconceptions on Material Systems af-
on CTT and IRT indicates the same level of ca- ter a Theoretical-Practical Instruction. Interna-
tegory, except for few sub-aspects. The data also tional Electronic Journal of Elementary Education,
show that few items of certain tests indicate 9(3), 499-510.
Çaparlar, C. Ö., & Dönmez, A. (2016). What is
higher difficulty for Grade 4 to 6 students than
Scientific Research and How Can It Be Done?
for Grade 1 to 3 students for the same tests. This Turkish Journal of Anesthesiology and Reanima-
case is not relevant to the expected learning out- tion, 44(4), 212.
comes. Based on this summary, the actualization Carin, A.A., & Sund, R.B. (1989). Teaching science
of scientific method-based learning in Natural through Discovery. Columbus: Merrill Publish-
Science subjects in elementary schools should be ing Company.
investigated. Dudu, W.T. (2014). Exploring South African High
School Teachers’ Conceptions of Thenature of
ACKNOWLEDGMENTS Scientific Inquiry: A Case Study. South African
Journal of Education, 34(1),1-19. Retrieved from:
http://www.sajournalofeducation.co.za
The researchers thank the Directora- Garrison, M. J. (2009). A Measure of Failure: The Po-
te of Research and Community Service of the litical Origins of Standardized Testing. Albany:
Ministry of Research, Technology, and Higher SUNY Press.
Education which has supported the researchers Gronlund, N.E. (1998). Assessment of Student Mastery
by providing financial support so that this study (9th ed.). Boston: Allyn and Bacon.
could be carried out. Gronlund, N.E. & Linn, R.L. (1990). Measurement and
Evaluation in Teaching (6th ed.). New York: Mac-
REFERENCES Millan Publishing Company.
Hairida & Junanto, T. (2018). The Effectiveness of Per-
formance Assessment in Project-Based Learn-
Adams, R.J. & Kho, Seik-Tom. (1996). Acer Quest Ver-
ing by Utilizing Local Potential to Increase Sci-
sion 2.1. Camberwell, Victoria: The Australian
ence Literacy. International Journal of Pedagogy
Council for Educational Research.
and Teacher Education, 2, 151-162.
Anderson, L.W. &Krathwohl, D.R. (2001). A Tax-
Le, Dai-Trang. (2013). Applying Item Response Theory
onomy of Learning, Teaching, and Assessing: A
Modeling in Educational Research. (Doctoral Dis-
Revision of Bloom’s Taxonomy of Educational Ob-
sertations, Iowa State University). Retrieved
jectives. A Bright Edition. New York: Addison
50 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51

from: http://lib.dr.iastate.edu/etd Assessing Science Process Skills (3rd ed.) Iowa:

LeBoffe, M. & Wisehart, G. (1989). Study Guide Biol- Kendall/Hunt Publishing Company.
ogy: Exploring Life. New York: John Wiley & Roid, G.H. & Haladyna, Th.M. (1982). A technology
Sons. for Test-Item Writing. Orlando: Academic Press,
Lederman, N. G., Lederman, J. S., & Antink, A. Inc.
(2013). Nature of Science and Scientific In- Shete, A. N., Kausar, A., Lakhkar, K., & Khan, S. T.
quiry as Contexts for the Learning of Science (2015). Item Analysis: An Evaluation of Multi-
and Achievement of Scientific Literacy. Inter- ple-Choice Questions in Physiology Examina-
national Journal of Education in Mathematics, Sci- tion. J Contemp Med Edu, 3(3), 106-109.
ence and Technology, 1(3), 138-147. Stage, C. (2003). Classical Test Theory or Item Re-
Luo, S.& Zhang, X. (2011). Multiple-choice Item and sponse Theory: The Swedish Experience (PDF
Its Backwash Effect on Language Teaching in file). Retrieved from: https://www.umu.se/
China. Theory and Practice in Language Studies, globalassets/organisation/fakulteter/samfak/
1(4), 423-425. institutionen-for-tillampad-utbildningsvetens-
McInerney, J. (1986). Curriculum Development at the kap/hogskoleprovet/publications/60581_em-
Biological Science Curriculum Study. Educa- no-42.pdf
tional Leadership: Journal of the Association for Subali, B. (2009). Pengukuran Keterampilan Proses Sains
Supervision and Curriculum Development, 44(4), Pola Divergen dalam Mata Pelajaran Biologi SMA
24-28. di Provinsi DIY dan Jawa Tengah. (Unpublished
Millard, S. (2012). Writing Multiple Choice and True/ dissertation). Yogyakarta State University, Yo-
False Exam Questions. A Good Practice Guide. gyakarta, Indonesia.
(Lecture Notes). Retrieved from: https://doc- Subali, B., & Mariyam, S. (2013). Pengembangan
player.net/50079039-Writing-multiple-choice- Kreativitas Keterampilan Proses Sains dalam
and-true-false-exam-questions.html Aspek Kehidupan Organisme Pada Mata Pela-
Miller, P.W. (2008). Measurement and Teaching. Mun- jaran IPA SD. Cakrawala Pendidikan,  3(3),365-
ster: Patric W. Miller & Associates. 381.
Miller, M.C.D., Montplaisir, L.M., Offerdahl, E.G., Sumantri, M. S., & Satriani, R. (2016). The Effect of
Cheng, F-Ch., & Gerald L., Ketterling, G.L. Formative Testing and Self-Directed Learning
(2010). Comparison of Views of the Nature on Mathematics Learning Outcomes. Interna-
of Science between Natural Science and Non- tional Electronic
science Majors. CBE—Life Sciences Education, 9, Tang, X., Coffey, J. E., Elby, A., & Levin, D. M.
45–54. (2010). The Scientific Method and Scientific
Northwest Evaluation Association. (2003). Idaho State Inquiry: Tensions in Teaching and Learning.
Aligned Learning Continuum Release 1.0. Nw- Science Education, 94(1), 29-47.
everest StPortland org, 4. Thorpe, G. & Favia, A. (2012). Data Analysis Us-
Nweke, C.O., Abonyi, O. S., Chinyere A. O., &Njoku, ing Item Response Theory Methodology: An
M.I.A. (2014). Effects of Experiential Teach- Introduction to Selected Programs and Ap-
ing Method on Pupils’ Achievement in Basic plications (PDF file). Retrieved from: https://
Science and Technology. International Journal of digitalcommons.library.umaine.edu/cgi/view-
Scientific & Engineering Research, 5(5), 875-881. content.cgi?article=1019&context=psy_facpub
Olatoye, R.A.and Adekoya, Y.M. (2010). Effect of Tsybulsky, D. (2019). Students Meet Authentic Sci-
Project-Based, Demonstration and Lecture ence: The Valence and Foci of Experiences
Teaching Strategies on Senior Secondary Stu- Reported by High-School Biology Students
dent achievement in an Aspect of Agricultural Regarding Their Participation in a Science
Science. International Journal of Educational Re- Outreach Programme. International Journal of
search and Technology, 1(1), 19-29. Science Education, 41(5), 567-585.
Pada, A.U.T, Kartowagiran, B., & Subali, B. (2016). Ulu, M. (2017). Errors Made by Elementary Fourth
A Separation Index and Fit Items of Creative Grade Students When Modelling Word Prob-
Thinking Skills Assessment. Research and Evalu- lems and the Elimination of Those Errors
ation in Education 2(1), 1-12. Retrieved from: through Scaffolding. International Electronic
http://journal.uny.ac.id/index.php/reid Journal of Elementary Education, 9(3), 553-580.
Petrillo, J., Cano, S.J., McLeod, L.D., & Coon, Ch.D. Watts, M., Bentley, D., & Hornsby, J. (1989). Learning
(2015). Using Classical Test Theory, Item Re- to Make it Your Own. In: Bentley, D. & Watts, M.
sponse Theory, and Rasch Measurement Theo- (ed). Learning & Teaching in Schools Science: Prac-
ry to Evaluate Patient-Reported Outcome Mea- tical Alternative. Philadelphia: Open University
sures: A comparison of Worked Examples. Press.
Value in Health, 18, 25 -34. Wenning, C.J. (2010). Levels of Inquiry: Using Inquiry
Popham, W.J. (2005). Classroom Assessment: What Teach- Spectrum Learning Sequences to Teach Sci-
ers Need to Know (4th ed.). Boston: Pearson Edu- ence (Shaded Sections Added January 2012). J.
cation, Inc. Phys. Tchr. Educ, 5(3), 11-20.
Rezba, R.J., Sparague, C.S., Fiel, R.L., Funk, H.J., Wright, B.D. (1999). Rasch Measurement Model. In:
Okey, J.R., & Jaus, H.H. (2007). Learning and Masters, G.N. & Keeves, J.P. (1999). Advances in
 B. Subali, Kumaidi, N. S. Aminah, B. Sumintono / JPII 8 (1) (2019) 39-51
51

Measurement in Educational Research and Assess- Zoghi, M., & Valipour, V. (2014). A Comparative Study
ment. Amsterdam: Pergamon, An imprint of of Classical Test Theory and Item Response
Elsevier Science. Theory in Estimating Test Item Parameters in
Wright & Masters, G.N. (1982). Rating Scale Analysis. A Linguistics Test. Indian Journal of Fundamen-
Chicago: Mesa Press. tal and Applied Life Sciences, 4(4), 424-435.
Zeidan, A. H., & Jayosi, M. R. (2015). Science Pro-
cess Skills and Attitudes toward Science among
Palestinian Secondary School Students. World
Journal of Education, 5(1), 13-24.