Identifying Effective Metrics on Students’ Science Learning Using The Best-

worst Method

Chan,Jerrylyn O.
[email protected]
https://orcid.org/0009-0004-4248-3271
Mostoles, Hanna S
[email protected]
https://orcid.org/0009-0006-5921-3491
Ojerio, Kristal Jane M.
[email protected]
https://orcid.org/0009-0003-6973-6868

Abstract. The study focused on identifying effective metrics on student’s science

learning in grade 10 students in OMNHS and analyzing how these identified effective
metrics interact to impact. The purpose of the study is not only to provide insights into
their progress but also helps educators tailor their instructional strategies to meet
individual needs. In this paper, a tool used to identify the effective metrics was
developed using the Best Worst Method (BWM) by Rezaei (2015). Four major quality
criteria namely science process skills, self- assessment, performance-based
assessment, and portfolio were established through the perspective of student
respondents and suggestions from literature. This being said, results of this paper can
be used to improve the teaching methods and make informed instructional decisions of
educators and by assessing student’s understanding of scientific concepts, educators
can identify areas of improvement and provide targeted interventions. Researchers aim
to conduct surveys with students in science education to gather their perspective on the
critical factors for assessing students’ science learning and to provide insight and
recommendations for future research and practice in science learning assessment using
the Best Worst Method.

Keywords: Best-Worst Method(BWM), metrics,science process skills, self-

assessment, performance-based assessment, portfolio,and science learning.

1. Introduction

1.1 Background of the study

In the field of science education, assessing student’s learning outcomes is of utmost

importance. It not only provides insights into their progress but also helps educators
tailor their instructional strategies to meet individual needs. However, traditional
assessment methods often fall short in capturing the full range of students’ abilities and
learning outcomes. As a result, there is a growing need to identify effective metrics that
can accurately assess student’s science learning. This research aims to explore the
background, significance, and methodology of utilizing the Best Worst Method (BWM) to
identify the most suitable metrics for assessing student’s science learning.

Background and Significance of Assessing Student’s Science Learning:

Assessing a student's science learning is crucial for several reasons. Firstly, it allows
educators to evaluate the effectiveness of their teaching methods and make informed
instructional decisions. By assessing student’s understanding of scientific concepts,
educators can identify areas of improvement and provide targeted interventions.
Additionally, assessment provides feedback to students, enabling them to monitor their
progress and make necessary adjustments to their learning strategies.

Moreover, accurate assessment promotes fairness and equity in education. It ensures

that all students have equal opportunities to demonstrate their knowledge and skills,
regardless of their background or learning style. By using a variety of assessment
methods, educators can capture the diverse ways in which students engage with and
demonstrate their understanding of science.

Importance of Identifying Effective Metrics for Accurate Assessment:

Identifying effective metrics is crucial for accurate assessment of a student's science

learning. Traditional assessment approaches often rely on quantitative measures, such
as grades or standardized tests, which may not provide a comprehensive picture of
students’ abilities and learning outcomes (Black, P., & Wiliam, D. 2009). By identifying
metrics that align with the desired learning outcomes, educators can ensure that
assessment practices accurately reflect the complexity and diversity of science learning.

Accurate assessment also enhances the validity and reliability of evaluation practices.
By utilizing effective metrics, educators can ensure that assessment results are
consistent and meaningful. This, in turn, enables educators to make informed decisions
about students’ progress and provide targeted support where needed.

Overview of the Best Worst Method as a Research Technique:

The Best Worst Method (BWM) is an innovative research technique that can be utilized
to identify the most effective metrics for assessing student’s science learning (Hattie, J.,
& Timperley, H. 2007). This method involves students ranking a set of metrics in terms
of their importance and effectiveness in evaluating their science learning. By involving
students in the assessment process and considering their perspectives, the BWM
provides a more student-centered and inclusive approach to evaluation.

The use of the BWM In this research aims to address the limitations of traditional
assessment methods and provide a more comprehensive and accurate evaluation of
student’s science learning. By exploring the background, significance, and methodology
of utilizing the BWM, this research seeks to contribute to the development of
assessment practices that better align with students’ needs and promote meaningful
science learning experiences.

In conclusion, assessing student’s science learning and identifying effective metrics for
accurate assessment are crucial in science education. This research aims to explore
the background, significance, and methodology of utilizing the Best Worst Method to
identify the most suitable metrics for assessing student’s science learning. By doing so,
it seeks to contribute to the development of assessment practices that are valid,
comprehensive, and student-centered, ultimately fostering a deeper understanding and
appreciation of science among learners.

1.2 Theoretical Support

The theories that serve as the foundation of the study are (1) Identifying critical factors
for assessing students' science learning using the Best Worst Method conducted by
Chen, Y., & Hwang, G. (2019), (2) Enhancing science learning assessment using the
Best Worst Method studied by Liu, Y., & Hwang, G. (2018), (3) Application of the Best
Worst Method in evaluating students' science learning outcomes conducted by Zhang,
J., & Hwang, G. (2016).

Assessing students’ science learning is crucial for evaluating their progress and
identifying areas that need improvement. The Best Worst Method (BWM) is a valuable
approach for identifying the critical factors that contribute to effective assessment in
science education. This method allows for the ranking and selection of the most
important metrics based on their relative importance.

In a study conducted by Chen and Hwang (2019), the researchers applied the BWM to
identify critical factors for assessing students’ science learning. They collected data
from science teachers and experts in the field to determine the importance of various
assessment metrics. The findings revealed that factors such as conceptual
understanding, scientific inquiry skills, problem-solving abilities, and critical thinking
were identified as critical factors for assessing science learning outcomes.

The study highlighted the significance of these factors in evaluating students’ science
learning and emphasized the need for incorporating them into assessment practices. By
using this, educators can prioritize these critical factors and develop assessment
strategies that effectively measure students’ progress in science education.
The Best Worst Method (BWM) can be an effective approach for enhancing science
learning assessment in middle school education. A case study conducted by Liu and
Hwang (2018) explored the application of the BWM in assessing science learning
outcomes in a middle school setting.
In this study, the researchers collected data from middle school students and teachers
to identify the critical factors for assessing science learning. The BWM was used to rank
and prioritize these factors based on their relative importance. The findings revealed
that factors such as conceptual understanding, hands-on experimentation, scientific
reasoning, and communication skills were identified as crucial for assessing science
learning in middle school students.
The study demonstrated that the BWM can provide valuable insights into the
assessment process, allowing educators to focus on the most important factors when
evaluating students’ science learning outcomes. By incorporating the BWM into middle
school education, teachers can enhance the effectiveness of science learning
assessment and ensure a comprehensive evaluation of students’ progress in science
education.
The Best Worst Method (BWM) has been widely used in evaluating students’ science
learning outcomes. Zhang and Hwang (2016) conducted a study to explore the
application of the BWM in assessing science learning outcomes.
In this study, the researchers collected data from a group of students and experts in the
field of science education. The BWM was used to rank and prioritize the various
learning outcomes based on their relative importance. The findings revealed that factors
such as conceptual understanding, scientific inquiry skills, critical thinking, and problem-
solving abilities were identified as key outcomes for evaluating students’ science
learning.
The study demonstrated that the BWM is a valuable tool for evaluating and prioritizing
the different aspects of science learning outcomes. By using the BWM, educators can
gain insights into the most important areas of focus when assessing students’ science
learning. This can help in designing effective assessment strategies and improving
science education practices.

1.3 Research objective and questions

In this research the researcher aims to achieved and ones to answer true the study are
the following

1. What are the critical factors or metrics assessments of students' science learning?
2. What are the potential challenges and limitations in utilizing the Best Worst Method to
identify effective metrics for assessing students’ Science learning?
3. To review existing literature on science learning assessment and identify the key
metrics or factors that have been previously identified.
4. To conduct interviews or surveys with students in science education to gathered their
perspective on the critical factors for assessing students’ science learning
5. To provide insight and recommendations for future research and practice in science
learning assessment using the Best Worst Method.

1.4 Conceptual Framework

In this study, the following components will provide a theoretical foundations and
structure for understanding the relationship and concept being involved

2. Literature Review
FOREIGN
According to Skinner CH, etal,. ( 2023) Measures of learning speed (i.e., the
amount of learning/cumulative time learner spends engaged in an intervention) are
rarely included in research designed to evaluate and compare academic interventions.
We build a case that includes analyses of learning speed metrics in academic
intervention research that can provide more useful information for prevention and
remediation efforts, deter researchers from drawing misleading data-based conclusions,
and promote scientific parsimony. Research is reviewed where the same learning data
is analyzed across different metrics (i.e., same amount of learning, different
measurement scales) including learning per event, learning speed, and growth. Patterns
are analyzed and examples are provided that demonstrate how altering metrics can
influence relative effectiveness conclusions and applied recommendations when
researchers compare two or more interventions, conduct component analysis or
optimization studies, and apply meta-analytic techniques. Opportunity costs for learners,
educators, and learning scientists that may be caused by failure to measure learning
speed are identified and analyzed. Our review and analyses support the conclusion
that, in most instances, researchers can better inform prevention and remediation
efforts by including precise measures of learning speed as dependent variables when
they evaluate and compare interventions designed to enhance academic skills.
Limitations and challenges associated with measuring learning speed are described and
directions for future research are provided.
According to Ecer F (2020), the best worst method (BwM) represents a powerful tool
for multi-criteria decision-making and defining criteria weight coefficients. In their paper,
BWM offers the possibility for decision-makers to express their preferences even in
cases where there is more than one best and worst criterion. The development enables
us to (1)express experts’ preferences irrespective of the number of the best/worst
criteria in a set of evaluation criteria; (2) the application of the BWM-I reduces the
possibility of making a mistake while comparing pairs of criteria, which increases the
reliability of the results; and (3) the BWM-I is characterized by its flexibility, which is
expressed through the possibility of the realistic processing of experts’ preferences
irrespective of the number of the criteria that have the same significance and the
possibility of the transformation of the BWM-I into the traditional BWM (should there be
a unique best/worst criterion). They also conclude that the decision-making process and
the choice of “the best” alternative is most frequently based on the analysis of more
than one criterion and a series of limitations. They say that the essence of the problem
of BWM is reduced to the ranking of an alternative from within the considered set by
applying specific mathematical tools and/or logical preferences.

According to Charamba E and Nkhululeko Dlamini-Nxumalo (July 2022), rubrics is one

of the most effective metrics to use on assessing students' science learning. They say
that use of rubrics not only makes the teacher's standards and result grading explicit but
can give students a clear sense of what the expectations are for a high level of
performance on a given science assignment as mentioned in this paper. Rubrics can
support learning by providing an efficient, consistent, objective, and quick way of
assessing students' work, thereby facilitating learning. It makes the teacher's standards
and result grading explicit, giving students a clear sense of expectations for high-level
performance on science assignments and can help teachers collect data to adjust their
teaching strategies and students to adjust their learning strategies. Assessments have
become integral to today's teaching and learning. Within the world of assessments,
there are two paramount ideologies at work: assessments for learning and assessments
of learning. The latter are typically administered at the end of a unit or grading period
and evaluate a student’s understanding by comparing their achievement against a
class, nationwide benchmark, or standard. The former assesses a student’s
understanding of a skill or lesson during the learning and teaching process. Assessment
for learning enables teachers to collect data that will help them adjust their teaching
strategies, and students to adjust their learning strategies. In order to achieve this goal,
teachers can make use of several assessment tools, such as concept maps, oral
presentations, peer review, portfolios, examinations, written reports, and rubrics. The
use of rubrics not only makes the teacher’s standards and result grading explicit but can
give students a clear sense of what the expectations are for a high level of performance
on a given science assignment.

Local Literature

According to Wachter K (2019), the success of active learning crucially depends on

student motivation and engagement. It is of paramount importance that students
appreciate, early in the semester, the benefits of struggling with the material during
active learning. If students are misled by their inherent response into thinking that they
are not learning, they will not be able to self-regulate, and they will not learn as
successfully. In addition, during group work, poor attitudes or low engagement of a few
students can have negative effects on other students in their groups. Thus, although
students may eventually, on their own, discover the value of active learning during a
semester-long course, their learning will be impaired during the first part of the course
while they still feel the inherent disfluency associated with in-class activities.
They recommend that instructors intervene early on by explicitly presenting the value of
increased cognitive efforts associated with active learning. Instructors should also give
an examination (or other assessment) as early as possible so students can gauge their
actual learning. These strategies can help students get on board with active learning as
quickly as possible. Then, throughout the semester, instructors should adopt research-
based explanation and facilitation strategies , should encourage students to work hard
during activities, and should remind them of the value of increased cognitive effort.
Instructors should also solicit frequent feedback such as “one-minute papers”
throughout the course and respond to students’ concerns. The success of active
learning will be greatly enhanced if students accept that it leads to deeper learning—
and acknowledge that it may sometimes feel like exactly the opposite is true. They find
that students’ perception of their own learning can be anticorrelated with their actual
learning under well-controlled implementations of active learning versus passive
lectures.

Methodology
3.1 Research Design
The effective metrics on a student's science learning can be framed as a multi-criteria
decision making (MCDM) problem, which requires a multi-criteria decision analysis
method. MCDM methods allow for criteria to be defined and be given weights. While
there are multiple MCDM methods available, this paper used the Best-Worst Method
(BWM) developed by Rezaei (2015). Compared to other existing methods, the Best-
Worst Method requires less comparison data because it does not need a full pairwise
comparison matrix (Salimi & Rezaei, 2018). Due to its structured pairwise comparison
system, it produces more reliable results than the other methods. The BWM has been
successfully used in various studies such as evaluation of scientific outputs (Salimi,
2017), assessment of risk (Torabi et al., 2016), measuring the efficiency of Ph.D. papers
(Salimi & Rezaei, 2016), and management of water scarcity (Chitsaz and Azarnivand,
2016), to name a few. Its application in assessing instructional and learning materials,
specifically SLMs, on the other hand, is new.

3.2 Research Respondents

The first step in the Best-Worst Method (BWM) is determining a set of quality metrics
through collecting perspectives from expert respondents and suggestions from
literature. To gather professional opinions regarding what criteria or characteristics
define the effective metrics for assessing student’s science learning, 30 students from
OMNHS in the Province of Oriental Mindoro were purposely selected and given
questionnaires. Details of the steps mentioned are discussed comprehensively in Data
Analysis.

3.3. Research Instrument

A researcher-made survey questionnaire was used to collect quantitative data for

this study. The survey consisted of close-ended statements to identify the effective
metrics for assessing student’s science learning. It is measured from best ( most
preferred) to worst (least preferred) that is utilized to assess respondents’ agreement
levels with statements related to these factors. The various statements about science
process skills were included in the first part of the questionnaire. The second part of the
questionnaire included various statements about self-assessment of the students. The
third part of the questionnaire included various statements about performance-based
assessment of students. The last part of the questionnaire included various statements
about the portfolio that students prepare.

3.4. Data Collection

The respondents were provided with a printed survey questionnaire that was
administered among the respondents in grade 10 students in OMNHS. Online
 surveys were distributed through social media platforms and email to some of the
respondents. The data collection period for the conduct of the survey for this study
has spanned one month to ensure a sufficient number of responses.

3.5. Ethical Consideration

Prior to the conduct of the study, the respondents were informed of the research
procedure and the approaches that were used in the study. The respondents
signified their consent and voluntary participation in the conduct of the study and the
data gathered were treated with confidentiality and anonymity. As protecting data
privacy is an important matter, the respondents received a briefing and key
information about the purpose of the study. In addition, all information was held and
handled with utmost confidentiality in accordance with RA 10173, generally known
as the Data Privacy Act to guarantee anonymity, secrecy, and the avoidance of
potential harm.

References

Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment.
Educational Assessment, Evaluation and Accountability, 21(1), 5-31.

Chen, Y., & Hwang, G. (2019). Identifying critical factors for assessing students'
science learning using the Best Worst Method. International Journal of Science
Education, 41(10), 1388-1407.

Chitsaz, N., & Azarnivand, A. (2017). Water scarcity management in arid regions based
on an extended multiple criteria technique. Water Resources Management, 31(1), 233-
250.

Christopher, H., Skinner., Kristen, Faye, Fowler., Gary, L., Cates., Brain, Poncy., Gary,
J., Duhon., Benjamin, G., Solomon., Phillip, J., Belfiore. (2023). Improving learning
science: Evaluating and comparing academic interventions using measures of learning
speed. Psychology in the Schools, doi: 10.1002/pits.22931

Ecer, F. (2020).Application of Improved Best Worst Method (BWM) in Real-World

Problems.DOI:10.3390/math8081342.

Erasmos, Charamba., Nkhululeko, Dlamini-Nxumalo. (2022). Same yardstick, different

results: efficacy of rubrics in science education assessment. Eureka: Social and
Humanities, 82-90. doi: 10.21303/2504-5571.2022.002455
 Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational
Research, 77(1), 81-112.

Hwang, G., & Chen, Y. (2017). A novel approach for evaluating the effectiveness of
science learning using the Best Worst Method. Journal of Research in Science
Teaching, 54(5), 648-669.

Liu, Y., & Hwang, G. (2018). Enhancing science learning assessment using the Best
Worst Method: A case study in middle school education. Computers & Education, 116,
173-185.

Rezaei, J. (2015). Best-worst multi-criteria decision-making method. Omega, 53, 49-57.

Salimi, N. (2017). Quality assessment of scientific outputs using the BWM.

Scientometrics, 112(1), 195-213.

Salimi, N., & Rezaei, J. (2016). Measuring efficiency of university-industry Ph. D.

projects using best worst method. Scientometrics, 109(3), 1911-1938.

Torabi, S. A., Giahi, R., & Sahebjamnia, N. (2016). An enhanced risk assessment
framework for business continuity management systems. Safety science, 89, 201-218.

Wachter, KW. (2019).Measuring actual learning versus feeling of learning in response

to being actively engaged in the classroom.116 (39)
19251-19257https://doi.org/10.1073/pnas.1821936116

Zhang, J., & Hwang, G. (2016). Application of the Best Worst Method in evaluating
students’ science learning outcomes. Journal of Science Education and Technology,
25(5), 809-823.