Reliability and its Types

How we measure reliability of a test ?

what factors influences reliability?
(By ALINA).
Meaning of Reliability

• The word "reliability" is derived from the Latin words

• - "re" meaning "again"

• "liabilis" meaning "liable" or "trustworthy"

• - "-ity" suffix indicating a state or quality

• So, the word "reliability" literally means "the state of being liable or trustworthy again and again", or "the quality
of being consistently trustworthy". In other words, something that is reliable can be trusted to perform or produce
consistent
Definition:

• Reliability: The ability of something to consistently perform or produce the same result, without errors or
inconsistencies.

• In other words, reliability means that something is:

• Consistent
• Dependable
• Trustworthy
• Accurate
• Stable
Types of Reliability

• Test Retest reliability

• Inter rater Reliability

• Internal consistency Reliability

• Split half

• Cronbach's alpha

• Parallel reliability
Test retest reliability.
Test-retest reliability is a type of reliability where the same test is administered to the same group at different time
intervals. If the results are similar or close to each other, the test is considered reliable. If there is a significant
difference in the results, it indicates that the test may not be reliable. This concept is applied to things that do not
change quickly, such as IQ levels, but not to factors that change frequently, like mood swings.

• Consider an IQ test designed to measure intelligence. To ensure this test is reliable, a group of people takes the
same IQ test twice, with a gap of three months between the tests. If the IQ scores from the first test are similar to
the scores from the second test for each individual, the test has high test-retest reliability.
Inter rater Reliability .
Inter-rater reliability measures the consistency of scores given by different raters or observers when evaluating the
same thing

• In a research study, two psychologists independently observe and rate the behavior of children in a classroom. If
both psychologists provide similar ratings for each child's behavior, this indicates high inter-rater reliability. For
instance, if Psychologist A rates a child's behavior as "4" on a scale of 1 to 5 and Psychologist B also rates the same
child's behavior as "4," their ratings are consistent, showing strong inter-rater reliability.
Internal consistency reliability.
Internal consistency reliability assesses the extent to which all items in a test measure the same concept or construct
and produce similar scores.

• For Example: Imagine a questionnaire designed to measure levels of anxiety. The questionnaire includes 10
different questions that all aim to assess various aspects of anxiety. To check internal consistency, researchers
look at how well these questions correlate with each other. If the questions are consistent, people who rate high
on one anxiety question should also rate high on other anxiety questions. For instance, if one question asks about
feeling nervous and another asks about experiencing worry, a person who strongly agrees with the first question
should also strongly agree with the second. High internal consistency reliability would be indicated by similar
responses across all anxiety-related questions, often measured using Cronbach's Alpha.
Split - half reliability.
Split-half reliability assesses the internal consistency of a test by dividing it into two halves and comparing the
results from each half. ring t

• For example: Suppose you have a 20-question math test designed to measure a student's math ability. To check
for split-half reliability, you randomly divide the test into two halves, with 10 questions in each half. If the scores
from the first half of the test are similar to the scores from the second half for each student, the test has high split-
half reliability. For instance, if a student scores 8 out of 10 on the first half and 7 out of 10 on the second half, this
indicates that the test is consistently measuring math ability.
Cronbach’s alpha.
Cronbach's alpha is a measure of internal consistency reliability used to assess the extent to which all items in a test
measure the same underlying construct.

• For Example: Let's say you have a questionnaire designed to measure job satisfaction, consisting of 15 different
statements about various aspects of job satisfaction. After administering the questionnaire to a group of
employees, you calculate Cronbach's alpha to see how well the statements correlate with each other. If
Cronbach's alpha value is high (typically above 0.70), it indicates that the items in the questionnaire are
consistently measuring job satisfaction. For example, a Cronbach's alpha of 0.80 suggests that the items in the
questionnaire are strongly related and consistently measure the construct of job satisfaction.
Parallel reliability.
Parallel reliability refers to the consistency between two different forms of a test that are intended to measure the
same construct. It involves administering two different versions of a test to the same group of individuals and then
comparing the results to see if they are consistent.

• For example: Parallel reliability is like when a teacher gives two different tests to check if they measure the same
thing. For example, if a teacher gives a math test on Monday and then a slightly different math test on Tuesday to
the same students, they can compare the results to see if both tests are consistent in measuring the students' math
skills. If the scores are similar, it shows that the tests have good parallel reliability.
Factors influencing reliability of test scores
• Extrinsic factors
Group variability
• Homogeneous examinee: lower reliability
• Heterogeneous examinee: higher reliability

• Guessing by the examinee

• Environmental conditions

• Momentary fluctuations in the examinee

Intrinsic factors.

• Length of the test

• Range of the total scores

• Homogeneity of items

• Difficulty value of items: close to 0,5 yields high reliability

• Discrimination value

• Scorer reliability
 Improving reliability of a test.
1. Standardize Test Administration

2. Increase the Number of Items

3. Use Clear and Precise Language

4. Provide Clear Scoring Guidelines

5. Pilot Test and Refine the Test

6. Increase Test Length and Avoid Overly Difficult or Easy Questions

7. Use Reliable Scoring Methods

8.Analyze and Improve Item Quality