LESSON 6: Establishing Test Validity and Reliability

APPLY:
An English teacher administered a spelling test to 15 students. The spelling test
is composed of 10 items. Each item is encoded, wherein a correct answer is marked as
“1”, and the incorrect answer is marked as “0”. The grade in English is also provided in
the last column. The first five are words with single stresses. The recording is indicated
in the table.
Your task is to determine whether the spelling test is reliable and valid using the
data to determine the following: (1) Split-half, (2) Cronbach’s alpha, (3) Predictive
validity with the English Grades, (4) Convergent Validity of between words with the
single and two stresses, and (5) difficulty index of each item (Item Analysis).

Studen Ite Ite Ite Ite Ite Ite Ite Ite Ite Ite Englis
t no. m m m m m m m m m m h
1 2 3 4 5 6 7 8 9 10 grade
1 1 0 0 1 1 1 0 1 1 0 80
2 0 0 0 1 1 1 1 1 0 0 81
3 1 1 0 0 1 0 1 0 1 1 83
4 0 1 0 0 1 1 1 1 1 0 85
5 0 1 1 0 1 1 1 0 1 1 84
6 1 0 1 0 1 1 1 1 1 1 89
7 1 0 1 1 1 1 1 1 0 1 87
8 1 1 1 0 1 1 1 1 1 1 87
9 1 1 1 1 1 1 1 1 0 1 89
10 1 1 1 1 0 0 1 1 1 1 90
11 0 1 1 1 0 1 1 1 1 0 90
12 1 0 1 1 1 1 1 1 1 1 87
13 1 1 1 1 1 1 1 0 1 1 88
14 1 1 0 1 1 1 1 1 1 1 88
15 1 1 1 1 1 0 1 1 0 1 85

1. Split Half
Item/Students 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

2 0 0 1 1 1 0 0 1 1 1 1 0 1 1 1
4 1 1 1 1
6 1 1 1 0
8 1 1 1 1
10 0 0 1 1
Total for 3 3 3 3 4 5 4 4 5 4
 each case
Xcase =

Item/Students 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

1 1 0 1 0 0 1 1 1 1 1 0 1 1 1 1
3 0 0 0 1
5 1 1 1 1
7 0 1 1 1
9 1 0 1 0
Total for 3 2 3 5 4 4 3 5 4 4
each case
Xcase = 59

Even Total
& 6 5 9 8 114
Odd

Odd Even
X Y X2 Y2 XY
3 3 9 9 9
2 3 4 9 6

3 3 9 9 9

5 3 25 9 15
4 4 16 16 16

4 5 16 25 20

3 4 9 16 12

5 4 25 16 20
4 5 16 25 20
4 4 16 16 16
∑X=59 ∑Y= ∑X2= ∑Y2= ∑XY=

FORMULA: PEARSON r ?
N ( ∑ xy )− (∑ x ) (∑ y)
r=
√¿¿¿
 15()− (59 ) ()
r=
√¿¿¿
000−3245
r=
√ [(3645−000)] [(000−3025)]
r= ❑
√(164 )()
r= ❑
√ 22960
40
r=
151.53
r =0.2639741305∨0.26
The value of the strength of correlation is 0.26 Indicates ______________________

Spearman Brown
2r
p=
1+r
2( )
¿
1+ ¿ ¿

¿❑
❑
= 0.41
Note: 4. Determining the strength of a correlation
The strength of the correlation also indicates the strength of the reliability of the test.
This is indicated by the value of the correlation coefficient. The closer the value to
1.00 or -1.00, the stronger is the correlation. Below is the guide:
0.80-1.00 Very Strong Relationship
0.60-0.79 Strong Relationship
0.40-0.59 Substantial/marked relationship
0.2-0.39 Weak relationship
0.00-0.19 Negligible relationship

What is test reliability?

Reliability is the consistency of the responses to measure under three conditions:
1. When retested on the same person;
2. When retested on the same measure; and
 3. Similarity of responses across items that measure the same characteristic.
Another statistical analysis mentioned to determine the internal consistency of the test is
the Cronbach’s alpha. Follow the procedure to determine the internal consistency.

2. Cronbach’s Alpha

Studen Ite Ite Ite Ite Ite Ite Ite Ite Ite Ite Total for Score (Score-
t no. m1 m2 m3 m4 m5 m6 m m8 m9 m each - Mean)2
7 10 case (X) Mean

1 1 0 0 1 1 1 0 1 1 0 6 -1.6 2.56
2 0 0 0 1 1 1 1 1 0 0 5 -2.6 6.76
3 1 1 0 0 1 0 1 0 1 1
4 0 1 0 0 1 1 1 1 1 0
5 0 1 1 0 1 1 1 0 1 1
6 1 0 1 0 1 1 1 1 1 1
7 1 0 1 1 1 1 1 1 0 1
8 1 1 1 0 1 1 1 1 1 1
9 1 1 1 1 1 1 1 1 0 1
10 1 1 1 1 0 0 1 1 1 1
11 0 1 1 1 0 1 1 1 1 0
12 1 0 1 1 1 1 1 1 1 1
13 1 1 1 1 1 1 1 0 1 1
14 1 1 0 1 1 1 1 1 1 1 9 1.4 1.96
15 1 1 1 1 1 0 1 1 0 1 8 0.4 0.16
Xcase=7. ∑(Score
6 -
Mean)2
= 25.6
α = 1.8285714
2 = 25.6
Cronbach’s α = Σ(Score-Mean)
N–1 14
 Total for 11 10 10 10 13 12 14 12 11 11 Xitem=7.6
each item
(∑X)
∑X2 11 10 10 10 13 12 14 12 11 11
SD2 ∑SD2
0.2095356

0.1673294

0.0666784

0.1673294

0.2095356

0.2095356
= 1.8680863
?

?
n a−∑ a
Cronbach ’ s α =( )( )
n−1 a
1 1.8285714−1.8680863
Cronbach ’ s α=( )( )
5−1 1.8285714
1 −0.0395149
Cronbach ’ s α =( )( )
4 1.8285714

Cronbac h' s a=( 1.0714285 ) (−0.0216097 )

'
Cronbac h s a=−0.0231532∨−0.02
Interpretation
The internal consistency of the responses in the attitude toward teaching is 0.02,
Indicating _____________________________

Note: The strength of the correlation also indicates the strength of the reliability of the
test. The closer the value to 1.00 or -1.00, the stronger is the correlation. Below is the
guide:
0.80-1.00 Very Strong consistency
0.60-0.79 Strong consistency
0.40-0.59 Substantial consistency
0.2-0.39 Low consistency
0.00-0.19 Very Low consistency

A Cronbach’s alpha value of 0.60 and above indicates that the test items have strong
internal consistency.
 3. Predictive validity with the English grade.
Total for each English Grades x2 y2 xy
cases
6 80 36 6400 480
5 81 25 6561 405
6 83 36 6889 498
6 85 36 7225 510
7 84 49 7056 588

8 90 64 8100 720
7 90 49 8100 630
9 87 81 7569 783
9 88 81 7744 792
9 88 81 7744 792
8 85 64 7225 680
∑x= ∑y= ∑ x2= ∑ y2= ∑xy=

PEARSONr ?
N ( ∑ xy )− (∑ x ) (∑ y)
r=
√¿¿¿
 1 ( 970 ) −( 14 ) (123)
r=
√¿ ¿ ¿
14050−14402
r=
√ [ 130−196 ] [16795−16749]
68
r=
√(34)(206)
68
r=
√ 78664
48
r=
886.3789
r =0.7310659∨0.73

The value of a correlation coefficient does not exceed 1.00 or – 1.00 indicates
perfect correlation. High positive correlation to mean that there is
____________________ in the student English grades.

4. Convergent validity of between words with single and two stresses.

Item/ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Stu.#
1 1 0 1 0 0 1 1 1 1 1 0 1 1 1 1
2 0 0 1 1 1 0 0 1 1 1 1 0 1 1 1
 3 0 0 0 0 1 1 1 1 1 1 1 1 1 0 1
4 1 1 0 0 0 0 1 0 1 1 1 1 1 1 1
5 1 1 1 1 1 1 1 1 1 0 0 1 1 1 1

x 3 2 3 2 3 3 4 4 5 4 3 4 5 4 5

6 1 1 0 1 1 1 1 1 1 0 1 1 1 1 0
7 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1
8 1 1 0 1 0 1 1 1 1 1 1 1 0 1 1
9 1 0 1 1 1 1 0 1 0 1 1 1 1 1 0
10 0 0 1 0 1 1 1 1 1 1 0 1 1 1 1

y 3 3 3 4 4 5 4 5 4 4 4 5 4 5 3

X+y 6 5 6 6 7 8 8 9 9 8 7 9 9 9 8

No. of x y x2 y2 xy
Students
1 3 3 9 9 9
2 2 3 4 9 6
3 3 3 9 9 9
4 2 4 4 16 8
5
6
7
8
9 5 4 25 25 20
10 4 4 16 16 16
11 3 4 9 9 12
12 4 5 16 16 20
13 5 4 25 25 20
14 4 5 16 16 20
15 5 3 25 25 15
∑x= ∑y= ∑ x2= ∑ y2= ∑xy=

PEARSON r ?
N ( ∑ xy )− (∑ x ) (∑ y)
r=
√¿¿¿
5 ( 28 ) −( 4 ) (0)
r=
√¿ ¿ ¿
30−30
r=
√ [ 320−216 ][ 320−360 ]
 30
r=
√(24)(10)
30
r=
√ 2440
30
r=
16.46085
r =0.191741257∨0.2
The value of the strength of correlation is 0.2 Indicates ______________________

5. Difficulty index of each item.

Student Item Item Item Item Item Item Item Item Item Item Total Rank
no. 1 2 3 4 5 6 7 8 9 10
1 1 0 0 1 1 1 0 1 1 0 6 4
2 0 0 0 1 1 1 1 1 0 0 5 5
3 1 1 0 0 1 0 1 0 1 1 6 4
4 0 1 0 0 1 1 1 1 1 0 6 4
5 0 1 1 0 1 1 1 0 1 1 7 3
6 1 0 1 0 1 1 1 1 1 1 8 2
7 1 0 1 1 1 1 1 1 0 1 8 2
8 1 1 1 0 1 1 1 1 1 1 9 1
9 1 1 1 1 1 1 1 1 0 1 9 1
10 1 1 1 1 0 0 1 1 1 1 8 2
11 0 1 1 1 0 1 1 1 1 0 7 3
12 1 0 1 1 1 1 1 1 1 1 9 1
13 1 1 1 1 1 1 1 0 1 1 9 1
14 1 1 0 1 1 1 1 1 1 1 9 1
15 1 1 1 1 1 0 1 1 0 1 8 2
Student’s Item Item Item Item Item Item Item Item Item Item Total
Number 1 2 3 4 5 6 7 8 9 10 score
8 1 1 1 0 1 1 1 1 0 1 9
9 1 1 1 1 1 1 1 1 0 1 9
12 1 0 1 1 1 1 1 1 1 1 9 UPPER
13 1 1 1 1 1 1 1 0 1 1 9
14 1 1 0 1 1 1 1 1 1 1 9
6 1 0 1 0 1 1 1 1 1 1 8
7 1 0 1 1 1 1 1 1 0 1 8
10 1 1 1 1 0 0 1 1 1 1 8
15 1 1 1 1 1 0 1 1 0 1 8
5 0 1 1 0 1 1 1 0 1 1 7
11 0 1 1 1 0 1 1 1 1 0 7
1 1 0 0 1 1 1 0 1 1 0 6
3 1 1 0 0 1 0 1 0 1 1 6
4 0 1 0 0 1 1 1 1 1 0 6 LOWE
2 0 0 0 1 1 1 1 1 0 0 5

Student’s Item Item Item Item Item Item Item Item Item Item Total
Number 1 2 3 4 5 6 7 8 9 10 score
8 1 1 1 0 1 1 1 1 1 1 9
9 1 1 1 1 1 1 1 1 0 1 9
12 1 0 1 1 1 1 1 1 1 1 9
13 1 1 1 1 1 1 1 0 1 1 9
TOTAL 4 3 4 3 4 4 4 3 3 4
Proportio
n of high 1 0.75 1 0.75 1 1 1 0.75 0.75 1
group
(PH)
1 1 0 0 1 1 1 0 1 1 0 6
3 1 1 0 0 1 0 1 0 1 1 6
4 0 1 0 0 1 1 1 1 1 0 6
2 0 0 0 1 1 1 1 1 0 0 5
TOTAL 2 2 0 2 4 3 3 3 3 1
Proportio 0.5 0.5 0 0.5 1 0.75 0.75 0.75 0.75 0.2
n of low 5
 group
(PL)

pH + pL
item difficulty=
2
Difficulty Index Remarks
0.76 or Higher Easy item
0.25 to 0.75 Average item
0.24 or lower Difficult item

Item Item
1+ 0.5Item 0.75+0.5
Item
1+ 0 Item 0 ,75+
Item
1+ 10.5 Item1+ 0.75
Item1+ 0.75
Item 0.75+0.75
Item 0.75+0.75
¿2 ¿ ¿ ¿ ¿ ¿ ¿ 1+ 0.25
1 2 3 24
2 5 ¿ 6 7 2 8 2 9 102 ¿ ¿2
2 2

Index of 0.5 0.625 1 0.75 0.75 0.625

difficulty
Item
Average

Average

difficulty
Easy

Easy

Easy

Easy

Easy

Easy

Easy

Easy
item discrimination= pH −pL

The value is interpreted using the table:

Index discrimination Remark
0.40 and above Very good item
0.30 -0.39 Good item
0.20 -0.29 Reasonably Good item
0.10 -0.19 Marginal item
Below 0.10 Poor item

Item Item 2 Item Ite Item Item Item Item Item Item
1 3 m4 5 6 7 8 9 10
 Index

Discrimination
Good item Discrimination
0.5 = 1 - 0.5

Good item = 0.75 - 0.5

Good item = 1- 0

Good item = 0.75-0.5

0.25

Good item =1-1

Good item = 1 - 0.75

0.25

Good item = 1 - 0.75

0.25

Poor item = 0.75 - 0.75

Poor item = 0.75 - 0.75

Good item = 1 - 0.25

0.75