O F T E S T S A ND T E S T I NG

P SY 112: PSYCHOLOGICAL TESTING & ASSESSMENT

 ASSUMPTIONS ABOUT
PSYCHOLOGICAL TESTING
AND ASSESSMENT
ASSUMPTION
“PSYCHOLOGICAL TRAITS
AND STATES EXIST”
A S S UMP T I ON 1. PSYCHOLOGICAL
T R A I T S A N D S TAT E S E X I S T

TRAIT – any distinguishable, relatively

enduring way in which one individual varies
from another (Guilford, 1959, p.6)

STATE – also distinguish one person from

another but are relatively less enduring
(Chaplin, et al., 1988)
 PSYCHOLOGICAL TRAITS COVERS A WIDE
RANGE OF POSSIBLE CHARACTERISTICS

CONSTRUCT – informed, OVERT BEHAVIOR –

scientific concept developed observable action or the product
or constructed to describe or of an observable action, including
explain behavior test – or assessment-related
responses.
The definition of traits and states we are
using also refer to “a way in which one
individual varies from another ”.
 ASSUMPTION
“PSYCHOLOGICAL TRAITS AND
STATES CAN BE QUANTIFIED
AND MEASURED”

“Whatever exists at all exists in some amount.” (E.L. Thorndike, 1918)

A S S UMP T I ON 2. PSYCHOLOGICAL TRAITS AND
S TAT E S C A N B E Q U A N T I F I E D A N D M E A S U R E D

Having acknowledged that psychological traits and states do exist, the

specific traits and states to be measured and quantified need to be
carefully defined.

Measuring traits and states by means of a test entails developing not

only appropriate test items but also appropriate ways to score the test
and interpret the results.

Item Content is the key to INSIGHT.

A S S UMP T I ON 2. PSYCHOLOGICAL TRAITS AND
S TAT E S C A N B E Q UA N T I F I E D A N D M E A S U R E D

• Domain Sampling refers to either:

(1) a sample of behaviors from all possible behaviors that could
conceivably be indicative of a particular construct.
(2) a sample of test items from all possible items that could conceivably be
used to measure a particular construct.

• Cumulative Scoring - a trait is measured by a series of test items.

Each response to a test item is converted to a number according to a test
“key”. The magnitude of the trait is assumed to correspond in some way
to the sum of the keyed responses.
 ASSUMPTION
“TEST-RELATED BEHAVIOR
PREDICTS NON-TEST-
RELATED BEHAVIOR ”

• One objective of tests is to provide some indication of

other aspects of the examinee’s behavior.
A S S UM P T I O N 3 . T E S T - R E L AT E D B E H AV I O R
P R E D I C T S N O N - T E S T - R E L AT E D B E H AV I O R

Obtained sample test behavior is In some forensic matters, psychological tests may
typically used to make be used not to predict behavior but to postdict it –
predictions about future aid in understanding of behavior that has already
behavior. taken place.
 ASSUMPTION
“ TESTS AND
OTHER
MEASUREMENT TECHNIQUES
HAVE STRENGTHS AND
WEAKNESSES”

• All Tests Have Limits and Imperfection

A S S UMP T I ON 4. TESTS AND OTHER MEASUREMENT
T E C H N I Q U E S H AV E S T R E N G T H S A N D W E A K N E S S E S

Competent test users understand the limitations

of the test

They understand, how a test was developed, the

circumstances under which it is appropriate to
administer the test
ASSUMPTION
“ VARIOUS
SOURCES OF
ERROR ARE PART OF THE
ASSESSMENT PROCESS ”
A S S U M P T I O N 5 . VA R I O U S S O U R C E S O F
E R RO R A R E PA RT O F T H E A S S E S S M E N T
P RO C E S S

Error – long standing assumption that factors other than what

a test attempts to measure will influence performance on the
test. Test scores are always subject to questions about the
degree to which the measurement process includes error.

Error variance – component of the test score attribute to

sources other than the trait or ability measure.
 S O U RC E S O F E RRO R VA RI A N C E

ASSESSEE

ASSESSOR

MEASURING INSTRUMENT
A S S U M P T I O N 5 . VA R I O U S S O U R C E S O F
E R RO R A R E PA RT O F T H E A S S E S S M E N T
P RO C E S S

• Classical or true score theory – assumption is made that

each testtaker has a true score on a test that would be
obtained but for the action of measurement error.

X =T + E

• The observed score X is related to the true score T and

the measurement error score E.
ASSUMPTION
“ TESTING AND
ASSESSMENT
CAN BE CONDUCTED IN A FAIR
AND UNBIASED MANNER ”
A S S UMP T I ON 6. TESTING AND ASSESSMENT
CAN BE CO N DUC TED IN A FAIR AN D
UNBIASED MANNER

More controversial than the remaining six assumptions

Decades of court challenges to various tests and testing

programs have alerted test developers and users to the
societal demand for fair tests used in a fair manner
A S S UMP T I ON 6. TESTING AND ASSESSMENT
CAN BE CONDUCTED IN A FAIR AND
UNBIASED MANNER

Sophisticated procedures and ethical guidelines have been set to

ensure test fairness.

One source of fairness-related problems is the test user who attempts

to use a particular test with people whose background and experience
are different from people for whom the test was intended.

“Tests are tools, and just like other tools, they can be used
properly or improperly.”
ASSUMPTION
“ TESTING AND
ASSESSMENT BENEFIT
SOCIETY”
A S S UMP T I ON 7. TESTING AND
ASSESSMENT BENEFIT SOCIETY

1 2
The prospect of a world without Considering the many critical
testing and assessment might seem decisions that are based on testing
appealing, yet a world without and assessment procedures, we
tests would most likely be more a can readily appreciate the need
nightmare than a dream. for tests, especially good tests.
W H AT I S A “ G OOD T E S T ” ?
OT H E R
C O N S I D E RA T I O N S

• A good test is one that trained

examiners can administer, score, and
interpret with a minimum of
difficulty. A good test is useful test,
one that yields actionable results that
will ultimately benefit individual test
takers or society at large.
 NORMS

• NORM-REFERENCED
TESTING AND ASSESSMENT –
method of evaluation and a way of
deriving meaning from test scores by
evaluating an individual test taker’s
score and comparing it to the scores
of group test takers.
• NORMS – test performance data of
a particular of test takers that are
designed for use as reference when
evaluating or interpreting individual
test scores.
NORMS…

NORMATIVE SAMPLE – group of people whose

performance on a particular test is analyzed for reference in
evaluating the performance of individual test takers

NORMING – process of deriving norms

Understanding how norms are derived through “formal

sampling methods” requires some discussion of the process of
sampling.
 SAMPLING TO
DEVELOP NORMS

STANDARDIZATION – process of
administering a test to a representative sample of
test takers for the purpose of establishing norms.

SAMPLING – process of selecting a portion of

the universe deemed to be representatives of the
whole population .

SAMPLE OF A POPULATION – portion of

the universe of people deemed to be a
representative of the whole population.
STRATIFIED SAMPLING – process of developing a
sample based on specific subgroups of a population

STRATIFIED-RANDOM SAMPLING – process of developing a

sample based on specific subgroups of a population in which every
member has the same chance of being included in the sample

PURPOSIVE SAMPLE – arbitrarily select some sample

because it is believed to be representative of the
population

INCIDENTAL SAMPLE/CONVENIENCE SAMPLE –

involves the sample being drawn from that part of the population
that is close to hand
DEVELOPING NORMS FOR A
S TA N DA R D I Z E D T E S T

In order to best assist future users of the test, test developers are encouraged to
“provide information to support recommended interpretations of the results,
including the nature of the content, norms or comparison groups, and other
technical evidence” (Code of Fair Testing Practices in Education, 2004, p. 4).

When the people in the normative sample are the same people on whom the test
was standardized, the phrases normative sample and standardization sample are
often used interchangeably. Increasingly, however, new norms for standardized
tests for specific groups of test takers are developed some time after the original
standardization.

That is, the test remains standardized based on data from the
original standardization sample; it’s just that new normative data are
developed based on an administration of the test to a new
normative sample.
TYPES OF NORMS

PERCENTILES AGE NORMS GRADE NORMS

TYPES OF NORMS

NATIONAL SUBGROUP LOCAL NORMS

NORMS NORMS
 FIXED
REFERENCE
G RO U P
SCORING
SYSTEMS
F I X E D R E F E R E N C E G RO U P S C O R I N G
SYSTEMS

Here, the distribution of scores

Norms provide a context for
obtained on the test from one
interpreting the meaning of a test
group of testtakers—referred to as
score. Another type of aid in
the fixed reference group—is
providing a context for
used as the basis for the
interpretation is termed a fixed
calculation of test scores for
reference group scoring system.
future administrations of the test.
N O RM - RE F E RE N C E D V E R S U S
C RI T E RI O N - RE F E RE N C E D
E VA L U AT I O N
One way to derive meaning from Derive meaning from a test
a test score is to evaluate the test score is to evaluate it on the
score in relation to other scores basis of whether or not some
on the same test. criterion has been met.
– norm reference – Criterion-reference
C RI T E RI O N - RE F E RE N C E D
T E S T I NG A ND A S S E S S ME NT

• May be defined as a method of evaluation and

a way of deriving meaning from test scores by
evaluating an individual’s score with reference
to a set standard.
EXAMPLE OF CRITERION REFERENCE

• To be eligible for a high-school diploma,

students must demonstrate at least a
sixth-grade reading level.

• To earn the privilege of driving an

automobile, would-be drivers must take
a road test and demonstrate their driving
skill to the satisfaction of a state-
appointed examiner.
EXAMPLE OF CRITERION REFERENCE

• To be licensed as a psychologist, the applicant

must achieve a score that meets or exceeds the
score mandated by the state on the licensing test.

• To conduct research using human subjects, many

universities and other organizations require
researchers to successfully complete an online
course that presents testtakers with ethics-
oriented information in a series of modules,
followed by a set of forced-choice questions.
C U L T U RE A N D I N F E RE N C E
C U LT U R E A N D I N F E R E N C E

In interpreting data from

So, in selecting a test for use, the
psychological tests, it is frequently
responsible test user does some
helpful to know about the culture
advance research on the test’s
of the testtaker, including
available norms to check on how
something about the era or
appropriate they are for use with
“times” that the testtaker
the targeted testtaker population.
experienced.
C U LT U R A L LY I N F O R M E D A S S E S S M E N T :
SOME “DO’S AND DON’TS”
B E AWA R E O F T H E
C U LT U R A L
ASSUMPTIONS ON
WHICH A TEST IS
BASED
 C O N S I D E R C O N S U LT I N G
WITH MEMBERS OF
PA RT I C U L A R C U LT U R A L
COMMUNITIES REGARDING
T H E A P P RO P R I AT E N E S S O F
PA RT I C U L A R
ASSESSMENT TECHNIQUES,
TESTS, OR TEST ITEMS
 STRIVE TO
I N C O R P O R AT E
ASSESSMENT METHODS
T H AT C O M P L E M E N T
T H E WO R L DV I E W A N D
LIFESTYLE OF ASSESSEES
W H O C O M E F RO M A
S P E C I F I C C U LT U R A L A N D
L I N G U I S T I C P O P U L AT I O N
 B E K N OW L E D G E A B L E
ABOUT THE MANY
A LT E R N AT I V E T E S T S O R
MEASUREMENT
P RO C E D U R E S T H AT M AY
BE USED TO FULFILL THE
ASSESSMENT OBJECTIVES
 B E AWA R E O F
E Q U I VA L E N C E I S S U E S
A C RO S S C U LT U R E S ,
INCLUDING
E Q U I VA L E N C E O F
L A N G UAG E U S E D A N D
T H E C O N S T RU C T S
MEASURED
 S C O R E , I N T E R P R E T, A N D
A N A LY Z E A S S E S S M E N T
DATA I N I T S C U LT U R A L
CONTEXT WITH DUE
C O N S I D E R AT I O N O F
C U LT U R A L H Y P O T H E S E S A S
P O S S I B L E E X P L A N AT I O N S
FOR FINDINGS
 TA K E F O R G R A N T E D
T H AT A T E S T I S B A S E D
O N A S S U M P T I O N S T H AT
I M PA C T A L L G RO U P S
IN MUCH THE SAME
WAY
TA K E F OR G RA NT ED T HAT
MEMBERS OF ALL
C U LT U R A L C O M M U N I T I E S
WILL
AU TO M AT I C A L LY D E E M
PA RT I C U L A R T E C H N I Q U E S,
TESTS, OR TEST
I T E M S A P P RO P R I AT E F O R
USE
TA K E A “ O N E - S I Z E - F I T S -
A L L” V I E W O F
ASSESSMENT WHEN IT
COMES
T O E VA L UAT I O N O F
P E R S O N S F RO M VA R I O U S
C U LT U R A L A N D
LINGUISTIC
P O P U L AT I O N S
SELECT TESTS OR OTHER
TOOLS OF ASSESSMENT
WITH LITTLE OR NO
REGARD FOR THE EXTENT
TO WHICH SUCH TOOLS
A R E A P P RO P R I AT E
FOR USE WITH A
PA RT I C U L A R A S S E S S E E .
S I M P LY A S S U M E T H AT A
T E S T T H AT H A S B E E N
T R A N S L AT E D I N TO
A N OT H ER L A N G UAG E I S
AU T O M AT I C A L LY
E Q U I VA L E N T I N E V E RY
WAY
TO THE ORIGINAL
 SCORE,
I N T E R P R E T, A N D
A N A LY Z E
ASSESSMENT IN A
C U LT U R A L VA C U U M