Research Methods, Design,

and Analysis
Eleventh Edition

CHAPTER 5
Measuring
Variables and
Sampling

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Variable and Measurement

• Variable – a condition or characteristic that

can take on different values or categories
• Measurement – the assignment of
symbols or numbers to something
according to a set of rules

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Scales of Measurement

• Nominal Scale
– use of symbols to classify or categorize
– e.g., using numbers to categorize gender
• Ordinal Scale
– rank-order scale of measurement
– e.g., finishing order in a race
– equal distances on scale not necessarily
equal on dimension being measured

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Scales of Measurement (cont'd)

• Interval Scale
– same properties of ordinal plus equal
distances between adjacent numbers
– e.g., temperature on Fahrenheit scale
• Ratio Scale
– highest scale of measurement
– same properties of other scales plus absolute
zero point
– e.g., weight, height
Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Psychometric Properties of Good
Measurement
• Reliability – refers to the consistency or
stability of the scores of your
measurement instrument
• Validity – refers to the extent to which your
measurement procedure is measuring
what you think it is measuring and whether
you have interpreted your scores correctly

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Psychometric Properties of Good
Measurement (cont'd)
• A measure must be reliable in order to be
valid but a reliable measure is not
necessarily valid

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Types of Reliability

• Test-Retest Reliability
– consistency of individual scores over time
– same test administered to individuals two
times
– correlate scores to determine reliability
– how long to wait between tests?

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Types of Reliability (cont'd)

• Equivalent-Forms Reliability
– consistency of scores on two versions of test
– each version of test given to different groups
of individuals

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Types of Reliability (cont'd)

• Internal Consistency Reliability

– consistency with which items on a test
measure a single construct
– involves comparing individual items within a
single test
– coefficient alpha is common index

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Types of Reliability (cont'd)

• Interrater Reliability
– degree of agreement between two or more
observers
– interobserver agreement is the percentage of
times different raters agree

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Validity

• Validity refers to the accuracy of the

inferences, interpretations, or actions
made on the basis of test scores.
• Involves the measurement of constructs
(e.g., intelligence or happiness)
• Do operational definitions accurately
represent construct we are interested in?

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Methods Used to Collect
Evidence of Validity
• Content-Related Evidence (content
validity)
– validity assessed by experts
 do items appear to measure construct of interest?
 were any important content areas omitted?
 were any unnecessary items included?

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Methods Used to Collect
Evidence of Validity (cont'd)
• Evidence Based on Internal Structure
– how well do individual items relate to the
overall test score or other items on the test
– factor analysis – statistical procedure used to
determine the number of dimensions present
in a set of items

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Methods Used to Collect
Evidence of Validity (cont'd)
• Evidence Based on Relations to Other
Variables
– criterion-related validity
 predictive validity – using scores obtained at one
time to predict the scores on a criterion at a later
time
 concurrent validity – degree to which scores
obtained at one time correctly relate to the scores
on a known criterion obtained at the same time

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Methods Used to Collect
Evidence of Validity (cont'd)
• Evidence Based on Relations to Other
Variables
– convergent validity – extent to which test
scores relate to other measures of the same
construct
– discriminant validity – extent to which your
test scores do not relate to other test scores
measuring different constructs

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Methods Used to Collect
Evidence of Validity (cont'd)
• Evidence Based on Relations to Other
Variables
– known groups validity evidence – extent to
which groups that are known to different from
one another actually differ on the construct
being developed

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Sampling Methods

• Sample – a set of elements selected from

a population
• Population – the full set of elements from
which the sample was selected

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Sampling Methods (cont'd)

• Sampling – process of drawing elements

from population to form a sample
– representative sample
– equal probability method of selection method
(EPSEM)
• Statistic – a numerical characteristic of
sample data

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Sampling Methods (cont'd)

• Parameter – a numerical characteristic of

population data
• Sampling error – the difference between
the value of the sample statistic and the
value of the population parameter
• Sampling frame – a list of all the elements
in a population

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Random Sampling Techniques

• Simple Random Sampling

– choosing a sample in a manner in which
everyone has an equal chance of being
selected
– sampling “without replacement” is preferred
– random numbers generators simplify the
process

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Random Sampling Techniques
(cont'd)
• Stratified Random Sampling
– random samples drawn from different groups
or strata within the population
 proportional stratified sampling involves insuring
that each subgroup in sample is proportional to the
subgroups in the population

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Random Sampling Techniques
(cont'd)
• Cluster Random Sampling
– involves random selection of groups of
individuals (clusters)
– one-stage cluster sampling involves randomly
selection clusters and using all individuals
within
– two-stage cluster involves randomly choosing
individuals within each chosen cluster

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Random Sampling Techniques
(cont'd)
• Systematic Sampling
– Involves three steps
 determine the sampling interval (k) – population
size divided by desired sample size
 randomly select a number between 1 and k, and
include that person in your sample
 also include each kth element in your sample

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Nonrandom Sampling
Techniques
• Convenience Sampling – using research
participants that are readily available –
e.g., college students
• Quota Sampling – identifying quotas for
individual groups and then using
convenience sampling to select
participants within each group

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Nonrandom Sampling
Techniques (cont'd)
• Purposive Sampling – involves identifying
a group of individuals with specific
characteristics – e.g., college freshmen
who have been diagnosed with ADHD
• Snowball Sampling – technique in which
research participants identify other
potential participants.
– particularly useful in identifying participants
from a difficult to find population
Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Random Selection and Random
Assignment
• Random selection involves selecting
participants for research
– purpose is to obtain a representative sample
• Random assignment involves how
participants are assigned to conditions
within the research
– purpose is to create equivalent groups to
allow for investigation of causality

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
 Determining Sample Size

• If less than 100 use entire population

• Larger sample sizes make it easier to
detect an effect or relationship in the
population
• Compare to other research studies in area
• Larger sample sizes are needed if
population is
– heterogeneous

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Determining Sample Size (cont'd)

• Larger sample sizes are needed if

population is
– heterogeneous
– you have multiple groups
– if you want increased precision
– when you expect a small effect

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Determining Sample Size (cont'd)

• Larger sample sizes are needed if

population is
– when you use less efficient methods of
sampling
– for some statistical techniques
– if you expect a low response rate

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Sampling in Qualitative Research

• Qualitative research focuses on in-depth

study of one or a few cases.
• Several different sampling methods are
available. It is common to mix several
different methods.

Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.
Research Methods, Design, and Analysis, Eleventh Edition Copyright ©2011 by Pearson Education, Inc.
Christensen • Johnson • Turner All rights reserved.