Bachelor of Social Sciences (Honours)

in Psychology

Research Methods in Psychology

Lecture 3
Survey Research

(Shaughnessy, J. J., Zechmeister, E. B., & Zechmeister, J. S., 2014)

 Survey Research
Survey is used to
 Describe people’s opinions, attitudes and
preferences
 Make predictions about behavior

Survey can be
 Specific and limited in scope, or
 More global in their goals
 Survey research involves selecting a sample (or
samples) to represent a population and
administering questionnaire with a
predetermined set of questions
 Correlational Research
 Assess relationships among naturally occurring
variables, e.g. attitudes, preferences, personality
traits, feelings, age, sex
 Correlation coefficients (r)
- strength and direction of predictive relationship
between two variables
- perfect positive correlation (r = +1.00)
- perfect negative correlation (r = -1.00)
- no correlation (r = 0)
 Obtaining a Sample
 Researchers are not interested simply in the
responses of those who complete a survey, but
they seek to describe the larger population from
which the survey was drawn.
 Careful selection of a sample allows researchers
to generalize findings from the sample to the
population.
 Basic Terms of Sampling
Population
 Set of all cases of interest, e.g. current students
at EdUHK

Sampling frame
 List of the members of a population, e.g.
registrar’s list of currently registered students

Sample
 Subset of the population used to represent the
entire population, e.g. students in this class as a
sample of all students at EdUHK
 Goals of Sampling
 Sample should represent the population
 Characteristics of participants in sample should
be similar to those of the entire population
 E.g. Which sample represents a population that
is 50% Bachelor students, 30% Master students,
20% Doctoral students?

Sample A Sample B
50 Bachelor students, 100 Bachelor students,
30 Master students, 60 Master students,
20 Doctoral students 40 Doctoral students
 Sampling
 “Sampling” refers to procedures used to obtain a
sample
 Two basic approaches
- probability sampling
- nonprobability sampling
 Probability Sampling
 The probability that any given individual will
appear in the sample is known

Some examples of probability sampling:

 Simple random sampling
 Systematic sampling
 Stratified random sampling
 Cluster sampling
 Simple Random Sampling
 All members of population have an equal
chance of being selected
 It is appropriate when the population is
homogeneous and sampling frame is available
 Steps:
- draw up the sampling frame
- choose the required number of units by
random number tables or random numbers
generated by computer programme
 Systematic Sampling
 Sampling units selected at regular intervals
 Arrange sampling units in some kind of
sequence
 Decide fraction of population to be chosen
 Choose a random starting point
 Select at regular intervals
 Assume no underlying patterns in sampling
frame
 Stratified Random Sampling
 Divide population into subpopulations, called
strata
 Draw random samples from the strata; best to
select samples proportional to the strata size

Example:
 Population: 60 Females vs 40 Males
 Sampling fraction: one-fourth
 Sample: 15 Females vs 10 Males
 Cluster Sampling
 A cluster may be a school, a class
 Taking a simple random sample of the clusters,
even if the size of the clusters is different
 All units in the selected clusters are included in
the sample
 Cluster sampling is preferable when:
- It is ethically or logistically inappropriate to
sample within cluster
- The primary sampling units are widely
dispersed
 Nonprobability Sampling
 No guarantee each member of population has
an equal chance to be in sample
 Convenience sampling – researcher selects
individuals who are available and willing to
respond to the survey
 Biased Samples
 A biased sample occurs when characteristics of
the sample differ systematically from those of
the population
 Samples can overrepresent or underrepresent a
segment of a population
 Two sources
- Selection bias
- Response rate bias
 Survey Methods
 Some methods for obtaining survey data
- mail surveys
- personal interviews
- telephone interviews
- internet surveys
 Each method has advantages and
disadvantages
 Choose method based on research question
Mail surveys
 Quick, convenient, self-administered, best for
highly personal or embarrassing topics
 Problem of low response rate
 Due to response-rate bias, the final sample may
not be representative of the population
 Little control over how people respond to the
questions
Personal interviews
 Researchers gain more control over how survey
is administered
 Interviewers can seek clarification of answers
and ask questions
 Potential problem of interviewer bias
 Interviews are costly; interviewers must be
highly motivated, carefully trained and
supervised
Telephone interviews
 Complete brief surveys efficiently and with
greater access to population
 Random-digit dialing to select random samples
 Supervise interviewers easily
 Problems of selection bias, low response rate
and interviewer bias
Internet surveys
 Efficient, low cost, potential for very large
samples
 Samples can be very diverse and access
typically underrepresented sample
 Potential problems of selection bias, response
rate bias and lack of control over the research
environment
Ways to increase response rate
 Questionnaire has a “personal touch”
 Responding requires minimal effort
 Topic of survey is interesting to respondents
 Respondents identify with organization or
sponsor of survey
 Survey Research Designs
 Cross-sectional design
 Successive independent samples design
 Longitudinal design
 Cross-Sectional Survey Design
Characteristics
 One or more samples are drawn from the
population at one time

Goals
 To describe the population at that time, make
predictions about the population

Problem
 Cannot assess change over time
Successive Independent Samples Design
Characteristics
 A series of cross-sectional surveys are drawn
from the population over time; different
representative samples are selected each time

Goals
 To describe changes in population over time
Problem
 Cannot determine whether individuals change
over time
 Noncomparable samples
 Longitudinal Survey Design
Characteristics
 The same sample of individuals is surveyed
more than once

Goals
 To describe changes in individuals over time
Problem
 Attrition
 Reactivity
 Cannot tell why people change over time (only
correlations)
 Test Year

2016 2017 2018 2019

2007 9 10 11 12

2008 8 9 10 11
Birth
Year
2009 7 8 9 10

2010 6 7 8 9
 Questionnaires
 Most frequently used to collect survey data
 Measure different types of variables
- demographic variables
- preferences and attitudes
 Self-report scales
 Respond using rating scales
 All measures must be reliable and valid
 Reliability refers to consistency of measurement
- test-retest reliability
 Ways to improve reliability
- more items
- greater variability among individuals on the
factor being measured
- testing situation free of distractions
- clear instructions
 Validity refers to the truthfulness of a measure.
A valid measure assesses what it is intended to
measure.
- construct validity
 Establishing construct validity
- convergent validity: extent to which two
measures of the same construct are correlated
- discriminant validity: extent to which two
measures of different constructs are not
correlated
Which correlation indicates test-retest reliability?
Which correlations indicate convergent validity?
Which correlations indicate discriminant validity?
 Constructing a Questionnaire
 Best choice for selecting a measure
- use measure already shown to be reliable and
valid in previous research
 If no suitable measure is found
- create a questionnaire or measure
 Creating a reliable and valid questionnaire is not
easy
Important steps
 Decide what information should be sought
 Decide what type of questionnaire should be
used
 Write the first draft of the questionnaire
 Have experts review questionnaire and then
revise it based on their suggestions
 Pretest the questionnaire using sample and
conditions similar to the planned administration
of the survey
 Review results and edit the questionnaire
 Establish reliability and validity
 Writing Survey Questions
 Choose how participants will respond
- free-response (open-ended): greater flexibility
in responses but difficult to code
- closed-response (rating scale, multiple-choice,
true-false): responses are quick and easy, easy
to score, but may not accurately describe
individuals’ responses
 Use simple, familiar vocabulary; keep questions
short
 Write clear and specific questions
- avoid double-barreled questions
- avoid double negatives
- avoid leading questions
- avoid loaded (emotion-laded) questions
- avoid biased questions
 Beware of social desirability tendency
 Correlation and Causality
 Correlation does not imply causality

Example
 Correlation between being socially active (A)
and life satisfaction (B)
 Three possible causal relationships
- A causes B
- B causes A
- Variable C causes both A and B (e.g. number
of friends) = spurious relationship
 Path Analysis
 Statistical procedure to tease apart complex
correlational relationships among variables
 Mediators: variables used to explain a
correlation between two variables
 Moderators: variables that affect direction or
strength of correlation between two variables
Example of path analysis (Evans et al., 2005)

Path A represents the direct relationship between

poverty and psychological distress.
Paths B & C represent the indirect relationship
between poverty and psychological distress.

“Chaos” mediates the relationship between

poverty and psychological distress.
 If the relationship between poverty and
psychological distress are not observed for all
people, then a moderator variable may exist.
 Moderator affects the direction and strength of
relationship.
 Possible moderators:
- sex (relationship may exist for boys but not
girls)
- population density (relationship may exist in
urban but not rural areas)
- personality features (relationship may exist for
low-resilient but not high-resilient people)
 Reference
 Shaughnessy, J. J., Zechmeister, E. B., &
Zechmeister, J. S. (2014). Research methods in
psychology (10th ed.). New York: McGraw-Hill.
(Chapter 5)