Social Science Research Methods
3. Data Collection
3.1 Survey Research
Key Topics
1. Questionnaire Surveys
2. Interview Survey
3. Biases in Survey Research
4. Choosing a Survey Approach
Survey Research
• Survey research is a method involving the
use of standardized questionnaires or
interviews to collect data about people and
their preferences, thoughts, and behaviors
in a systematic manner
Survey Advantages
1. Widely adaptable: preferences, traits,
attitudes, beliefs, behaviors, and factual
information (income, gender, etc.)
2. Suited for remotely collecting data about a
population too large to observe directly
3. Questionnaire surveys are unobtrusive
Survey Advantages
4. Interviews reach hard to reach populations
5. Large sample surveys may allow the
detection of small effects across a range of
variables and subpopulations
6. Economical (time, effort and cost)
Survey Disadvantages
1. Potential errors, which cause misleading
results
2. Subject to a large number of biases such
as non-response bias, sampling bias,
social desirability bias, and recall bias
10–7
1. Questionnaire Surveys
• A questionnaire is a research instrument
consisting of a set of questions (items)
intended to capture responses from
respondents in a standardized manner
• May be unstructured or structured
10–8
1. Questionnaire Surveys
• Unstructured questions ask respondents
to provide a response in their own words
• Structured questions ask respondents to
select an answer from a given set of
choices
• Responses on a structured questionnaire
may be aggregated into a composite scale
or index for statistical analysis
10–9
1. Questionnaire Surveys
• Self-administered mail surveys
• Group-administered questionnaire
• Online or web survey
10–10
1. Questionnaire Surveys
• Constructing a survey is an art
• Numerous decisions must be made about
the content of questions, their wording,
format, and sequencing, all of which can
have important consequences for the
survey responses
10–11
1. Questionnaire Surveys
• Response formats: survey questions may
be structured or unstructured. Responses
are captured using one of the following
response formats:
1. Dichotomous response: respondents are
asked to select one of two possible
choices, such as true/false, yes/no, or
agree/disagree
10–12
1. Questionnaire Surveys
2. Nominal response: respondents are
presented with more than two unordered
options
3. Ordinal response: respondents have more
than two ordered options
4. Interval-level response: respondents are
presented with a 5-point or 7-point scale
10–13
1. Questionnaire Surveys
5. Continuous response, respondents enter a
continuous (ratio-scaled) value with a
meaningful zero point, such as their age
10–14
1. Questionnaire Surveys
• Question content and wording
1. Is the question clear and understandable?
2. Is the question negatively worded?
– Should your local government not raise
taxes?
– Confusing and lead to inaccurate responses
10–15
1. Questionnaire Surveys
3. Is the question ambiguous?
– What is your annual income?
– Different interpretation by different respondents
will lead to incomparable responses that
cannot be interpreted correctly
4. Does the question have biased words?
– “Halting rising crime rates” or “law
enforcement”
– “assistance for the poor” or “welfare”
10–16
1. Questionnaire Surveys
5. Is the question double-barreled?
– Are you satisfied with the hardware and
software provided for your work?
– Does your family favor Manchester City FC?
6. Is the question too general?
– How well do you like this book?
– Better: Would you recommend this book to a
friend?
10–17
1. Questionnaire Surveys
7. Is the question too detailed?
– Do you need the age of each child in a
household or is just the number of children in
the household?
– Better to err on the side of details than
generality
8. Is the question presumptuous?
– What do you see are the benefits of a tax cut?
– Avoid questions with built-in presumptions
10–18
1. Questionnaire Surveys
9. Is the question imaginary?
– If you had a million dollars, how would you
spend it?
– Imaginary questions have imaginary answers
10.Do respondents have the information
needed to correctly answer the question?
10–19
1. Questionnaire Surveys
• Question sequencing
• In general, questions should flow logically
from one to the next
1. Start with non-threatening questions that
are easy to answer
– age, gender, education level, employee count,
annual revenues, industry
10–20
1. Questionnaire Surveys
2. Never start with an open ended question
3. Ask about one topic at a time
– When switching topics use a transition
4. Use filter questions
– “If you answered yes to question 3, proceed to
section 2”
1. Questionnaire Surveys
1. People’s time is valuable
– No more than 10-15 minutes
2. Assure confidentiality
– Tell them how data will be used
3. Thank respondents for their time
4. Pretest, pretest, pretest
10–22
2. Interview Survey
• Interviews are a more personalized form of
data collection method than questionnaires,
and are conducted by trained interviewers
using standardized set of questions
– The interviewer has the opportunity to clarify
any issues raised by the respondent or ask
probing or follow-up questions
– Interviews are time consuming and resource-
intensive
10–23
2. Interview Survey
• Face-to-face interview: the interviewer
works directly with the respondent to ask
questions and record their responses
• Focus group: a small group of respondents
(usually 6-10 respondents) are interviewed
together in a common location
10–24
2. Interview Survey
• Telephone interviews: interviewers contact
potential respondents over the phone,
typically based on a random selection of
people from a telephone directory
10–25
3. Biases in Survey Research
• Non-response bias. Survey research is
notorious for its low response rates
– A response rate of 15-20% is typical in a mail
survey, even after two or three reminders
• Example: dissatisfied customers tend to be
more vocal about their experience than
satisfied customers and are more likely to
respond to questionnaire surveys
10–26
3. Biases in Survey Research
• Sampling bias: surveys conducted by
calling a random sample of publicly
available telephone numbers will exclude
people with unlisted telephone numbers,
mobile phone numbers, and people who are
unable to answer the phone
10–27
3. Biases in Survey Research
• Social desirability bias: respondents may
avoid negative opinions or embarrassing
comments about themselves, their
employers, family, or friends
10–28
3. Biases in Survey Research
• Recall bias. Responses to survey
questions often depend on subjects’
motivation, memory, and ability to respond.
• When dealing with events that happened in
the distant past, respondents may not
adequately remember their own motivations
or behaviors
7–29
4. Choosing a Survey Approach
1. Is the assistance of the interviewer needed?
2. Are respondents interested in the issues
being investigated?
3. Will cooperation be easily attained?
4. How quickly is the information needed?
5. Will the study require a long and complex
questionnaire?
6. How large is the budget?
