03.1 Survey Research

Download as pdf or txt
Download as pdf or txt
You are on page 1of 29

Social Science Research Methods

3. Data Collection
3.1 Survey Research
Key Topics
1. Questionnaire Surveys

2. Interview Survey

3. Biases in Survey Research

4. Choosing a Survey Approach


Survey Research

• Survey research is a method involving the


use of standardized questionnaires or
interviews to collect data about people and
their preferences, thoughts, and behaviors
in a systematic manner
Survey Advantages

1. Widely adaptable: preferences, traits,


attitudes, beliefs, behaviors, and factual
information (income, gender, etc.)
2. Suited for remotely collecting data about a
population too large to observe directly
3. Questionnaire surveys are unobtrusive
Survey Advantages

4. Interviews reach hard to reach populations


5. Large sample surveys may allow the
detection of small effects across a range of
variables and subpopulations
6. Economical (time, effort and cost)
Survey Disadvantages

1. Potential errors, which cause misleading


results
2. Subject to a large number of biases such
as non-response bias, sampling bias,
social desirability bias, and recall bias
10–7

1. Questionnaire Surveys

• A questionnaire is a research instrument


consisting of a set of questions (items)
intended to capture responses from
respondents in a standardized manner
• May be unstructured or structured
10–8

1. Questionnaire Surveys

• Unstructured questions ask respondents


to provide a response in their own words
• Structured questions ask respondents to
select an answer from a given set of
choices
• Responses on a structured questionnaire
may be aggregated into a composite scale
or index for statistical analysis
10–9

1. Questionnaire Surveys

• Self-administered mail surveys


• Group-administered questionnaire
• Online or web survey
10–10

1. Questionnaire Surveys

• Constructing a survey is an art


• Numerous decisions must be made about
the content of questions, their wording,
format, and sequencing, all of which can
have important consequences for the
survey responses
10–11

1. Questionnaire Surveys

• Response formats: survey questions may


be structured or unstructured. Responses
are captured using one of the following
response formats:
1. Dichotomous response: respondents are
asked to select one of two possible
choices, such as true/false, yes/no, or
agree/disagree
10–12

1. Questionnaire Surveys

2. Nominal response: respondents are


presented with more than two unordered
options
3. Ordinal response: respondents have more
than two ordered options
4. Interval-level response: respondents are
presented with a 5-point or 7-point scale
10–13

1. Questionnaire Surveys

5. Continuous response, respondents enter a


continuous (ratio-scaled) value with a
meaningful zero point, such as their age
10–14

1. Questionnaire Surveys

• Question content and wording


1. Is the question clear and understandable?
2. Is the question negatively worded?
– Should your local government not raise
taxes?
– Confusing and lead to inaccurate responses
10–15

1. Questionnaire Surveys

3. Is the question ambiguous?


– What is your annual income?
– Different interpretation by different respondents
will lead to incomparable responses that
cannot be interpreted correctly
4. Does the question have biased words?
– “Halting rising crime rates” or “law
enforcement”
– “assistance for the poor” or “welfare”
10–16

1. Questionnaire Surveys

5. Is the question double-barreled?


– Are you satisfied with the hardware and
software provided for your work?
– Does your family favor Manchester City FC?
6. Is the question too general?
– How well do you like this book?
– Better: Would you recommend this book to a
friend?
10–17

1. Questionnaire Surveys

7. Is the question too detailed?


– Do you need the age of each child in a
household or is just the number of children in
the household?
– Better to err on the side of details than
generality
8. Is the question presumptuous?
– What do you see are the benefits of a tax cut?
– Avoid questions with built-in presumptions
10–18

1. Questionnaire Surveys

9. Is the question imaginary?


– If you had a million dollars, how would you
spend it?
– Imaginary questions have imaginary answers
10.Do respondents have the information
needed to correctly answer the question?
10–19

1. Questionnaire Surveys

• Question sequencing
• In general, questions should flow logically
from one to the next
1. Start with non-threatening questions that
are easy to answer
– age, gender, education level, employee count,
annual revenues, industry
10–20

1. Questionnaire Surveys

2. Never start with an open ended question


3. Ask about one topic at a time
– When switching topics use a transition
4. Use filter questions
– “If you answered yes to question 3, proceed to
section 2”
1. Questionnaire Surveys

1. People’s time is valuable


– No more than 10-15 minutes
2. Assure confidentiality
– Tell them how data will be used
3. Thank respondents for their time
4. Pretest, pretest, pretest
10–22

2. Interview Survey

• Interviews are a more personalized form of


data collection method than questionnaires,
and are conducted by trained interviewers
using standardized set of questions
– The interviewer has the opportunity to clarify
any issues raised by the respondent or ask
probing or follow-up questions
– Interviews are time consuming and resource-
intensive
10–23

2. Interview Survey

• Face-to-face interview: the interviewer


works directly with the respondent to ask
questions and record their responses
• Focus group: a small group of respondents
(usually 6-10 respondents) are interviewed
together in a common location
10–24

2. Interview Survey

• Telephone interviews: interviewers contact


potential respondents over the phone,
typically based on a random selection of
people from a telephone directory
10–25

3. Biases in Survey Research

• Non-response bias. Survey research is


notorious for its low response rates
– A response rate of 15-20% is typical in a mail
survey, even after two or three reminders
• Example: dissatisfied customers tend to be
more vocal about their experience than
satisfied customers and are more likely to
respond to questionnaire surveys
10–26

3. Biases in Survey Research

• Sampling bias: surveys conducted by


calling a random sample of publicly
available telephone numbers will exclude
people with unlisted telephone numbers,
mobile phone numbers, and people who are
unable to answer the phone
10–27

3. Biases in Survey Research

• Social desirability bias: respondents may


avoid negative opinions or embarrassing
comments about themselves, their
employers, family, or friends
10–28

3. Biases in Survey Research

• Recall bias. Responses to survey


questions often depend on subjects’
motivation, memory, and ability to respond.
• When dealing with events that happened in
the distant past, respondents may not
adequately remember their own motivations
or behaviors
7–29

4. Choosing a Survey Approach

1. Is the assistance of the interviewer needed?


2. Are respondents interested in the issues
being investigated?
3. Will cooperation be easily attained?
4. How quickly is the information needed?
5. Will the study require a long and complex
questionnaire?
6. How large is the budget?

You might also like