Quarter 2 – Module 5

Designing, Testing,
and Revising Survey
Questionnaires
This module is designed and
created to help you become
familiar with the concept of survey
questionnaire and how to design,
test, and revise questionnaire (CS-
EN11/12A-EAAP-lle-j-7).
Objectives:
By the end of this module you should be
able to:
1. define a questionnaire;
2. compare and contrast the two types of
questionnaire; and
3. design, test, and revise survey
questionnaires.
 Questionnaire: Definition,
Examples, Design and Types

By Saul McLeod, updated 2018

 A questionnaire is a research
instrument consisting of a series of
questions for the purpose of gathering
information from respondents.
Questionnaires can be thought of as a
kind of written interview. They can be
carried out face to face, by telephone,
computer or post.
 Questionnaires provide a relatively
cheap, quick and efficient way of obtaining
large amounts of information from a large
sample of people. Data can be collected
relatively quickly because the researcher
would not need to be present when the
questionnaires were completed. This
is useful for large populations when
interviews would be impractical.
 However, a problem with
questionnaires is that respondents
may lie due to social desirability. Most
people want to present a positive
image of themselves and so may lie or
bend the truth to look good, e.g.,
learners would exaggerate revision
duration.
 Questionnaires can be an effective means of
measuring the behavior, attitudes, preferences,
opinions and intentions of relatively large numbers
of subjects more cheaply and quickly than other
methods. An important distinction is between
open-ended and closed questions. Often a
questionnaire uses both open and closed questions
to collect data. This is beneficial as it means both
quantitative and qualitative data can be obtained.
 Closed Questions
Closed questions structure the answer by only
allowing responses which fit into pre-decided
categories. Data that can be placed into a category is
called nominal data. The category can be restricted
to as few as two options, i.e., dichotomous (e.g.
“yes” or “no” “male” or “female”)or include quite
complex lists of alternatives from which the
respondent can choose (e.g., polytomous).
 Closed questions can also provide ordinal data
(which can be ranked). This often involves using a
continuous rating scale to measure the strength of
attitudes or emotions. For example, strongly agree /
agree / neutral / disagree / strongly disagree /
unable to answer. Closed questions have been used
to research type A personality (e.g., Friedman &
Rosenman, 1974), and also to assess life events
which may cause stress (Holmes & Rahe, 1967), and
attachment (Fraley, Waller, & Brennan, 2000).
Strengths
•They can be economical. This means they
can provide large amounts of research data
for relatively low costs. Therefore, a large
sample size can be obtained which should be
representative of the population, which a
researcher can then generalize from.
Strengths
•The respondent provides information
which can be easily converted into
quantitative data (e.g., count the
number of 'yes' or 'no' answers),
allowing statistical analysis of the
responses.
Strengths
• The questions are standardized. All respondents
are asked exactly the same questions in the
same order. This means a questionnaire can be
replicated easily to check for reliability .
Therefore, a second researcher can use the
questionnaire to check that the results are
consistent.
Limitations

•They lack detail. Because the

responses are fixed, there is less
scope for respondents to supply
answers which reflect their true
feelings on a topic.
 Open Questions
Open questions allow people to express what they
think in their own words. Open-ended questions enable
the respondent to answer in as much detail as they like
in their own words. For example: “Can you tell me how
happy you feel right now?” If you want to gather more
in-depth answers from your respondents, then open
questions will work better. These give no pre-set answer
options and instead allow the respondents to put down
exactly what they like in their own words.
 Open questions are often
used for complex questions that
cannot be answered in a few
simple categories but require
more detail and discussion.
Strengths

•Rich qualitative data is obtained as

open questions allow the respondent
to elaborate on their answer. This
means the research can find out why
a person holds a certain attitude.
Limitations
•Time-consuming to collect the data.
It takes longer for the respondent to
complete open questions. This is a
problem as a smaller sample size
may be obtained.
Limitations
• Time-consuming to analyze the data. It
takes longer for the researcher to
analyze qualitative data as they have to
read the answers and try to put them
into categories by coding, which is
often subjective and difficult.
Limitations
• Not suitable for less educated
respondents as open questions
require superior writing skills and
a better ability to express one's
feelings verbally.
 Questionnaire Design
With some questionnaires
suffering from a response rate as low
as 5%, it is essential that a
questionnaire is well designed. There
are a number of important factors in
questionnaire design.
Aims
Make sure that all questions asked
address the aims of the research.
However, use only one feature of
the construct you are investigating
in per item.
Length
The longer the questionnaire, the
less likely people will complete it.
Questions should be short, clear, and
be to the point; any unnecessary
questions/items should be omitted.
Pilot Study
Run a small scale practice study to
ensure people understand the
questions. People will also be able
to give detailed honest feedback on
the questionnaire design.
Question Order
Questions should progress logically
from the least sensitive to the most
sensitive, from the factual and
behavioral to the cognitive, and from
the more general to the more specific.
 The researcher should ensure
that the answer to a question is
not influenced by previous
questions.
Terminology
• There should be a minimum of technical jargon. Questions
should be simple, to the point and easy to understand.
• The language of a questionnaire should be appropriate to the
vocabulary of the group of people being studied. Use
statements which are interpreted in the same way by
members of different subpopulations of the population of
interest.
• For example, the researcher must change the language of
questions to match the social background of respondents' age
/ educational level / social class / ethnicity etc.
Presentation
•Make sure it looks professional, include
clear and concise instructions. If sent
through the post make sure the envelope
does not signify ‘junk mail.’
 Ethical Issues
The researcher must ensure that the information provided
by the respondent is kept confidential, e.g., name, address, etc.
This means questionnaires are good for researching sensitive
topics as respondents will be more honest when they cannot be
identified. Keeping the questionnaire confidential should also
reduce the likelihood of any psychological harm, such as
embarrassment. Participants must provide informed consent
prior to completing the questionnaire, and must be aware that
they have the right to withdraw their information at any time
during the survey/study.
 Problems with Postal
Questionnaires
•The data might not be valid (i.e.,
truthful) as we can never be sure
that the right person actually
completed the postal questionnaire.
 Also, postal questionnaires may not be
representative of the population they are studying.
• This is because some questionnaires may be lost in
the post reducing the sample size.
• The questionnaire may be completed by someone
who is not a member of the research population.
• Those with strong views on the questionnaire’s
subject are more likely to complete it than those with
no interest in it.
 Benefits of a Pilot Study
A pilot study is a practice / small-
scale study conducted before the main
study. It allows the researcher to try out
the study with a few participants so that
adjustments can be made before the
main study, so saving time and money.
It is important to conduct a questionnaire pilot study
for the following reasons:
• Check if respondents can understand the terminology
used in the questionnaire.
• Check that emotive questions have not been used as
they make people defensive and could invalidate their
answers.
• Check that leading questions have not been used as
they could bias the respondent's answer. Ensure the
questionnaire can be completed in an appropriate time
frame.
 DESIGNING A SURVEY
QUESTIONNAIRE
Questionnaire research design proceeds in an
orderly and specific manner. Each item in the
flow chart depends upon the successful
completion of all the previous items. Therefore, it
is important not to skip a single step. Notice that
there are two feedback loops in the flow chart to
allow revisions to the process and questionnaire.
Here are the suggested steps on how to develop a
questionnaire:(Adapted from QuickMBA.com)

1. Determine the research problem or the target

information.
2. Choose the type of questions and methods of
administration (written, e-mail, telephone or
personal interview).
3. Determine the general question content needed to
obtain the desired information.
Here are the suggested steps on how to develop
a questionnaire:(Adapted from QuickMBA.com)

4. Determine the form of responses

5. Choose the exact question wording
6. Make sure that the sequence of the question
is effective.
7. Test the questionnaire and revise if needed.
Now that you know how to
design the questionnaire, are you
ready to form your questions?
Alright, let’s do this!
1. Write short and simple questions
Respondents are often unwilling to
study an item in order to understand it.
Assume that respondents will answer
the questionnaire quickly. Therefore,
provide clear, short items that will not
be misinterpreted.
Example:
Given the current trend of more hits, more
home runs, longer games in general, and
more injuries in baseball today, do you think
that steroid use should continue to be
banned even though it is not enforced?*

Problem: Long questions can be confusing

Better question:
Steroid use has both positive
and negative effects on
baseball. Do you think that
steroid use should be
banned?
2. Avoid leading questions
Wording that influences
respondents to consider a
subject in a weighted manner, or
injects a preference or opinion.
Example:
Do you hate the president of the
Philippines?”

Why is this leading? Because the

question itself includes an opinion
word
Who do you think of when you hear COVID -
19?
a. China b. Pres. Duterte c. Chinese d.
Lockdown

Why is this leading? Because it forces the

respondent to answer one of these choices,
even if none of them comes to mind.
3. Appropriately Open-Ended and Closed-
Ended Questions
Use open-ended questions when responses need to be
elaborated by the respondents for exhaustive and comprehensive
data gathering. They’re more suited to exploratory research that
looks to describe a subject based on trends and patterns.
Closed-Ended questions are popular because they provide
greater uniformity or responses and are easily processed
compared to open-ended questions. However, closed-ended
questions the response categories should be exhaustive and
mutually exclusive. In other words, all possible options should be
provided.
Example:
Why do you play sports?
1. Enjoyment 2. Health
3. Friends 4. Other-----
4. Questions must be non-threatening and
attempt to evoke the truth.
Example:
Who do you think consume more cigarettes: you or
your friends?

When a respondent is concerned about the

consequences of answering a question in a particular
manner, there is a good possibility that the answer
will not be truthful.
5. Question Clarity
Avoid ambiguities and
vague words (e.g. usual,
regular, normal)
Example:
What is your number of serving of eggs in a typical
day?

Problem: How many eggs constitute a serving? What

does ‘a typical day’ mean?

Better question:
On days you eat eggs, how many eggs do you usually
consume?
Example:
Do you watch television regularly?
*Vague questions are difficult to answer
(what is the meaning of "regularly"?)

Better question: How often do you

watch Television?"
Note: Questions should mean
the same thing to all
respondents. All the terms
should be understandable or
defined, time periods specified,
complex questions asked in
multiple stages.
6. Don’t use double-barrelled questions
Ask one question at a time. Avoid
asking 2 questions, imposing
unwarranted assumptions, or hidden
contingencies. Whenever you use
‘and’ on a question or a statement,
check if it is double-barrelled.
Example:
Do you find the classes you took
during your first semester in SHS
more demanding and interesting
than your JHS classes?
Yes No
* How would someone respond if they
felt their SHS classes were more
demanding but also more boring than
their JHS classes? Or less demanding but
more interesting? Because the question
combines “demanding” and
“interesting,” there is no way to respond
yes to one criterion but no to the other.
Do you find the classes
you took during your first
semester in SHS more
demanding than your JHS
classes?
7. Clearly define the response scale dimension
or continuum
When using a response scale,
clearly define the dimension or
continuum respondents are to use
in their rating task.
Example:
Response categories - Make them logical and
meaningful: NOT: Many......Some.......A
Few......Very Few.....None

DO a Bipolar or Unipolar rating scale: Bipolar

measures both direction and intensity of an
attitude: Unipolar scale measures one concept
with varying degrees of intensity.
8. Minimize presuppositions
An assumption about the world whose
truth is taken for granted.
Answering a question implies accepting
its presuppositions, a respondent may be
led to provide an answer even if its
presuppositions are false.
Example:
Are you a DDS or a Dilawan?

Problem: presupposes that one of

the alternatives is true.
Example:
What are your usual hours of work?”

Problem: Does respondent have usual hours

of work?

Better Question: What are your usual hours

of work, or do you not have usual hours?
Remember: Each question
should have a specific
purpose or should not be
included in the
questionnaire.
TYPE OF QUESTIONS AND ITS
USAGE
1. Open-Ended
Best Used for:
•Breaking the ice in an interview
•When respondent’s own words are important
•When the researcher does not know all the
possible answers
Example:
What changes do you recommend for the
school to do in order to help students perform
better?
_____________________________________
_____________________________________
_____________________________________
_____
2. Closed-Ended
Best Used for:
•Collecting rank ordered data
•When all response choices are known
•When quantitative statistical tool
results are desired
Example:
In which of the following do you
live?
O A house
O An apartment
O A condo unit
Other forms closed-ended questions:
a. Likert-Scale

Best Used for:

•Assessing a person’s opinion and
feelings about something.
Example:
Please circle the way you feel about the
following:
1 = Disagree 5 = Agree
b. Multiple Choice

Best Used for:

•When there are finite number
of options
Example:

Which of the following best describes your

current civil status?
Single
Married
Widowed
Divorced
c. Rating Scales

Best Used for:

•Rate things in relation to other
things
Example:
d. Ranking Questions

Best Used for:

●Ordering answer choices by way of
preference. This allows you to not only
understand how respondents feel about
each answer option, but it also helps you
understand each one’s relative popularity.
Example:
 ORDERING THE QUESTIONS

(Adapted from Contemporary

Communication Research by Smith,
M.J., 1988)
1. Adapt a general organizational pattern that
complements a survey’s research objectives.

Two general patterns:

• Funnel pattern – begins with broad questions
followed by progressively narrower or more
specific ones
• Inverted pattern – narrowly focused questions
are followed by more general ones
2. Topically related questions should be
grouped together.

A researcher should group together

questions pertinent to a single topic
then move to another topic. It is easier
for the answer questions this way.
3. Easy-to-answer questions
should be placed first. Easy
questions serve as motivation.
4. Questions should be ordered to avoid
establishing a response bias.
*Response Bias – a tendency of a
respondent to answer all closed-questions
the same way regardless of content.

Example: A respondent check “Somewhat

agree” to all criteria.
 TESTING AND REVISING
THE QUESTIONNAIRE
Have you experienced asking a dressmaker
or a tailor sew your school uniform? What does a
tailor or dressmaker usually do before finally
giving you your sewn uniform?
He or she would let you fit it first, right? Why
do you think so?
 That is the same as the questionnaire. You
are the tailor and the questionnaire is the school
uniform. You need to check if the questionnaire
fits the respondents and your target information.
No matter how carefully you design a
questionnaire, there is always the POSSIBILITY of
error. You are always certain to make some
mistake. The surest protection against such error
is to PRE-TEST the questionnaire in full or in part.
(Baxter, L. & Babbie, E., 2003)
 That is the last part of designing your
questionnaire before finally administering
and distributing it to your respondents.
There are no fixed steps on how to test
your questionnaire but here are some
general guidelines that might be helpful.
Keep in mind that you are aiming for the
questionnaire to be as effective as it can
be.
 Some Practical Tips on Testing a Questionnaire:
(Adapted from tools4dev.org)

1.Find 5 to 10 people from your target

group
2.Ask them to complete the survey while
thinking out loud.
3.Observe how they complete the survey.
4.Make improvements based on the results.
 Quick tips before you finish the
module:
All questionnaires need an introduction. Be sure to have one.
It is useful to begin every questionnaire with basic
instructions for completing it.
The format of a questionnaire is as important as the wording
and ordering. Be sure that it is spread out and uncluttered.
Physical aspects such as page layout, font type and size,
questions spacing, and the type of paper should be considered.
Always check on these.
Reference:
• https://www.studocu.com/ph/document/la-consolacion-u
niversity-philippines/philosophical-foundations-in-educati
on/eapp-q2-module-5-research/12965535