Data Collection

Data are a collection of information, evidence, or facts from which you can draw conclusions.

 Data can be quantitative – numeric information – or it can be qualitative – involving

stories, observation, etc.
 Data are important because they offer evidence for the outcomes your research is
producing.
 The data that you want to collect will be driven from the questions you want
answered.

There are a variety of different types of data collection methods. Some of the more common
types are:

Activity Logs/Skill Written documentation of participant’s

Sheets attendance, achievement or acquisition of skills.
Focus groups Moderated discussions on a particular topic or issue.
Interviews Data collection through oral conversations.
Observation Watching people engaged in activities and recording what
occurs.
Questionnaires Written responses to clearly defined questions.

Characteristics of good data

However we observe and record data, our record has to be both reliable and valid.
Reliable, or consistent, data records depend on the instrument used to collect them. Whether it
is a questionnaire, interview protocol, observation schedule, or whatever, it needs to give
consistent information every time it is used. That is, it needs to be reliable. The information
generated also must be worthwhile information for the purpose at hand – that is, it must valid.
It must inform us about the knowledge, skill or attitude of an individual, the dynamics of a
group, the size of a school system or the gross national product of a nation. It is easy to find
someone who can offer a number, but that is pointless unless the number means something in
the real world.

Collecting data can be one of the most enjoyable aspects of doing research, but unless
you know what you are doing, it can lead to research of poor quality.

1
 Questionnaires
The questionnaire has become one of the most used means of collecting information. If
well constructed, the questionnaire permits the collection of reliable and reasonably valid data
in a simple, cheap and timely manner. The decision to use a questionnaire is often motivated
by a need to collect routine data from a large number of respondents who may be in one or
several locations, for example, schools within a district. In this situation a questionnaire can
be administered to a group at one school or it may be widely dispersed throughout the district.

Basic steps in a questionnaire development

The task of developing and implementing a questionnaire can be accomplished by following

six essential steps:

Step 1: Determine your questions

To develop valid questionnaires, you must first clearly identify your general information
needs. What types of information do you require and from whom? Accurately defining your
needs is critical and will make it easier to write questions that will answer your research
questions and achieve your goals. Moreover, a clear understanding of the purpose of the
research will help you limit your questions to the relevant information and no more. In a busy
world, there is no place for questions which might be interesting but are not related to the
central theme.

Step 2: Draft the items

Once you have written your questions and sub-questions, questionnaire items will become
immediately apparent. Brainstorm as many items for each question as possible to help you
gain a full understanding of the breadth and depth of each question. Then, you can move on
and select the format(s) for the question (i.e., the question type: open questions, multiple
choice, fill in the blank, etc). There is an infinite range of question types, here are the six most
commonly used:

Fill-in-the-blank

This common format asks a question and leaves a blank for the response. The stem should be
a complete sentence rather than just a phrase. Normally the answer should not be more than
word, number or phrase. For example:

e.g.: How may management courses have you completed in the past two years? ……

(NB. The answer blank usually follows the question; it is on the same print line as the last
phrase of the question and of a length appropriate to the length of the response expected.

Multiple Choice

Multiple choice questions are similar to fill-in-the-blank questions, except the respondent is
given a choice of answers and must check one. Sometimes, there are discrete response options
(e.g. sex: male or female) in other instances a range of values is presented (e.g. annual
income: $10,000 – $50,000)

2
It is preferable to provide the response choices and this requires that you understand and be
aware of the range of possible responses. Normally four to eight defined response choices are
provided. Occasionally, a blank response option is included if you feel that the range of
responses may be larger than you have identified.

Examples:

How many days of training have you completed during the past 12 months?
- None
- 6-10
- 11-15
- 16 or more

What ways were you doing the year before you enrolled in this program?
- Full-time employment - Full-time student
- Part-time employment - part-time student
- Unemployed - other (specify) ……………

Comment-on

The comment-on question is similar in format to the fill-in-the-blank question except for one
critical difference. The comment-on question attempts to elicit an extensive answer by posing
a question and leaving enough space for the respondent to write a short paragraph. While this
format is essential for in-depth understanding, such question should not be overused. They
tend to bias the results by giving a greater weight to those who are verbally expressive and
tend to evoke responses only from those with extreme views. Furthermore, the information is
difficult to analyse and can often be obtained in other ways. Extensive use of these items is a
sign that the questionnaire writer has taken an easy route.

e.g.: To what extent did the training session relate to your job needs?

…………………………………………………………………………………………………
…………………………………………………………………………………………………

List

Asking the respondent to list things is a good way to find out views in an unbiased way. This
open format forces the respondent to think up answers without having a list of ‘acceptable’
options from which to choose. Here, the respondent is encouraged to list a specified number
of views, say three or five.

A useful variation on this type of question is to ask for the list in order of importance. This
provides additional information on the potency of each suggestion and enables you to weight
the order during the analysis. (for instance, in the following example, the most important skill
can be assigned a weight of 3, the second 2, and the third 1). Thus, in this example, first
choices are three times as potent as third choices.

3
e.g.: List the three important skills you acquired during this training session.
1. ………………………………………………………………………………………..
2. ………………………………………………………………………………………..
3. ………………………………………………………………………………………..

Likert Scale

The Likert Scale is one of the most useful question forms. The scale is named after Rensis
Likert who developed this format in 1932. In its most popular form, the respondent is
presented a sentence and is asked to agree or disagree on a three, five, or seven-point scale, as
shown in the example below.

Strongly not strongly

Disagree disagree sure agree agree

I use research methods in my job

Note that a question is not asked. Rather, a clear statement is made and the respondent is
asked to indicate whether the statement reflects his or her views.

While Likert scales can have many response points (three to seven being most common), a 5-
point scale is the most practical for most common purposes. It is easy to respond to,
straightforward to analyse, and sufficient for most needs.

Rank

In this type of question the respondent is given a list of items and asked to rank them in order
of importance (see the example below). When using rank questions one should not normally
present more than 10 or 12 items since people can become confused. If more items are needed
try modifying the question by asking people to rank the five most important from the whole
list and ignore the remainder.

e.g. Rank in order of importance the following five weaknesses of the training program. That
is, place a 1 beside the weakness you consider most important, a 2 beside the next most
important weakness and so forth, until you have ranked all five weaknesses.

Rank

The training program was too short. ………..

The content did not suit my needs. ………..
The content was too theoretical. ………..
The training group was too large. ………..
The training methods were poor. ………..

One difficulty with ranked items is the analysis since different respondents will rank
more items than will others. Thus, one person’s second choice may mean something quite
different from another’s

4
 When considering the visual presentation of the rank order question, it is imperative that
all response items are listed on one page. Do not split the list between pages or you will get
many response errors.

Step 3: Sequence the items

Good questionnaires, unless they are very short, generally contain subsections. Such
sections give structure to the questionnaire and communicate a sense of purpose and order to
the respondent. Within each section you will have to sequence the questions to accommodate
two different principles. The first principle, related to a common debate about questionnaire
writing, is whether or not to group questions on the same topic one after the other, or
disperses them throughout the questionnaires. In general, questions on the same topic are
grouped together under the same section. The second principle concerns the question of
whether questions of the same type should also be grouped. This principle is particularly
relevant for Likert scale items since one does not want to repeat the instructions too often.
Since these two principles are not always compatible, you may have to rewrite some questions
or perhaps change the sectioning of the questionnaire so that question types are not constantly
shifting back and forth.

Another sequencing consideration concerns the overall organisation of the

questionnaire. It should begin with easy, non-threatening questions. The various sections
should be organised in logical fashion generally progressing from descriptive type
information to more involved attitudes and opinions. Never start a questionnaire with an open
question that requires much writing, as this will discourage people and lower the response
rate.

Furthermore, the researcher should make sure that redundant or unnecessary questions
are removed. The questionnaire should be as short as possible to achieve its purpose.
Remember, lengthy questionnaires require lengthy analyses. Typically, questionnaires should
be limited to two or four pages unless the respondents are highly motivated, in which case up
to 16 pages are possible! What motivates a respondent? Certainly, if the respondents believe
that the results will affect them or something they believe in, they will be inclined to give in
their full attention.

Step 4: Design the questionnaire

The format of a questionnaire is extremely important because it is a major factor in

determining whether the questionnaire will be completed. An attractive, well laid-out and
easy-to-use questionnaire is taken more seriously than one which is thrown together with a
minimum of thought. There are two important aspects to design: individual questions and the
whole questionnaire. For individual questions, certain types of questionnaire layouts can
reduce confusion and contribute to valid responses. For example, response options for
multiple choice, rank or list formats should be in a single column following the question stem.
Fill-in-the-blanks or response boxes should follow the question rather than precede it; Likert
scale responses can follow the sentence being rated or appear at the right of it.

The second consideration concerns the overall questionnaire format. Some researchers
propose that a booklet format is best. This format is efficient for several reasons. First,
questionnaires printed on both sides will appear more slender and less onerous to complete
than those which are printed on one side and stapled in the corner. Second, a booklet is simple

5
to use and easy to produce. If there are more than four pages, it should be stapled in two
places along the fold in the manner of a magazine.

Unless the questionnaire is short (four pages or less), it is advisable to use the first page
as a cover. Whether or not a cover is used, the questionnaire should have a suitable title, and
there should be an introductory paragraph. The paragraph not only introduces the purpose of
the questionnaire, but it should mention that the responses are confidential, indicate that all
questions are to be completed and, in the case of mailed questionnaires, identify where to
return the completed questionnaire.

The various sections and questions should be organised to make efficient use of the
available space. Where possible, sections should be self-contained and begin at the top of a
page. Care must be taken to ensure that individual questions are not split but are intact on a
single page. Moreover, there should be a visually pleasing amount of white space. Questions
must not appear crowded. Open-ended comment questions can serve to open spaces and
number of response lies can be expanded to fill the page. Generally, questions are numbered
sequentially throughout the questionnaire;

Step 5: Pilot-test the questionnaire

It is always difficult to criticise your own written work and in developing questionnaires
it is essential to obtain comments from at least a small group of the intended respondents.
Pilot-testing will identify ambiguities in the instructions: it will help clarify the wording of
questions, and it may alert you to omissions or unanticipated answers in multiple choice or
ranking questions. The pilot-testing permits overall reactions including comments on the
length of the questionnaire.

An effective way to pilot-test a questionnaire is to assemble a group of 6 to 12

volunteers and have them complete the questionnaire individually. Encourage them to write
marginal comments on the actual questionnaire, then follow-up with a group discussion. After
the pilot-test, review the verbal and written comments, the questionnaire responses and
evaluate its effectiveness. Then, if required, revise the instrument. If major changes are
necessary a second pilot test is advisable.

Step 6: develop a strategy for Data Collection and analysis

Now you have a questionnaire ready to go! You will need to work out a strategy for how
and where to send it. Questionnaires can be printed and mailed to respondents. Respondents
fill them out and mail them back. Alternatively, it is also possible to administer them in a
group meeting with the targeted respondents. Results are manually entered into a database or
statistical program for analysis.

Conclusion

Well constructed questionnaires permit researchers to gather reasonably valid

quantitative and qualitative data in a simple, timely and cost efficient manner. By following
the above six steps you should be able to produce an effective questionnaire for your research
project.
Questionnaires are so important in educational research. Developing a questionnaire
requires thought, care and time, but the end product can be satisfying.

6
Summary: Six rules to aid questionnaire design

1. Lay out items to avoid confusion

 Use the formats shown in the examples;
 Do not change pages in the middle of a question;
 Give instructions on what you want the respondents to do for each type of question;
 Number the questions consecutively.

2. Use a booklet
 To make it professional.

3. Include a title and introductory explanation

 To let your clients know what you are doing;
 To help them fill it out properly.

4. Organise into sections, each with a title

 To help structure thinking;
 To facilitate analysis.

5. Group similar types of items together

 Especially rating scales;
 Fill-in-the-blank and multiple choice can be mixed.

6. Use all available space

 Try to limit to 4 pages;
 Use space for comments to fill in pages;
 Ensure it is visually appealing to the eye.