DATA COLLECTION AND ANALYSIS

Data Collection is an important aspect of any type of research study. Inaccurate data
collection can impact the results of a study and ultimately lead to invalid results. Data
collection is the process of gathering and measuring information on variables of interest, in
an established systematic fashion that enables one to answer stated research questions, test
hypotheses, and evaluate outcomes. The data collection component of research is common to
all fields of study including physical and social sciences, humanities, business, etc. While
methods vary by discipline, the emphasis on ensuring accurate and honest collection remains
the same.

Data collection methods for impact evaluation vary along a continuum. At one end of this
continuum are quantitative methods and at the other end of the continuum are Qualitative
methods for data collection.

In collecting the data, the researcher must decide:

1. Which data to collect

2. How to collect the data
3. Who will collect the data
4. When to collect the data

The selection of the chosen data collection method should be based on the following:

1. The identified hypothesis or research problem.

2. The research design
3. The information gathered about the variables being studied.

The methods of data collection vary according to:

1. The degree of structure

2. Degree of quantifiability
3. Degree of obtrusive (conspicuousness, un-mistakeableness)
4. Degree of objectivity

Different ways of collecting evaluation data are useful for different purposes, and each has
advantages and disadvantages. Various factors will influence your choice of a data collection
method: the questions you want to investigate, resources available to you, your timeline, and
more. (http://dmc.umn.edu/evaluation/data.shtml

The Quantitative data collection methods,

They rely on random sampling and structured data collection instruments that fit diverse
experiences into predetermined response categories. They produce results that are easy to
summarize, compare, and generalize.

Quantitative research is concerned with testing hypotheses derived from theory and/or being able
to estimate the size of a phenomenon of interest. Depending on the research question, participants
may be randomly assigned to different treatments. If this is not feasible, the researcher may collect
data on participant and situational characteristics in order to statistically control for their influence
on the dependent, or outcome, variable. If the intent is to generalize from the research participants
to a larger population, the researcher will employ probability sampling to select participants.

Typical quantitative data gathering strategies include:

 Experiments/clinical trials.
 Observing and recording well-defined events (e.g., counting the number of patients waiting
in emergency at specified times of the day).
 Obtaining relevant data from management information systems.
 Administering surveys with closed-ended questions (e.g., face-to face and telephone
interviews, questionnaires etc). (http://www.achrn.org/quantitative_methods.htm)

Qualitative data collection methods

They play an important role in impact evaluation by providing information useful to

understand the processes behind observed results and assess changes in people’s perceptions
of their well-being. Furthermore qualitative methods can beused to improve the quality of
survey-based quantitative evaluations by helping generate evaluation hypothesis;
strengthening the design of survey questionnaires and expanding or clarifying quantitative
evaluation findings. These methods are characterized by the following attributes:

 they tend to be open-ended and have less structured protocols (i.e., researchers may change
the data collection strategy by adding, refining, or dropping techniques or informants)
 they rely more heavily on iteractive interviews; respondents may be interviewed several
times to follow up on a particular issue, clarify concepts or check the reliability of data
 they use triangulation to increase the credibility of their findings (i.e., researchers rely on
multiple data collection methods to check the authenticity of their results)
 generally their findings are not generalizable to any specific population, rather each case
study produces a single piece of evidence that can be used to seek general patterns among
different studies of the same issue

Regardless of the kinds of data involved, data collection in a qualitative study takes a great
deal of time. The researcher needs to record any potentially useful data thoroughly,
accurately, and systematically, using field notes, sketches, audiotapes, photographs and other
suitable means. The data collection methods must observe the ethical principles of research.

The qualitative methods most commonly used in evaluation can be classified in three broad
categories:

 indepth interview
 observation methods
 document review

The following link provides more information on the above three methods.

http://www.worldbank.org/poverty/impact/methods/qualitative.htm#indepth

The Research Instrument

The research instrument or a tool is described as a device used to collect the data. It facilitates
variable observation and measurement. The type of instrument suitable for use by the
researcher depends on the data collection method selected. Instrument selection and or
development require a high degree of research expertise because the instrument must be
reliable and valid.

Ways of searching for Research Instrument

1. Read professional journals so as to learn (i) what kind of instruments are being used
for similar studies (ii) their format (iii) their style (iv) how they are used by the
writers.
2. Read books that provide a description or an actual copy of various instruments for the
reader.
3. Talk with other researchers who may know of certain tools they have developed for
themselves or who may have used tools developed by others.
4. Combine or adapt one or more tools used by other researchers.
5. Develop his/her own instrument to fulfil a specific need

Guidelines for developing an instrument

1. The instrument must be suitable for its function

2. The research tool will only be effective only as it relates to its particular purpose.
3. The instrument must be based on the theoretical framework selected for the study.
4. The instrument must be valid
5. The content of the instrument must be appropriate to test the hypothesis or answer the
question being studied.
6. The instrument should be reliable
7. The devised research tool should provide comparable data every time the subject uses
the instrument.
8. An instrument should include an item that directly asks questions on the hypothesis.
9. The research tool should be designed and constructed in such a way that cheating is
minimized.
10. The instrument should be free of bias
11. A good instrument is free of build-in clues.
12. The instrument should not contain measures that function as hints for desired
responses.
13. The researcher may need to read extensively to identify which aspects of the theory
are appropriate for investigation.
14. The researcher through the instrument must be able to gather data that are appropriate
in order to test the hypothesis or to answer the question under investigation.
15. The researcher should gather a group of items from such sources such as persons
knowledgeable in the field, accepted theories or hypothesis, personal experience, or
materials from studies reported in books and professional journals.
16. The response given by the respondent in the research study should solely be his/her
own. There should be no contamination through outside influences such as someone
 else’s ideas or products. Therefore the respondent who agrees to participate in a study
is responsible for supplying information or for exhibiting behaviour that is truly
his/her own.

Types of Research Instruments

1. Questionnaire
2. Check list
3. Distribution
4. Interview
5. Observation
6. Records
7. Experimental Approach
8. Survey Approach

QUESTIONNAIRE

Questionnaire is a series of questions designed to elicit information which is filled in by

all participants in the sample. This can be gathered either by oral interview or by written
questionnaire. Questionnaire is the most common type of research instrument.

Paper-pencil-questionnaires can be sent to a large number of people and saves the

researcher time and money.People are more truthful while responding to the questionnaires
regarding controversial issues in particular due to the fact that their responses are anonymous.
But they also have drawbacks .Majority of the people who receive questionnaires don't return
them and those who do might not be representative of the originally selected sample.(Leedy
and Ormrod, 2001)

Web based questionnaires : A new and inevitably growing methodology is the use of
Internet based research. This would mean receiving an e-mail on which you would click on
an address that would take you to a secure web-site to fill in a questionnaire. This type of
research is often quicker and less detailed.Some disadvantages of this method include the
exclusion of people who do not have a computer or are unable to access a computer.Also the
validity of such surveys are in question as people might be in a hurry to complete it and so
might not give accurate responses.
(http://www.statcan.ca/english/edu/power/ch2/methods/methods.htm)

Questionnaires often make use of Checklist and rating scales.These devices help simplify and
quantify people's behaviors and attitudes. A checklist is a list of behaviors, characteristics, or
other entities that the researcher is looking for. Either the researcher or survey participant
simply checks whether each item on the list is observed, present or true or vice versa. A
rating scale is more useful when a behavior needs to be evaluated on a continuum. They are
also known as Likert scales. (Leedy and Ormrod, 2001)

Advantages of Questionnaire

1. Relatively simple methods of collecting or obtaining data.

 2. Consumes less time
3. Allows researcher to collect data from a widely scattered sample.

Disadvantages of Questionnaire

1. Responses to a Questionnaire lack depth.

2. Respondent may omit or disregard any item he/she chooses.
3. Some items may force the respondent to select responses that are not his/her actual
choice.
4. Length of the questionnaire is limited according to the respondent’s interest.
5. Printing may be costly especially if it is lengthy.
6. Data are limited to the information that is voluntarily supplied by the respondents.
7. Some items may be misunderstood.
8. The sample is limited to those who are literate.

Techniques for developing Questionnaires

1. Researcher may read literature about the topic, look through available questionnaire
or obtain help from experts.
2. Open ended questions are preferable than closed ended questions since they reflect
respondent’s attitudes and feelings which are expressed in his own words.
3. If yes – no questions are used, additional information may be gained by leaving space
for respondent’s own ideas.
4. The possibility of a middle ground statement is also important.
5. Every item on the questionnaire should relate to the topic under study.

Criteria of a good Questionnaire

1. Clarity of Language. It must meet the level of understanding of the respondents in

order to generate needed responses.
2. Singleness of objective. An item must have one and only one answer.
3. One-to-One correspondence. The questionnaire as a whole must correspond with the
objectives of the study.
4. Correct Grammar, Spelling and Construction. The questionnaire must be constructed
by observing grammatically correct sentences, correctly spelled words, coherence in
construction of sentences, etc.

Characteristics of a good Questionnaire.

1. It has a well stated title.

2. It has a statement of purpose
3. It assures the respondents about the confidentiality of responses.
4. It is designed to achieve the objective of the study.
5. It has a clear direction
6. There are no double-negative questions.
7. It avoids double-barrels questions.
8. The design corresponds to an easy tabulation of data.
Degree of structure questions

1. Open-Ended Questions: This gives the respondents the ability to respond to the
questions in their own words.
2. Closed-Ended Questions: This allows the subject to choose one of the given
alternatives.

Specific types of Closed ended questions.

1. Dichotomous questions. This requires the respondents to make a choice between two
responses such as yes/no, male/female, married or unmarried, etc.
Example
(i) Have you travelled outside Nigeria (a) yes (b) No
2. Multiple questions. This offers the respondents more than one choice.
Example
(i) How favourable is it to you to become pregnant at this time? (a) Very
favourable (b) favourable (c) Not sure (d) unfavourable (e) Very unfavourable.
3. Cafeteria Questions. This is a special type of multiple choice question where the
respondents are required to select a response according to their own point of view.
Example
People have different views on ―family planning. Which of the following best
represents your view?
(a) Family planning is necessary to quality life.
(b) Family planning is immoral and should be totally banned.
(c) Family planning has undesirable side effects that suggest need for caution.
(d) Family planning has beneficial effects that merit its practice.
(e) Family planning is moral and should be practiced.
4. Rank-Order Questions. The respondents are asked to choose a response from the
―most‖ to the ―least‖.
Example
Why must family planning be practiced? Rank your answer from the 1-most
reasonable to 5-least reasonable?
(a) It limits maternal disabilities.
(b) Gives parents more time to meet family needs.
(c) Helps maintain financial viability of the family.
(d) Afford more working hours for couples.
(e) Ensures family capabilities to educate all the children in the future.
5. Rating Questions. The respondents are asked to judge something along an ordered
dimension. This is typically bipolar in nature with the end points specifying the
opposite extremes of a continuum.
Example:
On the scale of 1-5 where 1 means strongly disagree and 5 means strongly agree, the
health centre in Landmark University provides you with the best of health services.
Scale
5. Strongly agree 4. Agree. 3. Uncertain. 2. Disagree 1. Strongly disagree.
 CHECK LIST.

Check list are items that comprise several questions on a topic and require the same response
format.

Example: here are some characteristics of birth control devices that are of varying importance
to different people. How important are these in choosing a birth control method?

Characteristics of birth Of very great Of great Of some Of no

control devices importance importance importance importance
1. Comfort
2. Cost
3. Ease of use
4. Effectiveness
5. Non-interference
of spontaneity
6. Safety
7. Safety to partner

INTERVIEW

This involves either structured or unstructured verbal communication between the researcher
and the subject during which information is obtained for a study. In Quantitative research
(survey research), interviews are more structured than in Qualitative research.
(http://www.stat.ncsu.edu/info/srms/survpamphlet.html

Types of Interviews

1. Unstructured Interviews.

These are more conversational, very long (sometimes hours) and are conducted in a usual
situation. It allows the researcher flexibility in questioning the subject.

2. Structured Interviews.

These always operate within formal written instrument referred to as interview schedule.
The researcher designed the questions to be asked and their order prior to the interview.
The questions are asked orally either face-to-face or telephone format.

In a structured interview, the researcher asks a standard set of questions and nothing
more.(Leedy and Ormrod, 2001)

Face -to -face interviews have a distinct advantage of enabling the researcher to establish
rapport with potential participants and therefor gain their cooperation. These interviews yield
highest response rates in survey research. They also allow the researcher to clarify ambiguous
answers and when appropriate, seek follow-up information. Disadvantages include
impractical when large samples are involved time consuming and expensive.(Leedy and
Ormrod, 2001)
Telephone interviews are less time consuming and less expensive and the researcher has
ready access to anyone on the planet who has a telephone. Disadvantages are that the
response rate is not as high as the face-to- face interview but cosiderably higher than the
mailed questionnaire. The sample may be biased to the extent that people without phones are
part of the population about whom the researcher wants to draw inferences.

Computer Assisted Personal Interviewing (CAPI): is a form of personal interviewing, but instead of
completing a questionnaire, the interviewer brings along a laptop or hand-held computer to enter
the information directly into the database. This method saves time involved in processing the data,
as well as saving the interviewer from carrying around hundreds of questionnaires. However, this
type of data collection method can be expensive to set up and requires that interviewers have
computer and typing skills.

Advantages of Interview

1. Data from interview are usable.

2. Depth of response can be assured.
3. In an exploratory study, the interview technique provides basis for the formation of
questionnaire.
4. Clarification is possible.
5. No items are overlooked.
6. Higher proportion of responses is obtained.
7. Greater amount of flexibility is allowed.

Disadvantages of Interview

1. It consumes time
2. Biases may result
3. It is usually costly

OBSERVATION

Observation is most commonly used in qualitative research. In observation the researcher

usually has some prior knowledge about the behaviour or occurrence of interest.

Types of Observation:

1. Unstructured Observation. This is a method of collecting research data that has both
opponents and proponents.
2. Structured Observation. This involves preparation of record keeping forms such as
category systems, check lists and rating scales.

Advantages of observation.

1. Produces large quantities of data with relative ease.

2. All data obtained from observation are usable.
3. Relatively inexpensive.
4. All subjects are potential respondents.
5. Subjects are usually available.
 6. The observation technique can be stopped or begun at any time.
7. Observations may be recorded at the time they occur thus eliminating the possibility
of bias due to recall.

Disadvantages of observation.

1. Accurate prediction of a situation or event to be observed is unlikely to occur.

2. Interviewing selected subjects may provide more information, economically than
waiting for the spontaneous occurrence of the situation.
3. The presence of an observer gives the subject a quality normally absent.
4. Observed events are subject to biases.
5. Extensive training is usually needed.

RECORDS

A record refers to all the numbers and statistics that institutions, organizations and people
keep as a record of their activities. Sources of records include:

1. Census data
2. Educational data or records
3. Hospital/Clinic records.

Advantages of Records

1. Records are unbiased

2. Records often cover a long period of time.
3. Records are inexpensive.

Disadvantages of Records

1. All the researcher can have is what has been recorded. If the record is incomplete,
there is no way it can be completed.
2. No one can be sure of the conditions under which the records were collected.
3. There is no assurance of the accuracy of the records.

EXPERIMENTAL APPROACH

This is a powerful design for testing hypothesis of causal relationships among variables. In
experimental approach, the researcher controls the independent variable and watches the
effect on the dependent variable. There are two groups of experimental approach:

1. Treatment/Experimental group
2. Control group.

Disadvantages of Experimental Approach

1. It is difficult to minimize all the variables in which the control and experimental
groups might differ.
2. Causal relationships are difficult if not impossible to establish.
3. The time element may confound the results of experimental research.
 4. In an experimental Laboratory setting, it may be difficult to obtain subjects, especially
subjects who are unaware of the experiment.

SURVEY APPROACH

We have two types of survey approach:

1. Non experimental type in which the researcher investigates the community or a group
of people.
2. Formulative or explorative research

Advantages of survey Approach

1. It can provide information about the possibilities of undertaking different types of

research methods.
2. It provides data about the present.
3. It has a high degree of representativeness.
4. It is quite easy to get respondents who are willing to supply information by expressing
their true reaction to the questions.

Disadvantages of Survey approach

1. Yields a low degree of control over extraneous variables.

2. Verbal behaviour is quite unreliable especially when it involves people.

MEASUREMENT OF VARIABLES

Measurement is the process by which the researcher assigns specific number to the collected
data.

Levels of Measurement.

1. Nominal level (The lowest level). This level includes assignment of numbers simply
to classify characteristics into categories. It usually involves assigning a code to a
label. For example in relation to marital status, 0 might represent single and 1 might
represent married. Nominal level variables include sex, marital status, health status,
etc.
2. Ordinal level (the second lowest level). This level permits the sorting of objects on the
basis of their standing on an attribute relative to each other. Data are categorised and
ranked order from ―most‖ to ―least‖ according to frequency of occurrence. It
represents the rank order 1st, 2nd, 3rd, etc. Different scales used in ordinal level are: (i)
Likert scale (ii) Guttman scale (iii) Graphic rating scale (iv) Semantic differential
scale.
3.