Research Methodology (ECEG 7093)

Research Methods and Communications (ECEG 7035)

Chapter Three

Method of Data Collection

Saturday, May 27, Haramaya Institute of Technology, SECE

2023
 What is Data?
2

 Data is a existing   information/knowledge  

 represented or  coded in some form suitable for better usage
or processing
 Data is a set of values of qualitative or quantitative variables
 Quantitative data are anything that can be expressed as a number,
or quantified

 May be represented by ordinal, interval or ratio scales and lend

themselves to most statistical manipulation

 Quantitative – Numbers, tests, counting, measuring

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 What is Data?...
3

 Qualitative data is a categorical measurement expressed by

means of a natural language description
 In statistics, it is often used interchangeably with
"categorical" data
 Qualitative – Words, images, observations, conversations,
photographs
For example: favorite color = "blue“
 Quantitative and Qualitative data can be gathered from the
same data unit depending on whether the variable of interest is
numerical or categorical

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Quantitative Vs Qualitative Data
4

Example: 
Data unit Numerical = Quantitative Categorical = Qualitative data
variable data variable

"How “How good your

many children do 2  children are in Good
you have?" education”

“What is the
A person "How much do
10,000 Birr level of your Average
you earn?"
income ?"
"How many "Do you
40 hours per
hours do you work full-time or Full-time
week
work?" part-time?"

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Primary and Secondary Data
5

 Primary Data are collected by the researcher

 Secondary data collected by someone else and have already been
passed through the statistical process
 Sources of secondary data are existing literature, Reports of
professional agencies, Departments, Archives, Internet, etc
 While collecting secondary data one has to follow legal
procedures required and maintain the academic ethics
 A researcher as per requirement of study may decide to use
 primary data, secondary data or both

 Both primary and secondary data have their own pros and cons

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Methods of Collecting Data: Rules
6

 No one best way: decision depends on:

 What you need to know: numbers or stories
 Where the data reside: environment, files, people
 Resources and time available
 Complexity of the data to be collected
 Frequency of data collection
 Intended forms of data analysis

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Methods of Collecting Data: Rules…
7

 Rule to follow
 Use multiple data collection methods
 Use available data, but need to know
 how the measures were defined
 how the data were collected and cleaned
 the extent of missing data
 how accuracy of the data was ensured

 If must collect original data

 be sensitive to burden on others
 pre-test, pre-test, pre-test
 establish procedures and follow them (protocol)
 maintain accurate records of definitions and coding
 verify accuracy of coding, data input

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Characteristics of Good Measures
8

 Is the measure relevant?

 Does the measure capture what matters?
 Do not measure what is easy instead of what is needed
 Is the measure credible?
 Is the measure believable?
 Will it be viewed as a reasonable and appropriate way to capture the
information sought?
 Is the measure valid?
 How well does the measure capture what it is supposed to?
 Example: Are waiting lists a valid measure of demand?
 Is the measure reliable?
 A measure’s precision and stability- extent to which the same result
would be obtained with repeated trials
 Example: How reliable are speeds measured by a stopwatch?

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Quantitative Approach
9

 Data in numerical form

 Data that can be precisely measured

 age, cost, length, height, area, volume, weight, speed,
time, and temperature

 Harder to develop

 Easier to analyze
Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Qualitative Approach
10

 Data that deal with description

 Data that can be observed or self-reported, but not
always precisely measured
 Less structured, easier to develop
 Can provide “rich data” — detailed and widely
applicable
 Is challenging to analyze
 Is labor intensive to collect
 Usually generates longer reports
Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Triangulation to Increase Accuracy of Data
11

 Triangulation of methods
 collection of same information using different methods

 Triangulation of sources
 collection of same information from a variety of
sources

 Triangulation of evaluators
 collection of same information from more than one
evaluator
Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Data Collection Methods
12

 Participatory Methods
 Records and Secondary Data
 Observation
 Surveys and Interviews
 Focus Groups
 Diaries, Journals, Self-reported Checklists
 Expert Judgment
 Delphi Technique and
 Others

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Participatory Methods
13

 Involve groups or communities heavily

 Examples:
 community meetings, mapping
 Community Meetings:
 One of the most common participatory methods
 Must be well organized
 agree on purpose
 establish ground rules
 who will speak

 time allotted for speakers

 format for questions and answers

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Participatory Methods…Mapping
14

 Drawing or using existing maps

 Useful tool to involve stakeholders
 increases understanding of the community
 generates discussions, verifies secondary sources of
information, perceived changes
 Types of mapping:
 natural resources, social, health, individual or civic
assets, wealth, land use, demographics

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Records and Secondary Data
15

 Examples of sources:
 files/records
 computer data bases
 industry or government reports
 other reports or prior evaluations
 census data and household survey data
 electronic mailing lists and discussion groups
 documents (budgets, organizational charts, policies and
procedures, maps, monitoring reports)
 newspapers and television reports

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Using Existing Data Sets
16

 Key issues: validity, reliability, accuracy, response

rates, data dictionaries, and missing data rates
 Advantage
 Often less expensive and faster than collecting original
data again
 Challenges
 There may be coding errors
 Data may not exactly what is needed
 You may have difficulty in getting access
 You may have to verify validity and reliability of data

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Observation
17

 See what is happening  Observation is Helpful when

 traffic patterns  need direct information
 land use patterns  trying to understand ongoing
behavior
 layout of city and rural
areas
 there is physical evidence,
products, or outputs than can
 quality of housing be observed
 condition of roads  need to provide alternative
 conditions of buildings when other data collection is
 who goes to a health infeasible or inappropriate
clinic

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Filtering Observations
18

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Degree of Structure of Observations
19

 Structured
 Determine, before the observation, precisely what will be
observed before the observation
 Unstructured
 Select the method depending upon the situation with no
pre-conceived ideas or a plan on what to observe
 Semi-structured
 A general idea of what to observe but no specific plan
 Participatory and Non Participatory

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Ways to Record Information from Observations
20

 Observation guide
 printed form with space to record

 Recording sheet or checklist

 Yes/no options, rating scales

 Field notes

 least structured, recorded in narrative,

descriptive style

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Guidelines for Planning Observations
21

 Have more than one observer, if feasible

 Train observers so they observe the same things

 Pilot test the observation data collection

instrument
 For less structured approach, have a few key

questions in mind

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Advantages/Disadvantages of Observation
22

 Advantages:
 Subjective bias eliminated
 Researcher gets current information
 Independent of Respondents
 Disadvantages:
 Expensive, Time consuming
 Limited information
 Unforeseen factors may influence observation

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Surveys and Interviews
23

 Excellent for asking people about:

 perceptions, opinions, ideas
 Less accurate for measuring behavior
 Sample should be representative of the whole
 Big problem with response rates

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Mode of Survey Administration
24

 Telephone surveys
 Self-administered questionnaires distributed by mail,
e-mail, or websites
 Mail / Phone / Internet Surveys
 Literacy issues
 Consider accessibility
 reliability of postal service
 turn-around time

 Consider bias
 What population segment has telephone access? Internet access?

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Interviews
25

 The interview method of collecting data involves presentation

of oral-verbal stimuli and reply in terms of oral-verbal
responses.
 This method can be used through personal interviews or
telephone interviews
 Could be
 Structured

 Semi-Structured or

 Unstructured

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Interviews…
26

 Often semi-structured
 Used to explore complex issues in depth
 Forgiving of mistakes: unclear questions can be clarified
during the interview and changed for subsequent interviews
 Can provide evaluators with an intuitive sense of the
situation
 Challenges
 Can be expensive, labor intensive, and time consuming
 Selective hearing on the part of the interviewer may miss
information that does not conform to pre-existing beliefs
 Cultural sensitivity: e.g., gender issues

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Types of Interviews
27

 Personal Interviews:
 Interviewer asking questions generally in a face-to-face contact to the other
person or person
 Direct personal investigation or Indirect oral investigation
 Focused Interview
 is meant to focus attention on the given experience of the respondent and its
effects
 Clinical Interview
 is concerned with broad underlying feelings or motivations or with the course
of individual’s life experience.
 Non-directive Interview
 is that where the interviewer’s function is simply to encourage the respondent
to talk about the given topic with a bare minimum of direct questioning

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Skill of Interviewer
28

 The main game in interviewing is to facilitate an

interviewee’s ability to answer
 It involves:
 easing respondents into the interview
 asking strategic questions
 prompting and probing appropriately
 keeping it moving
 winding it down when the time is right

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Merits/Demerits of Interview
29

 Merits:
 More and in depth information obtained
 Personal Information
 Greater Flexibility
 Adaptation as per the respondent
 Demerits:
 Bias of Interviewer
 Expensive/Time Consuming
 Need expertise

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Sample Interview Questions
30

1. How do management and non-management employees

interact in the office?
2. How do you know when you have done an excellent job?
3. How do non-management employees learn about
organizational change?
4. If you could change one or two things about the way
management and non-management personnel interact, what
would you change?

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Questionnaire Method
31

 A questionnaire is sent (usually by post) to persons concerned

with a request to answer the questions and return the
questionnaire.
 A questionnaire consists of a number of questions printed in a
definite order
 The respondents have to answer the questions on their own.

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Steps in questionnaire construction
32

 Preparation
 Constructing the first draft
 Self-evaluation
 External evaluation
 Revision
 Pre-test or Pilot study
 Revision
 Second pre-testing
 Preparing final draft

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Characteristics of Good Questionnaire
33

 Questionnaire should be short and simple

 Question arranged in from simple to difficult.
 Personal and intimate questions should be left to the end.
 Technical term and vague expression should be avoided.
 Questions should be answered in yes or no ; multiple choice.
 Control question to cross check the information of the
responded.

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Advantages and Disadvantages of Questionnaire
34
 Disadvantage
 Advantages
 Can be used only for educated people
 Lower cost
 respondent’s may interpret questions
 Time saving
differently
 Accessibility to widespread
respondents
 do not provide an opportunity to collect
additional information
 No interviewer’s bias
 Greater anonymity
 Researchers are not sure whether the
person to whom the questionnaire was
 Respondent’s convenience
mailed has himself answered the
 Standard wordings questions
 No Variation  Many questions remain unanswered
 The respondent can consult other
persons before filling in the
questionnaire
Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Example: Questions in a Questionnaire
35  Employee / Management Relationship Survey
Team Goals and Objectives
Unclear; diverse; conflicting 1 2 3 4 5 Clear; understood; shared by
all.
Role Clarity
Employees are unclear about their 1 2 3 4 5 Employees are clear about
roles; responsibilities and authority what is expected of them; they
are ambiguous. know their responsibilities and
authority.
Communications
Employees are guarded and 1 2 3 4 5 Employees are open and
cautious when communicating authentic when
with management communicating with
management
Decision Making
Little opportunity for input; 1 2 3 4 5 Decisions made jointly
uninvolved; decisions made through group participation;
autonomously. plenty of opportunity for
input; persons affected asked
Haramaya University, Haramaya Institute of Technology, SECE for their opinion.
Saturday, May 27, 2023
 Focus Group
36

 It is qualitative research
 small homogenous groups of people are brought together
 informal discussion specific topics under the guidance of a
moderator
 Purpose: to identify issues and themes
 Not appropriate when
 language barriers are insurmountable
 evaluator has little control over the situation
 trust cannot be established
 free expression cannot be ensured
 confidentiality cannot be assured

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Diaries and Self-Reported Checklists
37

 Use when you want to capture information about events in

people’s daily lives
 Participants capture experiences in real-time not later in a
questionnaire
 Used to supplement other data collection

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Expert Judgment
38

 Use of experts, one-on-one or as a panel

E.g., Government task forces, Advisory Groups
 Establish criteria for selecting experts not only on
recognition as expert but also based on:
 areas of expertise
 diverse perspectives
 diverse political views
 diverse technical expertise

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Delphi Technique
39

 Enables experts to engage remotely in a dialogue

and reach consensus, often about priorities
 Experts asked specific questions; often rank

choices
 Responses go to a central source, are summarized

and fed back to the experts without attribution

 Experts can agree or argue with others’ comments

 Process may be iterative

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023
 Summary
40

 Choose more than one data collection technique

 No “best” tool
 Do not let the method drive your work but rather
choose the right tool to address the evaluation
question

Haramaya University, Haramaya Institute of Technology, SECE Saturday, May 27, 2023