Research methodology

Measurement
conceptualization
- Define construct
Operationalization:
- Measurement and scaling
Questionnaire Administration
- Content Validity
Measurement model
- (Convergent/discriminant validity) (EFA and CFA)
- Reliability

Measurement Definition
Measurement means assigning numbers or other symbols to characteristics of objects, we measure characteristic of
the object
Scaling Definition
 Scaling may be considered an extension of measurement.
 Scaling is the process of placing the respondents on a continuum with respect to their attitude towards banks
• The anticipated improved accuracy is based on the assumption that using several items to measure a single
concept is more likely to represent all the different aspects of the concept. This involves reducing
measurement error.

Diet (Reflective) Health (Formative)

 R1. I eat healthy food.  F1. I have a balanced diet
 R2. I do not each much junk food.  F2. I exercise regularly
 R3. I have a balanced diet.  F3. I get sufficient sleep each night
Direction of causality is from construct to measure Direction of causality is from measure to construct
Measures expected to be correlated No reason to expect the measures are correlated

New and existing measure?

Use an existing measure Develop a new measure
there is a strong theoretical basis access to existing measures is limited
past empirical research demonstrates reliability and there is room for improvement
validity
you are not interested in a measure-development study

Levels of Measurement
Nominal Scales ordinal scales Interval Scales Ratio Scales
can only provide the ranked in relation to the These two scales have
number of occurrences in amount of the attribute constant units of
each class measurement, so
differences between any
two adjacent points on any
part of the scale are equal
the researcher assign Every subject can be
numbers to each category compared with another in
terms of a ">" or "<"
relationship.
 gender (male or female) Price ----------- Length, Time, the highest
Safety ----------- price that you’d like to pay.
Design -----------
nonsensical to calculate an non quantitative because permitting nearly any All mathematical
average value they indicate only relative mathematical operation to operations are ratio-scale
positions in an ordinal be performed measurements
series.
Ordinal scales provide no
measure of the actual
amount in absolute terms,
only the order of the values

non-metric data non-metric data Metric data (Continuous Metric data (Continuous
(Categorical data) (Categorical data) data) data)

Multiple Item Scales

 Number of questions  >1

 Construct/Aim  understanding of the course materials

Likert Scales

• Respondent specifies a level of agreement or disagreement with statements express either a favorable or an
unfavorable attitude toward the concept under study.
• Each item measures some aspect of a single common factor.
• Usually consists of item part and evaluative part.
Advantages
• Scores from all statements are totaled to measure respondent’s attitude  summated scale.
 • Very popular in SC.
• Quick and easy to construct.
• Can be administered by either telephone or internet.
Disadvantages
• The “favorable/unfavorable” attitude toward each item has equally weight.
• Simply adding up all scores doesn’t make sense.
Semantic differential SDS

• The semantic differential is a seven-point rating scale with end points associated with bipolar labels that have
semantic meaning.
• In a typical application, respondents rate objects on a number of itemized, seven-point rating scales bounded at
each end by one of two bipolar adjectives, such as ‘cold’ and ‘warm’.
• The respondents mark the blank that best indicates how they would describe the object being rated.
• The negative adjective or phrase sometimes appears at the left side of the scale and sometimes at the right.
• Used to describe the set of beliefs that comprise a person’s image of an organization or brand
• Respondents rate each attribute object on a number of five or seven-point rating scales bounded by polar
adjectives or phrases.
• The midpoint is a neutral point.
• In profile analysis, means values on each rating scale are calculated and compared by plotting or statistical
analysis. This helps determine the overall differences and similarities among the objects.

Measurement accuracy
A variety of factors can cause measurement error which result in the measurement or observed score being different
from the true score of the characteristic being measured

• Systematic error: affects the measurement in a constant way (mechanical factors (poor printing, poor design in
questionnaire)
• Random error: (short term factors (health, emotion)) (situational factors (people, noise)
Validity
• The extent to which a measurement represents characteristics that exist in the phenomenon under
investigation.
• Perfect validity requires that there is no measurement error (Xo=Xt)
• Content validity
• Write clear conceptual definition of the constructs assessed by the set of measures
• Randomly order items on form
• Ask (5-20) experts to identify which item belongs to which construct, based on items’ content and
construct definitions
• Convergent validity
• Measure the extent to which the scale correlates positively with other measures of the same construct
• Discriminant validity
• Assess the extent to which a measure does not correlate with another construct from which it is
supposed to differ.

• establishing discriminant validity implies that a construct is unique and captures phenomena not
represented by other constructs in the model.

• One method for assessing discriminant validity is by examining the cross loadings of the indicators.
Specifically, an indicator's outer loading on the associated construct should be greater than all of its
loadings on other constructs (i.e., the cross loadings).

• Nomological validity
• Assess the relationship between theoretical constructs. It seeks to confirm significant correlations
between the constructs as predicted by a theory

Reliability
• Refers to the extent to which a scale produces consistent result if repeated measurements are made on the
characteristic

• In this case the Xr =0 . The measure is free from random error (situational factors)

• If Xr =0 the measurement is perfectly reliable

Relationship between validity and reliability

• If a measure is perfectly valid it is also perfectly reliable. In this case Xo=Xt , Xr=0 , Xs=0
• If a measure is perfectly reliable it may or may not be perfectly valid.
• Ex. A reliable scale can be constructed to measure “customer loyalty” but it may not necessarily be a valid
measurement of “customer loyalty”
Important
EFA CFA
when they search for latent patterns in the data in case when testing the hypotheses of existing theories and
there is no or only little prior knowledge on how the concepts
variables are related.
• A common measure to establish convergent validity on the construct level is the average variance extracted
(AVE). = the sum of the squared loadings divided by the number of indicators).

• the square root of each construct's AVE should be greater than its highest correlation with any other
construct.
Questionnaire
• A structured technique for data collection consisting of a series of questions, written or verbal, that a
respondent answer.

has three specific objectives

- it must translate the information needed into a set of specific questions

- must motivate the respondent to become involved in the interview
- questionnaire should minimize response error

Questionnaire design steps

1-Specify the information needed.
We have to review the research problem, hypotheses and characteristics that influence the research design.

Types of questions

Open questions: for the respondent’s own answer

Advantages:

- Enable you to get below the surface and explore .

- Encourages respondents to give honest opinions
- Encourages respondents to think and offer considered answers

Disadvantages:

- The responses can be hard to compare or collate

- Coding problem
- The research may be difficult for others to reproduce, so your findings may be open to doubt or question

Closed:

- where a limited number of alternative responses to the set question are provided.
- These can be in list, category, ranking, scale or other quantitative form.
- They can be precoded on a questionnaire to facilitate analysis.

Closed Question types

Dichotomous Questions:
- This is simplest form of a closed-ended question. Give only two choices for respondent to select.
- Usually is used to collect demographic information.
Multiple-Choice Questions:
- Multiple-Choice Questions permit the respondent to select from several alternatives.
Scaled-Response Questions
- Question format that permits the measurement of the “intensity” of a respondents’ answers.
Ranking Questions:
- Determine the relative importance to respondents of various options.

2-Specify the type of interviewing method:

1. In personal interviews
a. complex and varied questions can be asked.
2. Telephone interviews
a. short and simple questions
3. Mail and electronic questionnaires
 a. questions must be simple, and detailed instructions must be provided.

3- contents of Individual Question

1. Is the Question Necessary? filter questions that measure familiarity, product use and past experience should
be asked before questions about the topics themselves
2. Define the issue in terms of who, what, when, where, (the four W's). are particularly important.
3. Do not use jargon or specialist language, use ordinary words. Words should match the vocabulary level of the
respondents. Reading level
4. Avoid Leading or Biasing Questions
5. Concerns the handling of “don’t know” and neutral responses may be advisable to provide the interviewer with
an additional “no answer” category to identify these people correctly

4- Overcoming Inability To Answer: Is the Respondent Informed?

1. filter questions that measure familiarity and past experience should be asked before questions about the topics
themselves.
2. A “don't know” option appears to reduce uninformed responses without reducing the response rate.
3. Can the Respondent Remember?
4. Sensitive Questions: If the information is sensitive:
(a) Place sensitive topics at the end of the questionnaire.
(b) Preface the question with a statement that the behavior of interest is common. using the everybody
approach
(C) Hide the question in a group of other questions that respondents are willing to answer.

5-Questions Order
1- Beginning
a. Question that obtains a respondent’s interest
b. General questions
c. Most familiar questions
2- Middle
a. Questions that require “work”
b. Less familiar questions
3- End
a. Sensitive or threatening Questions
b. Easy questions
c. Demographic questions

6: Identify the Form and Layout

1. Divide a questionnaire into several parts.
2. Questions in each part should be numbered.
3. The questionnaire should be pre-coded.
4. The questionnaires themselves should be numbered serially

7-Reproduction of the Questionnaire

- The questionnaire should be reproduced on good-quality paper and have a professional appearance.
- Each question should be reproduced on a single page
- The tendency to crowd questions together to make the questionnaire look shorter should be avoided.
- Directions or instructions for individual questions should be placed as close to the questions as possible.

8-Pretesting
1. refers to the testing of the questionnaire on a small sample of respondents to identify and eliminate potential
problems.
 2. All aspects of the questionnaire should be tested, including question content, wording, sequence, form, layout,
difficulty, and instructions
3. The respondents for the pretest should be drawn from the same population
4. After the necessary changes have been made, another pretest could be conducted by mail, telephone, or
electronic means if those methods are to be used in the actual survey.
5. The pretest sample size varies from 15 to 30 respondents for each wave.
6. Finally, the responses obtained from the pretest should be coded and analyzed.
a. Coding means number for each possible answer ex. 1 for male and 2 for female

Population and sample

Census:
If all the respondents in a population are asked to provide information, such survey is called census.

Use when

- Population size is quite small.

- Information is needed from every individual in the population.
- Cost of making an incorrect decision is high.
- Sampling errors are high

Sampling:
Selection of a subset of individuals from within a population to estimate characteristics of the whole population.

Use when

- Population size is large.

- Both cost and time associated with obtaining information from the population are high.
- Quick decision is needed.
- Population being dealt with is homogeneous.
- If census is impossible.

Census
Sampling
1-Target population
• the collection of elements that have the information sought by the researcher and about which inferences are to
be made.
• The population must be defined precisely.
• Information about population parameters may be obtained by taking a census or a sample
• The inferences that link sample characteristics and population parameters are estimation procedures and
tests of hypotheses.

2-Determining a Sampling frame

• a representation of the elements of the target population.

• It consists of a list or set of directions for identifying the target population.

3-Select a Sampling Procedure

• Samples should be representative of the population.
• Two main domains of sampling procedure.
• Probability sampling
• Non-probability sampling

Probability Sampling
• sampling units are selected by chance.
• each element has a known and equal probability of selection
• This requires not only a precise definition of the target population but also a general specification of the
sampling frame. Because sample elements are selected by chance.
• If the sampling frame is a poor fit to the population of interest, random sampling from that frame cannot fix the
problem.

Simple Random Sampling

probability sampling technique in which each element has a known and equal probability of selection.
and the sample is drawn by a random procedure from a sampling frame random number table
Systematic Sampling
A probability sampling technique in which the sample is chosen by selecting a random starting point and then picking
every i th element in succession from the sampling frame

Stratified sampling
• process to divide the population into sub-populations, or strata.
• Elements are selected from each stratum by a random procedure.

Cluster sampling
• the target population is first divided into clusters, then a random sample of clusters is selected based on a
probability sampling technique such as simple random sampling.
• For each selected cluster all the elements are included in the sample.
Non-probability Sampling
Convenience
Use any lists by convenience. Used to obtain information quickly and inexpensively.

Judgmental
- “Expert” uses judgment to identify representative samples.
- Better than probability sampling when sample sizes are very small.
Snowball
A form of judgmental sampling. Each respondent, after being interviewed, is asked to identify one or more others in the
field.
- Appropriate when reaching small, specialized populations.

Quota
• Is used extensively in street interviewing.
• The first stage consists of developing control characteristics, or quotas, of population elements such as age or
gender.
• In the second stage, sample elements are selected based on convenience or judgemental.

Determine the Sample Size

Statistical techniques
– Estimated population standard deviation.
– The acceptable level of sampling error.
– Desired level of confidence that the sample result will fall within a certain range of true population values.

Descriptive statistics
- Statistics which summarize and describe the data in a simple and understandable way
Inferential statistics
- Using statistics to project characteristics from a sample to an entire population.

Gather Information & Data

Qualitative Research
• Time consuming
• Usually collected from a smaller sample

Direct -Focus Group and Depth interview

Focus group Depth interview

Definition Gather data, usually in the forms of opinions, depth interviews are an unstructured
from a selected group of people on a particular and direct way of obtaining
and pre-determined topic information.

Group size 8 – 12 1 one to one interview

The researcher creates a relaxed atmosphere
Group Homogeneous; respondents prescreened No group
composition
Physical setting Relaxed, informal atmosphere
Time duration 1 – 3 hours 30 minutes to more than an hour.
Recording Use of audio and video recording
Moderator Observational, interpersonal, and
communication skills of the moderator
Type The researcher encourages free discussion Substantial probing is done to surface
underlying motives, beliefs, and
attitudes
Useful useful way of finding out what the main issues They attempt to uncover underlying
and concerns motives, or attitudes toward sensitive
issues.
Application Automobile companies such as GM, Ford and
Chrysler make heavy use of focus groups to
understand consumer preferences for new car
models and features.
 Focus group Depth interview
Definition Gather data, usually in the forms of opinions, depth interviews are an unstructured
from a selected group of people on a particular and direct way of obtaining
and pre-determined topic information.

Group size 8 – 12 1 one to one interview

The researcher creates a relaxed atmosphere
Group Homogeneous; respondents prescreened No group
composition
Physical setting Relaxed, informal atmosphere
Time duration 1 – 3 hours 30 minutes to more than an hour.
Recording Use of audio and video recording
Moderator Observational, interpersonal, and
communication skills of the moderator
Type The researcher encourages free discussion Substantial probing is done to surface
underlying motives, beliefs, and
attitudes
Useful useful way of finding out what the main issues They attempt to uncover underlying
and concerns motives, or attitudes toward sensitive
issues.
Application Automobile companies such as GM, Ford and
Chrysler make heavy use of focus groups to
understand consumer preferences for new car
models and features.

Indirect -Projective Techniques

Diagnostic tools to uncover the true opinions and feelings of consumers when they are unwilling or unable to express
themselves

Free association
– What do you like best about the brand?
– What are its positive aspects?
– What do you dislike?
– What are its disadvantages?
– What do you find unique about the brand?
– How is it different from other brands? In what ways is it the same?
Completion Techniques
• Respondents are given incomplete sentences and asked to complete them. Generally, they are asked to use
the first word or phrase that comes to mind.

Construction Techniques
cartoon characters are shown in a specific situation related to the problem.

The respondents are asked to indicate what one cartoon character might say in response to the comments of another
character.

Quantitative Research
Observation
• Used in descriptive research.
• Only used to provide information on current behaviors
 • recording the behavioral patterns of people, objects and events in a systematic manner to obtain information
about the phenomenon of interest.
• The observer does not question or communicate with the people being observed.

Structured observation Unstructured observation

the researcher specifies in detail what is to be observed the observer monitors all aspects of the phenomenon
and how the measurements are to be recorded. that seem relevant to the problem at hand,
Structured observation is appropriate when the This form of observation can be used when a research
phenomena under study can be clearly defined and problem has not formulated precisely and when
counted flexibility is needed in observation to identify essential
components of the problem and to develop hypotheses.
Less time for observation process, and reduces the require a researcher to spend a large amount of time
potential for observer bias and enhances the reliability observing a particular group of people, by sharing their
of the data way of life.
Unstructured observation is most appropriate for
exploratory research

Disguised Undisguised
the respondents are unaware that they are being the respondents are aware that they are under
observed observation
using one-way mirrors, hidden cameras, or
inconspicuous mechanical devices

Survey
obtaining information is based on questioning respondents.
Structured interviews involve the use of questionnaires based on a predetermined and identical set of questions
The questions are usually read out by a researcher in a neutral tone of voice to avoid influencing a particular response
from a participant.
Advantages:

• (ease) the questionnaire is simple to administer.

• (reliability) the data obtained are consistent
• (simplicity) coding, analysis and interpretation of data are relatively simple.
Disadvantages
• that respondents may be unable or unwilling to provide the desired information.
• Respondents may be unwilling to respond if the information requested is sensitive or personal

Survey techniques
• Telephone interviews
• Mail interviews
• Personal interviews (home, office or street interview)
Traditional Telephone

• Phoning a sample of respondents and asking them a series of questions.

• The interviewer uses a paper questionnaire and records the responses with a pencil
• Telephone interviews are generally conducted from centrally located research facilities.
• Field service supervisors can closely monitor the telephone conversations.

Computer-Assisted Telephone Interviewing

• uses a computerized questionnaire administered to respondents over the telephone.
• The interviewer sits in front of a computer screen wearing a mini-headset.
• When contact is made, the interviewer reads the questions posed on the CRT screen and records the
respondent's answers directly into the computer.
• Interim and update reports can be compiled instantaneously, as the data are being collected.

Mail Survey

• Questionnaires are mailed to potential respondents, who complete and return them by mail.
• Mail survey process
• Identify/locate respondents
• Mail surveys
• Wait for feedbacks
• Initial tasks – obtain a valid mailing list of potential respondents
• A broad identification of the individuals to be samples before data collection begins is required.
• Difficult to obtain a useful mailing list.
Personal Survey

• Door-to-door interviewing (Face-to-face interviews with respondents in their homes)

• Executive interviewing (Interview business people at their office)
• Purchase intercept technique (PIT) (Face-to-face interviews with customers after observing them make
purchases in shopping centers)
• Self-administered interviews (Questionnaires filled out by respondents with no interviewer present.)
How to Choose A Survey Method

Personal Interview Telephone Interview Mail survey

Reading/Writing ability Not need Not need Yes
Population & Sample Language Speaking Speaking Written
Geographic restriction Very high Medium Low

Complicated question Best Good Poor

Detail in response Best Good Poor
Survey Long questionnaire Best Good Poor
Sequence is required Best Best Poor
Sensitive question Poor Good Best
Costs per respondent High Medium Low
Administrative issues
Interviewer Yes Yes Not need
Response rate High Medium Low

Secondary data
1. Purpose. the data must be carefully evaluated on how it relates to the current research objective
2. Accuracy. Accuracy is enhanced when data are obtained from the original source of the data. In addition, it is
important to evaluate whether or not the data are out of date
3. Consistency. seek out multiple sources of the same data to assure consistency
4. Credibility. Researchers should question the credibility of the secondary data source. Technical competence,
service quality, reputation, training, and expertise of personnel representing the organization are some of the
measures of credibility.
5. Methodology. The quality of secondary data is only as good as the methodology employed to gather it.
Therefore, the researcher must evaluate the size and description of the sample, the response rate, the
questionnaire, and the data collection method.
6. Bias. Researchers must determine the underlying motivation or hidden agenda, if any, of the organization that
collected the secondary data.

The whole works

Items linked to
Theory Constructs constructs Collect data
Contribute to theory
Data cleaning filter
Test structural
hypotheses

Goodness of fit filter EFA

Modify the
Structural
Model

Build/Run Conduct Link items to

Structural Modify the CFA constructs;
Model Measurement Model
Label
constructs
 • Data cleaning (missing data, linearity, normality, outliers)
• Testing Measurement models (EFA, CFA, Reliability test)
• Factor Analysis groups the columns into meaningful groups. Also called “R Factor Analysis”.
• Cluster Analysis groups the rows into meaningful groups. Cluster Analysis is usually used instead of the
more computationally complex procedure called “Q Factor Analysis”.
•

• Descriptive statistics (mean, s.d. , s.e.)

• Summarizes data
• Includes:
• Mean, median and mode
• Measures of dispersion (range, standard deviation, and variance)
• Correlation matrix
• Testing hypotheses
• Parametric tests: Hypothesis testing procedures that assume that the variables of interest are measured
on at least an interval scale.
• Non-parametric tests: Hypothesis testing procedures that assume that the variables are measured on a
nominal or ordinal scale

Frequency Distribution

• In a frequency distribution, one variable is considered at a time.

• A frequency distribution for a variable produces a table of frequency counts, percentages, and cumulative
percentages for all the values associated with that variable.
• The mean, or average value, is the most commonly used measure of central tendency.
• The variance is the mean squared deviation from the mean. The variance can never be negative.
• The standard deviation is the square root of the variance.
Dependence and interdependence techniques
Interdependence techniques
• Interdependence techniques: the variables cannot be classified as either dependent or independent. Instead,
all the variables are analyzed simultaneously in an effort to find an underlying structure to the entire set of
variables or subjects.

• If the structure of variables is to be analyzed, then factor analysis or confirmatory factor analysis is the
appropriate technique.

• If cases or respondents are to be grouped to represent structure, then cluster analysis is selected.

Dependence technique
• may be defined as one in which a variable or set of variables is identified as the dependent variable to be
predicted or explained by other variables
• The different dependence techniques can be categorized by two characteristics:
(1) the number of dependent variables: a single dependent variable, several dependent variables,
(2) the type of measurement scale employed by the variables: either metric (quantitative/numerical) or
nonmetric (qualitative/categorical) dependent variables.
• If the analysis involves a single dependent variable that is metric, the appropriate technique is multiple
regression analysis.
• if the single dependent variable is no metric (categorical), appropriate techniques are multiple discriminant
analysis
• if a set of dependent -independent variable relationships is postulated, then structural equation modeling is
appropriate