Module 4

Research Design
(Part 2)
Causal Research Design
 Necessary conditions for making
causal inferences
• Concomitant variation: The extent to which a cause X and effect Y occur/ vary
together. For one to be a cause of another the two variables need to be
strongly associated. However, strong association does not imply causal
relationship as other external factors might be at play too.

• Time order of occurrence of variables: The cause X must occur prior to or

simultaneously with effect Y.

• Absence of other possible causal factors: Ensure that all other external factors
that may influence the relationship between X and Y are controlled through
experimental designs.
 Concepts used in experiments
• Independent variables: Also known as explanatory variables. Researchers
often use these variables in their models to assess their association/ effect
on the outcome/ effect variable.

• Test units: Test units are those entities on which the treatment is applied.
Often these are the sampled individuals, organizations, geographic areas. It
is the unit of statistical analyses.

• Dependent variables: Also known as the outcome variable/ effect variable.

It is expressed as a function of all possible independent variables.
 Concepts used in experiments
(contd.)
• Experiment: In case of social science, business research statistical
manipulations and analyses carried out using available data is referred to
as an experiment.

• Extraneous variables: These are variable besides the mentioned

independent variables that may affect the outcome/ dependent variable.
Variables
Moderating Variables (MV)
• Variable that affects the direction and/or strength of the relation
between an independent or predictor variable and a dependent or
criterion variable

Drug / Corona Vaccine Recovery Rate

(IV) (DV)

Age

7
 Intervening variables (IVV)
• Is a temporal occurrence which follows the IV and precedes DV
• It is a conceptual mechanism through which the IV and MV might
affect the DV.
• This is a variable that probably does influence the relationship
between the independent and dependent variables, but it is one
that we do not control or manipulate.
Increase in
Job
Satisfaction
A flexi-time Work Schedule Individual’s Productivity
(IV) (DV)

8
Control Variable
• A control variable is a variable that affects the dependent variable
• When we "control a variable" we wish to balance its effect across
subjects and groups so that we can ignore it and just study the
relationship between the independent and the dependent variables.
• You control for a variable by holding it constant

Type of City
Advertising Media Sales
(IV) (DV)

9
 Goals of conducting an experiment
• A Researcher has two goals while conducting an experiment:
• To draw valid conclusions about the effect of independent variables on the
dependent variable.  internal validity
• To make generalizations about the results to a wider population.  external
validity
Validity in experimentation
examines whether the
observed effects on
Internal the dependent
variables is caused by
the independent
variables only
Validity examines whether
the results of an
External experiment can be
generalized beyond
the experimental
situations or not
Factors affecting internal validity

History Maturation Testing Instrumentation

Statistical regression Selection bias Test unit mortality

Factors affecting external validity

Environment Population

Time Mode of treatment

 Methods to control extraneous
variables
Randomization Matching Use of experimental designs
• Random assignment of test units and • Matching using background • Systematic method of eliminating effects
treatments to experimental groups. characteristics, demographic, socio- of extraneous variables.
• Requirement: large sample economic profile.
• Benefit: No selection bias • Requirement: Similar group composition
• Benefit: Control of behavourial patterns/
confounding variables like age, incomes,
work experience.

Statistical control
• Statistical methods like Analysis of
Covariance (ANCOVA) might be used to
deal with confounding effects and make
causal inferences.
 Classification of experimental
designs
Experimental
designs

Pre-experimental Quasi experimental True experimental Statistical

One-shot case Pre-test post-test Completely

Time series randomized
study control groups

One-group pre- Multiple time Post-test only control Randomized

test Post-test series groups blocks

Static group Solomon four group Latin Square

Factorial
Definition of symbols:
• 0t = Observation in time ‘t’ of experimental group
• X = Treatment
• R = random assignment of test units or groups to separate
treatments.
• 0c = Control group or CG
Pre-experimental design
• One-shot case study: Only one time observation is made on the respondents after the treatment
has been administered. This is not randomized. Eg: The sales manager might have selected the
group of sales persons he/she wanted to get the training and then evaluated their performance
post training.
• One-group pre-test Post-test : Observation on the test units are made twice, once before the
treatment and once after. This is usually not randomized.
• Static group: Observations are taken in the experimental and the control group. Experimental
group is the group where treatment (X) is given and control group is a group where treatment is
not given. The design appears as follows:
• Group 1 (experimental group)- X O1
• Group 2 (control group)- O2
Quasi experimental
• Researchers can control when measurements are taken and on whom they are taken. However,
the lack complete control in the sense that they cannot control the scheduling of the test and also
lacks the ability to randomize test units’ exposure to treatment.
• Time series: O1 O2 O3 O4 X O5 O6 O7 O8
Series of observations are made on a group of test unit without randomization. The timing of
treatment and the test units who are getting exposed to the treatment is not within the researcher’s
control. Due to multiple observations maturation, testing effect, instrumentation and statistical
regression can be ruled out. Randomizing this can deal with selection bias and strong incentives can
also reduce attrition.
• Multiple time series: Same as time series but multiple time series has a control group where no
treatment is given.
Group 1 (experimental group)- O1 O2 O3 O4 X O5 O6 O7 O8
Group 2 (control group)- O’1 O’2 O’3 O’4 O’5 O’6 O’7 O’8
True experimental
• Researchers are able to randomly assign test units and treatment to an experimental group.
• Pre-test post-test control groups: Also called before-after with control group.
Experimental group: R O1 X O2
Control group: R O3 O4
Let, A=O2- O1  treatment+ extraneous variables (interactive testing effect)
and B= O4 - O3  Extraneous variables
Therefore, A-B=(O2- O1 )-(O4 - O3 )  treatment effect
True experimental (Contd.)

• Post-test only control groups: Also called after-only with one control group
Experimental group: R X O1
Control group: R O2
Let, A=O2- O1
 treatment+ extraneous variables (interactive testing effect)-extraneous variables treatment effect
• Such experiments are largely used in research where the goal might be to evaluate the
effectiveness of a programme, say online education. Researchers could randomly choose to
educate 10 children of the same class through online mode and 10 through traditional mode and
evaluate their performance post treatment. Here, they don’t need to worry about the effect of
instrumentation or interactive test effects.
 True experimental (Contd.)
• Solomon four group: Also called four group six study design. Guarantees
maximum internal validity and useful for cause-effect relationship in studies
where survival is crucial.
Experimental group 1: R O1 X O2
Control group 1: R O3 O4
Experimental group 2: R X O5 No pre-test measurement
Control group 2: R O6
 Solomon 4 group- mathematics
Here, Experimental group 1:
A= O2 - O1= treatment effect+ extraneous effect without interactive
testing effect + interactive testing effect
= T+I+E
Control group 1:
B= O4 – O3= Extraneous effects= E
X=A-B= T+I (treatment effects along with interaction effects)
Experiment Group 1 and 2:
C=O2 – O5 = (treatment effect+ interaction effects+ extraneous effect) –
(treatment effect+ extraneous effect) = interaction effects
= (T+I+E)-(T+E)= I (Only interaction effects)
D= X-C= (T+I)-I= T (Pure treatment effects)
Statistical designs
• Advantages of a statistical design:
• The effect of more than one level of independent variable on the dependent
variable can be manipulated.
• The effect of multiple independent variables can be examined.
• The effects of specific extraneous variables can be controlled.
Statistical Design: CRD (contd.)
• Completely randomized design (CRD):
• Used when effect of one independent variable on dependent variable is being examined.
• Independent variable should have categories- each category is a treatment.
• Assumption: No difference in test units  no extraneous variables can influence the
outcomes
• One-way ANOVA (Analysis of Variance)
• Example: effects of three categories of price on sales

Price
Low
Sales
Medium
High
Statistical Design: RBD (contd.)
• Randomized block designs (RBD):
• Main limitation of CRD was that all
Store: Medium
extraneous variables were assumed
to be constant over treatment Price:
• Low
groups, which may not be true. • Medium
• In RBD it is possible to separate out Store: Small
• High
Store: Large
the influence of one extraneous Price: Price:
variable on the dependent variable. • Low • Low
• Eg. All stores taken as test units • Medium • Medium
• High • High
may not be of the same size, and
might be small, medium or large.
This blocking will be done prior to
the experimentation. Now, we
analyze the treatment effect Sales
(categories of price) on the
outcome (sales) by checking their
effect in homogenous blocks (store
sizes).
Statistical Design: LSD(contd.)

• Latin Square Design:

• LSD is applied when the researcher wants to separate out the
influence of two extraneous variables.
• Necessary condition: No. of categories of the two extraneous
variables=No. of categories of treatment variable.
• Eg: Suppose along with store size we are also interested to explore
type of packaging as an extraneous variable.
 Statistical design: LSD (contd.)

Store Size Packaging

I II III

1. Small Price 1 Price 2 Price 3

2. Medium Price 2 Price 3 Price 1

3. Large Price 3 Price 1 Price 2

• Treatment should be assigned in such a manner that one treatment occurs once and only one in each row and
column
 Statistical Design: FD (contd.)

• Factorial Design:
• Measure the effect of two or more independent variables at various levels.
• Interaction is suppose to take place when simultaneous effect of two variables is
different from a simple addition of their individual effects (say an individual might
love mangoes and also love ice-creams but might not like mango flavoured ice-
cream!)
 Statistical Design: FD (contd.)
A two-factor design looks like:
Price Store
Small (B1) Big (B2) Low price in small
store
1. Low (A1) A1 X B1 A1 X B2
3X2= 6 cells
2. Medium (A2) A2 X B1 A2 X B2
3. High (A3) A3 X B1 A3 X B3 High price in big
store

• Respondent receive a specified treatment combination.

• Advantages:
• Possible to measure main effects and interaction effects of two or more independent variables
• Saves time and effort as all observations are employed to study the effects of each factor
• Has wider acceptance as the design acknowledges the different combinations of factors
• Challenge: when the number of categories and factors increase
No-Sampling Error
 This is an error occurring for reasons other than sampling.
 Example – If the questionnaire is poorly designed with many ambiguities, the
response will be poor. Non-sampling error consists of non-response errors and
response errors.
Non-response Error
 This happens when chosen respondents do not respond
 This may be due to
 Non availability of respondent
 Refusal to answer
 This will cause the resulting sample to be of different size compared to original
sample.
Response Error
 This occurs due to any of the eventualities
A. Mis-recording
B. Inaccurate answers
C. Wrong analysis errors made by researcher include
i. Surrogate information
ii. Measurement
iii. Population Definition
iv. Sampling Frame
v. Data Analysis Error