Unit 4: Experimental

Research Design
By Vuong Yen
Source: N01- Griffe, D. T, An introduction to second
language research methods: Design and Data. TESL- EJ
Publication, 2012 (p70-93)
 Objectives:
• After finishing the lesson, students can grasp:
1. Experimental design defined
2. Key components of experimental design
3. The basic structure and function of experimental research
4. Advantages and disadvantages of ERD
Experimental design
1. Some technical terms used in Experimental research Design
1. Control group (Nhóm kiểm soát): A group of participants used for comparison purposes, similar
to the experimental group, but to whom no treatment (tác động) is administered. What
happens in the control group, it is argued, is what would happen if no experiment or treatment
were done. Results from the control group are known as counter-factual evidence.

2. Experimental group (nhóm thí nghiệm) : Also known as the treatment group, the group of
participants with whom the innovation or treatment is applied. The experimental group is
where change of some sort will be compared with the performance in the control group.

3. Dependent variable (Biến phụ thuộc): A variable that is usually the test used to measure results.
In terms of cause and effect, this is a measure of the effect.

4. Independent variable (Biến độc lập): The variable under investigation. For example, if a TREE
were using EXD to investigate listening comprehension, then listening comprehension would be
the independent variable. If a TREE were investigating how students acquire vocabulary, then
vocabulary acquisition would be the independent variable.
5. Hypothesis A hypothesis is a possible explanation. The null-hypothesis states
that there is no relationship between variables of interest. The research
hypothesis is the preferred explanation put forward by the TREE to explain
the results of the experiment. An alternative or rival hypothesis is an
explanation other than the research hypothesis that also might explain the
results.

6. Inferential statistics (thống kê suy luận) A class of statistics that does not
directly describe scores, but allows the TREE to infer or suppose something
that cannot be directly observed, usually the probability
that two scores are related (correlation) or not related (a significance test).

7. Intact class A class that is whole or intact before the TREE comes into
contact with it. A class is intact because either it forms itself (students select
the class and then enrol for some reason such as convenient time) or the class
is formed by others (administrative control). Intact classes are the norm
because most teachers cannot randomly assigned students.
8. Mean scores (Giá trị trung bình): The average score for a group of taking a test. A
mean score of 82.7 indicates that the average score for the whole class was about 83
points.

9. Non-parametric statistics (Thống kê phi tham số ): A family of statistical

procedures used when the requirements of parametric statistics are not available.
Most often this is because the scores do not form a normal distribution or the test
does not use continuous scale. A popular non-parametric statistic is the Chi-square,
which uses frequencies.

10. p-value (probability value)The small letter p stands for probability. A TREE
administers a test to two groups of students and wonders if the difference between
the resulting scores is likely the result of chance
variation. The answer comes in the form of a percent called the p-value. If the p-
value is small, traditionally .05 or smaller, the TREE can make the claim there is a
small probability, 5%, that they would get these scores if there were no relationship;
therefore, they can infer that there is a relationship. If the point of the experiment
was to show score improvement, the TREE hopes the mean difference is large and
the p-value is small.
11. Parametric statistics (Thống kê tham số): A family of statistical procedures
that requires normal distribution and a continuous scale. Some familiar
parametric statistics are the t-test, ANOVA, and Pearson correlation.

12. Pretest (tiền kiểm): A test that is administered before the experiment,
intervention, treatment, or teaching takes place. The purpose of the pretest is to
establish baseline data; in other words, it establishes what the situation is before
the intervention or treatment is administered.

13. Posttest (hậu kiểm) A test that is the same or very similar to the pretest,
and is administered after the experiment or teaching takes place.

14. Random assignment (Phân công ngẫu nhiên ): A type of placement that
refers to the process by which any participant has an equal chance to be
assigned to any group. One way of random assignment is to write names on
pieces of paper, put them into a container, and have somebody draw them out
one by one without seeing the names. Another way is to use a computer
program that can randomize a list of names or numbers.
15. Reliability (độ tin cậy) The idea of consistency of results or getting the same
or similar results each time data is collected. It was originated by Spearman as a
comparison of a hypothesized true score and observed score. It is often given in
terms of a percent marked with a decimal point, such as .82.

16. Scale Refers to unit of measurement. Every measuring device must have a
scale in which the measurement is reported. Tests are usually reported in one of
three scales: nominal (e.g., gender, nationality), ordinal (e.g., frequency counts), or
continuous (e.g., test scores).

17. Significance test (Kiểm định độ tin cậy) A statistical procedure that estimates
the difference between two (sometimes more depending on the type of test) sets
of scores. The results of a significance test is a p-value, which is an indicator of
the likelihood of obtaining this difference given the null hypothesis. A common
significance test for testing the difference between two and only two scores is
called a t-test. Another significance test is called analysis of variance - Phân tích
phương sai (ANOVA) and is used for testing the difference between two or more
groups.
18. Stratified sampling (Chọn mẫu phân tổ) Although identified with
SRD, stratified sampling can be used in EXD to equalize groups. For
example, a TREE considers ethnic groups important variable. The
TREE has a control group with 50% one ethnic group and 50%
another group. In order to make the treatment group similar to the
control group, the TREE wishes to use only those students in the two
ethnic groups in approximately a 50-50 proportion.
19. Threats The name given to a rival hypothesis used to explain the
difference between treatment groups.
For example, if a researcher were investigating the effects of an
innovative writing curriculum, a threat would be that, unknown to the
researcher, many students went to the writing center, and those visits
accounted for the score increase, not the innovative curriculum.
Attendance at the writing center, usually called a threat of history,
threatens the research hypothesis that it was the innovative writing
curriculum that caused the increase in scores.
20. Time-Series (dãy số thời gian) A variation of EXD where
instead of comparison of two groups, the TREE uses only one
group, takes several measurements to establish baseline data (to
establish which is usually the case), introduces an innovation or
treatment, and continues with additional measurements
(to establish the effect of the treatment). The main advantage is
only one group is necessary and can function as its own control
group.
21. Unit of Assignment The persons or things being studied.
Things being studied can be classes, schools, or even cities and
towns. An EXD can study one person, a group of persons, an
entire class, or a school.
22. Validity (Độ chuẩn xác) Test validity is traditionally defined
as providing evidence that shows or indicates that what the
TREE believes he or she is testing is actually being tested. In
practice, validity means offering an explanation of how one
checked or verified instrument results.
23. Variable (biến) A trait or quality that can be measured by
a test or other kind of data collection instrument. A variable
can be anything a TREE is interested in.
 2. What is Experimental Research?
• Experimental research is a scientific approach to research, where one or
more independent variables are manipulated and applied to one or more
dependent variables to measure their effect on the latter. The effect of
the independent variables on the dependent variables is usually observed
and recorded over some time, to aid researchers in drawing a reasonable
conclusion regarding the relationship between these 2 variable types.

• The experimental research method is widely used in physical and social

sciences, psychology, and education. It is based on the comparison
between two or more groups with a straightforward logic, which may,
however, be difficult to execute.

• Mostly related to a laboratory test procedure, experimental research

designs involve collecting quantitative data and performing statistical
analysis on them during research. Therefore, making it an example of
quantitative research method.
• Experimental research is the most familiar type of research
design for individuals in the physical sciences and a host of
other fields. This is mainly because experimental research is a
classical scientific experiment, similar to those performed in
high school science classes.
• Eg: Imagine taking 2 samples of the same plant and exposing
one of them to sunlight, while the other is kept away from
sunlight. Let the plant exposed to sunlight be called sample
A, while the latter is called sample B.
• If after the duration of the research, we find out that
sample A grows and sample B dies, even though they are
both regularly wetted and given the same treatment.
Therefore, we can conclude that sunlight will aid growth in
all similar plants.
 2. Experimental design defined
✓ Experimental research is defined as “OBSERVATIONS UNDER
CONTROLLED CONDITIONS”.
✓ In experimental design the researcher is active agent rather than a
passive observer.
✓ Experimental designs are concerned with examination of the effect
of an independent variable on dependent variable, where the
independent variable is manipulated through treatment or
intervention(s).
✓ According to Riely, experimental design is a powerful design for
testing hypotheses of causal relationship among variables.
 3. What are The Types of Experimental
Research Design?
• The types of experimental research design are
determined by the way the researcher assigns
subjects to different conditions and groups. They are
of 3 types: pre-experimental, quasi-experimental,
and true experimental research.
3.1. Pre-experimental Research Design
• In pre-experimental research design, either a group or
various dependent groups are observed for the effect of the
application of an independent variable which is presumed to
cause change. It is the simplest form of experimental
research design and is treated with no control group.
• Although very practical, experimental research is lacking in
several areas of the true-experimental criteria. The pre-
experimental research design is further divided into three
types
 a. One-shot Case Study Research Design (Mô hình nắm
bắt một tình huống)
• In this type of experimental study, only one dependent
group or variable is considered. The study is carried out
after some treatment which was presumed to cause
change, making it a posttest study.
 b. One-group Pretest-posttest Research Design
(thiết kế một nhóm tiền kiểm- hậu kiểm)
• This research design combines both posttest and pretest study by
carrying out a test on a single group before the treatment is
administered and after the treatment is administered. With the
former being administered at the beginning of treatment and later
at the end.

Eg: Một giảng viên muốn tìm hiểu về sự thay đổi về nhận thức của sinh viên đối với môn
học Pháp luật đại cương thông qua một bài trắc nghiệm được áp dụng trước và ngay sau
khi kết thúc học phân. Tuy kết quả lần trắc nghiệm sau cao hơn lần trắc nghiệm đầu,
giảng viên khó có thể đưa ra kết luận về sự thay đổi do môn học mang lợi bởi trong suốt
thời gian học, các phương tiện báo- đài có những chương trình về giáo dục pháp luật cho
thanh niên.
 c. Static-group Comparison
(Thiết kế so sánh nhóm tĩnh)
• In a static-group comparison study, 2 or more groups are placed under
observation, where only one of the groups is subjected to some
treatment while the other groups are held static. All the groups are
post-tested, and the observed differences between the groups are
assumed to be a result of the treatment.
 3.2. Quasi-experimental Research Design (Thiết kế bán
thử nghiệm)
• The word “quasi” means partial, half, or pseudo. Therefore, the quasi-
experimental research bearing a resemblance to the true experimental
research, but not the same. In quasi-experiments, the participants are not
randomly assigned, and as such, they are used in settings where
randomization is difficult or impossible.
• This is very common in educational research, where administrators are
unwilling to allow the random selection of students for experimental
samples.
• Some examples of quasi-experimental research design include: the time
series, no equivalent control group design, and the counterbalanced
design.
 3.3. True Experimental Research Design
(thiết kế thử nghiệm đích thực)
• The true experimental research design relies on statistical analysis
to approve or disapprove a hypothesis. It is the most accurate type
of experimental design and may be carried out with or without a
pretest on at least 2 randomly assigned dependent subjects.
• The true experimental research design must contain a control
group, a variable that can be manipulated by the researcher, and
the distribution must be random. The classification of true
experimental design include:
 a. The posttest-only Control Group Design
(Thiết kế chỉ kiểm tra sau)
• In this design, subjects are randomly selected and assigned to the
2 groups (control and experimental), and only the experimental
group is treated. After close observation, both groups are post-
tested, and a conclusion is drawn from the difference between
these groups.

Eg: Sau khi học hết HP NCKH trong giảng dạy tiếng Anh (X), sinh viên
năm 3 ngành sư phạm Anh, HPU2 (A) tham gia một cuộc kiểm tra (O)
để được đánh giá khả năng tiếp thu môn học.
 b. The pretest-posttest Control Group Design
(T/kế kiểm tra trước sau)
• For this control group design, subjects are randomly assigned to the 2 groups,
both are presented, but only the experimental group is treated. After close
observation, both groups are post-tested to measure the degree of change in
each group.
 c. Solomon four-group Design
(Thiết hiết kế 4 nhóm)
• This is the combination of the pretest-only and the pretest-
posttest control groups. In this case, the randomly selected
subjects are placed into 4 groups.
=> The first two of these groups are tested using the
posttest-only method, while the other two are
tested using the pretest-posttest method.
 4. Examples of Experimental Research

• Experimental research examples are different, depending on the

type of experimental research design that is being considered.
The most basic example of experimental research is laboratory
experiments, which may differ in nature depending on the subject
of research.
 Administering Exams After The End of Semester
• During the semester, students in a class are lectured on particular
courses and an exam is administered at the end of the semester. In this
case, the students are the subjects or dependent variables while the
lectures are the independent variables treated on the subjects.
• Only one group of carefully selected subjects are considered in this
research, making it a pre-experimental research design example. We
will also notice that tests are only carried out at the end of the
semester, and not at the beginning.
• Further making it easy for us to conclude that it is a one-shot case
study research (Mô hình nắm bắt một tình huống).
 Employee Skill Evaluation
• Before employing a job seeker, organizations conduct tests that are used to
screen out less qualified candidates from the pool of qualified applicants.
This way, organizations can determine an employee’s skill set at the point of
employment.
• In the course of employment, organizations also carry out employee
training to improve employee productivity and generally grow the
organization. Further evaluation is carried out at the end of each training to
test the impact of the training on employee skills, and test for
improvement.
• Here, the subject is the employee, while the treatment is the training
conducted. This is a pretest-posttest control group experimental research
example.
 Evaluation of Teaching Method
• Let us consider an academic institution that wants to evaluate the teaching
method of 2 teachers to determine which is best. Imagine a case whereby
the students assigned to each teacher is carefully selected probably due to
personal request by parents or due to stubbornness and smartness.
• This is a no equivalent group design example because the samples are not
equal. By evaluating the effectiveness of each teacher’s teaching method
this way, we may conclude after a post-test has been carried out.
• However, this may be influenced by factors like the natural sweetness of a
student. For example, a very smart student will grab more easily than his or
her peers irrespective of the method of teaching.
5. What are the Characteristics of Experimental Research?
5.1. Variables
• Experimental research contains dependent, independent and
extraneous variables. The dependent variables are the variables
being treated or manipulated and are sometimes called the
subject of the research.
• The independent variables are the experimental treatment being
exerted on the dependent variables. Extraneous variables, on the
other hand, are other factors affecting the experiment that may
also contribute to the change.
5.2. Setting

• The setting is where the experiment is carried out. Many

experiments are carried out in the laboratory, where control can
be exerted on the extraneous variables, thereby eliminating
them.
• Other experiments are carried out in a less controllable setting.
The choice of setting used in research depends on the nature of
the experiment being carried out.
5.3. Multivariable

• Experimental research may include multiple independent

variables, e.g. time, skills, test scores, etc.
6. Why Use Experimental Research Design?

• Experimental research design can be majorly used in physical

sciences, social sciences, education, and psychology. It is used to
make predictions and draw conclusions on a subject matter.
• Some uses of experimental research design are highlighted below.
• Medicine: Experimental research is used to provide the proper
treatment for diseases. In most cases, rather than directly using
patients as the research subject, researchers take a sample of the
bacteria from the patient’s body and are treated with the
developed antibacterial
=> The changes observed during this period are recorded and
evaluated to determine its effectiveness. This process can be carried
out using different experimental research methods.
• Education: Asides from science subjects like Chemistry and Physics which
involves teaching students how to perform experimental research, it can
also be used in improving the standard of an academic institution. This
includes testing students’ knowledge on different topics, coming up with
better teaching methods, and the implementation of other programs that
will aid student learning.
• Human Behavior: Social scientists are the ones who mostly use
experimental research to test human behaviour. For example, consider 2
people randomly chosen to be the subject of the social interaction
research where one person is placed in a room without human
interaction for 1 year.
• The other person is placed in a room with a few other people, enjoying
human interaction. There will be a difference in their behaviour at the
end of the experiment.
 7. What are the Disadvantages of Experimental Research?
• It is highly prone to human error due to its dependency on variable control
which may not be properly implemented. These errors could eliminate the
validity of the experiment and the research being conducted.
• Exerting control of extraneous variables may create unrealistic situations.
Eliminating real-life variables will result in inaccurate conclusions. This may
also result in researchers controlling the variables to suit his or her
personal preferences.
• It is a time-consuming process. So much time is spent on testing
dependent variables and waiting for the effect of the manipulation of
dependent variables to manifest.
• It is expensive.
• It is very risky and may have ethical complications that cannot be
ignored. This is common in medical research, where failed trials may
lead to a patient’s death or a deteriorating health condition.
• Experimental research results are not descriptive.
• Response bias can also be supplied by the subject of the conversation.
• Human responses in experimental research can be difficult to
measure.
• BTVN (làm ppt đề thuyết trình): mỗi nhóm chọn 1 trong
hai bài cô đã gửi và phân tích dựa vào trả lời 6 câu hỏi
Unit 5: Case Study Design
By Vuong Yen
Source: N01- Griffe, D. T, An introduction to second
language research methods: Design and Data. TESL- EJ
Publication, 2012 (p94-106)
 Objectives:
• After finishing the lesson, students can grasp:
1. Case study design (CSD) defined
2. Key components of Case study design
3. Practical steps of Case study design
4. Advantages and disadvantages of Case study design