Quadrant II – Transcript and Related Materials

Programme: Bachelor of Arts (Third Year)

Subject: Psychology
Paper Code: PSC109
Paper Title: Psychological Research
Unit: 03 (Experimental Research)
Module Name: Experimental error and reliability and validity of experimental
research
Name of the Presenter: Michelle Fernandes (Ph.D)

Notes
Experimental error
Experimental error is the difference between the data actually obtained and that which
would have been obtained if it could have been possible to ensure that the IV was the sole
influence on the DV, and that any changes to the DV have been measured with complete
accuracy.
The presence of some degree of error is inevitable simply because it may emerge from so
many different sources. This means that much of the effort in designing an experiment must
be directed at devising methods of controlling those sources of error that seem most likely
to interfere with the action of the IV on the DV. If this is not done then one or more of the
uncontrolled variables are said to be confounded with the effects of the IV.

Random and constant error

Random error is error that sometimes affects one condition in an experiment and
sometimes another causing the measures of the DV to be sometimes greater and
sometimes less than their true value in an entirely random fashion.
Constant error is error which has a consistent character throughout an experiment,
producing the same degree of overestimate or underestimate of the true value of the DV.
The main sources of experimental error
Each of the sources of error are capable of contributing both random and constant error to
an experiment. Constant error makes it impossible to see whether the IV caused the DV to
change at all, poses the greater problem and should be the first concern in experiment
design, however, the aim is always to try to reduce both types of error to as close to zero as
possible.
1. Error due to sampling
This form of error is caused by a failure of the sampling procedure to generate a sample that
is adequately representative of the population of interest. If the planned sample is small a
highly representative sample is unlikely to be achieved, so the only effective remedy is to
take a random sample to ensure that any constant error present in the sample is converted
to random error.
2. Participant-related error
The participant variables are those, such as intelligence, motivation, expectations, manual
dexterity, handedness that, in a repeated measure design, account for the variance within
conditions and in an independent group design, for variance occurring both within and
between conditions.
Participant related error also arises from treatment variables – the way participants are
allocated to different treatments, or the order on which tasks are taken – which may
constitute a source of error if they are able to confound the IV.
3. Task-related error
Task variables are those concerned with the procedures that participants are required to
perform in an experiment. They include such things as: degree of difficulty of the task, the
time allowed for its completion, the nature of the instructions given to the participants and
in repeated measures designs, any effects on performance attributable to fatigue or
practice.
4. Error due to measurement
This error arises when an experiment produces an inaccurate estimate of the true value of
the variable being measured.
This may be divided into:
Error inherit in the measuring technology (present in most psychometric tests)
 Human error such as using a manual stopwatch, rather than one triggered
electronically to record response time
5. Error due to the environment of the experiment
Many environment variables can also act as a source of experimental error. These include
not only the physical characteristics of space, such as its temperature or noise level, but also
features such as demeanour of the experimenter, presence of other people. The purely
physical variables can be dealt with by standardization or counterbalancing as appropriate
but this involving the other human participants in the experiment are less easy to reduce
since they arise from the social psychology of the situation.
6. Error due to demand characteristics
Demand characteristic refers to an experimental artifact where participants form an
interpretation of the experiment's purpose and subconsciously change their behavior to fit
that interpretation.
A negative version of demand effect is called the ‘screw-you-effect’ – participants
deliberately try to generate data that would disconfirm the research hypothesis
7. Error due to experimenter effects
Error introduced (unconsciously) into the research by the researcher through interaction
with participants Effect on the subjects’ behaviour caused by an experimenter’s
presence, actions or comments
Rosenthal 2020 identifies four general types of experimenter effect:
a. Biosocial effects are produced by the personal characteristics of the experimenter
and participants, such as sex, age and ethnicity. Male participants have been found
to be less smiled at than female participants and smiling at participants was also
found to predict the results of the experiment.
b. Psycho-social effects are produced by the personality and perceived competence
and authority of the experimenter. Researchers with a warm demeanour were found
to elicit higher intelligence test scores than those with a cooler demeanour.
c. Situational effects, such as the level of experience of the researcher and the trend
visible in incoming results (i.e. whether they appear to conform to the research
hypothesis) are also good predictors of the eventual result.
d. Expectancy effects also called the Pygmalion effect, refers to the self-fulfilling effects
of an experimenter’s expectation about the results of an investigation.
Counter measures against experimental error
1. Randomization: Randomization distributes the effects of a potential source of error
at random throughout the experiment, or through one particular condition, thus
converting constant error into the somewhat less serious random error. Allocating
participants to conditions at random controls for individual difference by ensuring
that such differences are randomly distributed.
2. Standardization: Standardization involves ensuring that some aspects of the
experiment is the same for all participants within a condition and perhaps across
conditions as well. For example, the environment in which an experiment takes place
should normally be the same for all participants in all conditions and the instructions
and experimental procedures should be identical within each condition.
3. Counterbalancing: Counter balancing attempts to ensure that error form one
possible source is neutralized by being balanced by an equal amount of error from
another, similar source, for example, in a repeated measure design the order in
which two tasks are performed is a possible source of error. Counterbalancing the
order in which they are performed so that half the participants complete task 1
before task 2, while the other half take the tasks in the reverse order, removes or
reduces error from this source.

Reliability and validity of experimental research

In seeking to demonstrate that a causal relationship exists between variables, the
researchers’ goal is to design an experiment that both delivers a true view of that
relationship and also gives an identical picture whenever the experiment is performed. That
is, the aim is to design an experiment that possesses the essential qualities of validity and
reliability.
• Reliability The extent to which a given experimental phenomenon can be replicated
and thus shown to be independent of a specific situation
• Validity The extent to which any conclusions drawn from an experiment are justified,
i.e., it deals with the value of the inferences made on the basis of experimental
results
Types of validity
Questions of validity deal with the extent to which an experiment ca be relied upon to
present a true picture of the relationship between two variables. Data from an experiment
are assessed as valid if they meet the following criteria.
• Theoretical validity: this form of validity is also called construct validity, looks at
whether the concepts (constructs) that are implemented in a piece of research are
derived from a theory for which empirical support exists, and can therefore be linked
to other well-researched phenomena.
This quality is important as psychological knowledge slowly moves beyond the
accumulation of unconnected empirical findings to become more systematic.
• Internal validity: this deals with the internal features of an experiment.
The main threats to internal validity come from the repeated testing of participants
(affecting within-participant validity) the presence of systematic differences between one
treatment group and the other(s) between-groups validity.
If left uncontrolled either of these factors can seriously weaken the validity of any
conclusions that are drawn from the results.

References:

1. Dyer, C. (2013). Research in Psychology: A practical guide to methods and statistics,

Delhi: Wiley India.