Research Methods Study Guide for Test Three

Donald F. Kendrick, MTSU

Topics for Multiple Choice Questions Identification of Independent Variable and Dependent Variable in all Designs, completely randomized design One of the simplest between-subjects design Also called: Simple randomized design, or the simple random subject design The assignment of participants is completely randomized between groups. In its simplest form, it is composed of two level of independent variable.

Group A receive a high level of IV and Group B receive a zero level. The most common way to analyze a Completely Randomized Design is to use a SingleFactor analysis of Variance (ANOVA)

multilevel completely randomized design The completely randomize design can contain more than two levels of IV, which is called the multilevel completely randomized design and can be diagrammed as:

Example: As an example of completely randomized design with four levels of the IV, consider a study of the effects of drinking coffee on the performance of airplane pilots. Participants in four groups would be given a prescribed number of cups of coffee and then tested in a laboratory equipped with a flight simulator. The number of errors made by the participant in each group would be the dependent variable. In this hypothetical example, level 4 could equal 6 cups of coffee, level 3 could equal 4 cups, level 2 could equal 2 cups, and level 1 could equal 0 cups. If you actually were to run this study,

This is a multilevel completely randomized design with pretest-postest measures

analysis of variance,

post hoc tests To determine whether there is a statistically significant difference between any combination of groups, we must use a post hoc test that compares each group to another in a pairwise manner. A variety of post hoc tests, such as the Newman-Keuls procedure or Tukeys HSD post hoc test Performing a post hoc test, is similar to performing a t test between Group 1 and Group 2, and then a t test between group 2 and group 3, and then a t test between group 1 and group 3. The number of tests that you do influences the probability of significant results and must be included in your analysis. That is to say, if you perform a large number of post hoc tests, you would expect more of them to be significant by chance than if you performed only a few tests, and this needs to be taken into account when you perform the statistics. This is referred to as Familywise Error Rates. Familywise Error Rates: the error is the possibility of a type I error, which can result from performing multiple tests that can lead us to reject the null hypothesis when it is true.

a priori tests, A priori tests: planned comparison Alternative to using post hoc tests is to plan particular comparison before the study is performed. Planned comparision do not take advantage of chance in the same way as post hoc comparisons. This is the statistical approach BRansford and Johnson Took. They compared the context before condition with each of the four other conditions.

Factor Level, DV The Independent Variables are also called Factors

Identify number of Factors and Levels from design notation (2x4 factorial, or 3x2 mixed, etc.), 2 x 2 factorial design: in this notation, 2x2 refers to the two levels of the one IV and two levels of another. 6 x 2: if we had a design involving six levels of Factor A and two levels of Factor B 2 x 3 x 4 Factorial Design: A design that consisted of two levels of Factor A, three levels of Factor B, and four levels of Factor C Thus, 2 x 4 would represent two different IV with the first having two levels and the second having three levels.

Main Effects The treatment differences (between levels of a given factor) in a factorial design are called main effects. This would be equivalent to performing two single-factor experiments; one would compare the amounts of money given for recall, and one would compare the levels of pictorial context.

Interaction Effects, In a Factorial design, the IV may combine in various ways in their effects on the DV. Such a combination effect is called Interaction effect. Interaction effects cannot be determined by performing separate single-factor studies and thus are unique to factorial design An interaction effect is the result of two IV combining to produce a result different from that produced by either variable alone. Occurs when the effect of one IV depends on the level of another IV. o Example: an interaction would be present if the two IV had a greater influence in combination than would be expected from adding the main effects of each variable together.

Ordinal interaction, Treatment-by-order interaction: with such an interaction, the effects of the treatments would depend on the order of presentation. This type of confound would also be inevitable in drug or clinical treatment studies as the Strupp and Hadley psychotherapy study. Differential order effect.

disordinal interaction, subject variables Subject Variable: a variable that refers to an existing attribute of an individual (for example, sex) that an experimenter selects rather than manipulates. Using subject variables in factorial designs help us to answer questions when characteristics of our participants in the research are being addressed

Within subjects designs: Within-subjects Design: Design in which participants receive more than one level of the IV

intrasubject counterbalancing Intrasubject counterbalancing: a procedure in which all participants receive the tasks in the same counterbalanced order. ABBA, or BAAB

advantages and disadvantages of within-subjects designs, counterbalancing, In many cases, order and time can be controlled (but not eliminated_ through such procedures as counterbalancing. Complete Counterbalancing: certain conditions must be met. o 1. First each condition must occur equally often o 2. Second each condition must precede and follow all other conditions an equal number of times. o Counterbalancing ensures that every possible sequence appears at each presentation of the treatment. A method for equating groups to ensure that differences in an experiment are not a result of biased participant assignment Intrasubject Intragroup

intragroup counterbalancing, Counterbalancing ensures that every possible sequence appears at each presentation of the treatment. A procedure ensuring that every possible equence appears at each presentation of the treatment

repeated measures One common application of within-subjects design uses repeated measures as one factor in a factorial design. Useful in studying psychological processes that occur over time and is widely used in studying human and animal learning processes. Example: we might present the same subject with different levels of problem difficulty (for ex, easy versus difficult problems), which would be one factor, and the other factor might be the number of trials (the repeated-measures factor).

mixed designs Mixed design: a design that includes both within-subjects and between subjects components. Example: assume that we want to perform a biofeedback experiment to determine the effect of feeback on our ability to control hear rate.

matched subjects designs A procedure for equating groups in an experiment and reducing within-groups variance by using participants who are similar Advantage resulting from the prior matching of participants to groups: o 1. The first is that such matching ensures that the groups in a study are equal (on the measure by which they were matched) before the treatment is introduced. Equating groups in this way is especially valuable for situations in which we have a limited number of participants but cannot use a within-subjects design. o 2. The second advantages is that matching participants reduces the within-group variance (error variance0 and thus making our design more sensitive to treatment than a comparable between-subjects design. However, this is possible only when the matching factor has a high correlation with the dependent variable.

correlated groups designs, order effects The effect in which participation in one condition of the experiment influences results in a later condition

practice effects The effect brought about by the continued repetition of a task

carry-over effects Transfer of training from one condition to the other, especially if a research participant learned to control heart rate with total feedback

latin square design, Special case of Incomplete counterbalancing To form a Latin square, we must have the number of rows equal the number of columns, which equals the number of conditions. The selection of the conditions traditionally has been performed randomly One common use of Latin square design is with psychopharmacological research.

mixed designs, use of matched-subjects design as control and as experimental procedure As control: o We know that a particular individual variable or characteristic has a high correlation with the DV, equal groups may be obtained by matching along this characteristic.

o The task of forming matched groups of participants consists of two steps. 1. Firsts, pairs of participants are matched on some measure that is correlated with performance on the DV. 2. Second, one member of each pair is assigned randomly to either the experimental or the control group; the other member is then assigned to the other group. As Experimental o It is possible to analyze the matching factor. o Randomized block design: When matching factor is analyzed, the resulting procedure is traditionally called a Randomized Block Design.

Participant-scientist interaction, Danziger, Suggests that two historical models dating from 19th century having influenced psychological research in this country: o The Leipzig model and the Paris model

Leipzig model, Typified by the research of Wilhelm Wundt and others about normal cognitive processes, makes little differentiation between the participant and the experimenter in terms of expected roles or social status. In the model, the same person might serve in both capacities at different times.
Psychophysics, in its study of the relationship between physical properties of a stimulus and our sensory experience of it, is one example of the Leipzig model in its methods

Paris model,
typified by research about hypnosis and abnormal psychology, makes a clear distinction between the roles of the scientist and the research participant. In this model scientists and participants are different, and one would never play the role of

the other.

ecology, Ecology is the scientific study of the relationship of living organisms with each other and their environment. It is applied to the relationship between humans and their environment. Ecology of nature; the interrelationships between humans or animals and their environment. Ecology of the psychological experiment: by examining the relationships among the scientist, the research participant, and the experimental situation Ecological Validity: seeks to determine whether the impacts of the important relationships among the scientist, the participant, and the context have been considered fully in evaluating a given piece of research.

ecology, experimenter effects, Experimenter effect: an effect upon participants in an experiment that can be traced to the behavior of the experimenter but is not designed to be apart of the experimental manipulation

bias, A prejudice in the design, performance, analysis, or presentation of a research project

personal equation, A constant factor related to a persons psychology and physiology that results in individual differences in the measurement of variables

interrater reliability, The correlation between ratings of two or more raters in a research study

experimenter bias, The inherent differences between experimenters in the way they perceives and respond to the world, personality may also create bias. It cannot only effect the data the experimenter records but also can exert strong influence on the theoretical interpretation of the findings Based on data collection and conveyed by interactions between the experimenter and the research participant

biased data collection, biased interactions with subjects, ways to reduce/eliminate experimenter bias, subject factors, A characteristic of the participants behavior that causes unexpected behavior in the experimental setting

Hawthorne effect, The situation in which the participants knowledge that they are part of an experiment influences the results differentially

Rosenthal effect, placebo effects, double-blind experiment,

The situation in which both the participant and the experimenter or agent of the experimenter are blind to certain key aspects of the experiment. Double-blind studies are often used in drug research in which neither the participant nor the person who administers the drug knows whether it contains the active ingredients or a placebo.

Demand Characteristics, The overall effect of the situation on a participants behavior; generally used to denote aspects of an experiment that bias the results and prevent adequate influence of the IV

reactivity, social roles of subjects (4), quasi-control group, cultural and social bias, paradigm The worldview and set of assumptions that direct what a scientist examines, what measurements are made, and how these are to be understood

From Memory Questions 1.Discuss post hoc tests and when to use them 2. Discuss main effects and interaction effects 3. What are the advantages of the within-subject designs 4. Know which design is most appropriate for which hypothesis (I'll state a hypothesis, you choose and explain which design is best to test it). 5. Discuss Barber's (1976) 13 ways to improve Psychological Research (page 257, Box 11.1) 6. What does it mean to say that experimenter bias does not affect internal validity? 7. Provide three examples of experimenter bias 8. Be able to design an experiment to test hypothesis (between-subject, within-subject, factorial, mixed), identify the IV's, the DV, control variables, potential carry-over effects (if any), state a testable hypothesis including operational definitions, etc.

Within-subjects designs are good for Increasing the sensitivity of a study

Which of the following best illustrates why would we expect a reduction of error variance in a within-subjects design? Stephens behavior in situation A, B, and C is more similar than Marys, Kevins, and Stuarts behaviors in situation X, Y, and Z.

In a within-subjects design Each participant serves as his or her own control

The procedure that eliminates order and time effects is Not: o o o

Intrasubject counterbalancing Intragroup counterbalancing Complete counterbalancing

In a randomized block design, the term ___ is usually used when matching is on interval or ratio scales. Leveling

What effect do within-subjects designs have on F-ratios? F-ratios are more sensitive to changes in the treatments

Matched-subjects procedures reduce error variance More than between-subjects designs

In a within-subjects design Smaller treatment differences are needed to reject the null hypothesis

In a within-subject design, as compared to a between-subjects design, you are More likely to obtain a large F ratio

In a Mixed design Error variances of the between and within comparisons are calculated differently

Matched subjects procedures are more sensitive than between-subjects designs

if complete counterbalancing is used for a three condition experiment, there will be___ possible orders. Six

A researcher wants to conduct an experiment that uses a 2 X 3 X 3 design and requires ten participants in each group, then _______ participants would be needed for a between-subjects design but only _______ would be needed if a within- subjects design is used. 180, 10

Likely confounds in a within-subjects design are All the these o Order effects o Carryover effects o Time-related effects

______ effects refer to any increases or decreases in performance due to continued repetition of tasks. Order

The design that allows for the greatest reduction of error variance is Within-subjects

In the within-subjects design, the effects of treatment order can be controlled by counterbalancing

The design below best illustrates which of the following designs? R Participant 1 to 10 Level 1 M R Participant 1 to 10 Level 2 M within-subjects design

Another term for error variance is within-group variance

In a randomized block design, the term blocking is usually used when matching is on a(n) nominal scale

Different experimenters observe the world in different ways. This is referred to as

the personal equation.

Save

Question 2 (1 point)

Ecological validity may be threatened by the Question 2 options:

experimenters expectations. experimenters personality. experimenters research assistant. All of these

Save

Question 3 (1 point)

The term that refers to the relationship between organisms and their environment is called Question 3 options:

ecology.
Save

Question 4 (1 point)

Demand characteristics can result from all of the following except Question 4 options:

the experimental situation. expectations. cultural bias. faking.

Save

Question 5 (1 point)

To combat cultural bias in research, the U.S. National Institutes of Health implemented a policy in 1994 that requires all federally funded research involving human participants to include Question 5 options:

women and minority groups.

Save

Question 6 (1 point)

A scientist can subtly influence the outcome of an experiment by making biased decisions about Question 6 options:

unintended variations in the experiment that tend to favor the hypothesis whether to accept or discard a participants data conducting unplanned ad hoc analyses all of these
Save

Question 7 (1 point)

Demand characteristics that are especially strong may introduce error and thus threaten Question 7 options:

internal validity.
Save

Question 8 (1 point)

Ecological validity can be threatened by Question 8 options:

experimenter bias. the Hawthorne effect. demand characteristics.

all of these
Save

Question 9 (1 point)

If there are multiple observers used in experimental design, it is especially important to Question 9 options:

operationally define variables and provide training.

Save

Question 10 (1 point)

Between-subjects and within-subjects designs are most consistent with the _______ model of ecology.

Paris

Save

Ecology of the psychological experiment is concerned with the interactions among Question 1 options:

scientist, participant, and situation

Save

Question 2 (1 point)

The Leipzig model of research involved ______, while the Paris model emphasized ______. Question 2 options:

little distinction between participant and R; clear distinction between participant and researcher
Save

Question 3 (1 point)

To reduce bias, Barber suggests that _______ should specify exactly what _______ should and should not to. Question 3 options:

those who plan studies; those that run those studies

Save

Question 4 (1 point)

The higher the correlation between the ratings of two independent observers in an experiment Question 4 options:

the less the ratings represent inconsistent subjective factors of the raters.
Save

Question 5 (1 point)

To control for the Hawthorne effect, you need Question 5 options:

a control group that is given special treatment but does not get the experimental treatment.
Save

Question 6 (1 point)

Which of the following is not a way of avoiding experimenter bias?

use a placebo condition

Save

Question 7 (1 point)
One perspective argues that all human experimentation is really a(n)

social event.

Question 8 (1 point)

The case of Samuel George Morton is strong evidence that Question 8 options:

a scientist can subtly influence the outcome of an experiment.

Question 9 (1 point)
Research participants performance may be altered as a result of special attention given to them rather than because of any effect of the independent variable. This is called Question 9 options:

the Hawthorne effect

Save

Question 10 (1 point)

Biased data collection is an example of Question 10 options:

experimenter effects.
Fudging the data refers to

intentional altering of experimental results.

Save

Question 2 (1 point)

If the experimenter knows which treatments the participants have been administered, but the participant do not, the study is called Question 2 options:

blind.
Save

Question 3 (1 point)

When participants in an experiment behave in a way that they think is expected by the experimental situation, they are responding to Question 3 options:

the Hawthorne effect.

Save

Question 4 (1 point)

To correct for the possibility that a high interrater reliability is actually due to different but consistent ratings from raters, a statistic called _____ can be used. Question 4 options:

Cohens kappa
Save

Question 5 (1 point)

An experiment in which neither the experimenter nor the participants knows which treatment has been administrated to the participant is called Question 5 options:

double blind.
Save

Question 6 (1 point)

I want to examine my subjective reactions to environmental variables. This statement best demonstrates the ______ model. Question 6 options:

Leipzig
Save

Question 7 (1 point)

As a suggestion to improve research, Barber advises that it is important for students to learn that Question 7 options:

good research methods are invaluable in answering questions.

Save

Question 8 (1 point)

In studying animals, overinterpreting issues such as animal language or animal consciousness can Question 8 options:

threaten ecological validity.

Save

Question 9 (1 point)

The placebo effect is Question 9 options:

both psychological and physiological in origin.

Save

Question 10 (1 point)

The Leipzig model was typified by the research of _______.

Wundt

A researcher wants to test the effects of a drug but suspects it might interact with gender. She gives men and women either a placebo or one of three doses of the drug (5 mg, 10 mg, or 20 mg). This experiment

requires 8 cells.

The following is what type of experimental design? R Group A Level 1 T M R Group B Level 2 T M R Group C Level 3 T M R Group D Level 4 T M -

multilevel completely randomized design

If one uses a subject variable in a study

causality cannot be attributed to the subject variable.

In a 2 X 2 design, cell means represent

the combined effect of one level of one independent variable and one level of another independent variable.