Research Assessment

AZA 2490
Introduction
Introductions
• Lecturer:
• Catherine Rogers
• M.A. Research Psychology, PhD candidate
• Research background
• Email: [email protected]
• Consultation times: Wed 12.00 to 13.00

• Students write down: Keep your

• Why I am here answers in mind
• My short term and long term academic goals – how does this
course help you
• My academic interests with your future
aspirations?
Course Information:
• Unit Guide
It is your
responsibility to
• Tutorial Guide read these
documents and
to be informed.
• Moodle
 Read each chapter of G&W
before/after class: complete
Prescribed textbooks: the learning check examples
and the problems at the end
of each chapter

• Gravetter F. J., & Wallnau, L. B. (2013). Statistics for the Behavioral

Sciences (9th ed.). Wadsworth, Cengage Learning

• Allen, P., Bennett, K. & Heritage, B.(2014). SPSS Statistics Version 22: A
practical guide. South Melbourne, Australia: Cengage Learning

• Findlay, B. (2014). How to write psychology research reports and

essays. (7th ed). Australia: Pearson.
Course Overview
1. Introduction to the course & Revision of the Research Process.
2. Revision of Descriptive Statistics.
3. Hypothesis testing.
4. One-Sample t statistic.
5. Two related samples t-test (including Wilcoxon Test)
6. Two independent samples t statistic continued (including Mann-Whitney Test)
7. Correlation (including Spearman)
8. Intro to ANOVA (including Kruskal-Wallis test)
9. One-way Repeated-Measures ANOVA (including Friedman test)
10. Two Factor ANOVA
11. Chi-square
12. Revision
Assessments
Assessment Task Value Due Date
Lecture Topic Quiz 1 5% Opens: 31 March 2017
Based on lectures 1 to 5 Closes: 07 April 2017 at 16.00
Lecture Topic Quiz 2 5% Opens: 19 May 2017
Based on lectures 6 to 10 Closes: 26 May 2017 at 16.00
Class Participation 4 x 2.5% = 10% Variable dates (see unit schedule)
Assignments

Assignment 1 15% 13 April 2017 at 16:00

Assignment 2 15% 12 May 2017 at 16:00

Examination 50% During Examination Period (see

exam timetable)
Hybrid Learning
Read each
chapter NB!
• 2 online tutorials before/after
class to
• diarise all events – see unit guide reinforce your
This course
• Found on Moodle requires that you
learning reinforce your
learning by going
through examples
• 8 face-to-face tutorials in your own time.
There are
additional
• 12 face-to-face lectures readings,
resources and
videos on Moodle
as well.
Why study research methods?
2 key objectives:
1. You should be able to understand research you read (know if it is
sound, know it’s limitations) so that you can make informed
decisions as a practitioner

2. You should be able to complete your own rigorous and statistically

sound research study so that you can contribute to knowledge
production.
How do you feel about
Research and
Statistics?
Why am I here? Research
generates
information
that society
• Why are you studying psychology? needs
• Do you wonder why people do what they do?
• Do you ask why, what, how?
• Do you want to help people, to counsel people?

Of all the many things that we teach our students, perhaps the most
important is how to think like scientists – Buskist, 2007
 Lecture &
I challenge you: tutorial
attendance is
key to
• Give research and statistics a chance mastery

• Keep application in mind – conducting a full research project to answer a fascinating

research question

• Research = systematic problem solving

• Research and statistics are there to help you write an honest story.

• Statistics = logic (and not as mathematics). Learn the logic and apply it systematically.

• Develop these important skills -> increase your employability

I invite you to learn the unique language of
research methods

How do you PRACTICE

learn a new &
language? PERSISTENCE
 Revision:
The Research Process
 Humans are curious creatures.

Why do things happen?

How do they happen?
Why do they happen?

We conduct research in order to better

understand our world.
https://www.bcps.org/offices/lis/researchcourse/develop_writing_method_qualitative.html
The Selection of a Research Approach:
• We start with a topic we want to study.

• Then we need to choose the most suitable approach for studying the topic.

• The research approach should be chosen with careful consideration of the

research problem/topic

• Behind each approach are philosophical assumptions (worldviews/paradigms)

• These philosophical assumptions should be carefully considered when choosing

the research approach, method and design

Adapted from Creswell, 2013

Answering the Research Question
• Depending on the research question being asked, we can
either conduct research that is:
• Quantitative (numbers)
• Qualitative (words) or
• Mixed Methods (both)

• It is essential that the research question being asked and the

method used to answer that question are matched correctly.
Qualitative Methods
• An approach for exploring and understanding the meaning individuals or
groups ascribe to a social phenomena.

• An inductive approach to research involving emerging questions and

procedures with data typically collected in the participant’s setting.

• The final written report has a flexible structure.

• Those who engage in this form of inquiry support a way of looking at

research that honours an inductive style, reflexive engagement with data, a
focus on individual meaning, and the importance of rendering the
complexity of a situation.

Adapted from Creswell, 2013

Qualitative: Methods for collecting data
• Interviews
• Focus groups
• Ethnographic observation
• Narratives
• Naturally occurring talk, text and video
Qualitative designs & methods for analysing
data
• Thematic analysis
• Narrative analysis
• Phenomenological analysis
• Grounded Theory analysis
• Discourse Analysis
Qualitative
• Ethnomethodology research is a
• Conversation Analysis lot more than
just finding
• Membership categorization analysis themes.
Quantitative Methods
• An approach for testing objective theories by examining the relationship
among variables.

• These variables, in turn, can be measured, typically on instruments, so that

numbered data can be analysed using statistical procedures.

• The final written report has a set structure.

• This form of inquiry has assumptions about testing theories deductively,

building in protections against bias, controlling for alternative explanations,
and being able to generalize and replicate the findings.

Adapted from Creswell, 2013

Quantitative: Methods for collecting data
• Surveys (quantitative or numeric description of trends, attitudes, or
opinions)
• Psychometric tests
• Measurements (e.g. heart rate, test marks etc)

Adapted from Creswell, 2013

Quantitative: Designs & methods for
analysing data
Designs:
• Experimental (with the random assignment of subjects to treatment
conditions)
• Quasi-experimental (nonrandomized assignment)
• Non-experimental (for example correlational research)

Methods of analysis: The focus of

this course.
• Various statistical techniques
Mixed Methods
• An approach to inquiry involving:
• collecting both quantitative and qualitative data,
• integrating the two forms of data, and
• using distinct designs that may involve philosophical assumptions and
theoretical frameworks.

• The core assumption: that the combination of qualitative and

quantitative approaches provides a more complete understanding of
a research problem than either approach alone.

Adapted from Creswell, 2013

Mixed Methods Designs & methods for
analysing data
• Convergent parallel
• Explanatory sequential
• Exploratory sequential
• Transformative
• Embedded
• Multiphase

Methods of analysis:
• Various statistical and text-analysis techniques

Adapted from Creswell, 2013

Worldviews & choosing a research approach:
• Although philosophical ideas remain largely hidden in research, they
influence the practice of research and need to be identified.

• In planning a study, researchers need to think through the

philosophical worldview assumptions that they bring to the study, the
research design that is related to this worldview, and the specific
methods or procedures of research that translate the approach into
practice.

Adapted from Creswell, 2013

What is the nature of our reality?

What is the nature of knowledge?

Worldviews/Paradigms/Epistemologies/Ontol
ogies
Epistemology: the study of the nature of knowledge (different
paradigms or ways of understanding knowledge)

• We will discuss 4 paradigms:

1. postpositivism,
2. constructivism,
3. transformative, and
4. pragmatism
 Quantitative

The scientific method/positivist/postpositivist

• Deterministic: causes determine effects. Conduct experiments to identify and assess the
causes that influence outcomes.

• Reductionist: reduce ideas into a small, discrete sets to test, such as the variables that
comprise hypotheses and research questions.

• An Objective Reality: knowledge generation is based on careful observation and

measurement of the objective reality that exists “out there” in the world.

• Theory driven: there are laws or theories that govern the world, which should be tested
and refined so that we can understand the world. A researcher begins with a theory,
collects data that either supports or refutes the theory, and then makes necessary
revisions.

Adapted from Creswell, 2013

The Constructivist Paradigm Qualitative

• Complexity: Individuals develop subjective meanings of their experiences. These meanings are varied and
multiple, leading the researcher to look for the complexity of views.

• Participant oriented: The goal of the research is to rely on the participants’ views of the situation.

• Contextual: These subjective meanings are negotiated socially and historically. They are not simply
imprinted on individuals but are formed through interaction with others and through historical and cultural
norms that operate in individuals’ lives.

• Reflexive: Researchers recognize that their own backgrounds shape their interpretation. They position
themselves in the research to acknowledge how their interpretation flows from their personal, cultural, and
historical experiences.

• Inductive: The researcher’s intent is to make sense of the meanings others have about the world. Rather
than starting with a theory, inquirers generate or inductively develop a pattern of meaning.

Adapted from Creswell, 2013

The Transformative Paradigm Qualitative

• Political reality: Research inquiry needs to be intertwined with politics and a

political change agenda to confront social oppression. Focus is on the needs of
groups and individuals in our society that may be marginalized or
disenfranchised.

• Action agenda: Research contains an action agenda for reform that may change
lives of the participants, the institutions, and the researcher’s life. Addressing
important social issues such as empowerment, inequality, oppression,
domination, suppression, and alienation.

• Participant involvement: collaborative research involving the participants so as to

not further marginalize the participants during the inquiry. The participants may
help design questions, collect data, analyse information, or reap the rewards of
the research. The research provides a voice for these participants.

Adapted from Creswell, 2013

The Pragmatic Paradigm Mixed Methods

• Application focused: Pragmatism is focused on solving real world

problems and designing research according to the needs of the
context in question.

• Problem-focused: Researchers emphasize the research problem and

use all approaches available to understand the problem (see Rossman
& Wilson, 1985).

• Pluralistic solutions: Using a variety of methods to derive knowledge

about the problem.

Adapted from Creswell, 2013

Worldviews/Paradigms/Epistemologies/Ontol
ogies
Postpositivism Constructivism
• Determination • Understanding
• Reductionism • Multiple participant meanings
• Empirical observation and • Social and historical construction
measurement • Theory generation
• Theory verification
Transformative Pragmatism
• Political • Consequences of actions
• Power and justice oriented • Problem-centered
• Collaborative • Pluralistic
• Change-oriented • Real-world practice oriented
Adapted from Creswell, 2013
 The research design you choose should be based
on the research question you are trying to answer.
The focus of this course:

QUANTITATIVE
METHODS
Underlying philosophical
assumptions of Positivism
 1. Research
Question
Interpretation of
2. Hypothesis
the statistical
results

1. Recruit
Statistics participants
used to test the 2. Collect
hypothesis data
https://www.bcps.org/offices/lis/researchcourse/develop_writing_method_qualitative.html
Back to basics:
• What are statistics?

• The term statistics

refers to a set of
mathematical
procedures for
organizing,
summarizing, and
interpreting
information.

http://edition.cnn.com/election/results
Statistics
• Population: the set of all individuals of interest in a particular study
• Example: All Monash Students

• Sample: a set of individuals selected from the population, usually

intended to represent the population in a research study
• Example: 200 Monash students

• Research typically involves measuring each individual to obtain a

score.
• Every sample of individuals produces a corresponding sample of
scores.
Copyright © 2017 Cengage Learning. All Rights Reserved.
The relationship between the population and
the sample
What we measure:
• Variables: a characteristic or condition that changes or has different
values for different individuals.
• Example of a characteristic of an individual in a population: weight, gender,
personality
• Example of an environmental condition in the population: weather, economic
climate, time of the day.
To demonstrate
changes in a variable,
we take
measurements of the
variable – a
measurement of the
variable is a datum.
What we collect:
• Data (plural): are measurements or observations.

• A data set is a collection of measurements or observations.

• A datum (singular): is a single measurement or observation

(score/raw score)
Parameters and Statistics
• A parameter is a value, usually numeric, that describes a population.
A parameter is usually derived from measurements of the individuals
in the population.
• For example: the average matric score for all matrics in 2016

• A statistic is a value, usually a numerical value, that describes a

sample. A statistic is usually derived from measurements of he
individuals in the sample.
• For example: the average matric score for a sample of matric students in 2016
Types of Statistics
• Descriptive: statistical procedure used to summarise, organise, and
simplify data
• Example: Mean, Median, Mode, Frequency, Probability

• Inferential: techniques that allow us to study samples and then make

generalizations about the population from which they were selected
• Example: t-test, correlation, ANOVA, regression

• Sampling error is the naturally occurring discrepancy, or error, that

exists between a sample statistic and the corresponding population
parameter.
 Why inferential statistics exist?
Samples are
Researchers use generally
Populations are
So a sample is sample statistics representative of
usually very
selected from the as the basis for a population, it is
large. Measuring
population – to drawing not expected to
everyone in a
represent the conclusions give a perfectly
population is not
population. about population accurate picture
possible.
parameters. of the
population.

This discrepancy is called the sampling error. Sampling error is the

fundamental problem that inferential statistics aims to address.
Sampling Error Example
The Role of Statistics in Experimental Research
 We are looking
Types of Research at types of
RESEARCH now
and not types of
statistics.
• Descriptive Research:
• Describes individual variables
• Research studies that are conducted simply to describe individual variables as
they exist naturally.
• Statistical techniques such as measures of central tendency and variability
• Example: the number of hours spent studying every day

• Relational Research
• Examine the relationships between two or more variables.
Two distinct data structures

Research on the relationship

between variables

1 2

Comparing 2 or more groups of

One group with two variables
scores
measured for each individual
–
-
Experimental and
The Correlational Method
Nonexperimental methods
Two distinct data structures

Research on the relationship

between variables

1 2

Comparing 2 or more groups of

One group with two variables
scores
measured for each individual
–
-
Experimental and
The Correlational Method
Nonexperimental methods
One group with two variables measured for
each individual
• The correlational method
• In the correlational method, 2 different variables are observed to
determine whether there is a relationship between them.
• Statistics for the correlational method:
• Numerical data: correlation
• Non-numeric data (e.g: gender): chi-square
• Limitations of the correlational method:
• It does demonstrate the existence of a relationship between 2 variables
• It does NOT provide an explanation of the relationship, i.e. a cause-an-effect
relationship.
The Correlational Method

Figure 1.4 Note that there are two separate measurements for each individual (wake-up time and academic performance). The
same scores are shown in a table (a) and in a graph (b).

Table 1.1 Correlational data consisting of non-numerical scores. Note that there are two measurements for each individual:
gender and cell phone preference. The numbers indicate how many people are in each category. For example, out of the 50
The danger of mixing up causality and correlation:
Ionica Smeets at TEDxDelft

https://youtu.be/8B271L3NtAw
Two distinct data structures

Research on the relationship

between variables

1 2

Comparing 2 or more groups of

One group with two variables
scores
measured for each individual
–
-
Experimental and
The Correlational Method
Nonexperimental methods
Comparing 2 or more groups of scores
• Experimental & Nonexperimental Methods

• The relationship between variables is examined by using 1 of the

variables to define the groups, and then measuring the 2nd variable to
obtain scores for each group.

• Statistics for comparing 2 or more groups of scores:

• Descriptive statistics summarize and describe the scores in each group.
• Inferential statistics determine whether the differences between the groups
can be generalized to the entire population – statistical tests for comparing
the averages between groups include t-tests and ANOVA
Comparing 2 or more groups of scores
Experimental vs Nonexperimental Methods
• The results from an experiment allow a cause-and-effect explanation.
• We can say that changes in one variable are responsible for causing
differences in a second variable

• A nonexperimental study does not permit a cause-and-effect

explanation.
• We can say that changes in one variable are accompanied by changes in a
second variable – but we can’t say why
The Experimental Method
• The goal of an experimental study is to demonstrate a cause-and-
effect relationship.
• The experiment has two distinguishing characteristics:
1. The researcher manipulates one variable by changing its value from one
level to another. For example: manipulating the amount of violence
exposure by giving one group a violent video game and another a non-
violent video game.

2. The researcher must exercise control over the research situation to ensure
that other, extraneous variables do not influence the relationship being
examined.
 Whenever a
research study
Control allows more than
1 explanation for
results – the
study is
• There are two general categories of variables that confounded.
researchers must control in experiments:
1. Participant variables such as age, gender, and intelligence that vary from
one individual to another
2. Environmental variables such as lighting, time of day, and weather
conditions
• Researchers use three methods to control variables:
1. Random assignment – each participant has an equal chance of being
assigned to each group
2. Matching – ensure equivalent groups
3. Holding constant (such as only including individuals with specific
characteristics)
In the experimental method, one variable is manipulated
while another is observed and measured.

To establish a cause-and-effect relationship between the

two variables, an experiment attempts to control all
other variables to prevent them from influencing the
results.
Variables in the Experimental Method
• The independent variable (IV) is manipulated by the researcher.
• In behavioral research, the IV usually consists of the two (or more) treatment
conditions to which subjects are exposed.
• The IV consists of the antecedent conditions that were manipulated prior to
observing the dependent variable.

• The dependent variable is observed to assess the effect of the

treatment.

• Example: which is the IV?

• Exposure to violent video games causes violent behaviour?
Conditions in the Experimental Method
• Individuals in a control condition do not receive the experimental
treatment.
• Instead, they either receive no treatment or they receive a neutral/placebo
treatment.
• The purpose of a control condition is to provide a baseline for comparison with the
experimental condition.

• Individuals in the experimental condition do receive the experimental

treatment.

• The goal of this kind of research is to determine that treatment has an

effect by showing that the scores in the treatment condition are
substantially different from the scores in the no-treatment condition.
Experimental Design - Psychology

https://youtu.be/Wvd-oqAeBLA
The Non-Experimental Method
• There are research designs that are not true experiments but still
examine the relationship between variables by comparing groups of
scores:
• Non-equivalent groups
• Pre-Post Studies

• Correlational studies are non-experimental but compare two

variables within the same group

• In a nonexperimental study, the IV that is used to create the groups is

often called the quasi-independent variable (it isn’t manipulated).
The Non-Experimental Method
Non-equivalent
groups:
Comparing pre-
existing groups

In these studies,
the researcher
Pre-post study has limited
control over
extraneous
variables.
Understanding variables
• Constructs are internal attributes or characteristics that cannot be directly
observed but are useful for describing and explaining behavior.
• For example: anxiety, intelligence

• An operational definition identifies a measurement procedure for

measuring an external behavior and uses the resulting measurements as a
definition and a measurement of a hypothetical construct.
• An operational definition has 2 components:
1. It describes a set of operations for measuring a construct
2. It defines the construct in terms of the resulting measurements
• Example: intelligence measured and defined by your performance on a IQ test
Types of variables
• The variables in a study can be characterized by the type of values
that can be assigned to them:

• A discrete variable consists of separate, indivisible categories; no values exist

between two neighboring categories.
• Can count it

• A continuous variable contains an infinite number of possible values that fall

between any two observed values, and is divisible into an infinite number of
fractional parts.
• Can measure it on a scale
Discrete and Continuous Data

https://youtu.be/_yAQb8gWBpU
Measuring Variables
• To establish relationships between variables, researchers must
observe the variables and record their observations:

• This requires that the variables be measured.

• The process of measuring a variable requires a set of categories called a scale

of measurement.

• Scales of measurement are important because they help determine the

statistics that are used to evaluate data.
Four Types of Measurement Scales
1. A nominal scale consists of a set of categories that have different names.
• Label and categorize observations, but do not make any quantitative distinctions between
observations.
2. An ordinal scale consists of a set of categories that are organized in an ordered
sequence.
• Ranks observations in terms of size or magnitude
3. An interval scale consists of ordered categories that are all intervals of exactly
the same size.
• Equal differences between numbers on scale reflect equal differences in magnitude.
• However, the zero point on an interval scale is arbitrary and does not indicate a zero amount
of the variable being measured.
4. A ratio scale is an interval scale with the additional feature of an absolute zero
point.
• Ratios of numbers reflect ratios of magnitude.
Scales of Measurement - Nominal, Ordinal, Interval,
Ratio (Part 1) - Introductory Statistics

https://youtu.be/_yAQb8gWBpU
Scales of Measurement - Nominal, Ordinal, Interval,
Ratio (Part 2) - Introductory Statistics

https://youtu.be/yJpiUHbLKLU
References:
• Many of these slides are directly from or adapted from your textbook and
generic textbook slides:

Gravetter, F. J. & Wallnau, L. B. (2017). Statistics for the Behavioural Sciences.

10th ed. Cengage.

• Other sources:

Creswell, J. W. (2014). The Selection of a Research Approach In Research Design:

Qualitative, Quantitative, and Mixed Methods Approaches. Sage.

See individual videos for the URL.