Research methods tutorial notes

Tute 1:

Why is it important for graduates of a psychology degree to know how to (systematically)

critically evaluate research???

- To evaluate new therapies

- To avoid inaccurate research
- Identifying gaps and limitations in research
OSC paper 36% of studies replicated results when initially 87% met criteria for
replicability

Preregistering your honours project state hypothesis, method before you collect the data
easier to publish
Let your supervisor choose your study for you

1. Is the theory behind the research presented adequately & in sufficient detail?

Good start but they should have explained ENT better not clear what the IV and DV were

2. Are the hypotheses presented clearly?

Prediction = correlational/ pearsons correlation  presented clearly but there is a mismatch

between the hypothesis and analysis

3. Is the study designed in a way that will allow the hypotheses to be tested?
Yes, the design appears fine

4. Is the dependent variable operationalised appropriately?

DV: distance perception given that it’s not a lab study, but the real-world measurements of
height estimation were taken in the real world on heights  Horizontal illusion
5. Are the results presented appropriately?
Yes, the IV and DV appear to be inverted on the axis
6. Are the statistics appropriate?
Used correlation when they should have used regression. Correlations are quite low/weak
would prefer to use r rather than the strange statistic of probability of replicating
7. Are the conclusions made warranted by the analysis?
Distance misperception produces height fear implies causality very no causal analysis
done. The alternative is equally plausible due to the correlational nature of the study

Tute 2:
- Game theory context of RR economics created games for people to play spend or
save to predict fiscal behaviour
- Now used psychologically to explain certain behaviours like altruism, cooperation,
and punishment
- Dictator game: simple game derived from game theory Player A is given $10 A
decides how many $ to give to player B
- Play with every person
What did we learn?
- Impacts of social desirability
- Told allocations will be revealed so it affected scores
- What factors might affect scores? Incentives, quality of conversation, individual
differences, reputation, how much you talked, time spent of interactions
- Manipulated social distance/emotional proximity empathise with the recipient
see them more as a real person
- Variable 1 = social proximity Condition 1: small emotional proximity vs Condition 2:
high emotional proximity
- Variable 2 = reputation = 2 condition Also manipulated if told that allocations were 1 st
condition: confidential vs 2nd non-confidential
- 2X2 FACTORIAL design between subjects ANOVA would be used
- Hypotheses: 1. If social distance impacts generosity, interaction condition should give
more than non-interaction condition
- 2: if reputation impacts generosity, revealed should give more than concealed
- DV = generosity and operationalised through dictator game
- Quasi experimental design because were using pre-existing classes for our
conditions Quasi random therefore we can infer causality important for
discussion

Tute 3

- Different conditions per tute class

- We were given 1 min to talk
- Other classes had no communication
- High and low emotional distance conditions
- Filter on multi-search exclude medicine, health, and epidemiology
Hypothesis
1. If social distance impacts generosity, contact should give more than no contact
2. If reputation impacts generosity, revealed should give more than concealed

Analysis
- 2X2 Between subject ANOVA (factorial ANOVA)
- 2 IVS with 2 levels
- 2 Categorical variables and 1 numerical variable
Assumptions
- Independence of Observation (give)
- Normally distributed DV withing each level of each IV (SWILK test)
- Homoscedastic variance in each condition equal (Levine’s test)
- DV is numeric (interval/ratio)
- No Multi-collinearity (don’t want IVS to be doing the same thing) (VIF)

Varaibles:
- Sex
- Amount given DV 1-10
- Social distance contact/no contact
- Reputation  concealed/revealed
- Age

Main effects = marginal means

 - Averaging across reputation of concern does contact/non-contact affect amount given
- Interaction differences of differences of revealed and concealed are seen
- If there is an interaction than we can’t look at main effects

Assumptions
- Failed to meet the assumption of equal variances (Levine’s test = significant)
- Violated the assumption of normality given that 3 of the 4 conditions are non-normal
- 3rd condition = bimodal (two groups)
- 2nd condition = relatively normal
- 1st condition = leptokurtic
- 4th condition = negative skew
- Highlight why its non-normal in the results as its part of the assumptions
- Can bring up in discussion if its relevant to limitation and future research.
- SK test for normality tells us why its non-normal Skewness and Kurtosis are both
significantly skewed/kuritc report kurtosis and skewed statistics and p-value from
the SK test for each of the cells (conditions)
- Interval data in the DV can assume it’s right
- VIF is good so no multicollinearity
- Highlight which assumptions are violated but state that ANOVA is robust to
violations

ANOVA
- Main effect of reputation = non-significant
- Main effect of social distance = non-significant
- Interaction between reputation/ social distance = significant the difference b/w
contact and no contact in the revealed conditions is larger than the difference between
the contact and no contact in the concealed condition (can tell what is larger from the
cell mean) can check with the CI plot
- No ad-hoc estimations as it’s a 2X2
- Don’t just report significance but the effect size as well  partical eta squared for the
model effect sizes for main effects not interested in as it wasn’t significant but
interested in the interaction eta squared
- Small effect size (+ .1) for the interaction that was statistically significant

Discussion:
- Hypotheses were not supported
- BUT there was a significant interaction, with meaningfully higher values present only
with Contact + revealed condition, so main effects couldn’t be interpreted in isolation
anyway! This is our top-line finding
- Generosity is dependent on the complex interplay of the variables of contact and
revealed. Generosity when both the face-to-face interaction and the threat to
reputation is when we see a nudge to generosity.
- Our study delineated the effect of these variables, but we found that they couldn’t be
separated as they need to interact
- Make it clear in introduction that other studies confounded these variables while we
found them to interact
- Future research differences in the rewards, culture (perceived value: high SES and
Low SES/ Individualism and collectivism) , reputation of others effect reward
 explain why it would be interesting to follow this up is there past research associated
with it to go to that possible direction.
- Pseudo randomisation not a true experiment as student chose their classes even if
classes were allocated at random making it hard to infer causality instead our
study is quasi-experimental, so we suggest a true experiment in the future where there
is random allocation.
- Consider the possibility of potential confounds because it is a quasi-experiment
without true random allocation to be confounding it has to systematically (e.g., all
the females and males are in one condition) vary with the conditions/levels of the IV
and influences/correlated the DV  anything that wasn’t measured is an extraneous
variable only becomes confounding when it meets 2 criteria above.

READ RUBRIC FOR NEXT TUTORIAL

Tute 4
- Report due on the 30th of September
- 2000 words (not including abstract) all up 2200 words
- Tittle needs to include the variables avoid having a tittle that is a question need
clear indication of what the results were in some clear way don’t use causal
language indicate the experimental design
- Abstract 200 words 2 sentences on background, hypotheses, methods (procedure),
results, discussion cross reference with other people don’t discuss limitations just
interpret meaning of findings
- Introduction ask someone without a psych background to read introd and see
if they understand it approx. 500 words
- start broad introduce the topic then highlight the signficance of the topic like real
world application (what is the dictator game by explaining paradigm briefly therefore
any allocation can be seen as an act of geneoristy linking the game with
generosity review literature note what we don’t know build a case for why you
needed to do your study clearly state aims and hypotheses good integration of
both empirical and theoretical research give reader enough info about the study to
understand how they came to the conclusion e.g. random allocation and measured
using x and found yz  bfreif explanation of our study design and how we got the
aim address the limitations of previous studies
- what mistakes do people make?  review literature generally without making case
for you study  fail clearly to state hypotheses
- What should hypotheses look like?  specific (to the tested situation) actually
related to the game conditions and meaning make sense if you explain it in the
intro how you operationalise it?
- Directional
- Positive (make an affirmative claim)
- Testable
- Logical (flow intelligibly from the theoretical rationale)
- Refutable
- Hypothesis: a statement that describes or explains a relationship b/w or among
variables
- a proposal to be tested and evaluate
- Research hypotheses look on slides
- do they refer to scores and predicted direction and is it logical?
- Methods: clearly describe how the study was done provide enough info that
someone else could replicate your study
- Tips: present clearly don’t include irrelevant info figures can be a good way to
illustrate how things were done—
- . user sub-sections, e.g., participants, materials, procedures, designs
- Participants mean and standard deviation for age
- Experimental design (2x2groups between subjects’ factorial)
- How social distance was manipulated
- How reputation (anonymity) was manipulated
- Enough detail about element of procedure so someone can replicate it
- Can use the diagram
- Try to avoid repetition try to be conscience in naming your variables

Results
- Flag to the reader whether the methods were supported or not
- Describe your data, analyses and results
- Very briefly state whether results supported your hypotheses
- Use tables to represent your data
- Only report what is relevant from STATA
- TABLES dont count towards the words count
- Need APA tables not Stata tables
- Graphs can be used can use a bar chart or line one
- Always report non-significant results if they are relevant (e.g., the main effects)
- Also report the interaction
- Be sure not to omit the descriptive broken down by condition
- Report the means averaged across conditions (marginal means) but also the cell
means for the interaction
- Report overalls for sex and age only not each one
- Assumption tests violated but we trust in ANOVA robustness
- Just report swilk for normality
- Accuracy and omission related to doing well in results

Discussion
- Sumarise the results
- Relates your results back to your hypotheses
- Briefly discusses strengths/limitations of the study
- Discusses your results’ implications for theory
- Tips:
- Make sure you include all these aspects
- Most frequent mistake is to just state results and limitation without discussing
hypotheses and theory
- Make sure to prioritise the limitations (why it is a limitation, how it affected the
results and what future implications to rectify that) that most influence the
interetations of the results ( & hold off from interpretations until those limitations
- Need to go beyond quasi experimental and mention why it is a limitation and mention
another limitation to get a D
- Is problematic by creating more noise in the data nobody talks in the same room
compared to nobody talking in different locations
 - Identify the problems explain how the problems changed or biased the results
what the implication of the could be and how we could rectify thatdiscuss any theory
related in the intro
- having more than one explanation of why results were not expected
- need to have a conclusion that summarises the discussion take home message
just have a separate paragraph no subheading needed
- opposite of intro starts specific and ends general
- References 8-12 the title is not capitalised of the actual ariticle in APA but a
generated library does this
- Make sure the page number is in the same font as your report
- Don’t need an appendix
- Abstract  200
- Intro 500
- Methods 500
- Results 300
- Discussion  700

Project 2- organ donation

1. Finding a research idea
2. From theory to hypotheses
3. Defining & operationalising variables
4. Identifying participants
5. Research strategies (types of research)
6. Research designs (between/withing subjects etc

Research strategies
- Describing events (what is happening?)
- Describing the relationship b/w variables
- E.g. is there a relationship b/w x & y

Correlational: assessing the relationship b/w 2 or more variables

Experimental methods: attempting to infer causality manipulating some aspect of a
situation to see what influence it has on other variables
Quasi experimental (established groups) non-equivalent groups are compared

Formulating hypotheses: a statement that describes or explains a relationship b/w or among

variables
Research hypothesis: specific testable prediction refers to specific measures, situation, or
events
Hypotheses: 
- logical: follows logically from ‘theory’
- falsifiable/refutable: it must be possible to obtain results contrary to the prediction
- testable: should be observable, assessable
- positive  evidence is there rather than the absence of evidence (negative) e.g. black
swan (HA VS HO)
- directional

RQ: Why might people be unwilling to be organ donors?

1. Generate a hypothesis
2. Experimental between subject manipulate two groups  control for empathy give
info session to one group hypothesis: people given the info session are going to be
more likely to want to donate organs
3. Correlational: is trait empathy associated with attitudes on organ donation yes
empathy will be correlated with greater organ donation
4. Quasi experimental: do different majors in university effect attitudes on organ
donation yes, with medical majors understanding information on organ donation
then majors not in the medical field.

Tute 5
DO methods and results now
Survey research design
- Developing self-report instruments
- Items should be easy to understand
- Should avoid ambiguous and vague items
- Should be culturally appropriate
- Should consider ethical issues

Assessing scale items

Test item 1 “Would you take drugs which may have strange or dangerous effects?”
- Ambiguous
- Double barrelled question ‘strange or dangerous’
- Social desirability
- Doesn’t specify illicit drugs

Test item 2 “I tend to vote for liberal political candidates” (US scale)
- Culturally bound to the US
- ‘tend’ casual wording

Test item 3 “I am usually not highly curious”

- Confusing wording may lead to misinterpretation of question

Test item 4 “I would like to kill myself”

- Bad wording ‘like to’
- Ethical issues duty of care to report  obligation to ensure participants are out of
harm’s way

Test item 5”Would you call yourself happy-go-lucky?”

- Culturally and cohort bound idiom

Test item 6 “When did you first notice that you were homosexual”
“What do you think caused you to be homosexual?”
- Offensive assumptions

Surveying attitudes

Create 6 scale items related to organ donation

1. Cognitive: “I believe organ donation can save lives”

 “I believe organ transplantation should be punishable”
2. Behavioural: “I would except an organ transplant if needed one”
“I would never donate my organs to another person after death” reverse coded
3. Affective: “I am a bad person if don’t donate my organs” reverse coded
“I feel positively towards people who donate organs”
1 strongly disagree
2 strongly agree

•Our study: we will be examining the relationship between knowledge, religiosity, &
attitudes associated with organ donation (OD)

Tute 6 Ethics

 ethics application need to put it in before mid feb

- Have all your data completed by the end of sem 1 and writ ethe thesis in the break
between semesters

Human participants
Human research ethics committees HREC (Human Sciences and Humanities) and Hrec
(Medical sciences)
- National ethics application form for Psych

Why is it important?
- To protect the welfare, rights, dignity and safety of participants
- Protect rights
- Assesses and understands risks (informed consent form included)
- Ensure release of grant funds
- Ensure that research is appropriately insured

Committee Constituiton
- Made up of 8 people not affiliated by MQU
- Chairperson
- 2 lay people
- One person with knowledge of and experience in the professional care
- Pastoral care person
- Lawyer
- Two people responsible for professional care

HREC:
- Responsible for reviewing, approving and monitoring
- The university has established Faulty Ethics Subcommittees to review low risk ethics
applications
- They also review research in which the foreseeable risk is one of discomfort only

Application:
- Occurs online using the online ethics management system
- Assessed as low risk or high risk
 - Expect to get a notification for their decision from 7-10 days- minor = HREC
executive or major changes = HREC review which is a month later try to get the
ethics in the first or second meeting
- Low risk = 15–20-day feedback need to respond between 6 weeks after first
feedback
- Include: participants info and consent forms, proposed interview questions,
Questionnaires/ survey, advertisements, telephone scrips, recruitment letter/emails,
letters of approval from organisations

Plain language statement

- The study involves understanding the role of reputation and social interaction on
generosity. Based on Game theory the game represents real life generosity and allows
us to understand real life influences on generosity.

Tute 7 Qualitative studies

- Non-numerical data
- Types & categories
- E.g., dreams, hallucinations
- Content & thematic. Analyses
- Theory development
- Useful when limited cases, new research area, understanding subjective experience
- Historically, every well-defined concept we now take for granted in quantitative
measurement had an initial phase of qualitative work, where important distinctions
and categories were explored and verifies

Conducting qualitative research

- Research question: e.g., what types of strategies do DOCS workers cope with the
challenges of dealing with child abuse cases?
- Formulating method for answering the question
- Non-numerical data collection
- E.g., open-ended questions
- E.g. interviews, focus groups etc.

Steps:
i) Recording data
ii) Transcription (if necessary)
iii) Familiarisation
iv) Coding (creating categories) & analysis
v) Theory development (grounded theory)
vi) Testing/validation (quantitative/qualitative/mixed)

Analyses (many types)

- Conceptual analysis: investigating meaning
- Content analysis: examination of concepts within communication
- Thematic analysis: examination of themes & patterns within communication
Qualitative coding
- Open coding: select & naming categories from the analysis of the data
- Axial coding: identify the relationships b/w sub-categories & categories (themes,
patients)
- Category examples: how the categories and subcategories are related to each other

1. Individualistic
2. Collectivist
3. Individualistic
4. Both
5. Individualistic
6. Individualistic
7. Collectivist
8. Individualistic
9. Individualistic
10. I
11. C
12. B
13. I
14. B
15. C
16. I
17. B
18. I
19. I
20. I
21. I

23=
16 = I
3=B

Tute 7: Observational methods & Pedestrian behaviour

Today we will design an observational study of pedestrian street-crossing behaviour
56% of the pedestrians killed were aged 60 or more, although only 21% of the population is
represented by people of this age
- Speed/judgement/mobility/jaywalking
- Stopwatch/video/radar
- Traffic conditions/time of day/weather
- DON’T NEED TO REMBER THE STATS OF THIS TUTE
- Most common in weekends, 5-9pm speed has an additive impact
- Mob mentality
- Mid-block no signals/ ped crossing most fatalities  ass speed bump/wombat
crossing to force driver to slow down
- More likely to die at a crossing than jaywalking
- Misjudges buses rather than cars
- Start-up time glancing prior to crossing entering crosswalk early
 Plan an observational study with intervention for the herring road crossing:
- How long the time is between the light
- How long people wait on average
- How much people go while it is flashing
- Video recording of peak hours school zone hours  naturalistic/disguised
observation

Research report:
- Indicate the two variables interacted in the title don’t use tittles
- Don’t use a question
- Abstract: say it’s a 2X2 between subject’s factorial design
- Intro: why was it important  introduce the key terms first paragraph should have
the aim of the study P2: social distance and reputation concern identify the gap in
the research e.g., the previous research has inflated the variables? Explain how they
inflated the variable not enough to just say they were confounded need to highlight
how. Aim how we manipulated the variables then hypotheses
- Methods: done generally pretty well optimise replication, write logically put the
design before the procedure can then just talk about the manipulations instead of
going through each single conditions.
- Results: didn’t interpret the direction of the interaction greater in the contact than
no contact
- Discussion: interpret the results more clearly for the reader inconsistent in past
literature can’t separate these variables
- Confounding variables: systematically varies (e.g., all females were in the contact
condition and all males in the no-contact condition) across levels of the IV and effects
the DV has to meet both criteria otherwise it’s just an extraneous variables impacts
both groups equally  can identify a relevant confound (like friendship established in
low social distance and high social distance leading to more giving in the low social
distance but we didn’t find a main effect anyway) or that the quasi experimental and
random allocation at the group level and were not that worried about confounds
when discussing limitation make sure they are logical needs to be a reason of why
thought it effected the results explain how and why you want to incorporate other
past variables so make them specific and testable find something original for
limitation not just that we used university students structure your arguments for the
marker topic sentences and concluding sentences use the same terms don’t say
altruism and generosity

Tute 8: Using existing datasets

1. Determine RQ (possibly tentative hypothesis)
2. Find appropriate dataset for you are/ RQ
3. Refine hypotheses in light of actual variables
4. Identify hypotheses in light of actual variables
5. Analyse data
6. Make conclusion and write up results
NOTE: have to write method as normal

RQ: Greater emotional stability and perceived social support at wave 1 associated with
significantly greater psychological wellbeing at wave 5
Greater emotional stability at wave 1 associated with significantly greater psychological
wellbeing at wave 2

Emotional stability at wave 2 will significantly predict greater wellbeing at wave 5

controlling for perceived social support at wave 2

There will be a significant interaction between gender and emotional stability at a wave 1 on
health related QOL at wave 5

The overall model was significant but only explained at small amount of unique variance for
psych wellbeing (0.5). The standardized co-efficient demonstrated both emotional stability
and social support as significant predictors of psych wellbeing at wave 5.

Missing data:
- Omit them completely
- Can predict their scores based on previous scores
- Include the average as their scores

DON’T RELY ON P-VALUES as their reliant on SS (high SS= high chance of significance)
- Strengths: longitudinal data (no directionality problem but still 3 variable problems
as can’t impute causation)
- High psychometric validity
- Very low attrition rates
- Limitations: missing age, education, socioeconomic status
- Using existing data brings some new challenges still need to follow the scientific
methods generate hypotheses/RQ based on literature and theory data analysis
follows

Tute 9: research methods in Psychology

i. Finding a research idea

ii. From theory to hypotheses
iii. Defining and operationalising variables
iv. Identifying participants
v. Research strategies (type of research)
vi. Research designs (BSD/ WSD)

Correlational:
- Examining the relationship between 2 or more variables (strength/relation)

Experimental:
Examining cause-effect relations using manipulation measurement. Comparison &
control

Describe the Quasi-experimental strategy

- Typically compares groups/condition defined by a non-manipulated variable

What are the major differences between the strategies?

 What is a hypothesis?
A statement that describes or explains a relationship b/w or among variables
A proposal to be tested & evaluated

What is a research hypothesis?

A specific testable prediction
Refers to a specific situation or event that can be directly observed

Logical: follows from & is consistent with past theory & research

Testable: all variables observable/measurable

Refutable: can be demonstrated to be wrong

Positive: proposed a relationship or a difference between variables as opposed to no

relationship

Specific: refers to an actual testable situation from the present study. Not just variables in
abstraction

Internal validity: the extent to which a study produces a single, unambiguous explanation
for the relationship b/w 2 variables

External validity:
The extent to which we can generalise to factors outside of our study (can be related to
sampling or ecological validity the extent to which your actually capturing a natural
phenomenon)

What you do to improve the internal validity sometimes reduces the external validity

Do correlational studies have high internal validity?

No, it’s ambiguous and there is a directionality/third variable problem
Can still have an amount of internal validity by measuring things correctly using valid scales
and can control and measure other things and put them in our statistical model but there is
still a directionality problem (A & B correlate: A causes B or B causes A) and third-variable
problem (A & B correlate: relations cause by C)

Strengths of correlational research:

- Can study things that we wouldn’t be able to manipulate it
- Good way to start off our studies to see if there is a possible effect before doing
experimental research
- Potentially high external validity; can be non-intrusive and more ethical

Extraneous variable: Any variable in a study other than the variable being studies
Confounding variable: an extraneous variable that changes systematically along with the iv &
has the potential to influence the DV very hard to get a confounding variable

Experiments:
Need to consider
Manipulation: manipulating a variable to create 2 or more treatment conditions (IV)
Measurement: measuring scores in each treatment condition
Comparison: comparing treatment condition scores
Control: controlling for other variables

Holding constant: standardising an extraneous variable across all treatment conditions (e.g.,
testing only females; same time of day)

Matching: balancing levels of the variables across treatment conditions e.g., block
randomisation or counterbalancing

Both methods disrupt the ability the systematic variations

Randomisation & random assignment: disrupting systematic relationships b/w variables;

equal chance of assignment per condition

Control group: A condition that involves no treatment/ placebo (don’t necessarily have to
have a control group e.g., colour)

Manipulation check: a measure used to assess the effect of the manipulation (e.g., survey, did
we operationalise it correctly did we manipulate it correctly?)

Research exercise: this is the 10 mark question

Correlational: RQ: Is there a negative association between time spent studying and test
anxiety? (Need relationship/association)

Hypothesis: Standardized hours are negatively correlated with anxiety scores

 operationalise and mention the analysis for RH
Research hypothesis: self-reported cumulative hours during the semester will be studying will
be significantly negatively correlated with less anxiety scores on an anxiety scale

Operationalising: test scale for anxiety and self-report for cumulative hours spent studying

Sample: 100 university students recruited sampling people that have exams and study

Analysis: Pearson’s correlation

Experimental: RQ Does hours of study cause a decrease in test anxiety?

Hypothesis: the higher amounts of time allowed to study will cause lower anxiety scores on a
pre-existing scale

Research Hypothesis: Participants in the 10min study time condition will have significantly
lower means score anxiety scores then participants in the 2min condition

Between subject’s design: 1st year undergraduate students who are participating for course
credit
Computer randomly allocates them to the conditions
15 question exams composition of soil (we want to make sure they haven’t studied it before)
Sample: 50 in each group  100 total

Independent samples t-test (compare means)