EUN678 Survey Research

Module 4

CRICOS No.00213J
CRICOS No.00213J
ACKNOWLEDGEMENT OF TRADITIONAL OWNERS
QUT acknowledges the Turrbal and Yugara, as the First Nations
owners of the lands where QUT now stands. We pay respect to
their Elders, lores, customs and creation spirits. We recognise
that these lands have always been places of teaching, research
and learning. ​
QUT acknowledges the important role Aboriginal and Torres
Strait Islander people play within the QUT community.​

CRICOS No.00213J
Research designs in
survey research
(part 2)

CRICOS No.00213J
Overview
In this module you continue down the pathway to designing a survey that will work best
to answer your research question. We will learn about data analysis methods for survey
designs that use quantitative, qualitative, and mixed methods (those that combine both
quantitative and qualitative methods).

CRICOS No.00213J
Content
• Revisiting previous modules
• Module 4
• Research design
• Participants
• Measures
• Procedures
• Data analysis (deciding on analyses; writing about analyses)
• Time to work

CRICOS No.00213J
Revisiting previous modules

CRICOS No.00213J
Warning - important message
The assessment in this unit will not
require you to administer a survey.

You will be developing a survey and

writing a proposal but you will stop
there. You will not and must not
administer your survey to your
target sample.

CRICOS No.00213J
By now you should have:
• Drafted an introduction that clearly identifies the research topic and research problem,
explains the significance of the research topic and places this in the context of the
research problem. It will provide a rationale for why this is important to investigate this
topic.
• Drafted a rough literature review that The literature review summarises and synthesises
relevant empirical research undertaken by others on the topic, as you would typically
see in peer reviewed journal articles reporting on research studies. It should conclude
with a statement of the research question(s).
• Identified existing instruments that you can use and/or adapt for your questionnaire.
• Kept a list of citations (sources) that will help compile your reference list later down the
track.

CRICOS No.00213J
CRICOS No.00213J
 Module 4

CRICOS No.00213J
EUN678 process for survey design

• Step 1: Decide if a survey is the best design to use.

• Step 2: Identify the Research Questions or Hypotheses
• Step 3: Identify the Population, the Sampling Frame, and the Sample EUN678
• Step 4: Determine the Survey Design and Data Collection Procedures
• Step 5: Develop or Locate an Instrument
• Step 6: Administer the Instrument
• Step 7: Analyse the Data to Address the Research Questions
• Step 8: Write the Report

(Creswell & Guetterman, 2021, pp 457-458)

CRICOS No.00213J
The relationship between constructs, operational
definitions, and instruments
• A construct is a broad concept or topic for study. They can be concrete (e.g., attendance – directly observable) or abstract
(e.g., engagement – not directly observable). Constructs can and should be defined. Usually from the existing literature
(some of which will be empirical research).

• Variables are created when we develop a construct into a measurable form. Each variable represents a characteristic or
construct that varies (i.e., it has at least 2 possible values). Variables can be defined as well. We call these “operational
definitions”. Operational definitions specify how variables are measured for the purpose of the research (how it is recorded
and how it is calculated as a numeric value).

• Examples of operational definitions of constructs…..

• attendance = the number of people attending class in person or online tonight. This will be recorded by taking an
attendance role. An attendance roll will be printed on a spreadsheet, and a teaching assistant will identify each student
in attendance marking 1 for attendance, or 0 for no attendance.
• engagement = the sum of student self-report responses to items on the ‘Masters Engagement Scale’ with 3 items in
which students rate the quality of their own involvement in class tonight on a 5-point Likert-type scale (1 = not at all
engaged; 5 = highly engaged). Scores will be summed for a total engagement score (or scores will be summed, and the
mean of the sum of scores will be calculated).

CRICOS No.00213J
The relationship between constructs, operational
definitions, and instruments
Construct Operational definition How to measure it
Attendance Attendance = the number of days a Take a class roll. Count the number of days a
student attends school in a school year student attends school. Present the students’
attendance as a fraction in which the
attendance days is the numerator and total
school days is the denominator (i.e.,
145/200)
Engagement Engagement = is viewed as a multi- Administer the Student Engagement
dimensional construct comprising four Instrument (Appleton et al., 2006). Adhere to
subtypes: academic, behavioural, the scoring procedures detailed by Appleton
cognitive, and psychological. Each subtype et al (2006) in which xxxxxx. [See here:
comprises indicators (Appleton et al., https://checkandconnect.umn.edu/docs/SEI%
2006) 20Administration,%20Scoring,%20and%20R
esults%20Use%20ICI.pdf ]

CRICOS No.00213J
Finding instruments
• Week 3 task: were you able to identify at least
one instrument that could be used or adapted
for your questionnaire?
• No, I haven’t started looking yet
• No, I am still looking
• Yes, I found something suitable

CRICOS No.00213J
Writing the method section

CRICOS No.00213J
Writing the method section

research design

participants

measures

procedures

data analysis

CRICOS No.00213J
Research design
• This study examines xxxx using yyyyy… It will zzzzz….
• This study uses a survey design to collect data from xxxx using yyyy.
• I will conduct a cross-sectional survey of xxxxx in xxxxx using xxxxx….
• To (add aim of study) I will create and administer a cross-sectional survey that will …..
• The current study explores xxxx using xxxx (add research design)….
• Data will be collected via xxx …
• Cross-sectional survey research design will be used to gather information in relation to xxxx
• The study will adopt a cross-sectional survey design with data collected from xxxx using yyy……
• Using a mixed-methods survey, this study will ….
• This study will utilize a mixed-methods survey design using an online questionnaire…

CRICOS No.00213J
Examples – from Module 3 – Research Design
• Examine current knowledge, attitudes, practices • Motivation of parent involvement in secondary-level
schooling
• Examine beliefs, opinions, views, perspectives.
• Young children’s online learning during COVID-19
• Assess needs for xxxx (needs analysis) pandemic: Chinese parents’ beliefs and attitudes
• Assess the feasibility and acceptability of an • Parental perspectives on inclusive education for
intervention (policy, program, practice) children with intellectual disabilities in Greece
• Compare two or more groups on xxxx • A cross-sectional study of demographic, environmental
and parental barriers to active school travel among
• Learn about a sample’s characteristics children in the United States
• Gather perceptions on xxxx • What do parents need to enhance participation of
their school-aged child with a physical disability? A
• Understand what people need to xxxx cross-sectional study in the Netherlands
• Who uses online parenting support? A cross-sectional
survey exploring Australian parents’ internet use for
parenting

CRICOS No.00213J
Now you try…
Go to discussion forum
Copy and paste your approved post to a new Word doc

5:00 minutes

Add a heading “Method”

Write a sentence or two describing your method!

CRICOS No.00213J
Participants
• Who will your participants be? Now you try…
• How can you describe them with precision (if
students, what year level and corresponding age • Write down rough answers to these questions
ranges)? • Then use examples from Module 3 as a starting
• How many will you need (sample size)? [sample point for phrasing.
size calculator] (see also Creswell & Guetterman,
2021, pp.666-669)
• How will you recruit them? How will you invite
them to participate?
• You will also need to explain ethical considerations
5:00 minutes
for participant recruitment, voluntary
participation, and consent (more about that in
Module 5, but as you draft up the section now, you
can add a placeholder/note to self)

CRICOS No.00213J
Examples – from Module 3 - Participants
• Examine current knowledge, attitudes, practices • Motivation of parent involvement in secondary-level
schooling
• Examine beliefs, opinions, views, perspectives.
• Young children’s online learning during COVID-19
• Assess needs for xxxx (needs analysis) pandemic: Chinese parents’ beliefs and attitudes
• Assess the feasibility and acceptability of an • Parental perspectives on inclusive education for
intervention (policy, program, practice) children with intellectual disabilities in Greece
• Compare two or more groups on xxxx • A cross-sectional study of demographic, environmental
and parental barriers to active school travel among
• Learn about a sample’s characteristics children in the United States
• Gather perceptions on school issues • What do parents need to enhance participation of
their school-aged child with a physical disability? A
• Understand what people need cross-sectional study in the Netherlands
• Who uses online parenting support? A cross-sectional
survey exploring Australian parents’ internet use for
parenting

CRICOS No.00213J
Measures
• How will you collect the data from your
participants (sample)?
Now you try…
• What type of survey will you use (mail/paper- • Write down rough answers to these questions
based, online, interview F2F/online, telephone)? • Then use examples from Module 3 as a starting
• What instruments will you use? point for phrasing.
• Have these been adopted or adapted from an OR
existing instrument(s)? Which ones? • Use examples from your favourite empirical
• How many items in each instrument? studies that are relevant to your topic and
research question
• What kind of response options will be used?
• Have the psychometric properties of the
instruments been assessed? (is there
information on reliability and validity?) 5:00 minutes

CRICOS No.00213J
Examples – from Module 3 - Measures
• Examine current knowledge, attitudes, practices • Motivation of parent involvement in secondary-level
schooling
• Examine beliefs, opinions, views, perspectives.
• Young children’s online learning during COVID-19
• Assess needs for xxxx (needs analysis) pandemic: Chinese parents’ beliefs and attitudes
• Assess the feasibility and acceptability of an • Parental perspectives on inclusive education for
intervention (policy, program, practice) children with intellectual disabilities in Greece
• Compare two or more groups on xxxx • A cross-sectional study of demographic, environmental
and parental barriers to active school travel among
• Learn about a sample’s characteristics children in the United States
• Gather perceptions on school issues • What do parents need to enhance participation of
their school-aged child with a physical disability? A
• Understand what people need cross-sectional study in the Netherlands
• Who uses online parenting support? A cross-sectional
survey exploring Australian parents’ internet use for
parenting

CRICOS No.00213J
Procedure
• Plan it out step by step. Put yourself in the shoes of the participant.
• Data will be collected from teachers/parents/students in xxxx (place?).
• The survey will be administered as a self-report form in xxxx (when? what time
period? add any contextual details that might be important).
• Participants will be invited to complete the survey (how? add format e.g., online via
the xxxx survey platform OR during class time allocated for Year 9 mathematics).
• They will receive an invitation to participate via xxx (add mechanism for distributing
invitations).

5:00 minutes
• Add further details, step by step.
• Use examples to guide you.

CRICOS No.00213J
Examples – from Module 3 - Procedures
• Examine current knowledge, attitudes, practices • Motivation of parent involvement in secondary-level
schooling
• Examine beliefs, opinions, views, perspectives.
• Young children’s online learning during COVID-19
• Assess needs for xxxx (needs analysis) pandemic: Chinese parents’ beliefs and attitudes
• Assess the feasibility and acceptability of an • Parental perspectives on inclusive education for
intervention (policy, program, practice) children with intellectual disabilities in Greece
• Compare two or more groups on xxxx • A cross-sectional study of demographic, environmental
and parental barriers to active school travel among
• Learn about a sample’s characteristics children in the United States
• Gather perceptions on school issues • What do parents need to enhance participation of
their school-aged child with a physical disability? A
• Understand what people need cross-sectional study in the Netherlands
• Who uses online parenting support? A cross-sectional
survey exploring Australian parents’ internet use for
parenting

CRICOS No.00213J
Data analysis
• Whatever data are collected, the data cannot speak for themselves.
• All data require data analysis methods.
• These approaches align with the type of data you have collected
• Data analysis requires interpretation.
• This is another reason why it is important to think ahead, to keep the whole process in
mind, as you build your questionnaire and design your survey.

CRICOS No.00213J
 Quantitative Data types
(based on level of
measurement)

Don’t label a variable as

categorical or
Continuous/ quantitative without
Categorical thinking about the
Quantitative question you want it to
answer

Names categories
Answers questions
and answers
about the quantity
questions about
of what is
how cases fall into
measured
those categories

CRICOS No.00213J
Categorical variables N.O.I.R. – Nominal, Ordinal,
Interval, Ratio data
• Binary or dichotomous variables (only 2 categories)
Do you hold a driver licence?
___ Yes
___ No
• Nominal variables (more than 2 categories)
In which course are you enrolled?
MEd (Inclusive Education)
MEd (Educational Leadership)
MEd (Early Childhood)
• Ordinal variables (same as nominal, except there is a logical order e.g. fail, pass, credit, distinction, high distinction)
Rank the movies in order of preference where 1 indicates best and 4 indicates worst
___ Indiana Jones and the Temple of Doom
___ Little Miss Sunshine
___ The Girl with the Dragon Tattoo
___ The African Queen

CRICOS No.00213J
Quantitative variables N.O.I.R. – Nominal, Ordinal,
Interval, Ratio data

• Interval variables (equal intervals on the variable represent equal intervals in

the property being measured)
All MEd students should study survey research methods

1 2 3 4 5
Strongly Disagree Neutral Agree Strongly
disagree Agree

• Ratio variables (actual ‘real’ numbers, includes zero)

Uses distances between values on scale
Allows researcher to make absolute comparisons between the raw responses.

CRICOS No.00213J
Data analysis: Summary tables
Summarising data: Descriptives example Summarising data: Various examples

• https://apastyle.apa.org/style-grammar-
guidelines/tables-figures/sample-tables

CRICOS No.00213J
Data analysis: Inferential statistics

Source: Creswell & Guetterman, 2021, p.670

CRICOS No.00213J
 Data analysis 5:00 minutes
You’ll need to describe how your data will be analysed step by step. Explain how you will:
• Download raw data (in what form?)
• Clean the raw data (i.e., look for errors and missing data)
• Score/code the data
• Decide if/how you will group the data (e.g., will you sum knowledge scores for an
overall total score? What does the original measure require you to do?)
• Name the statistical analysis program you would use (hypothetically)
• Conduct descriptive analyses (“descriptives” – frequencies, percentages, means ± SD)
• Conduct inferential analyses (“inferentials” – statistical tests to answer research
questions)

CRICOS No.00213J
Examples – from Module 3 – Data analysis
• Examine current knowledge, attitudes, practices • Motivation of parent involvement in secondary-level
schooling
• Examine beliefs, opinions, views, perspectives.
• Young children’s online learning during COVID-19
• Assess needs for xxxx (needs analysis) pandemic: Chinese parents’ beliefs and attitudes
• Assess the feasibility and acceptability of an • Parental perspectives on inclusive education for
intervention (policy, program, practice) children with intellectual disabilities in Greece
• Compare two or more groups on xxxx • A cross-sectional study of demographic, environmental
and parental barriers to active school travel among
• Learn about a sample’s characteristics children in the United States
• Gather perceptions on school issues • What do parents need to enhance participation of
their school-aged child with a physical disability? A
• Understand what people need cross-sectional study in the Netherlands
• Who uses online parenting support? A cross-sectional
survey exploring Australian parents’ internet use for
parenting

CRICOS No.00213J
CRICOS No.00213J
 Time to work

CRICOS No.00213J