Research Methods

for Engineering Educators:

A Primer
Dr Prathiba Nagabhushan
AUSTRALIA
22nd August 2016
PURPOSE

 To familiarise the audience with the process of scientific

research in an engineering education environment

 To outline the structure of a research project

 To encourage the participants to critically read, understand and

evaluate research papers
Why Engineering Educational Research?

 To facilitate learning
 To achieve maximum output from students
 To maintain the achievement standards as required by the
governing body
 To inspire and inculcate in students a sense of higher order
thinking
We should understand Research Methods because

 all use the same scientific method.

 all follow the same ethical principles.

 all use the same language and terms.

 all provide information to the world-wide community reported in

a full and open manner.

 all acknowledge the previous work of others.

Ethics in Engineering Educational Research

To protect the rights of the participants

 Participation is voluntary
 Informed consent & risk information
 Contact details to ask more questions
 The participants can withdraw at any time
 Anonymous, without identification of individual participants
 Results are provided to the participants – debriefing
Types of Research Questions in Engg. Education

1. Why? – Why did students lose interest in engineering

courses over time?

2. What? – What was the effect of the strength of a particular

teaching strategy used in a particular unit?

3. How? – How could the faculty promote student engagement

in learning?
Steps in Engineering Educational
Research

1. Introduction
2. Specification
3. Experimentation
4. Verification
5. Conclusion
Steps in Scientific Approach to
Research
1. Problem-posing
2. Review of Literature
3. Problem Specification
4. Statement of Aims & Hypothesis
5. Choice of Research Methods
6. Data Collection
7. Data Analysis
8. Discussion
9. Report writing
Literature Search & Review
 Archival literature
 Databases
 Types of publication
- journal articles
- books
- infomercials
- short journal articles
- conference papers
- patents
- theses
- websites
- trade magazine articles
- newspaper articles
- keyword searching on the internet
What do we do in literature review?

 Who has done what and how?

 What is their plan for “further work”?

 Have they reported more recent work in a conference?

 What opportunities are available for confirming the results of

others and expanding their results and conclusions?

 Is there any gap in literature review?

Quantitative vs Qualitative
Research
 The General Characteristics of
Quantitative Research
 Systematic scientific investigation of a phenomenon through
data and their relationships.

 Quantity is the unit of analysis

 Amounts
 Frequencies
 Degrees
 Values
 Intensity

 Uses statistics for greater precision and objectivity

Steps in Quantitative Research
(Ryerson University, 2016)
 Research Hypotheses 15

for Quantitative Research

 Educated guess or presumption based on literature

 States the nature of the relationship between two or

more variables

 Predicts the research outcome

 Research study designed to test the relationship

described in the hypothesis
 16

Types of Variables

Variable

 Element that is identified in the hypothesis or research

question

 Property or characteristic of people or things that varies in

quality or magnitude

 Must have two or more levels

 Must be identified as independent or dependent

 Independent Variables (IV) 17

 Manipulation or variation of this variable is the cause of

change in other variables
Eg. Visual aids influence students’ focus on learning.

Visual aids Students’ focus on learning

Cause Effect
IV DV
 Technically, independent variable is the term reserved for
experimental studies
 Also called antecedent variable, experimental variable,
treatment variable, causal variable, predictor variable
 18

Dependent Variables (DV)

 The variable of primary interest
 Research question/hypothesis describes,
explains, or predicts changes in it
 The variable that is influenced or changed by the
independent variable
Eg. Visual aids influence students’ focus on
learning.

Visual aids Students’ focus

Cause Effect
IV DV
 19

Relationship Between Independent and

Dependent Variables

 Cannot specify independent variables without

specifying dependent variables

 Number of independent and dependent variables

depends on the nature and complexity of the study

 The number and type of variables dictates which

statistical test will be used
 20

Operationalizing Variables
 All variables need an operationalization

 Specifies the way in which variable is observed or

measured

 Practical and useful?

 Justified argument?
 Coincides with the conceptual definition?
 21

Establishing Reliability and Validity

 Reliability = consistency in procedures and in

reactions of participants

 Validity = truth - Does it measure what it was

intended to measure?

 When reliability and validity are achieved, data are

free from systematic errors
The General Characteristics of
Qualitative Research
Steps in Qualitative Research
(Ryerson University, 2016)

Steps in Qualitative Research

Design for Outcomes

 One-dimensional data

Eg. Conductivity of silicon carbide (V1) at

500 degree Celsius (temperature - V2)
- represented through a bar graph

 2-dimensional data

Eg. A variable measured as a function of time.

- represented through a line graph/scatter plot
Design for Outcomes

 3-dimensional data

+ 1 more variable to the above – represented through a

scatter plot

 N-dimensional data

More than 3 parameters are measured – use of a 2-

dimensional cross-sectional image with the ‘cut’ positioned
through the features of interest.
Statistical Procedures

 One-dimensional data – ‘t’ tests, paired and unpaired ANOVA

 2- dimensional data – Repeated Measures Procedures,

correlational coefficients, regression

 Multi-dimensional data – partial correlational coefficients

 Null Hypothesis testing

Statistical Analysis

 Resolution – the smallest change in input that can be detected

at the output

 Sensitive – change in output divided by a small change in input

 Inference effects – likely error

 Absolute error – difference between the measured value and

the true value

 Sample selection
Designing a Survey for Engineering
Education

 Role of the researcher

 Develop the research plan
 Design the survey instrument
 Select the sample population
 Issue/distribute the survey
 Prompt the sample population for responses
 Analyse the data
 Generate conclusions
General Survey Guidelines

Key features in Survey Questionnaire construction:

 Introduction
 Instruction
 Statements – Likert Scale
 Demographic details
 Open-ended questions and Comments

Survey delivery – paper/online/telephone

Survey Statements

Qualitative data:

 This can be a challenge as everyone will use a different

description.
 One approach is to convert qualitative data to quantitative data
(eg. rate from very bad to very good on a score of 1 to 10).
 Asking the right questions without leading the person (survey
instruments - questionnaires).
 Survey results (Is 35% return good enough?).
Writing survey questions

 Keep it very simple – avoid jargon

 Use one concept per question – avoid multiple concepts
 Keep wording positive – avoid negative words and phrases,
double negatives
 The first question should be the “over-all question” – Never
place a controversial question at the beginning.
 Place demographics questions at the end – Demographics at
the beginning can raise suspicions.
 Keep related questions together – Difficult for the respondent
to remain coherent
 Use three questions per topic – Do not over question, don’t
waste people’s time.
Sample Statements
 I don’t concentrate most of the time (negative wording).
I have good concentration.
 I am satisfied with the ventilation and the lighting
environment (double-barrelled question).
I am satisfied with the ventilation.
I am satisfied with the lighting.

 The University does not do a bad job of keeping us

informed about work place health and safety issues.
(double negative)
The University does a good job of keeping us
informed about work place health and safety issues.
 Many students feel ill as soon as they walk into the
building. (projecting the feelings of others).
Students enjoy working in this building.
 Sampling
 Population – every possible candidate.
 Sample population – a small number of candidates selected
from the population.

Eg. Selecting a representative sample for a

telephone poll?
 Post code?
 Telephone book?
 In the street or shopping centre?
Respondent Selection
 Clustered Sampling: Select a sample from only those parts of
the population which are relevant; eg choose only those
students who use computers for revision.

 Stratified Sampling: Select a sample proportionally to those

who are likely to use computers for revision and those that
don’t. (4/6 use computer for revision and 2/6 don’t, so use a
sample of 3, two who use computers for revision and one that
does not)
Random Sampling
 How can I choose a team of 6 people randomly from this
class?
 Family name?
 Student number?
 Type of course opted by students?
 Every second person?
 Every person must have an equal probability of being
chosen.
Developing a Research Plan

RESEARCH PROPOSAL

Structure of a Research Plan/Proposal

 Proposal title
 Research Team Members
 Project Summary
 Project description
 Aim, background, method, data collection & analysis and
deliverables
 Reporting
 Budget
Research Plan in a Nutshell
Reference
 Thiel, David V. (2014). Research Methods for Engineers. Cambridge
University Press: Australia
 Borrego, M. 2007a. Conceptual hurdles experienced by engineering faculty
becoming engineering education researchers. Journal of Engineering
Education 96 (2): 91-102.
 Borrego, M. 2007b. Development of engineering education as a
rigorous discipline: A study of the publication patterns of four
coalitions, Journal of Engineering Education (1): 5-18.
 Borrego, M.,J. Froyd, and D. Knight. 2007. Accelerating emergence of
engineering education via the International Conference on Research in
Engineering Education (ICREE). Journal of Engineering Education, 96 (4):
281-82.
 Borrego, M., R.A. Streveler, R.L. Miller, and K.A. Smith. 2008. A new
paradigm for a new field: Communicating representations of engineering
education research. Journal of Engineering Education 97 (2): 147-62.
My email id:

[email protected]