Research

Comprehensive Exam 2024

>>Research as a Script How to collect data

1. Formulate the idea (introduction)
2. Formulate the design (method)
—proposal—
3. Collect the data
How to analyze data
4. Analyze the data (results)
5. Communicate the findings (discussion)
—thesis defense—

How to formulate the IDEA

How to communicate findings

How to formulate the DESIGN

>>Approaches to Research
Research
Comprehensive Exam 2024

PHILOSOPHICAL WORLDVIEWS 3.Data, evidence, and rational considerations shape knowledge.

4.Research seeks to develop relevant, true statements, ones that
can serve to explain the situation of concern or that describe the
causal relationships of interest.
5.Being objective is an essential aspect of competent inquiry;
researchers must examine methods and conclusions for bias.

CONSTRUCTIVISM -Berger & Luckmann’s “The Social

Construction of Reality” (1967)
Lincoln & Guba’s “Naturalistic Inquiry” (1985)
•Understanding . Individuals seek to understand the world they
live in.
•Multiple participant meanings . Individuals develop subjective
meanings to their experiences.
•Social and historical construction . Subjective meanings are
POSTPOSITIVISM negotiated socially and historically through interaction with
the “scientific method”, “science”, empirical science, others.
positivism, quantitative research •Theory generation. Research aims to develop a theory or
pattern of meaning.
•Determination . Examine causes that
influence outcomes. ASSUMPTIONS - Crotty (1998)
•Reductionism . Reduce ideas into variables 1.Human beings construct meanings as they engage with the world
that constitute hypotheses and questions.
they are interpreting.
•Empirical observation and measurement . Carefully observe
2.Humans engage with their world and make sense of it based on
objective reality “out there”.
their historical and social perspectives—we are all born into a world
•Theory verification. Begin with a theory; collect data of meaning bestowed upon us by our culture.
to support or refute theory; revise theory.
3.The basic generation of meaning is always social, arising in and
out of interaction with a human community.
ASSUMPTIONS
1.Knowledge is conjectural (and TRANSFORMATIVE (1990s, from Marx, Adorno,
anti-foundational)—absolute truth can never be Marcuse, Habermas, & Freire)
found.
2.Research is the process of making claims and then refining or Postpositivism imposed theories that marginalize
abandoning some of them for other claims more strongly warranted.
Research
Comprehensive Exam 2024

•Political. Research needs to be intertwined with • Transformative research uses a program theory of beliefs about
politics and a political agenda.
how a program works and why the problems of oppression,
•Empowerment issue-oriented. Specific social domination, and power relationships exist
issues need to be addressed (e.g. inequality).
•Collaborative. Research includes participants PRAGMATIC (Peirce, James, Mead, Dewey, Rorty, Murphy,
as “researchers” or collaborators.
Patton, CherryHolmes)
•Change-oriented. Advocacy gives participants a
“voice” for change (e.g., consciousness-raising, •Consequences actions . Knowledge claims arise out of actions,
advancing an agenda). situations, and consequences rather than antecedent conditions (as
in postpositivism).
• Feminist perspectives. Problematize women’s diverse
situations & oppressive contexts. •Problem-centered . Concern is on “what works” or on applications
and solutions to problems.
•Racialized discourses . Question control & production of •Pluralistic . Researchers use all approaches to understand the
knowledge, esp. knowledge of people of color. problem.
• Critical theory . Asks how human beings can be •Real-world practice oriented . “Truth is what works at the time”.
empowered and transcend constraints due to race, •Pragmatism is not committed to any one system of philosophy
class, & gender. and reality.
• Queer theory . Gives voice to lesbians, gays, bisexuals, & •Individual researchers have a freedom of choice
transgendered (LGBT) people.
• Disability inquiry . Addresses the meaning of inclusion •Pragmatists do not see the world as an absolute unity
for individuals with disabilities. •Truth is what works at the time. It is not based in a duality
between reality independent of the mind or within the mind.
KEY FEATURES- Mertens (2010)
•The pragmatist researchers look to the what and how to research
• It places central importance on the study of lives and based on the intended
experiences of diverse groups that have traditionally been •Pragmatists agree that research always occurs in social, historical,
marginalized. political, and other contexts.
• In studying these diverse groups, the research focuses on •Pragmatists have believed in an external world independent of the
inequities based on gender, race, ethnicity, disability, sexual mind as well as that lodged in the mind
orientation, and socioeconomic class that result in asymmetric
•For the mixed methods researcher, pragmatism opens the door to
power relationships. multiple methods, different worldviews, and different assumptions,
• The research in the transformative worldview links political as well as different forms of data collection and analysis.
and social action to these inequities.
————————
Research
Comprehensive Exam 2024

-Philosophical worldviews (also referred to as philosophical QUANTITATIVE

worldviews or paradigms) refer to ”a basic set of beliefs that guide
• Experiments . True experiments, quasi- experiments, correlational
action”
studies, structural equation models.
-it serves as a guide what kind of research to pursue (quanti, • Surveys . Use of questionnaires or structured interviews for data
quali) and how to do it collection, with the intent of generalizing from a sample to a
population.
-The types of beliefs held by individual researchers will often
lead to embracing a qualitative, quantitative, or mixed methods QUALITATIVE
approach in their research.
• Ethnographies . Studies of intact cultural groups in their natural
-pag empirical ka, postpositivist setting over a prolonged period.
• Grounded theory . Multiple stages of data collection to derive an
pag gusto mo mag understand and explore a phenomenon and abstract theory of a process or action.
generate theory, constructivist
• Case study . In-depth exploration of an event, activity, process, or
pag about sa marginalized and political matters, transformative one or more individuals.
• Phenomenological research . Understanding “lived experiences”
pag problem-centered and open to both methods quali and
or the “essence” of a phenomenon.
quanti (mixed) pragmatist
• Narrative research . Re-telling of individuals life stories.
————————————
MIXED METHODS
DESIGNS
• Explanatory Sequential. Begin with QUANTI to test theory then
proceed with QUALI to explore a few cases in detail.
• Exploratory Sequential. Begin with QUALI to explore then
proceed with QUANTI with large sample;
• Convergent parallel. Collect both QUALI and QUANTI data at the
same time and integrate information in the interpretation of the
results.
• Transformative, embedded, or multiphase. The use of a
theoretical lens to provide a framework and methods of data
collection that contain both QUALI and QUANTI data.
Research
Comprehensive Exam 2024

RESEARCH METHODS
In this situation, the researcher seeks to establish the meaning
of a phenomenon from the views of participants. This means
identifying a culture-sharing group and studying how it develops
shared patterns of behavior over time (i.e., ethnography). One of
the key elements of collecting data in this way is to observe
participants’ behaviors during their engagement in activities.

Qualitative approach:
Transformative worldview, narrative design, and
open-ended interviewing

For this study, the inquirer seeks to examine an issue related to

oppression of individuals. To study this, stories are collected of
TYPICAL SCENARIOS OF RESEARCH individual oppression using a narrative approach. Individuals are
interviewed at some length to determine how they have
Quantitative approach:
personally experienced oppression.
Postpositivist worldview, experimental design, and pretest and
posttest measures of attitudes Mixed methods approach:

In this scenario, the researcher tests a theory by specifying Pragmatic worldview, collection of both quantitative and
qualitative data sequentially in the design
narrow hypotheses and the collection of data to support or refute
the hypotheses. An experimental design is used in which
The researcher bases the inquiry on the assumption that
attitudes are assessed both before and after an experimental collecting diverse types of data best provides a more complete
treatment. The data are collected on an instrument that understanding of a research problem than either quantitative or
measures attitudes, and the information is analyzed using qualitative data alone. The study begins with a broad survey in
statistical procedures and hypothesis testing. order to generalize results to a population and then, in a second
phase, focuses on qualitative, open-ended interviews to collect
Qualitative approach: detailed views from participants to help explain the initial
quantitative survey.
Constructivist worldview, ethnographic design, and
observation of behavior
Research
Comprehensive Exam 2024

CRITERIA FOR SELECTING A RESEARCH APPROACH ● By establishing that the question is theoretically significant,
i.e. situated in the context of current knowledge

o Establishes link from past studies to the current study

using a review of related literature (rrl)

o Establishes why the research question is the logical

question to ask now given past studies
o Past studies have found.. hence, this study will find…
● By establishing that the research question is clear, i.e.
defined and explained by a theory or a theoretical framework
● By establishing that the research question is feasible, i.e.
can be answered empirically using a sound research
methodology
● By establishing that the research question is socially
significant, i.e. situated in the context of Philippine social
reality
o where the research questions come from
o not an essential argument for the academic community
o what ordinary people wish to know: How is this useful?
How is this relevant? How can this solve social problems?
o assumed or apparent to the reader

Quantitative Introduction
o Presents that the research problem is best addressed by
>>Theoretical Argumentation understanding what factors or variables influence an outcome
o Advances a theory to test
● The introduction as a single argument o Incorporates a substantial review of literature to identify research
● To argue that we have a good research question questions that need to be answered or hypotheses that need to be
● How? tested
Research
Comprehensive Exam 2024

o Forms of theories in quanti research o Placement of quantitative theories

o Script out the theory section: “The theory that I will use is
(name of theory). It was developed by (identify the origin, source,
or developer of the theory), and it was used to study (identify the
propositions or hypotheses in the theory). As applied to my study,
this theory holds that I would expect my independent variable(s)
(state IVs) to influence or explain the dependent variable(s) (state
DVs) because (provide a rationale based on the logic of the
theory).”

Qualitative Introduction
o Presents that the research problem is best understood by
exploring a concept or phenomenon
o Explored a topic when the variables and theory base are
unknown
Research
Comprehensive Exam 2024

o Established a review of literature that there is a lack of theory, Ø Studies that have addressed the problem
research, understanding of the concept; hence, the need to explore
Ø Deficiencies in the studies (past literature)
the phenomena and to develop a theory
o Use of theory Ø The importance of the study for an audience

o First, like in quanti, used as a broad explanation for Ø The purpose statement
behavior and attitudes o Identifying deficiencies
-provide ready made series of hypotheses to be tested from o Literature is deficient…
the literature (e.g., qualitative health science, ethnography,
anthropology) ● …because past studies have not studied specific areas or
topics or variables
o Second, use of a theoretical lens or perspective in
● …because past studies have not explored the topic with a
quali research particular group, sample, or population
-provides an overall orienting lens for the study of questions ● …because there is a need to replicate the findings given new
of gender, class, and race samples
o Third, theory (or some other broad explanations) ● …because past studies have not used specific theories or
becomes the end point theoretical frameworks
● (often get cues from recommendations for future research in
past studies)
Mixed Method Introduction
o Presents that the research problem is best understood both o Addressing deficiencies
quantitatively (understand relationship among variables) and
● Explain how current study will remedy or address
qualitatively (explore in depth) deficiencies
o Can follow quantitative intro – when explaining relationships
among variables (with qualitative data to support or further explore >>CONCEPTUALIZING AND FRAMING A QUANTITATIVE
variables); using theory deductively RESEARCH PROBLEM (FRAMEWORKS)
o Can follow qualitative intro – when exploring a phenomenon in Where do research problems come from?
depth (with quanti data to support of validate with larger
sample);using theory inductively as in an emerging qualitative theory • From intuitive sources
or pattern. • From existing research
o Deficiency model - an approach to writing an introduction to a • From theory
research study that builds on gaps existing in the literature. Parts:
A well-conceptualized problem derives from (or is situated in) a
Ø The research problem well-defined theoretical or conceptual framework.
Research
Comprehensive Exam 2024

Scientific inquiry hinges on the logic of inductive and deductive •organizes knowledge in the form of precise statements of
reasoning. relationships among the concepts
•provides basis for explaining the phenomenon & making
Deduction is reasoning from a set of general statements to a
logically valid conclusion. further predictions

- general premises that are held to be true are applied to the What is a theoretical/conceptual framework?
prediction of specific instance
The system of concepts, theories, assumptions, and
- Process of theory- or hypothesis-testing
expectations that supports and informs your research.
Induction is reasoning from specific data to reach a probable •Forms the basis of your argumentation that your research
conclusion or generalization that may explain the data. problems, hypotheses, measures, analyses are sound.
-note that one can never reach a logically certain conclusion
-Process of theory-building

Scientific reasoning

What is a theory?
Elements of a theoretical framework
A set of propositions or logical statements about a
phenomenon that… ● It starts with concepts or constructs
● The framework defines the concepts
•summarizes existing empirical knowledge
Research
Comprehensive Exam 2024

● The framework then specifies the proposed patterns of Self-esteem will be determined according to scores on the
relationships among the concepts Rosenberg Self-esteem Scale, where higher scores are
indicative of higher self-esteem.
● The framework specifies the boundaries or scope of the
study: your particular lens • congruence between conceptual and operational
definition: validity
1. Concepts: It starts with concepts or constructs
3. Relationships: The framework then specifies the proposed
•Abstractions that refer to internal motives or clusters of behaviors patterns of relationships among the concepts
•Are inferred from observations of behavior
•Must be conceptually and operationally defined •Exogenous concepts / Independent or Predictor variables
•Endogenous concepts / Dependent or Outcome variables
ex: from school pressure to parental control to parental
•and mediators! and moderators!...!
psychological control

2. Definitions: The framework defines the concepts Framework: Types of relationships

•Conceptual definition- defines a concept by using

other concepts
Self-esteem refers to the subjective feeling of satisfaction
with one’s attributes or characteristics.

Psychological control is the assertion of parental

authority that intrudes in the psychological and emotional
world of the child—that is, her thoughts, emotions, and
sense of self.
• Good conceptual definitions…
● use clear & precise terms whose meaning is generally
agreed upon by everyone (i.e. “primitives”)
● are stated positively (i.e., what the concept is, rather than
what it is not)
● Concepts & their definitions must be amenable to
operationalization
• Operational definition- indicates the measurements,
procedures, or behaviors to be used as indicators of the concept
Research
Comprehensive Exam 2024

•can integrate propositions derived from diverse theoretical and

empirical sources (but have to make logical sense!)

3. What if there seems to be no “theory”?

Then via induction, the researcher must construct a framework by:
•systematically outlining the major, relevant concepts that have been
studied in relation to the topic
•integrating the research findings pertaining to the
A theoretical / conceptual framework is indispensable! concepts/variables
•Supports your argument for why your research problem is •constructing a diagram consisting of these concepts and their
legitimate, researchable, & significant proposed relationships

•Situates your questions in the context of existing theory &

knowledge 4. How do I present my theoretical framework in my research
paper?
•It is the source of your conceptual (and operational) definitions
•Define key concepts/variables according to the framework
•It justifies your hypotheses
•Discuss relationships suggested by the framework (causal,
Practical Questions mediating, etc); the psychological processes & mechanisms, as
indicated or inferred from related literature
1. Where do I find theories?
•In the research literature! (Introduction and Discussion sections) Common Problems
•Psychological Review; Psychological Bulletin
1. Lack of clarity or consistency in the concept/s being studied
Consider the existing models and frameworks on the topic and
select among them… 2. Relationships among concepts are ambiguous (Causal?
Correlational? Mediated?)
…according to relevance, empirical support, applicability, testability, 3. No clear logical build-up (or storyline!) to the problems,
interest -> developing your theoretical LENS hypotheses, methods
4. Incomplete
2.Do I have to use just one theory? 5. Outdated
•can focus on the work of a single theorist or the propositions 6. Premature commitment
from a single theory
Research
Comprehensive Exam 2024

>>THEORY AND QUESTION IN QUALITATIVE Expect the research questions to evolve and change during the
study in a manner consistent with the assumptions of an
RESEARCH emerging design.
Often in qualitative studies, the questions are under continual
QUALITATIVE RESEARCH QUESTIONS review and reformulation (as in a grounded theory study).
Qualitative studies often have a central question and associated
• Use open-ended questions without reference to the literature or
subquestions.
theory unless otherwise indicated by a qualitative strategy of
The central question is the question in its most general form. inquiry.

The subquestions are specific topics to be explored. • Specify the participants and the research site for the
Relate the central question to the specific qualitative strategy of study if the information has not yet been given
inquiry (e.g. phenomenology), theory (e.g. social representations),
A typical script for a qualitative central question: (How or
or theoretical perspective or lens (e.g. feminist theory). what?) is the (“story for” for narrative research;
“meaning of” the phenomenon for phenomenology; “theory that
Use “What” or “How” or questions signifying an open and explains the process of” for grounded theory; “culture-sharing
emerging design. pattern” for ethnography; “issue” in the “case” for case study) of
Focus on a single phenomenon or concept. Use exploratory (central phenomenon) for (participants) at
(research site).
verbs like:
Qualitative theories
• Discover (grounded theory)
The use of theory in qualitative research is much more varied
• Seek to understand (ethnography) compared to quantitative research.
• Explore a process (case study)
The theory can come at the end as in the case of grounded theory
• Describe the experiences (phenomenology) (which seeks to generate a theory).
• Report the stories (narrative research)
The theory can come at the beginning to provide a lens that
shapes what is looked at and the questions asked.
Question can be further explained in the text (in a paragraph).
Similar to quantitative research, the theory can serve as a broad
Avoid quantitative words like “affect”, “influence”, “impact”,
explanation for the phenomenon.
“determine”, “cause”, “relate”, and “change”.
Research
Comprehensive Exam 2024

Unlike quantitative research, the theory often does not explain the phenomenon.
hypotheses or relationships between variables.
Find a theoretical perspective that can be your lens in
understanding the phenomenon (e.g., feminist theory, critical theory,
queer theory).

EXAMPLES OF QUALITATIVE APPROACHES

● Ethnography Action Research
● Conversation analysis
● Discursive psychology
● Foucauldian discourse analysis
● Psychoanalytic approach to qualitative psychology
● Memory work
● Narrative psychology
● Phenomenological psychology
● Interpretative phenomenological analysis
● Social representations
● Q methodology
● Grounded theory

Use of a specific qualitative approach often comes with a

theoretical perspective or lens that shapes your understanding of A QUALITATIVE EXAMPLE (IPA)

Interpretative Phenomenological Analysis

The aim of IPA is to explore in detail how participants are making

sense of their personal and social world. IPA looks at the meaning of
particular experiences, events, and states for participants. The
approach is phenomenological in that it involves the detailed
examination of the participant’s lifeworld.
Research
Comprehensive Exam 2024

Interpretative Phenomenological Analysis **Note that the theoretical lens is always linked back to the
phenomenon of interest (i.e., intimate violence).
In the theoretical framework, explain the theoretical assumptions of
IPA. What is phenomenology? Explain how phenomenology as a Research questions are stated according to the theory:
lens will shape your understanding of the phenomenon. • What are the storylines…?
Examples of research questions: • What are the positions…?

• How do people come to terms with the death of a partner? >>Quantitative Designs
• What does it mean to be transgender?
SURVEY
• How does being HIV-positive shape personal relationships?
● Provides a quantitative or numeric description of trends,
• What model of the person do priests have?
attitudes, or opinions of a population using a sample (e.g.,
• How do parents experience a rare illness affecting their child?
population’s attitudes, opinions) ž describe a population
• What is the subjective experience of the Filipino male tambay? ● Goal is to generalize from sample to population
A QUALITATIVE EXAMPLE (POSITIONING)
EXPERIMENT
Outline of theoretical framework
● Tests the effect of a treatment on an outcome,
• A social constructionist paradigm (the wider theoretical lens) controlling for all other factors -> test the effect on
• Primacy of relational processes an IV on a DV, controlling for all EVs
• A relational view of intimate violence** ● Uses randomization
• A discursive approach (a specific approach within social ● Goal is to generalize from sample to population
constructionism)
SURVEY PLAN
• Centrality of language/talk
• Focus on talk in episodes involving intimate violence** [S1] Survey design
• Positioning theory* (a specific discursive theory) [S2] Population & sample
• Analysis of social episodes [S3] Instrumentation
• The position/act-action/storyline triad
[S4] Variables in the Study
• Positioning in intimate violence**
*Note that positioning theory is a discursive approach within a social [S5] Data analysis and Interpretation
constructionist paradigm.
 Research
Comprehensive Exam 2024

[S1] SURVEY DESIGN

• Stratified • Purposive
● Identify purpose of the survey • Clustered • Snowball
● Indicate why survey is the preferred data collection
method [S3] INSTRUMENTATION
● Indicate if cross-sectional or longitudinal
● Specify form of data collection and why this is preferred Name the survey instrument (developed for the study, modified,
adapted an existing instrument -> permission to use!)
• Self-administered questionnaire (individual or group,
face-to-face or airmail/email) For an existing instrument, describe the established
validity and reliability
• Interview (face-to-face or telephone/chat)
• Structured records For a modified instrument, re-establish validity and reliability
• Structured observation For a developed instrument, establish validity and
reliability
[S2] POPULATION & SAMPLE
QUICK REVIEW OF VALIDITY AND RELIABILITY
● Identify population
● Identify sampling design (process of selecting sample or Validity (“are we measuring what we are claiming to measure?”)
participants)
• Content validity (face, expert)
• Random/ Probability/ Systematic vs Nonrandom/ • Criterion validity (predictive, concurrent)
Nonprobability/ Convenience
• Construct validity (confirmatory factor analysis or structural
• Randomization ensures a representative sample from a equations modeling, exploratory factor analysis, comparison groups
population to allow for generalization or known groups, convergent, discriminant or divergent)
● Discuss sampling procedure
● Indicate sample size or number of people in the study (n) Reliability (“are we getting consistent scores over repeated
observations?” i.e., across time, across situations)
QUICK REVIEW OF SAMPLING DESIGNS
• Stability or test-retest
Nonrandom/ Convenience • Equivalence or parallel/ alternative forms
Random/Probability • Internal consistency (split-half or Cronbach’s Alpha)

• Simple • Convenience [S3] INSTRUMENTATION

• Systematic • Quota
Research
Comprehensive Exam 2024

Include sample items from the instrument; append instrument to [E1] PARTICIPANTS
proposal
Describe process for selecting participants
Indicate major content sections of the instrument (cover letter,
instructions, items, scales) • Random sampling?
• Convenience sampling?
Discuss plans for pilot testing to improve questions, format, scales
and to determine content validity
Describe process for random assignment of individuals to groups
Include plans for incorporating pilot test results to final version of
Identify controls in the experimental design
survey instrument
• Matching participants
[S4] VARIABLES IN THE STUDY • Controlling for covariates
• Selecting homogeneous samples
Usually presented in the purpose statement and research • Blocking into subgroups
questions/hypotheses sections
Relate variables, questions, or hypothesis and sample items on the Identify the number of participants in each group and the procedure
survey instrument for determining group size (alpha level, power analysis, effect size)

[S4] DATA ANALYSIS [E2] VARIABLES

● Descriptive statistics Clearly identify the independent variable/s (IV/s)

● Correlational statistics • The variable manipulated (the “cause”) & how it was manipulated
● Other statistical procedures Clearly identify the dependent variable/s (DV/s)
• The variable measured (the “effect”) & how it was measured
EXPERIMENTAL PLAN Clearly identify the extraneous variable/s (EV/s)
[E1] Participants (Subjects) • The variable/s controlled for because they may affect the DV (the
“confounds”) & how they were controlled
[E2] Variables
[E3] MATERIALS
[E3] Materials (Instruments)
Describe the materials or instruments used (developed for the study,
[E4] Procedures
modified, adapted an existing instrument ž permission to use!)
[E5] Experimental design • Items
Research
Comprehensive Exam 2024

• Scales • Internal validity threats – issues with the experimental procedure

• Reports of reliability & validity and participants (e.g., changes in the procedure or participants
Discuss the materials used for the experimental treatment condition during the experiment that may affect validity)
• Pilot test • External validity threats – issues with the persons, settings, and
situations to which the sample data applies (e.g., generalizing
• Training for administration
beyond the sample of the study)
• Manipulation check
Identify threats and how they will be addressed in the
[E4] PROCEDURE study
Discuss step-by-step procedure for the experiment
Discuss statistical analysis
• Descriptive statistics
• Inferential statistics (e.g., t-tests and anovas)
• Other statistical procedures

[E5] EXPERIMENTAL DESIGNS

Pre-Experimental Designs

● One-Shot Case Study (Grp A X---O)

● One-Group Pretest-Posttest
Identify what is being compared in the experiment Grp A O1---X---O2
• Between subjects (2 or more groups)
● Static Group Comparison or Posttest-Only With Nonequivalent
• Factorial designs (2 or more treatments/IVs)
Groups
• Within subjects or repeated measures
Grp A X---O
Provide a diagram or figure of the experimental design Grp B--------O
● Alternative Treatment Posttest-Only With Nonequivalent
Threats to validity (“can we really conclude that the IV affected Groups
the DV?”)
Grp A X1-- O
Grp B X2-- O
Research
Comprehensive Exam 2024

Quasi-Experimental Designs Grp A R--------X---O

● Nonequivalent (Pretest & Posttest) Control-Group Grp B R-------------O

Grp A O---X---O SURVEY
Grp B O--------O
Survey Design

● Single-Group Interrupted Time-Series Population & Sample

• Representative Sample
Grp A O--- O---O---X---O---O--O
•Generalizability
● Control-Group Interrupted Time-Series Instrumentation
• Reliability
Grp A O--- O---O---X---O---O---O
• Validity
Grp A O--- O---O---O---O---O---O
EXPERIMENT
True Experiments
Experimental Design
● Pretest-Posttest Control-Group
Participants
Grp A R---O---X---O • Random Assignment
Grp B R---O--------O Variables
● Posttest-Only Control-Group • Manipulation
• Measurement
Grp A R---X--O
• Control
Grp B R-------O
● Solomon Four-Group Materials/Instruments
• Reliability
Grp A R---O---X---O • Validity
Grp B R---O--------O Procedures
Research
Comprehensive Exam 2024

• Threats to Internal Validity 3. Case studies

• Threats to External Validity 4. Discursive psychology
5. Foucauldian discourse analysis
KEY ISSUES FOR QUANTITATIVE DESIGNS
6. Memory work
Given its objectives of
• Describing a population Quality in qualitative research (Henwood & Pidgeon, 1992)
• Determining a relationship between X & Y of a population 1. the importance of fit
• Testing for the effect of an IV on a DV of a population 2. integration of theory
and the end goal of generalizing from sample to population, 3. reflexivity
the key issues for Quantitative designs are internal validity and 4. documentation
external validity. 5. theoretical sampling & negative case analysis
6. sensitivity to negotiated realities
>>QUALITATIVE METHODS 7. Transferability

QUALI (WILLIG, 2008) Quality in qualitative research (Elliott et al, 1999)

1. owning one’s perspective
From recipes to adventures
2. situating the sample
• How, and what, can we know? 3. grounding in examples
• EPISTEMOLOGY 4. providing credibility checks
• Positivism 5. coherence
• Empiricism 6. accomplishing general vs specific research tasks
• Hypothetico-deductivism 7. resonating with readers
• Critique of the ‘scientific method’
• Feminist critique of established epistemologies QUALI (SMITH, 2003)
• Social constructionism
• Epistemology and methodology 1. Phenomenology
• Qualitative research 2. Interpretative phenomenological analysis
3. Grounded theory
1. Grounded theory
4. Narrative psychology
2. Interpretative phenomenology
5. Conversation analysis
Research
Comprehensive Exam 2024

6. Discourse analysis Emerges rather than prefigured (patterns of understanding

emerges or unfolds as one conducts the study)
7. Focus groups
Is interpretive (acknowledges researcher’s personal and
8. Cooperative inquiry
theoretical lens)

QUALI (CRESWELL) Views social phenomena holistically (gives broad views)

Is reflexive ž reflexivity (researcher reflects on her role in shaping
1. Proposal the study – biases, values, interests – represents honesty &
2. Characteristics openness)
3. Strategies of inquiry Uses complex reasoning that is multifaceted, iterative, &
simultaneous ( both inductive & deductive, cycles back & forth from
4. Procedure in the proposal data to theory/problem, simultaneous processes)
5. Researcher’s role
Uses a strategy of inquiry as a guide for procedures
6. Data collection
7. Data recording QUALI (SMITH, 2003)
8. Data analysis & interpretation
Validity and qualitative psychology (Yardley, 2000)
9. The Quali narrative (Quali results)
10. Validity in Quali 1. sensitivity to context (sensitive to the literature, to the method, to
the data, to the socio-cultural situation, to the relationship between
researcher & participant)
1. PROPOSAL
2. commitment, rigour, transparency, & coherence (degree of
● Convince audience of the legitimacy of quali
● Review needs of potential audience (do they know quali? engagement, thoroughness, clarity and detail, logic in arguments)
what do they need to know about quali?)
● Present basic characteristics of quali if needed 3. impact & importance (says something useful and makes a
difference)
2. CHARACTERISTICS (ROSSMAN & RALLIS, 1998)
3. STRATEGIES: RESEARCH DESIGNS (CRESWELL)
Takes place in the natural setting
Uses multiple methods that are interactive & humanistic (often [1] Narrative
involve active participation by participants and sensitivity to the [2] Phenomenology
participants)
-> for studying individuals
Research
Comprehensive Exam 2024

[3] Case study Purposively or purposefully select participants or sites that will best
[4] Grounded theory help understand the research question
->for exploring processes, activities, & events • Setting (where)
• Actors (who)
[5] Ethnography
• Events (what)
->for learning about cultural behaviors of
individuals or groups • Process (changes in what)

Indicate type or types of data to be collected

4. QUALI PROCEDURE IN THE PROPOSAL
• Observations (field notes)
• Interviews (open-ended questions)
Identify the specific design • Documents (public or private)
Provide background information about the design (origin, • Audio & visual material (photos, videos)
applications, definition)
7. QUALI DATA RECORDING
Discuss why it is the appropriate strategy for the study
Identify how the use of the design shapes the questions, data Plan the approach for data recording
collection, analysis, etc. Use an observational protocol for observation (e.g., a form divided
into descriptive notes, reflective notes, & demographic info)
5. QUALI RESEARCHER’S ROLE Use an interview protocol for interviews (e.g., instructions,
questions, space for recording comments & notes)
Identify biases, values, and personal interests in the topic and
Record interviews using handwritten notes, audiotaping, or
process videotaping
Include background of topic, setting, or participants Record documents and visual materials’ key content
Comment on the relationship between the researcher and the
participants and sites. e.g. “backyard” research 8. QUALI DATA ANALYSIS & INTERPRETATION
Indicate steps taken to obtain permission from the institutional
review board Step 1: Organize and prepare the data for analysis (e.g.,
transcribing interviews, sorting data)
Discuss steps to gain entry to a setting & securing permission to do
research from “gatekeepers” Step 2: Read through all the data (e.g., obtain a general sense of
the information and reflect on its overall meaning), writing notes in
Comment about sensitive ethical issues and how are addressed
margins or recording general thoughts about the data
6. QUALI DATA COLLECTION • What general ideas are the participants saying?
Research
Comprehensive Exam 2024

• What is the tone of the ideas? • Narrative passage (e.g., detailed discussion of themes,
• What is the general impression of the depth, credibility, & sub- themes, & examples)
use of the information? • Visuals, figures, tables
• Process models, drawings, tables of case information
Step 3: Begin detailed analysis with a coding process (organizing
Step 6: Interpret or make meaning of the data.
data –categories or labels – before giving meaning to the data)
• “What were the lessons learned?”
Notes on coding: • Researcher’s personal interpretation
1. Get a sense of the whole. Read transcriptions carefully. Jot down • Relationship with past literature and theory
ideas. • Raising questions
2. Pick one and discover its underlying meaning. Write thoughts on • Advancing an action agenda
margins. • Implications to theory & practice!
3. Complete for all and make a list of topics. Cluster topics.
Notes on the generic processes in quali data analysis:
Organize into major, minor, common, unique, etc.
• It is an ongoing process involving continual reflection about the
4. Go back to data and code using the list of topics. See if new
data, asking analytic questions, and writing memos throughout the
categories or codes emerge.
study. (not sharply divided steps or processes)
5. Find the most descriptive wording for categories. Reduce into
• It involves using open-ended data.
interrelated categories.
• It often entails tailoring the data analysis to specific types of
6. Make a final list of codes.
qualitative research strategies. (e.g., narrative, phenomenology,
7. Code data and arrange by codes. Do preliminary analysis.
case study, grounded theory, ethnography)
8. Recode data if needed.

Step 4: Identify themes during the coding process. 9. THE QUALI NARRATIVE (QUALI RESULTS)
• Themes appear as major findings.
Have a plan on how to present the data analysis. Use examples
• Find interconnections among themes (e.g., a storyline for
narrative, a theoretical model for grounded theory, for each case from past studies.
and across cases for case studies, a general description for
phenomenology). Consider the general form of narrative (e.g., analysis by case or
across cases? detailed description? a process model? a story? a
• Go into complex theme connections beyond description.
chronology?)
Step 5: Advance how the description and themes will be Specific conventions:
presented in the narrative.
• Vary long, short, and text-embedded quotations
Research
Comprehensive Exam 2024

• Script conversation & state in local language of an account. Validity is equated with trustworthiness, authenticity,
• Present text in table or matrix and credibility.
• Use words from participants
• Mix quotes with researcher’s interpretations KEY ISSUES FOR QUALI
• Use indents or special formats
• Use first person “I” or “We” Validity and qualitative psychology (Yardley, 2000)
• Use metaphors
1. sensitivity to context (sensitive to the literature, to the method, to
• Use the narrative approach of your selected strategy of
inquiry. the data, to the socio-cultural situation, to the relationship between
researcher & participant)
10. VALIDITY IN QUALI 2. commitment, rigour, transparency, & coherence (degree of
engagement, thoroughness, clarity and detail, logic in arguments)
Strategies for checking accuracy of findings: 3. impact & importance (says something useful and makes a
• Triangulate data sources. difference)
• Check back with participants.
>>MIXED METHODS
• Use rich, thick descriptions (to make reader share the
experience). Mixed methods research is expanding with the development &
• Clarify researcher’s biases & reflections. legitimacy of both quanti & quali
Procedures for mixing quanti & quali was developed to make
• Present negative information that runs counter to themes. analyzing complex & multiple forms of data more understandable
• Spend prolonged time in the field to gain in-depth NATURE OF MIXED METHODS RESEARCH
understanding of the phenomenon.
• Ask a person to review and ask questions about the study. Because it is new, it is useful to give a basic definition and
description of mixed methods in a proposal
• Ask an external person to review the entire project.
• Trace its brief history (e.g., late 80s and early 90s,
Reliability (e.g., inter-rater) and generalizability (e.g., multiple cases)
play a minor role in quali from data triangulation)
Validity is the strength of quali but it means accuracy of the findings
from the standpoint of the researcher, the participant, or the readers • Define mixed methods research (i.e., collecting & analyzing both
quanti & quali data)
• Give reasons for “mixing”
Research
Comprehensive Exam 2024

• Note challenges & requirements of “mixing” At data analysis (e.g., transforming quali themes into numbers or
vice-versa)
CRITERIA FOR CHOOSING A STRATEGY
At data interpretation (e.g., comparing quanti & quali results)
[1] What is the implementation sequence of quanti & quali data
With some combination
collection?
[2] What priority will be given to quanti & quali data collection & [4] THEORETICAL PERSPECTIVE
analysis?
[3] When will quanti & quali data & findings be integrated? Explicit - a theory is presented as a guiding framework
Implicit - (all research designs have implicit theories)
[4] Will an overall theoretical perspective be used? Perspective may be from the social sciences or from an
advocacy/participatory lens (e.g., gender, race, class)
[1] IMPLEMENTATION SEQUENCE
TYPES OF MIXED METHODS STRATEGIES
● No sequence -> Concurrent
● Sequential -> Quali first ● Convergent Parallel
● Explanatory Sequential
● Sequential -> Quanti first
● Exploratory Sequential
[2] PRIORITY ● Embedded
● Transformative
● Equal ● Multiphase
● Quali is priority
● Quanti is priority 1. CONVERGENT PARALLEL

Priority is in terms of which data is emphasized, the extent of

treatment, & the use of theory (inductive or deductive)

[3] INTEGRATION

At data collection (e.g., combining open- ended with closed

questions in a survey)
Research
Comprehensive Exam 2024

Most familiar (i.e., the traditional mixed methods model or data 3. EXPLORATORY SEQUENTIAL
triangulation)
To confirm, cross-validate, or corroborate findings
Combining strength of quanti & quali
Quanti & quali data collection is concurrent
Ideally equal emphasis on quanti & quali
Quali before quanti, priority on quali
Integration during data interpretation phase, noting convergences &
Quanti used to generalize quali results or help explain quali
divergences
Goal is to explore a phenomenon
Challenge is how to make results comparable & how to resolve Often used for developing tests, scales, or instruments
discrepancies Useful for exploring an emerging theory

2. EXPLANATORY SEQUENTIAL 4. EMBEDDED

Quanti and quali data are collected simultaneously but one

method guides the project
Most straightforward; Quanti before quali, priority on quanti The embedded method is given less priority
Integration is during the data analysis stage
Quali used to support quanti results or explain unexpected results
A primarily quali design can use quanti to enrich description of
Goal is to explain relationships sample participants
A primarily quanti design can use quali to describe an aspect that
cannot be quantified
Research
Comprehensive Exam 2024

Multilevel designs can involve different methods for different Longitudinal study with a focus on a common objective
groups or levels; one method could be used within another (e.g., for the multiple projects
case studies within an experiment)
Applications: evaluation or program implementation
5. TRANSFORMATIVE MIXED

Either quanti or quali first

Results are integrated during interpretation
Has a theoretical perspective that guides the study (e.g., framework,
ideology, advocacy)
Not yet clear how theory or advocacy guides method

6. MULTIPHASE MIXED

Conduct of several mixed methods projects, sometimes including

mixed methods convergent or sequential approaches or only
quantitative or qualitative studies
Research
Comprehensive Exam 2024

Explore outliers -> [for sequential strategies] quanti data can yield
extreme cases which can be interviewed qualitatively

Instrument development -> [for sequential strategies] obtain themes

from participants in quali phase and then use themes as scale items
in quanti phase
Examine multiple levels -> [for embedded strategies] conduct a
survey at one level (e.g., families) to get quanti & conduct interviews
at another level (e.g., individual members of families) to get quali

DATA VALIDATION

Check validity of quanti data ->reliability & validity of instruments,

issues relating to internal validity & external validity
Check validity of quali data ->check accuracy of findings
(triangulation, validation, detailed description, etc.)

REPORT STRUCTURE

· For sequential studies ->organize results in terms of 2 distinct

DATA COLLECTION
phases (e.g., quanti then quali, or quali then quanti); how data from
2nd phase supports data from 1st phase in conclusion
Identify & be specific about the type of data
· For parallel studies ->quanti & quali may be 2 separate sections
Recognize that quanti often involves random sampling and that but the analysis & interpretation are combined to show
quali often involves purposeful or purposive sampling convergences
Relate the procedures to the visual model & give the details for each
phase · For transformative studies ->advocacy issue is presented and
data is used to advance the advocacy issue or agenda for
DATA ANALYSIS change

Data transformation -> [for parallel strategies] quantify the quali data
(e.g. coding quali themes & making frequency counts) or qualify the
quanti data (e.g., creating themes from clusters of factors on a
scale)
Research
Comprehensive Exam 2024

KEY ISSUES IN MIXED METHODS ○ Be professional; constantly be aware of responsibility

to society
QUANTI data ->issues relating to the internal validity & external ● Integrity
validity of the study; as well as reliability & validity of instruments ○ Be scrupulously honest
● Justice
QUALI data ->issues relating to validity or accuracy of the ○ Always treat people fairly
findings; i.e., the trustworthiness, authenticity, and credibility of ● Respect for peoples’ rights and dignity
the study ○ Safeguard individual rights; protect rights of
privacy and confidentiality
>> Ethics: in Psychological Research
10 STANDARDS OF PRACTICE
Ethics: “... concerned with that which is deemed acceptable in
1. Resolving ethical issues
human behaviour, with what is good or bad, right or wrong with
human conduct in pursuit of goals or aims.” 2. Competence
3. Human relations
Ethical Issues 4. Privacy and confidentiality
5. Advertising and other public statements
● Psychologists are obliged to consider the psychological 6. Record keeping and fees
well-being, health, values and dignity of their participants. 7. Education and training
● If they do not do this properly, their research is described as 8. Research and publication
unethical. 9. Assessment
● Researchers should strive to ensure that their research is as 10. Therapy
ethical as possible.
● Problems arising from conflict between: PRACTICES THAT RAISE ETHICAL ISSUES
■ What is necessary for the research
■ The moral obligations towards participants. ● Getting informed consent
● Deceiving participants
APA CODE OF ETHICS: 5 GENERAL PRINCIPLES ● Putting PPs at elevated risk of harm during or after
participation
● Beneficence and non-maleficence ● Obtaining confidential information
○ Constantly weigh costs & benefits; protect from harm; ● Invading PPs’ privacy
produce greatest good
● Fidelity and responsibility CONSENT

● Participants should give informed consent.

Research
Comprehensive Exam 2024

DEBRIEFING

● In studies involving children, informed parental consent should ● After the study:
be obtained. ○ Inform participants about the true nature of the research
● Payment should not be used to induce risk taking behaviour. they have participated in..
● Special safeguarding procedures are necessary with subjects ○ Ensure no harm has occurred.
with limitations in communication or understanding or where ○ Obtain feedback about the study; check for negative
unable to give real consent. effects or misunderstandings; invite and answer
questions.
DECEPTION ○ Offer the right to withdraw and destroy data if so.
● Should be avoided if at all possible, especially where it
would raise other issues WITHDRAWAL
● Will it cause stress?
● Can stop participating at any time, including during and after the
● Would the participants participate if they knew?
● Only permissible where research is very important and no study. For any reason.
alternative method is available (APA). ● Their results will not be included.
● Participants must be fully informed as soon as possible ● This should be made clear at the start & end of the study.
(BPS).
PROTECTION OF PARTICIPANTS
CONFIDENTIALITY
● Risk should be no more than participants expect in everyday life.
All data should be confidential, all PPs should be anonymous
and unidentifiable unless prior informed consent given. ● Physical and psychological harm (e.g. stress, damage to self
image)
○ Can be problematic in case studies. ● Any negative effects of an investigation should be dealt with
○ Procedures should routinely anonymise PPs (e.g. afterwards.
through use of numbers; not recording names etc.). ● If evidence of psychological or physical problems arise from
○ If confidentiality cannot be ensured e.g. a participant study: the researcher must offer advise or recommend them
discloses something illegal, they must be warned of to a professional.
this at the start of the investigation. ● There must be opportunity to get in touch with the researcher
afterwards.
Research
Comprehensive Exam 2024

UNETHICAL RESEARCH ● Unethical research can be justified if the costs are outweighed
by the benefits of the findings.
● Research that breaches the ethical guidelines may still be
carried out if it’s outcome can be justified. *********
● Deception is very common: this may cause distress or Pointers:
embarrassment for the participants. This may be justified if they
-Philosophical worldviews, how they connect to your research
are treated ethically after the event e.g. methodologies and designs, strategies
○
counselled
○
ensured privacy -characteristics of the different Research methods
○
given the right to withdraw.
○
Need to deceive participants when investigating -application of the different methods in a given problem
prejudice, memory etc.
-reliability and validity of research methods
NON-HUMAN ANIMALS

● Use the smallest number of animals possible

● Avoid procedures which cause pain, suffering, distress or lasting
harm. If not possible need a project license.
● Ensure that animals are well housed and cared for.
● Normal feeding and breeding habits should not be disrupted.

Investigate free-living animals as much as possible, with as little

interference with their natural environment.

SUMMARY

● Ethical research protects the welfare of participants.

● The main ethical guidelines covering psychological research are:
Can Do Can’t Do With Participants.