Paper ID #11839

Using Phenomenography: Reflections on Key Considerations for Making Method-

ological Decisions
Emily Dringenberg, Purdue University, West Lafayette
Emily Dringenberg is a PhD Candidate in Engineering Education at Purdue University. She holds a
Bachelor of Science in Mechanical Engineering (Kansas State ’08) and a Master of Science in Industrial
Engineering (Purdue ’14). Her current dissertation research focuses on using qualitative methods to ex-
plore the experiences of students engaging with engineering design problems. Additionally, her research
interests include transfer of learning, personal epistemology, and design learning.
John Alexander Mendoza-Garcia, Purdue University, West Lafayette / Pontificia Universidad Javeriana - Bo-
gota, Colombia

John Mendoza-Garcia is a Colombian Systems Engineer (Bachelor’s and Master’s degree) that currently is
a Ph.D candidate in Engineering Education at Purdue University. His advisors are Dr. Monica E. Cardella
and Dr. William C. Oakes. He is interested in understanding the development of systems thinking to
support its assessment and teaching. Currently, he works for the first year engineering program at Purdue
where he has taught the engineering introductory courses in design and algorithmic thinking, and has also
developed content for these courses. He has an appointment with the department of systems engineer-
ing at Pontificia Universidad Javeriana in Colombia since 2005 and is currently on a leave of absence.
There he taught systems thinking and coordinated the professional internships, the social internships and
the graduation project. He worked as Software Engineer in different companies for seven years before
transitioning to academia.

Mariana Tafur, Purdue University, West Lafayette

Mariana Tafur is a Ph.D. candidate and a graduate assistant in the School of Engineering Education at
Purdue University. She has a M.S., in Education at Los Andes University, Bogota, Colombia; and a B.S.,
in Electrical Engineering at Los Andes University, Bogota, Colombia. She is a 2010 Fulbright Fellow. Her
research interests include engineering skills development, STEM for non-engineers adults, motivation in
STEM to close the technology literacy gap, STEM formative assessment, and Mixed-Methods design.
Mr. Nicholas D. Fila, Purdue University
Nicholas D. Fila is a Ph.D. candidate in the School of Engineering Education at Purdue University. He
earned a B.S. in Electrical Engineering and a M.S. in Electrical and Computer Engineering from the
University of Illinois at Urbana-Champaign. His current research interests include innovation, empathy,
and engineering design.
Ms. Ming-Chien Hsu, Purdue University, West Lafayette

Page 26.1676.1

c American Society for Engineering Education, 2015

 Using Phenomenography: Reflections on Key Considerations for
Making Methodological Decisions

Abstract

Phenomenography is an emerging method in the engineering education research community.

Critical variations exist across studies regarding the use of phenomenography as a
methodological approach. Beyond many of the documented variations, including the Swedish
and Australian approaches, many nuances are not necessarily explicit in existing studies. The
authors of this study contributed to this area of scholarship through individual reflection,
collaborative discussion, and synthesis of their experiences of making methodological decisions
while designing and conducting phenomenographic studies in engineering education. The results
of this work include the identification of five key considerations: Nature of phenomenon,
Application of outcome space, Development of outcome space, Treatment of data, and Stance
towards participants. Reflections on each of these considerations are summarized and provided
in full. Our analysis of the reflections revealed that the application of the phenomenographic
methodology must respond to nuances in individual contexts. Thus, the primary contribution of
this paper is the identification of five key considerations that can help guide a phenomenographic
research approach, and transparency of how five emerging phenomenographic researchers have
dealt with these considerations.

Introduction

Phenomenography is a qualitative methodological approach that seeks to identify the variety of

ways people experience a given phenomenon 1-3. Due to its widespread applicability to both
classroom practice and educational research, this approach has steadily gained popularity since
its official introduction in 1981 3,4. Today, phenomenography is recognized as one of the top
emergent methodologies in engineering education 5. As the use of phenomenography has grown,
so too has the variation in phenomenographic methods 6,7. While literature commonly draws
attention to the distinctions between work by Ference Marton in Sweden (e.g. pure
phenomenography and later “new” phenomenography 1,3,4,8) and John Bowden in Australia (e.g.
developmental phenomenography 2), individual studies often demonstrate subtle variations or
combine elements from multiple approaches. Further, more nuanced methodological decisions
and the rationale behind them are often left implicit.

Prior phenomenographic studies in engineering education include explorations of student

understanding or students’ ways of experiencing human-centered design 13, size and scale in
nanotechnology 14, repetitive and conditional structure in programming languages 15, workplace
problem solving 16, and transition from pre-college engineering to first year engineering 17.
Another study focused on faculty members’ experience of including simulation tools in
instruction 18. Studies of practitioners’ (e.g., professional engineers and designers) ways of
Page 26.1676.2

experiencing cross-disciplinary practice 19, design 20, and sustainable-design 21 also demonstrate
applicability to engineering education. Many of these studies cite both Marton and Bowden,
indicating influence from both broader approaches. At the same time, they also demonstrate
more nuanced methodological variation.

The purpose of this paper is to explicitly articulate some key considerations when designing
phenomenographic studies within engineering education, and further, to identify how variations
may affect the outcomes of a particular study. The key considerations discussed herein were
identified through an iterative process of individual reflection and group discussion on our own
phenomenographic work (as Ph.D. candidates in engineering education). In this paper, we first
provide a brief summary of phenomenography, including its history, core practices, and
acknowledged variations. Next, we describe the five distinct considerations and provide our
personal reflections as examples of how key methodological decisions have been approached and
justified. We do not suggest that these reflections demonstrate a definitive way of conducting
phenomenographic research—our own ideas have changed as a result of this project and will
likely change further as we progress as researchers. Still, we imagine this work may be helpful
to fellow researchers interested in phenomenography or those encountering similar
methodological decisions.

Background

This section provides an overview of the history and evolution of phenomenography as a

research approach.

Swedish origin

Marton and his research group developed the phenomenographic approach for educational
research during the 1970s when he and his team conducted empirical studies on the way Swedish
students learned from academic texts 8. At that time the research questions were:

(1) “What does it mean that some people are better learners than others?”
(2) “Why are some people better at learning than others?” 8

Throughout these empirical studies, a new research approach, phenomenography was gradually
specified. The guiding research questions of this approach were:

(1) “What are the different ways of experiencing the phenomenon?” and
(2) “How are these related to each other?” 8.

When addressing these questions, a second-order perspective is utilized for “describing people’s
experience” 3 (p. 1) of a phenomenon (e.g., learning). This second-order perspective focuses on
the interactions between the person who experiences the phenomenon and the phenomenon
itself. Phenomenographic research results in an outcome space which comprises uniqueness and
variation among different ways of experiencing a phenomenon, which translates into categories
Page 26.1676.3

of description 9.
During his early work, Marton and his colleagues identified three lines of phenomenographic
research: (1) general aspects of learning, where variation focuses on learners approaches and
experiences; (2) learning situated within context domains, where variation focuses on learners
preconceptions; and (3) pure phenomenography, where variation describes on how people
conceive various aspects of their reality 9 (p. 189). Marton proposed a systematic analysis
method that included a pool of meaning, a selection of transcript excerpts relevant to research
questions. The researcher’s attention shifts from the individual to the meanings embedded in the
pool of meaning. An iterative process of analysis alternates between the individual participants’s
context and the broader context of the pool of meanings, aiming to define boundaries between
categories that are developed during the analysis 9 (p. 198).
Ference Marton’s current approach (seeking variation)

The 1990s saw the introduction of the “new” phenomenography 8,10. With this development, the
research question changed from, “What is a way of experiencing something?”—which was more
related to understanding human meaning—to, “What is the actual difference between two ways
of experiencing the same thing?” or, ”What are the different ways the ‘thing’ can be
experienced?” 8. It can be interpreted that this “thing” is an instance of the phenomenon.
According to Marton, you cannot experience a concept (e.g., the concept of a circle 4), only an
instance of that concept (e.g., a circle drawn on paper). Accordingly, studying how people
experience the same thing means studying how people experience the same instance of the
phenomenon under investigation, which is different than asking participants to describe their
experiences of different instances of the phenomenon they have experienced in the past (See
Marton’s Note 36 4). Each way of experiencing the phenomenon is described by a set of what
Marton calls “critical aspects,” and “critical features” that a learner is able to “discern,” “be
aware of,” or “focus on” simultaneously. These critical aspects have a structural aspect 11 and a
meaning 4,11, or referential aspect. The structural aspect is composed of an internal horizon
(constituted by the parts that are related and interacting all together carrying an overall meaning 1
(p. 148)) and an external horizon, which is explained as, “the way the phenomenon is discerned
from its context and how it is related to its context as well” 1 (p. 89). These ways of
experiencing the phenomenon are related in a hierarchy from less to more complex. They
are developed to inform teachers “what is to be learned” in terms of critical aspects, or "what
learners need to learn in order to achieve the" educational objectives or learning goals4.

Australian approach

John Bowden is another contributor to the development of phenomenographic research. He

primarily distinguishes himself from Marton’s pure approach by emphasizing the practical
application of his work. He aims to create outcome spaces that are “developmental” in that they
can enable others to change their own experiences, usually in an educational setting 12. As he
states, “in developmental phenomenography, as opposed to pure phenomenography, the research
is designed with the intention that there will be practical outcomes. Implications for learning and
for practice abound. The research is intended to inform and influence practice” 2 (p. 35).
Additionally, Bowden describes, “the aim is to describe variation in experience in a way that is
useful and meaningful, providing insight into what would be required for individuals to move
Page 26.1676.4

from less powerful to more powerful ways of understanding a phenomenon” 2 (p 72).

Additional distinctions about Bowden’s approach to phenomenographic research is his belief that
analysis should be performed by a team of researchers, as multiple perspectives allow for more
comprehensive analysis 12. While the team approach cannot be achieved in all research settings
(e.g., dissertation research), conversations with another playing the role of devil’s advocate can
introduce alternative perspectives into analysis. Bowden also reports his preference for waiting
until the categories have stabilized before addressing the structural relationship between them.
He argues, “If the development of the structural relationships is undertaken simultaneously with
the development of the categories of description, there is potential to distort the categories by
including the relation of the researcher to the phenomenon in addition to the true focus of the
study, the relation between the subjects and the phenomenon” 12 (p. 16).

Documented variations

As noted above, the Swedish and Australian approaches represent variants of phenomenographic
research, but variation extends beyond one’s general approach. Methods decisions differ from
study to study, context to context. Åkerlind 10, for example, reviewed a selection of methods
writings by authors from both schools and identified four dimensions along which
phenomenographic studies commonly vary. These variations include: handling interview
transcripts, emphasis on collaboration, ways of managing data, and ways of constituting
structure.

In addition to these variations related to data analysis, other researchers have noted variations in
the type of data collected, research assumptions, and interview technique. Marton 22, for
example, acknowledges that interviews are the primary, but not necessarily the only, means of
collecting data for phenomenographic analysis. Richardson 23 even suggests that observations are
more appropriate for the types of questions phenomenography attempts to answer. Ashworth and
Lucas 24,25 stress the importance of bracketing presuppositions, knowledge, and beliefs related to
the nature of research, the phenomenon, and the participants, but also note that many
phenomenographic studies do not do this.

Additional variation

Despite the documented variation in phenomenography, additional nuances exist that have not
been explicitly articulated in the literature. Others have previously used phenomenographic
studies in order to explain these variations in phenomenography 26. Our approach, as novice
phenomenographers was to reflect on our own experiences with and interpretations of
phenomenography. The goal of this approach was to provide a more immediate and intimate
portrayal of variation in phenomenography, with a focus on its practical implementation. Our
reflection was guided by the following research questions:

1. What methodological considerations, beyond what is documented in the literature, are

important for researchers utilizing a phenomenographic approach?
2. How do we, as first-time phenomenographic researchers, individually address on these
methodological considerations?
Page 26.1676.5
Research Approach

The concept for this paper came out of our informal conversations as five doctoral students, each
at a different stage in a phenomenographic dissertation project. As we read about, designed, and
conducted phenomenographic studies, we each grappled with different subtleties and challenges.
This emergent discussion evolved into the formal research questions posed in this study. To
address these questions, we underwent several rounds of individual reflection followed by group
discussion and analysis. This section outlines our approach chronologically.

Initial brainstorming and reflections

To begin, we all met to discuss our evolving understandings of the phenomenography literature
and also to familiarize each other with our respective studies, including the methodological
issues that we were facing. To begin generating our data, we identified some common themes of
our initial conversations and then each wrote a reflection about our own process of developing an
understanding of phenomenography. This included a summary of our study as well as our
general rationale for selecting phenomenography. We also reflected on how we had come to
understand the “schools of thought” within phenomenography, e.g., those outlined in the
Background section. In addition, each author intentionally documented how their particular
method decisions informed their analysis and the implications for the overall design of their
study. We shared these reflections with each other and then began to analyze our respective
experiences with phenomenography.

Analysis of reflections

As a result of analyzing our initial reflections, we identified five initial themes. These were
chosen based on considerations that occurred in most of the first round of reflections and those
that we judged as important in a group discussion. These initial categories were:

1. Specificity/nature of the phenomenon/what does it mean to experience?

2. Expectation of outcome space
3. Portion of transcript to use for analysis
4. Data analysis construction vs. discover...formation of structural relationships
5. Considering participant background/empathy/naturalistic approach

We each wrote a second reflection addressing these five categories. Again, we shared our
reflections with each other and entered another round of analysis.

Clarification of five considerations for final reflections

By reviewing and discussing our reflections, we were able to clarify how we were each thinking
about phenomenographic research. This allowed us to refine the five initial categories into more
meaningful and cogent considerations that would be applicable to all phenomenographic
Page 26.1676.6

research. We each wrote a final reflection addressing our interpretation of each of these
considerations and how they manifested in our individual dissertation studies. Finally, we
analyzed each of these final reflections and provided a synthesis of the variation that was
captured across our experiences with our respective studies.

Results

This section presents the five major considerations that our research revealed as important for
phenomenographic researchers: Nature of phenomenon, Application of outcome space,
Development of outcome space, Treatment of data, and Stance towards participants. Each
section has a brief summary describing the consideration and an overview of the perspectives of
each of the five authors. Then, the verbatim reflections are included. We have formatted the
reflections as indented quotations for clarity. We encourage interested readers to explore the full
reflections to add depth and context to the summaries.

1. Nature of phenomenon

The first major consideration that our research and discussions lead us to was thinking about how
the nature of the phenomenon of interest relates to the design of a phenomenographic study. Our
collaboration revealed that we had all struggled with defining our phenomenon and ensuring that
our participants are commenting and reflecting on that particular phenomenon of interest. We
also discussed the issue of what it means to experience something. For example, if a participant
is not aware of any of the characteristics we use to define our phenomenon, but they were in a
situation that we would consider to be a concrete representation of our abstracted phenomenon,
have they experienced it? Also, we explored how these questions about the nature of the
phenomenon being investigated influence our methodological decisions.

As a summary of the following reflections on this consideration of phenomenographic research,

Nick started out with a brief summary and discussed his experience with defining innovation as
his phenomenon, but not providing that definition to his participants. Next, Ming-Chien
provided a similar discussion for her study on interdisciplinary learning as well as a reflection on
what she would change about her interview protocol now that she has completed her analysis.
Emily and John both provided their reasoning for their belief that the nature of the phenomenon
doesn’t necessarily indicate a particular method of investigation (e.g. approach of Marton vs.
Bowden). John provided examples from his area of systems thinking and personal
correspondence with Ference Marton while Emily commented on her decision to allow
participants to select their own situation to represent the phenomenon of engaging with problems
with multiple possible solutions. Finally, Mariana’s reflection discussed how the focus of
developing the bounds between the categories describing the participants’ experiences remains
consistent regardless of how broad or narrow the phenomenon of interest may be.

Nick: Phenomenographic studies explore the variety of the ways particular phenomena are
experienced by particular groups of people 1,2. These studies are predicated on a non-dualist
ontology which emphasizes the relationships between the individual participants and the
Page 26.1676.7

phenomenon being studied 2. The relationship between people and the phenomenon is not
characterized solely by the psychological (internal world) nor the physical (external world), but
the specific aspects of the phenomenon that an individual holds in attention at any one time 1.

While the notion of relationality 2 is common to all phenomenographic work, the nature of the
phenomenon can vary from study to study. For example, the authors of this paper are investigating
phenomena as various as solving ill-defined problems, systems thinking, technological challenges,
interdisciplinary learning, and innovation. The context, which can affect the specific aspects of the
phenomenon that are available for participants to experience, also differs from study to study.

The nature of the phenomenon and the context(s) in which the phenomenon is experienced have
serious implications for a study. When the nature of the phenomenon is more concrete (such as
solving a particular physics problem) and the context is limited (such as experiencing this
problem-solving in the same middle school science class), the types of variation uncovered may be
limited in scope but they will also be cohesive. In other words, the researchers can be relatively
certain that participants are discussing the same phenomenon and that the aspects of the
phenomenon they are discussing are connected in some way. When the nature of the phenomenon
is more abstract and context is more widespread (such as my study of innovation among
undergraduate engineering students), the perspectives on the phenomenon may be wider in scope
but there is less guarantee that the participants are even referring to the same phenomenon.

In my dissertation study, I seek to investigate the qualitatively different ways engineering students
experience innovation. For the convenience of the study proposal, I have defined innovation as the
process of developing conceptually novel, technologically feasible, economically viable, and
desirable design solutions that address human needs and change the way people act in and
perceive the world around them. This is not the only possible definition, but one meant to convey
the extent of potential interpretations of the activity I am calling innovation while still situating it
as something engineering students are doing or at least learning to do.

I will not provide a definition of innovation to participants, of course. To do so would be to unduly

bias them towards the definition I have concocted, and thus lead me to miss out on at least some of
the conceptions of innovation they hold (as indicated by Bowden 2). Instead, the interview guides
them to a general topic of innovation through an economy of words: “Please describe an
engineering project in which you have experienced innovation in some way.” The responses of
study participants will then shape the actual definition of the phenomenon that I am studying. I am
studying innovation, whatever that is, as experienced by engineering students. But as I mentioned
earlier, what guarantee do I have that participants are discussing the same phenomenon rather than
different phenomena they happen to call innovation?

The answer is that I have no guarantee beyond. Instead I must be aware of this possibility during
the interviews and guide straying participants towards the phenomenon I am attempting to
investigate through follow-up questions, as well as being aware during analysis when participants
are referring to the target phenomena and other phenomena. This is the tradeoff I must live with
due to my choice in phenomenon. But it should be a fun ride!

Ming-Chien: The phenomenon I am investigating is interdisciplinary learning. I did not define the
term "interdisciplinary learning" for the participants either during recruitment or interviews.
During recruitment, the potential participants took a screening survey to list of learning situation
that they would consider interdisciplinary that they had participated. During the interview, they
would describe a learning situation in a concrete way beginning. An important follow-up question
was why they would consider the learning experience interdisciplinary. Although participants
were in different learning situations, they all had learning experiences that required them to work
across disciplinary differences, which became how I defined the phenomenon that I am
investigating. Therefore, I consider that my participants have experienced the same phenomenon.
Page 26.1676.8

Looking back, I now see the virtue of asking for an experience working across disciplinary
difference instead of asking for an interdisciplinary learning experience. The former is a more
concrete phenomenon and the latter. I would still include in my protocol of how they would define
interdisciplinary learning and what made their learning experience interdisciplinary. My intention
for doing so would be to understand how the term we regularly mean to the participants and how
they interpret that.

The reason that I would change my protocols now is that I came across a transcript in which the
student definitely were in an interdisciplinary situation, but he was not aware that the differences
in disciplinary expertise was part of the interdisciplinary learning experience. I debated on whether
to still include the transcript in data analysis, since I would argue that you could not experience
something if you are not aware of it. Despite my reasoning, I have decided to include the
transcript, and it constitutes a category by itself. My rationale is based on the practical implication
of presenting the category, which relates to the purpose of my study, which is included in another
section of this paper.

John: My understanding is that the same phenomenon can be studied using any of the two
approaches. In both you want to study how people experience something, and specifically, in
Bowden’s and in current Marton’s, both are really interested in studying how people experience
phenomena that are objects of learning, like concepts (i.e. Newton’s laws of physics cited in 1) or
skills (i.e. long distance jumping cited in 4), professional vision driven by a pedagogical
knowledge interest 27, or in the current studies of the co-authors of this paper like my study on
systems thinking or Emily’s study of ill-defined problems.

In a communication I exchanged with Dr. Marton by mail in December 17, 2013, he said that for
him, to experience the world, is related to the way a person “sees it,” and that way of seeing “is
characterized in terms of critical aspects and features of the phenomenon in question [an object of
learning] that are discerned by the person and focused on simultaneously”. The first part,
experiencing as a way of seeing, I think is close to the Australian understanding of experience and
in fact, is close to schools of thought in soft systems thinking that explains that everyone of us sees
or perceive or experience the world differently based on “the lenses” we use to see it. What you do
(your practice), is based on what you see (in the world), and in this, Marton (see for example
Marton 4) also agrees with soft systems thinkers. See for example Checkland’s 28 soft systems
thinking methodology in which he states that it is necessary that everyone in the organization
“see” and participate in the design of the new organization. Similarly, Senge’s approach to
learning organizations 29, states that it is needed that every member of the organization share their
vision regarding the organization, which I understand as a way of seeing as well. In the same way,
Lleras 30-32, who has also developed his own methodology for the design of participative
organizations, have as central feature the development of care between individuals within the
organization, to share not only the way of seeing the organization, but also to care about every
member personal development. I will finally mention Bohm’s approach to dialogue in which he
define it as, “stream of meaning flowing among and through us and between us” 33 (p. 6).

After the meetings I have had with the other co-authors, and after reflecting about what I have
found in techniques Checkland and Senge use to help people in organizations to see other people’s
perception of the company, the workshops I remember with Lleras in which he uses Bohm’s
dialogue techniques to help members of the organization to “see” what the other person was
“seeing” and according to him, my conclusion is that there is no difference in the phenomenon
under study, but there is a difference in how you studied, how you describe it, and the kind of
evidences you look for when trying to describe it. I remember that for example Valero-Silva 34
presented a “dispositif” that he used to explained that a particular historical experience can be
described as a result of the interaction of social/power relationships, forms of self-understanding
(identities), and bodies of knowledge (discourses), and I think that whenever he is trying to look
for evidences of experience in a specific moment of time, he will look for these elements.
Likewise, Lleras used a modified version of the language games 35,36 to understand people
Page 26.1676.9

practices, which are related to how they perceive the world, congruent with “they do based on
what they see.”
I understand that like Valero-Silva and Lleras, Marton’s has also developed a framework
(Variation Theory of Learning 1,4,37) that guides him and his followers when trying to find how
someone experiences an object of learning. As I said above, for him, to “experience something” is
to be able to “identify and focus on critical aspects and critical features of the object of learning
simultaneously”, which is ones qualitative way of experiencing the world. The goal would be then,
using Marton’s framework, to find all the different qualitative ways in which that object of
learning can be experienced, or in other words, all the different critical aspects and features of the
object of learning, and the research approach that have been successfully used to find them is
phenomenography. Later, in my reflection about the development of the outcome space, I will talk
about finding evidences of how people experience a phenomenon.

Emily: In general, I interpret the existing phenomenographic literature as suggesting that there is a
sort of spectrum for the nature of phenomena that range from a specific concept (e.g. in physics
education research) to a general concept (e.g. experiencing a social setting). The most recent work
of Marton seems to fall towards the former end, especially with the idea of conducting think-
alouds during interviews for data collection where each participant is provided the exact same
problem for a prompt. However, Marton also lists a “pure” approach to phenomenography as
focusing on descriptions of how people conceive of more general aspects of their reality (e.g.
political power) 1. Bowden seems to fall more towards the end of the spectrum with more general
concepts, I haven’t been able to find a clear distinction. I believe that either approach could
ultimately be used for any phenomenon. I have also come to realize that the specificity is not the
only way that phenomena differ…they may vary from abstract to concrete, or well-defined to ill-
defined, for example. My phenomenon is working on engineering design tasks early in
engineering training, specifically within the First-year Engineering Program at Purdue University.
To access this phenomenon, I asked my participants to describe an experience they have had
working on a problem with multiple possible solutions. I don’t think this necessarily indicates an
alignment with Marton or Bowden, and because I wanted to allow my participants to talk about
any situation that was relevant and accessible to them, I did not provide them with a concrete
example in the vein of Marton; I am more aligned with Bowden in this regard.

Mariana: “Phenomenography is a research method adapted for mapping the qualitatively different
ways in which people experience, conceptualize, perceive, and understand various aspects of, and
phenomena in, the world around them” 22 (p. 31) in 26. Phenomenography is not only about
experiences, but also perceptions, conceptualizations and understandings according to Marton and
Bowden. But what is to experience? To me is the contact or action related to the real world. This
implies present time, which means that a person is interacting with the phenomenon. The
interaction itself is the same for those performing the same task (e.g. crossing a bridge), but the
experience may be perceived in different ways by different individuals. Furthermore, after having
an experience, the person may conceptualize it, which implies creating an abstraction of the
experience, which may lead to an understanding.

Phenomenography studies the experience through the lens of the participant, it means through her
or his perspectives, perceptions, and understandings. The experience being studied can be broad or
specific, depending on researcher interests and questions. For instance, a researcher may want to
analyze a specific experience such as the ways people use or understand the expression “a piece of
cake”; contrary, he or she may want to analyze a broad phenomenon such as using or learning a
foreign language. The first approach may lead to small variations within certain part of a broader
experience; therefore, the result of this analysis may bring more detail to a fraction of another
outcome space. Although the phenomenon being studied may be specific or general,
phenomenographic analysis is focused on establishing the boundaries between ways of
experiencing the phenomenon, which are identified by studying differences among similar
experiences (i.e. between categories), or similarities among different experiences (i.e. within
categories).
Page 26.1676.10

For my research, the phenomenon I am analyzing is broad: technological challenges experienced

in the everyday life. It is broad due to its general nature; a technological challenge can be
 experienced by interacting with a computer, the Internet, hi-tech lab equipment, or a complex
procedure. All of them are different experiences but my analysis focus on those similarities among
them.

2. Application of outcome space

Within this section we reflected on our expectations about the format and purpose of the
outcome space, including whom should use it and how it should be used. In other words, we
were considering why performing phenomenography may be valuable in our field. All of us are
aware of the importance of building a more comprehensive understanding of a phenomenon or
situation. The outcome space provides a rigorous way for describing a phenomenon using
different experiences that allows a more inclusive understanding of that phenomenon. Further,
we all recognized phenomenography's potential in educational practice due to the identification
of learning progressions, which are elicited by the structure of the outcome space created by the
relationships between the categories in it.

A more practical approach to this connection with education was made explicitly by the majority
of us; Emily, Nick and John acknowledged that this approach provide value for educators. They
present the outcome space as a tool for promoting a more comprehensive design of learning
activities, ensuring alignment between content, assessment and pedagogy, and other educational
activities. Similarly, Emily and Ming-Chien highlighted, with a pragmatic approach, that the
outcome space is a great visualization that allows capturing variation in a clear way. In
particular, Emily and Mariana identified the challenges of reaching saturation, and the dichotomy
between seeking to describe a phenomenon totally and the inability to capture it completely.
Regarding to the nature of the outcome space, Nick and Mariana also reflected that it may be
undetermined and replicable, and should be parsimonious. In our reflections we expressed the
relevance of this method for expanding people’s awareness in diverse fields; further, we expect
that the outcome space offers a new or a shifted framework for understanding a phenomenon.

Emily: My expectation for the eventual application of my outcome space was another key
consideration in my decision process. I felt that my desire to capture variation in students’
experiences solving problems with multiple possible solutions should be of immediate practical
value to engineering educators who wish to provide their students with more comprehensive
design learning experiences. At the same time, I knew that my sample from a Purdue University
would not enable me to saturate the outcome space...additional interviews would almost certainly
represent experiences outside of what I will be able to capture. With these two main expectations,
I found Bowden’s approach to be more aligned with my work. Marton describes the outcome
space as “finite but not closed; in particular, scientific discoveries frequently introduce new ways
of seeing the phenomenon in question” 1 (p. 117). Further, Marton describes the goal of
phenomenography as seeking a totality (or a pertinent and accessible subset of the totality) of the
ways that the phenomena is experienced 1 (p. 121). I am unable to make that claim based on my
sample, which comes from a single first-year engineering program with in the United States. With
that being said, I still believe that either approach can be used, as authors provide evidence that
they don’t strictly adhere to these differences. For example, Marton and Booth 1 acknowledge the
inability to capture all the ways, or the most complete ways that an individual experiences a
phenomenon, which means no outcome space can ever be exhaustive. To further blur the lines,
Bowden and Walsh 38 make a claim that ultimately the outcome space becomes synonymous with
the phenomenon itself! Ultimately, I want my outcome space to be of pragmatic use to
Page 26.1676.11

engineering educators to promote the development of design capability within undergraduate

engineering programs. This echoes the emphasis of Bowden on his desire to “find out how people
experience some aspect of their world, and then to enable them or others to change the way their
world operates…in a formal education setting” 2 (p. 3).

Nick: In my dissertation proposal, I suggested a variety of ways the findings of my study might be
used in education and research settings. These included expanding instructor awareness of student
perspectives, suggesting pedagogical activities, identifying learning progressions and course
sequences, and developing improved assessments, all of which have been suggested by previous
phenomenographic studies (1,13,22,39,40. While these outcomes are all possibilities, and certainly the
study results will represent at least some progress in each categories, the overall value of the study
in each of these areas depends greatly on the as-yet-undetermined structure of the outcome space.
As I describe in the “Development of the Outcome Space” section, I am not designing my study
with a particular outcome space structure in mind.

If the outcome space that takes the shape of a set of hierarchical relationships, which can suggest a
learning progression 22, the study has great potential to inform instruction, curriculum design, and
assessment design. For example, a sequence of courses can be designed to help students move
towards more comprehensive levels. The differences between levels can also be used in
assessments to differentiate students at differing levels of understanding or achievement.

Regardless of outcome space, however, I believe the study’s results can be used equally by
students, instructors, and researchers. All outcome spaces describe the different ways participants
experienced the phenomena and the structural relationships between those ways of experiencing.
As other researchers 39,40 have suggested, these findings alone can be presented to students to help
them reflect on their own perspectives and become better aware of others’ perspectives. Thus, this
study can be used to help students develop more comprehensive understandings of the
phenomenon and to broaden their perspectives and understanding of others in general. Less
discussed in the literature is the possibility of also helping instructors better understand their
students. For example, a senior design instructor who emphasizes innovative projects will better
understand the variety of perspectives his or her students may be coming for and tailor activities,
discussions, and feedback towards those potential individual student differences. Researchers can
also use this expanded understanding to develop more appropriate innovation studies and situate
their results in more comprehensive contexts.

John: I anticipate many possible impacts of my study. I have read several statements from
different communities about the key role the development of systems thinking has in society and I
hope they find it useful. Additionally, new knowledge that contributes to the understanding of
systems thinking development will create better conditions to assess it and promote it. This is
because, on one hand, knowledge about how people learn or develop a skill is key when designing
effective assessment instruments 41 that would positively increase the chances of developing
curricula in systems thinking that align content, assessment and pedagogy. I also expect that
companies have another way to determine minimum levels of competency required for different
positions in which systems thinking is desired. Both industry and the academy could use the
results from this research to create instructional interventions (or learning activities/learning
experiences) that promote the development of systems thinking and industry and academia would
be able to evaluate the effectiveness of these interventions. Finally, I also hope the results from
this research will also inform practitioners from other disciplines like management who have also
used the systems thinking paradigm to enhance their practice. Finally, this study will also
contribute to create more evidences to support the new learning proposed by Marton: Variation
theory of Learning 4 in the context of learning skills.

Ming-Chien: I would like to create an outcome space of how engineering undergraduate student
experience interdisciplinary learning and how they construct their interdisciplinary learning
experience. We called a lot of learning situations interdisciplinary (or multidisciplinary, cross-
Page 26.1676.12

disciplinary, etc. as some would argue). We put students from different disciplines together and
expect them to benefit from it. My intent is to find the variance in students' conception of their
experiences, and the outcome space would be a great visualization to show that there are different
 conceptions across the board given the same learning situation that we dubbed interdisciplinary
learning. It would provide us with awareness of the difference of experiences. It would also
provide a framework to examine the kinds of experiences that would contribute to students'
interdisciplinary learning.

Mariana: In Learning and Awareness, Marton and Booth 1 explain phenomenography as a

strategy for understanding a phenomenon through individuals experiences that build a whole
description of the phenomenon (Kindle Loc. 2952). They also explain that the outcome space
should be parsimonious, looking for the minimum of categories that explain the whole
phenomenon (Kindle Loc. 3008). But, now I ask myself, how to assure saturation, or how to know
those are all? What happen if one category of the outcome space (for let’s say phenomenon A) is
set as the new phenomenon (let’s say phenomenon B) to study? will it be subcategories explaining
that particular way of seen the phenomenon A, now called phenomenon B, all the ways
experiencing phenomenon B, therefore was the first outcome space all the ways experiencing
phenomenon A? Or is this a better level of awareness for the researcher? This seems to be aligned
with what the authors said about the inability for describing an experience entirely (Kindle Loc.
2952). This process of understanding the phenomenon through people’s eyes allows to build an
understanding of the concept linked to the phenomenon, which form me, is the process of learning
how adults approach technological challenges. Because the method is rigorous, this learning is
made through research, and produces an outcome space. Marton 9 expect the outcome to be in
some sense replicable, which I translate to the search for a generalized (as possible)
characterization of the phenomenon. This also implies that the different ways of experiencing the
phenomenon is the unit of analysis, rather than an individual or a group of people. When the goal
of the research is this characterization itself the process is called by researchers (e. g. Marton,
Bowden, Walsh) Pure Phenomenography, but when the goal of research is to understand and
improve the learning process, the characterization of the outcome space should be connected to the
individual in order to address her learning process, which is unique and should be differentiated
from other processes. This later approach is commonly called developmental phenomenography
26
. For my research, I use a pure approach because my goal is to complement the STEM views
about what is a technological challenge and how it should be approach, which is a predecessor
activity before linking it with individual’s learning process about technology. The aim of creating
this outcome space is to change the framework in which technology literacy is built to propose
more inclusive approaches to technology learning, which may lead to higher levels of non-STEM
adults’ positive involvement with technology.

3. Development of outcome space

The outcome space is the deliverable expected after conducting a study using phenomenography
as research approach. It is expected to identify different categories of description of the
phenomenon, and how they relate to each other. We all agree that this outcome space will be
constructed following a rigorous process that involves several iterations, and reinterpretations of
the categories of description based on the evidence captured from the transcripts. We also agree
that it is difficult to be unbiased when analyzing the data. However, there is variation in the way
we all have expressed how to get there.

Ming-Chien, the only one of us who has already produced an outcome space, used Åkerlind’s
approach to data analysis, using the whole transcript as unit of analysis and creating piles for
each category of description she found. There was an expectation at the beginning of these
categories to be hierarchically related, but she found out that the relationships appeared only
after all the categories stabilized, which will be closer to Bowden’s claim of defining the
Page 26.1676.13

relationships after finding the categories. The rest of us based our expectation about the outcome
space on what other researchers have done in the past, and we all expect to create an outcome
space related to our research question. Mariana’s study is looking to describe how people
experience technology. Accordingly, she is assuming her process as a discovery, and she expects
her categories to be related, but not necessarily as a hierarchy. Emily and Nick defined their
expected processes as following Åkerlind’s approach. Accordingly, they would allow
relationships to form before categories of description are finalized. Since they both want to
inform education, they might expect their categories to be hierarchically related. However, they
both expressed differences in the way a transcript informs the categories of description found.
Emily’s transcripts will be informing different categories, and Nick’s transcripts will be
primarily informing one category with the understanding that they may inform aspects of
additional categories as well. John expressed his intention of using the framework of variation
theory of learning to structure his outcome space. Consequently, he will analyze the transcripts to
find people’s views of the problem and then form what is called “a pool of meanings,” which
will be sorted in categories of description. He expected his outcome space to be hierarchically
related, and that it expresses the way in which a learner identify variation.

Emily: Considering the construction of the outcome space has brought about awareness of a detail
in which I feel I follow the argument of Åkerlind more closely than Bowden. Despite the
argument that the relationship between the categories should only be developed after the full
development of categories, Åkerlind describes her method of developing the relationship between
the categories of description as she goes 42. She describes placing emphasis on meaning over
relationships early in the transcript in order to resist allowing the development of the structural
relationships to inform the category descriptions too much 42. I am focused on developing an
outcome space that is of practical use to educators, so I believe that it is important to develop a
meaningful relationship between the categories in addition to their descriptions. However, I
realize that I will have to remain aware of the possibility of letting those relationships unduly
influence the way in which I develop the descriptions. I also went through a couple of rounds of
“practice” analysis with the data from my pilot interviews, and I felt that getting a sense of the
relationship between the changing categories helped me to clarify the updating lenses with which I
read and re-read the transcripts, which lead me to believe that this strategy will be helpful in my
upcoming dissertation analysis as well.
John: When I started my research, I was not clear about what to expect of the outcome space, and
it was not in my preliminary defense either, I guess that it was because of my mixed views of the
Australian approach and Marton’s approach. Now I think I can see better the difference between
both research approaches to phenomenography, and they are related with the kind of evidence you
need to collect when trying to find how people can experience an object of learning, and the
outcome space you expect from studying them.

How can I know how someone experiences an object of learning? Phenomenographers commonly
use interviews. The main distinction between the two approaches is explained by Marton 4, and
my explanation is based on my current understanding of what he means after reflecting on his
work several times. On one hand, Bowden’s approach asks participants about how they experience
the object of learning and the meaning it has to them. Participants talk about experiences in their
past that have built their understanding of the object of learning. Obviously, these experiences are
most likely different because they all have experience different instances of the same object of
learning, and it is the job of the researcher to try to find similarities and differences among these
ways to describe their experiences with the object of learning. For example, Zoltowski asked them
to talk about one or more experiences designing for others 13, Emily is asking them to describe an
experience when they solved a problem that had multiple possible solutions, and Mariana asks
about interaction with technology.
Page 26.1676.14

On the other hand, Marton’s current view is that to collect evidence of how an object of learning
can be experienced, it is necessary that all the participants experience the same instance of the
object of learning. Based on studies using this approach, what it can be seen is that participants are
asked to engage in a task that requires them to use what they know to, for example, explain
decisions they make, such as the study with primary school students being asked about increasing,
decreasing or keeping the price of a hot dog in their school (see Marton and Pang 43) to study their
understanding of the relationship between price, supply and demand. Likewise, Fredlund, Airey,
& Linder 44, when studying the role of physics representations, asked their students to “provide an
appropriate and adequate explanation for the refraction of light to a hypothetical peer student,” and
they were asked to do it in the same physics laboratory in which they could count with water, a
glass tank, laser pointers, etc, and a blackboard and a chalk. As we see in these two examples,
researchers always created an instance of the object of learning and all the participants are asked
about it. In my study, after trying with Bowden’s approach, for me it was more tangible to study
systems thinking using Marton’s approach and, since systems thinking is about dealing with
problems in complex systems, I am asking my participants to address two different problems in
complex systems, an ill-defined one, and a semi-structured one.

Mariana: In my study I use pure phenomenography to identify a more comprehensive approach to

what technology literacy means; I am not addressing a learning process but a characterization of a
phenomenon itself. Therefore, I aim to build an outcome space without judging which ways of
experiencing technological challenges are better or worse than others. In my design this is
important because the goal is to broaden the understanding of what technology literacy means, and
how make technology more inclusive to non-engineers. This means that I expect an outcome space
with relationships but not necessarily hierarchies. Because the characterization is constructed with
little judgment as possible (acknowledging that a bias will be always present due to researcher’s
background and lens), I see the analysis as a discovering process, rather than a construction from a
predefined structure.

Nick: As I mentioned in the “Application of the Outcome Space” section, I have no definite
structure of the outcome space in mind for this study. A hierarchical progression might lead to
more practical benefits of the results, but I will not compromise the study in an attempt to fit the
data into a predetermined structure. Like Åkerlind 42, and to a lesser extent Bowden 12, my goal is
to let the structure emerge organically as part of the study to better uncover the ways of
experiencing and relationships between those ways of experiencing as they exist among the
participants at the time of the study. Ensuring this fidelity to the participants and the process is one
of the key ways to ensure validity in phenomenography 45,46.

Similar to Åkerlind 42, this study will allow structural relationships to form before categories of
description are finalized. The first few iterations will focus on developing the categories of
description until they are relatively stable, and then I will alternate focus on categories of
description and their structural relationships with each subsequent iteration. Åkerlind argues for
the process of co-construction in order to highlight critical aspects of variation during analysis,
and thus move beyond simple descriptions in order to present results that are more detailed,
meaningful, and applicable in educational settings. I would also argue that elements of structural
relationships naturally become evident to the researcher when transcripts are placed into
categories. When one selects a category for a transcript, he or she is not only saying it is similar to
transcripts in that category, but different in some critical way from transcripts in other categories.
Thus, unlike Bowden 12, I recognize the improbability of remaining unbiased regarding the
structure throughout the development of categories of description.

Ming-Chien: The tenant of phenomenography is a hierarchically structured outcome space. I

thought it was a helpful guideline during data analysis: the outcome space might take a
hierarchical form. However, that does not weigh in at all until I have gone through a couple of
iterations of sorting the transcripts into pile, writing description for each pile, reread the transcripts
in each pile, and readjust the piles and descriptions. It was until the very end when the piles were
Page 26.1676.15

not really changing that I started to describe relationships between the piles, sort the piles, look for
common themes across the piles, resort according to the themes, adjust the description according
to the themes, and repeat. In the end, I found many themes that were changing, but only two
 themes that vary through all the categories. I realized that was what "critical variations" means.
This would be the "discovery" part of the data analysis, but I cannot deny there would be aspects
that were constructed- like the aspects I would pay attention to or neglect during reading the
transcripts.

I do not believe that we can be truly objective as we analyze the data. As researchers, we play the
role of interpreting the participant accounts. We want to stay true to the data, but we cannot deny
that we play a role in interpreting the participants' account as well as constructing the outcome
space. For example, after a couple of iterations, I discussed my results with a seasoned
phenomenographer, and I was told that I would need to interpret the experiences, conceptions, or
ways of experiencing, instead of just summarizing the participants' story, which I did not realize
was what I was doing. It was a very difficult step in the analysis, and it also made me realize how
much I would need to insert myself in the process.

During the iterative process of discovering and constructing the outcome space, I found the
following guidelines useful regarding keeping my interpretive hat on while being attentive to the
data.

1. What each participant conceived as interdisciplinary learning

a. Meaning of a particular statement in relation to the context of the surrounding statement
b. Meaning of a particular statement in relation to the transcript as a whole
2. Compare across transcripts
a. Compare individuals within each group
b. Compare them between groups
3. How each participant conceived interdisciplinary learning?
a. How individuals delimited and organized what they conceived as interdisciplinary
learning
b. What each participant conceived of as interdisciplinary learning in relation to how they
learn?
4. Cross-check interpretation of each conception by reading through the transcript expressing a
particular conception while testing whether an alternative interpretation held until the
structure stabilizes
5. Focus on the meaning or structural components of the categories of description
6. Focus on the "how" and "what" aspects of the phenomenon
7. Focus on the similarities and differences between categories and between transcripts
associated with particular categories
8. Focus on borderline transcripts
9. Focus on transcripts in which there are aspects that do not fit the proposed categories of
descriptions
10. Look for the implications for all the categories of a change in any one of the categories.

Items 1-5 were adapted from Sandberg's procedure 47. Items 6-11 were from Åkerlind's approach
42
, which was based on the work of Drs. John Bowden and Eleanor Walsh

4. Treatment of data

In this section, each of us reflected on how we treated or plan to treat the data collected.
Specifically, we addressed the portion of data used for analysis and how we arrived at that
decision. We used two different approaches when analyzing the data, with the critical difference
being the degree that the whole transcript was considered when grouping categories.
Page 26.1676.16

Using the first approach, John and Mariana focused on extracting quotes that refer to particular
ways of perceiving a phenomenon to "pools of meanings". The context within the transcript was
considered when extracting quotes but not afterwards when grouping the quotes into the
categories. The rationale for this approach, as John points out, was that participants might shift
their focus during interview. Using the second approach, Emily, Nick and Ming-Chien focused
on each transcript as a whole throughout the analysis. The tenet of using this approach was to
understand a statement in the larger context of the transcript that the statement belongs to. Nick
offered three reasons this approach will afford him. Although Emily, Nick, and Ming-Chien
focused on using whole transcripts throughout the analysis, they acknowledged that a single
transcript could inform multiple categories since an individual can be aware of different aspects
of a phenomenon; for example, Ming-Chien used evidence of different conceptions within a
transcript to develop or confirm hierarchical structure among the categories.

Note that there were variations within the second approach regarding the relationship between
individual participants and the categories of description. In Nick's study, each individual was
associated with only one category of description, whereas Emily and Ming-Chien acknowledged
that a single transcript could inform different categories since a single student can be aware of
different aspects of a phenomenon. The difference between how Emily and Ming-Chien planned
to present the results was that in Emily's results, each category would have contributions from
different transcripts and the transcripts contributing to a category would be noted. Ming-Chien
would only use elements of different conceptions within a transcript to develop or confirm
hierarchical structure between the categories but would not label them along with the different
categories.

John: After doing and transcribing the interviews, Marton and other researchers have stated that
you create a “pool of ways of seeing the object of learning” 1,4,10,48. As it was said above, a way in
Marton’s words, is a set of critical aspects and critical features the participant focus on
simultaneously when addressing the task. This collection is stripped from the participant because a
participant, Marton says, can change the aspects he/she focused on during the interview (see more
at 1 and specially in the work of Marton and Pong 11.

Mariana: As some authors had stated, data collection and analysis are an intertwined process,
“The particular aspect is that of the interviewee reflecting over his experience in a state of “meta-
awareness,” being aware of his awareness of something” 1 (Kindle Loc. 3112). The interviewee
may be discovering her level of awareness about certain topic, while the researcher is constructing
the outcome space that describes the experience as a whole. While the participant is making sense
of the experience, the researcher is building the experience itself; therefore the unit of analysis is
each qualitative way in which participants conceptualize, reflect, and perceive the phenomenon,
which help to build the entire outcome space about how adults approach technological challenges.

As Marton 9 stated in his early work about phenomenography, those elicited conceptions and
perceptions are extracted from individual interviews, in relation with the context. However, once
the quotes that respond to the research questions, in my case how adults approach technological
challenges, are extracted from the data, a second step of the analysis is to identify those meanings
similar across the quotes. This analysis, that goes beyond individual experiences, aims to discover
those categories in which similar ways of experiencing the phenomenon are grouped. Due to the
nature of my design, where variation is based on STEM background and lifelong learning, I
should be careful when extracting utterances within individual’s context while keeping the pool of
meanings consolidated for the analysis of similarities and differences across utterances.

Emily: Again, for this consideration, I am largely aligned with the phenomenographic practices of
Bowden. This was influenced, in part, by the initial phenomenographic studies that I read, which
Page 26.1676.17

came out of Engineering Education and cited Bowden (and Åkerlind) for the analysis portion of
their methods sections—in particular, the consideration of the transcript as a whole. While
Bowden does talk about using the entire transcript for contextualization, a single transcript can
still inform multiple categories of description 2. Bowden clarifies, “using the whole transcript
means always having the whole transcript at your disposal all the time—never separating any
particular utterance from its context” 2 (p. 25). Even earlier in his work, Bowden explained that
while the ‘pool of meanings’ approach initiated by Marton can lead to appropriate categories of
description, he believes the de-contextualization makes the analysis more difficult 38. Åkerlind is
the author that does talk about separating entire transcripts to build categories 42. I have found
that throughout the interview process, the point is to continue to prove for understanding, so
something that a participant shares in the beginning is difficult to interpret without the follow-up
questions. Therefore, meaning is often only understood in a way that is most true to the
experience of the participant by considering the entire interview and the meaning that unfolds over
time. At the same time, based on the interviews that I have done for this study, and the fact that
the participants selected their own concrete experiences to reflect on the phenomenon of interest, I
believe that there is variation within each transcript that should lead to the categories I will
discover in the data. In other words, because students are aware of different aspects of their
experience for each example they describe, I want to allow that variation to contribute to the
variation in the outcome space. At the same time, I believe that the context must be considered
too, so I won’t do the ‘pools of meaning’ approach where utterances are removed from the
transcript as in the practice of Marton.

Nick: In my study, the unit of analysis is selected as the entire interview transcript 42, as compared
to decontextualized quotes/excerpts 1,22 or larger excerpts considered in the context of the whole
transcript 2. The rationale for this decision is that transcripts represent a set of interrelated
meanings 10. Thus, even if participants may make certain statements that can be represented by
different categories of description, these statements are understood within the larger context of the
individual at that point in time. Further, the slight variation between individual transcripts can
highlight critical border cases that aid in differentiating between categories of description and
determining structure. Finally, more nuanced views of the categories of description can be
developed when whole transcripts are considered over even contextualized quotes. For example,
Åkerlind 49 identifies multiple themes of expanding awareness within each category of description
that otherwise might not have been uncovered with a less holistic approach.

One concern of this approach is that it emphasizes the individual when phenomenography’s nature
is to focus on the collection of individuals. This distinction is most clear in the difference between
a “way of experiencing” and a “category of description.” A way of experiencing represents the
unique relationship between an individual participant and the phenomenon. A category of
description refers to a composite of related ways of experiencing, and is what phenomenographic
findings actually represent. As Cherry 50 (p. 58) notes, categories of description “while not the true
story of any one of us – at some level help to define the story of all of us.”

It would seem to go against phenomenography’s nature to focus on the individual by utilizing

entire transcripts. Yet, I remain firm in my insistence that using whole transcripts is the correct
decision. I have highlighted some of the benefits in the first paragraph of this section, namely that
whole transcripts provide better fodder for case-wise comparison than either contextualized or
decontextualized excerpts, and thus more nuanced understanding of the categories of description.
In essence, this approach acknowledges that each individual is unique but certain individuals will
share certain core aspects in common with relationship to the phenomenon. Further, with a
phenomenon as abstract and potentially varied as innovation, it is important to consider every
statement in a transcript in relation to every other statement in that transcript. This is necessary to
(a) ensure the participant is in fact discussing innovation (see my “Nature of the Phenomenon”
section), (b) build understanding of the participant’s perspective beyond what they discuss in
individual statements, and (c) identify connections between aspects of the phenomenon that might
otherwise be seen as unrelated.
Page 26.1676.18

Ming-Chien: I adopted the whole transcript approach during data analysis as described by
Åkerlind 42. Quotes and utterances from an individual will be considered with respect to rest of the
transcript. This approach would reduce the risk of the researcher trying to make sense of a quote
 regardless of the context from which they emerge, and it allow the transcripts to be understood as
inter-related meanings 42. Although phenomenography focus on the experience and not the
individual, I do not believe that I can understand the experience without knowing more about the
individuals' thoughts and other experiences. For that reason, I chose to use the "Australian"
approach. I am able to interpret a learning experience according to other experiences in the same
transcript as well as the reflection portion. First of all, this approach helped me with interpreting
some transcripts that I was initially unsure of. For example, there was a participant that talked very
passionately about one of her experiences, and I thought that transcript stood alone by itself as a
category, but after referring to another experience she talked about, I realized/interpreted that she
was in a category that already existed. Also, after initial construction of the outcome space, I saw
characteristics of prior categories in a transcript, which was verification that the outcome space
was indeed a hierarchical structure.

5. Stance toward participant

The final key consideration of phenomenographic research that was evident in our initial
reflections was the stance we take as researchers toward participants. This refers not only to our
interactions with participants during interviews but who we choose to interview and why, how
we consider them and handle their responses during data analysis, how we represent their
perspectives in papers and presentations, and what we consider the nature of the participant in
phenomenography. Unlike the previous aspects, the themes in this section present less like
tensions or tradeoffs to be negotiated, but critical considerations that are addressed in slightly
different ways.

Each of us stressed the importance of open communication with participants. This includes
transparency on the part of the researcher and freedom for the participants to voice their
perspectives. Mariana referred to the necessity to build rapport and the importance of “break[ing
the] defensive wall” a participant might have towards sharing. Ming-Chien noted that some
participants might be inclined to voice perspectives that were not their own but that they believed
the interviewer was expecting. Emily, Ming-Chien, and Nick responded to this concern by
suggesting upfront clarification of the research purpose, focus on their perspectives, and the lack
of right or wrong answers. Nick further stressed the importance of relaxed and conversational
interview atmosphere. John also noted the importance of building a dialogue with the participant
and avoiding impressions of being evaluated on the part of the participants.

Bracketing our own perceptions was another important aspect of our stance towards participants.
This not only facilitated the open communication described above, but allowed more thorough
and accurate analysis and appropriate research framing. Toward this end, John described
following six principles proposed by Bohm, including being open to new ideas, listening without
judgment, and suspending assumptions. Nick further discussed the importance of empathic
understanding through bracketing one’s view of the phenomenon, careful listening, and
reflecting on one’s own understanding of what the participant is trying to say. Ming-Chien
specifically focused on the challenge of asking thorough follow-up questions without interjecting
one’s own conception of the phenomenon or topic at hand.

A final theme in these reflections was accurately and appropriately representing the participants
Page 26.1676.19

to external audiences. Both bracketing and open communication support accurate representation
of the participants, but this extends beyond stances taken during data collection and analysis.
Emily describes an ethical responsibility to treat participants’ responses with respect and without
judgment, especially in light of experiences of other researchers making fun of their participants’
responses during conference presentations. John notes that inaccurately representing (including
inappropriately assigning a participant to a particular category or any category at all) the
participants may cause feelings of being evaluated and potentially distress at lack of satisfactory
responses. Thus accurate and appropriate representation is a critical aspect of ethical treatment of
participants.

Mariana: “The interviewer has power of an external kind, being deliberate and calculated,
whereas the interviewee has a power that can act from within, being spontaneous and reactive. If
the interviewer-interviewee relationship were to break down, then the loss would be to the
research effort; avoidance lies in the interviewer’s sensitivity to the potential of the relationship
and the interviewer’s ability to prepare and maintain it” 1 (Kindle Loc. 3155).

Due to the nature of “meta--awareness” of interviewee, she needs to break defensive wall that
prevents the interviewer, and even herself, to become aware of her level of awareness or lack of
awareness. This may be a tough process for interviewee and uncomfortable situation for both. The
importance of empathy during data collection resides in the ability to rapport with the interviewee
in a way that both people are comfortable being honest, even when reality won’t seems real to one
or both of them. This process is connected with the tension between discovering and constructing
the ways a phenomenon was experienced.

Ming-Chien: This is an interesting topic of discussion for phenomenography, in my opinion. The

interesting thing about using a phenomenography is that we are focusing on characterizing the
experiences and not the individuals. I found it to be very difficult during my data analysis to think
of a transcript as a collection of experiences and not the individual who provided the experience. I
was glad that I chose to use the whole transcript approach so I could make meaning of a part of a
transcript according to the other part. For example, the part where they defined why each
experience they decided to talk about was interdisciplinary might not provide a lot of meaning
("there were people with different background and expertise" for example). However, if I look at
the part where they talked about why that was important to them, I usually got something beyond
the simple definition (such as, it would push them out of their comfort zone to see the different
array of ideas). The whole transcript approach was important aspect of me being empathetic to my
participants' experience.

After experiencing this difficulty with data analysis and interpretation, I looked back at the data
collection stage and re-realize how important it was to be good at conducting interviewing. Being
familiar and better at interviewing would be the way that I could do the participants and their
experiences justice. What I am referring to is to know when to ask follow up questions that would
get at the meaning associated with the experience, and at the same time not interject our
presupposition of what is significant aspects of the experience, which is really hard!

Another point is the power relation during data collection. It could be uncomfortable to share
anything with an almost stranger and thinking that you might be judged. On another level, the
participants could think that the researchers were looking for some type of answers, and they
would try to tell the researcher what they wanted to hear. I tried taking measures to make sure they
understood that I was in a position to learn from them and I valued their experiences and insights.
For example, I did not define what interdisciplinary meant for my participants, I did emphasized at
the beginning of the interviews that there were no right or wrong answer, and I was in a position to
learn about their experiences. I occasionally got questions like: "Are you looking for
interdisciplinary in engineering or something else"? I would answer: "I am interested in whichever
Page 26.1676.20

experience that were important to you and that you would like to talk about".
Emily: A big motivator for me doing this particular method of research is to capture the voice of
the students. I have had multiple experiences during research seminars where findings, often
quotes from undergraduate students, are presented and then ridiculed for being naïve. Personally,
this really hurts my feelings. I understand that as educators we want to expect the best from our
students, and I believe that means understanding where they are and working with them to make
progress. This means that I take a stance that throughout my recruitment, data collection, data
analysis and sharing of my study I have been and will continue to be consciously trying to give my
participants the benefit of the doubt. Phenomenography is specifically aimed at studying the
relationship between the participants and the phenomenon of interest 2, so it is a great place of
privilege that I get to, as a researcher, be the one who interprets the shared experiences of my
participants and present them as I see fit. I am aware of this privilege and constantly reminding
myself that my epistemological assumption is that the way that participants describe their
experiences is, in truth, their reality and reflects the conception that I am attempting to understand.

In terms of the way I actually interacted with my participants during the interviews, the thing that I
found most helpful was to be as clear and transparent as possible. From my pilots to my
dissertation interviews, I became much more direct, clearly explaining the purpose of the study as
well as what they should expect during the process—including being explicit about how it may be
uncomfortable, and that my goal was to understand what it was like for them during the
experiences we would talk about. I had many students who stayed after the interview was over to
chat with me about the study, engineering, or other things that came up during the interviews. I
had two students that even asked if they could read the study! I believe that was evidence that
they felt they understood what was going on and were happy to be represented in the study. They
all have my card, so hopefully some will show up at my defense!

John: I understand that when you use Marton’s current approach to phenomenography, you don’t
look for participant’s right or wrong answers, but for the aspects of the object of learning they are
able to discern and focus on simultaneously while addressing the task. It makes me think that it is
necessary that I provide a space in which they don’t feel evaluated and the task could be better
used as an excuse to generate a dialogue with the participant about their perception of the object of
learning. I will follow six out of nine of the requirements proposed by Bohm 33 for an effective
dialogue: To act as colleagues (avoid considering hierarchical relationships), to create an empty
space in our mind to allow new ideas to come, to listen without judgment, to suspend assumptions
in order to listen to the assumptions of others, to focus on learning from the other, to inquire and
reflect by asking open-ended questions and by doing deep listening, concentration and reflection
upon what has been heard, and to respect and value differences. To postpone agenda and goals and
to observe self, will not happen because first the participants were selected to describe what they
see and that’s a clear goal, and because I will not show them my view on how I think the task can
be addressed. Still I hope I can engage them in a conversation that describes the nature of what
they see, and that they feel comfortable during the process.

Finally, it is relevant to say that not too many days ago, I have heard about cases in which
participants who have known the results of a phenomenographic study felt uncomfortable when
the researcher put the aliases of the participants in one of the categories they found. I think that
one of the benefits of using Marton’s approach is that your unit of analysis (the way of seeing the
world) is stripped from the participant who said it, and my hope is that if any of them reads the
results of my study don’t feel they should have tried to say something more or different about
what they see or do.

Nick: What initially drew me to qualitative research was the stories. An excerpt, assertion, or
theme at a time, I felt the voice of real people more so than I ever did in quantitative research.
Instead of seeing volume and dichotomy, I understood a little better these people’s lives and
situations and was able to truly connect a research finding to something tangible in the world.
Page 26.1676.21

Whenever I’m learning about a new qualitative research method, I’m always interested in what
aspect of the human experience will be uncovered and how the voices of the participants come
through. While phenomenography is about highlighting the variation in people’s experience with a
particular aspect of the world, as I noted in the Treatment of Data section, individual voice tends
to become lost in these limited number of composite ways of experiencing we call categories of
description.

The loss of individual voice in the final outcomes might suggest an impersonal stance towards
participants in phenomenography, but it is just the opposite. Thorough understanding of each
individual participant, as illuminated during the interview and further developed during data
analysis, is an essential element in the phenomenographic process 2, and also one of the critical
indicators of the validity of a phenomenographic study 49. It is up to the individual researcher to
determine what should be done to achieve such understanding in their individual study.

My approach is to take an empathic stance towards participants. While some have identified a
multitude of phenomena under the heading of empathy 51, here I refer to empathy as a
contextualized and nuanced understanding of another’s perspectives and internal state. I recognize
that I will never be able to know exactly another’s internal understanding of the phenomenon of
innovation, or anything else for that matter, but steps can be taken to facilitate a high level of
empathic accuracy. Ashworth and Lucas 25, for example, described nine distinct guidelines for
conducting empathic phenomenographic research that extend from research design, to data
collection, to data analysis. These can be synthesized to a few key principles. First, the researcher
must keep an open mind throughout the research process regarding the nature of the phenomenon
being studied and toward the participants and their contexts. This includes bracketing one’s own
view of the phenomenon 1 and not placing assumptions on or prematurely finalizing elements of
the outcome space 2. Second, the interview should be set up to maximize a participant’s ability to
openly voice his or her perspectives. This involves setting a conversational and relaxed
atmosphere 42, clarifying the purpose of the interview 2, and building trust with the participant.
Third, the researcher’s focus, during data collection and analysis, should be empathic
understanding of the participant. This involves careful listening, constantly reflecting on
understanding of what the participant is trying to say, ensuring thorough coverage and
understanding of the participant’s current perspective before concluding the interview, and
attention to non-linguistic cues 25,42. Finally, the research process should involve regular internal
and external appraisal of the researcher’s performance in developing and maintaining empathic
understanding. At this point, it would not be appropriate to discuss the precise and comprehensive
process I am utilizing to develop empathic understanding, but I encourage interested parties to
contact me.

I will also note that another critical aspect of empathy is empathic concern. This has less to do
with understanding and more on action related to that understanding 51. Åkerlind 42 notes that
some questions during phenomenographic interviews can be particularly difficult to answer. Some
participants may be frustrated by their inability to respond in a way they feel voices their true
perspective. The interview at best should be a pleasant and revelatory experience and at worst do
no harm. Thus, during the interview, it is important to not to be aware of the participant’s
perspective but also their emotional state, and to know when to move on to another topic or even
comfort a participant. Toward this end, I state clearly at the beginning of the interview that:

I am going to be asking you some questions over the next hour or hour and a half with the goal of
understanding your unique experiences and perspectives related to innovation. There are no right
or wrong answers to these questions, no right or wrong perspectives. I simply want to understand
what you think.

Some of these questions may be difficult to answer immediately. If you wish to not answer a
particular question, that is okay. We can always come back to the question at a later time, or skip
it entirely.
Page 26.1676.22

Thus far, the three students I have conducted pilot interviews with have noted afterwards that these
statements were helpful to them during the interview.
Conclusions and Recommendations

In this study, we developed five key considerations for making methodological decisions when
conducting phenomenographic studies. These included: Nature of Phenomenon, Application of
Outcome Space, Development of Outcome Space, Treatment of Data, and Stance towards
Participants. We identified these five key considerations through an iterative process of group
brainstorming sessions, individual reflection write-ups, and analysis and synthesis of reflections.

At the conception of this project, we expected that this process would lead to agreement on a
framework for linking the nature of the research to a specific school of thought within the
various practices of phenomenographers. However, after engaging in reflections and multiple
discussions, we discovered that although we were able to identify key considerations common to
all of our experiences, each researcher’s interpretation of these considerations varied within the
context of our respective studies. As a result, we presented our five key considerations along
with the detailed reflections to make transparent the thought process of each researcher.

We ultimately offer a single recommendation: engineering education researchers can strengthen

their phenomenographic work by reflecting on these five key considerations throughout the
research process. The considerations we presented build upon other variations already captured
in literature10 as well as nuances of their own individual research context. We hope that by
illuminating the variations among our interpretations of these key considerations, researchers
who are exploring phenomenography as a research approach can use this work to guide their
processes.

Acknowledgements

We would like to acknowledge the many previous researchers who have used phenomenographic
methods who have led the way for us to think deeply about this area of educational research. We
are also appreciative of our committee members who have encouraged us to spend time
embracing the difficulties and nuances of our research.

References

1 Marton, F. & Booth, S. A. Learning and awareness. (Psychology Press, 1997).

2 Bowden, J. A. & Green, P. in Qualitative Research Methods vi (RMIT University Press, 2005).
3 Marton, F. Phenomenography - Describing conceptions of the world around us. Instructional Science 10,
177-200 (1981).
4 Marton, F. Necessary conditions of learning. (Routledge, 2014).
5 Case, J. M. & Light, G. Emerging research methodologies in engineering education research. Journal of
Engineering Education 100, 186-210 (2011).
6 Åkerlind, G. Learning about phenomenography: Interviewing, data analysis and the qualitative research
Page 26.1676.23

paradigm. Doing developmental phenomenography, 63 (2005).

7 Åkerlind, G. S. Variation and commonality in phenomenographic research methods. Higher Education
Research & Development 31, 115-127 (2012).
8 Pang, M. F. Two Faces of Variation: On continuity in the phenomenographic movement. Scandinavian
Journal of Educational Research 47, 145-156, doi:10.1080/00313830308612 (2003).
9 Marton, F. Phenomenography: Exploring different conceptions of reality. Qualitative approaches to
evaluation in education: The silent revolution, 176-205 (1988).
10 Åkerlind, G. Variation and commonality in phenomenographic research methods. Higher Education
Research & Development 31, 115-127 (2012).
11 Marton, F. & Pong, W. Y. On the unit of description in phenomenography. Higher Education Research &
Development 24, 335-348, doi:10.1080/07294360500284706 (2005).
12 Bowden, J. A. Reflections on the phenomenographic team research process. (2005).
13 Zoltowski, C. B., Oakes, W. C. & Cardella, M. E. Students' Ways of Experiencing Human-Centered
Design. Journal of Engineering Education 101, 28-59 (2012).
14 Swarat, S., Light, G., Park, E. J. & Drane, D. A typology of undergraduate students' conceptions of size and
scale: Identifying and characterizing conceptual variation. Journal of Research in Science Teaching 48,
512-533 (2011).
15 Bucks, G. A phenomenographic study of the ways of understanding conditional and repetition structures in
computer programming languages, (2010).
16 Pan, R. Engineering students' experiences and perceptions of workplace problem solving, (2014).
17 Salzman, N. A phenomenographic study of students' experiences with transition from pre-college
engineering programs to first-year engineering, (2014).
18 Magana, A. J., Brophy, S. P. & Bodner, G. M. Instructors' intended learning outcomes for using
computational simulations as learning tools. Journal of Engineering Education 101, 220-243 (2012).
19 Adams, R. S., Forin, T., Srinivasan, S. & Mann, L. Cross-disciplinary practice in engineering contexts: A
developmental phenomenographical perspective. Proceedings of the 9th … 1 (2010).
20 Daly, S. R. Design across disciplines. (ProQuest, 2008).
21 Mann, L., Dall'Alba, G. & Radcliffe, D. 28-28 (American Society for Engineering Education).
22 Marton, F. Phenomenography—a research approach to investigating different understandings of reality.
Journal of thought, 28-49 (1986).
23 Richardson, J. T. E. The concepts and methods of phenomenographic research. Review of Educational
Research 69, 53-82 (1999).
24 Ashworth, P. & Lucas, U. What is the 'world' of phenomenography? Scandinavian Journal of Educational
Research 42, 415-431 (1998).
25 Ashworth, P. & Lucas, U. Achieving empathy and engagement: A practical approach to the design, conduct
and reporting of phenomenographic research. Higher Education 25, 295-308 (2000).
26 Bowden, J. A. & Walsh, E. Phenomenography. Phenomenography, v (2000).
27 Sandberg, J. Human competence at work: an interpretative approach, Företagsekonomi, (1994).
28 Checkland, P. Systems Thinking, Systems Practice. (John Wiley & Sons, 1981).
29 Senge, P. M. The fifth discipline: The art of the learning organization. 1st Editio edn,
(Doubleday/Currency, 1990).
30 Cabrera Velosa, M. F. & Lleras Manrique, E. Implementación de estrategia empresarial mediante el
enfoque TESO : caso Viajes Especiales S.A, (2002).
31 Lleras, E. Towards a methodology for organisational intervention in Colombian enterprises. Systems
Practice 8, 169-182, doi:10.1007/BF02253411 (1995).
32 Lleras, E. Notification of Communities of Learning in Local Development. World Futures 60, 91-98,
doi:10.1080/725289196 (2004).
33 Bohm, D. & Nichol, L. On dialogue. (London ; New York : Routledge, 1996).
34 Valero-Silva, N. 7-7 (ISSS).
35 Winograd, T. & Flores, F. Understanding computers and cognition : a new foundation for design.
(Addison-Wesley, 1987).
36 Xanthos, N. Wittgenstein's Language Games. Louis Hébert (dir.), Signo [online], Rimouski (Quebec)
(2006).
37 Marton, F. Sameness and difference in transfer. The Journal of the Learning Sciences 15, 499-535 (2006).
38 Bowden, J. A. & Walsh, E. Phenomenographic Research: Variations in Method: the Warburton
Symposium. (Royal Melbourne Institute of Technology (RMIT), 1994).
Page 26.1676.24

39 Daly, S. R., Adams, R. S. & Bodner, G. M. What Does it Mean to Design? A Qualitative Investigation of
Design Professionals' Experiences. Journal of Engineering Education 101, 187-219 (2012).
40 Reid, A. & Solomonides, I. Design students' experience of engagement and creativity. Art, Design &
Communication in Higher Education 6, 27-39 (2007).
41 Pellegrino, J. W., Chudowsky, N. & Glaser, R. Knowing what students know: The science and design of
educational assessment. (National Academies Press, 2001).
42 Åkerlind, G. Phenomenographic methods: A case illustration. (2005).
43 Marton, F. & Pang, M. F. 533-559 (Routledge, Taylor & Francis, 2008).
44 Fredlund, T., Airey, J. & Linder, C. Exploring the role of physics representations: an illustrative example
from students sharing knowledge about refraction. European Journal of Physics 33, 657-666,
doi:doi:10.1088/0143-0807/33/3/657 (2012).
45 Cope, C. Ensuring validity and reliability in phenomenographic research using the analytical framework of
a structure of awareness. Qualitative Research Journal 4, 5-18 (2004).
46 Sin, S. Considerations of quality in phenomenographic research. International Journal of Qualitative
Methods 9, 305-319 (2010).
47 Sandberg, J. Understanding human competence at work: an interpretative approach. Academy of
management journal 43, 9-25 (2000).
48 Berglund, A. Learning computer systems in a distributed project course : The what, why, how and where,
Acta Universitatis Upsaliensis, (2005).
49 Åkerlind, G. Appendix B: Interim Stages of the Analysis of Ways of Experiencing Being a University
Researcher. Doing Developmental Phenomenography, 171 (2005).
50 Cherry, N. in Doing Developmental Phenomenography (eds John A. Bowden & Pam Green) 56-62
(RMIT Press, 2005).
51 Batson, D. in The Social Neuroscience of Empathy (eds Jean Decety & William Ickes) 3-15 (MIT Press,
2009).

Page 26.1676.25