Field Notes and Observation - Validity, Reliability and Triangulation
Field Notes and Observation - Validity, Reliability and Triangulation
Field Notes and Observation - Validity, Reliability and Triangulation
10-27-2018
Frank Hernandez
Southern Methodsist University, [email protected]
This How To Article is brought to you for free and open access by the The Qualitative Report at NSUWorks. It has
been accepted for inclusion in The Qualitative Report by an authorized administrator of NSUWorks. For more
information, please contact [email protected].
Enhancing the Value of Qualitative Field Notes Through Purposeful Reflection
Abstract
This commentary outlines the efforts taken to provide doctoral students with purposeful reflection
questions to help them enhance the value and utility of qualitative data. It is based upon experiences
teaching a doctoral level qualitative research methods course for students enrolled in an executive format
doctoral program. Reflexivity of the researcher, reflection, and research design decisions are discussed.
Suggestions for purposeful reflection questions are also discussed and listed in four categories. The
categories of purposeful reflection questions include (1) Research Setting Access, (2) Examining Norms
and Cultures, (3) Positionality of Research Subjects, and (4) Positionality as an Observer. These four
categories of questions provide a paradigm that could help many qualitative researchers take a more
systematic in-depth approach to the collection, transcription, and analysis of field notes and other forms
of qualitative data.
Keywords
Field Notes, Reflection, Data Validation, Positionality
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International
License.
This commentary outlines the efforts taken to provide doctoral students with
purposeful reflection questions to help them enhance the value and utility of
qualitative data. It is based upon experiences teaching a doctoral level
qualitative research methods course for students enrolled in an executive format
doctoral program. Reflexivity of the researcher, reflection, and research design
decisions are discussed. Suggestions for purposeful reflection questions are
also discussed and listed in four categories. The categories of purposeful
reflection questions include (1) Research Setting Access, (2) Examining Norms
and Cultures, (3) Positionality of Research Subjects, and (4) Positionality as an
Observer. These four categories of questions provide a paradigm that could
help many qualitative researchers take a more systematic in-depth approach to
the collection, transcription, and analysis of field notes and other forms of
qualitative data. Keywords: Field Notes, Reflection, Data Validation,
Positionality
Professors of qualitative research methods seek new methods to help novice researchers
embrace their philosophical lens, personal epistemology, and bias as they embark on their first
efforts to collect, interpret, and analyze qualitative data. Students as novice researchers, seem
to be in conflict with their ideologies regarding data validity, trustworthiness, or credibility.
We recently co-taught a doctoral level qualitative research methods course for students
enrolled in an executive format doctoral program. The course included both K-12 and higher
education practitioners with various career experiences including instructional and school
leaders, student affairs professionals, and postsecondary faculty. Throughout the course, we
sought methods to help students think more deeply about their qualitative field notes thus
developing their critical thinking skills. Our efforts were motivated by the desire to help
doctoral students more truly understand the research phenomenon they would investigate,
develop a robust data source for triangulation purposes, and perhaps, most importantly,
improve qualitative data validity and reliability. This commentary outlines the efforts we took
to provide our students with purposeful reflection questions to help them enhance the value
and utility of qualitative data.
contradictory. Creswell and Miller (2000) asserted that triangulation is a validity method where
the researcher seeks convergence among multiple and difference sources of data. While only
the researcher’s lens is utilized, the method provides for corroborating evidence. The
practicality of triangulation as an analysis technique was emphasized by Leech and
Onwuegbuzie (2007) who called for at least types of data analysis tools to triangulate data.
Given the documented importance of triangulation as a data validation strategy, we felt it was
important to offer suggestions to improve the quality of field notes as a data source that could
be triangulated by researchers.
Research Journals
We concur with Ortlipp’s (2008) statement that using reflective journals allows the
researcher to make their experiences, opinions, thoughts, and feelings visible. It was our
concern that our students, like others in classes we have each taught before, would simply gloss
over qualitative data, particularly their field notes. We felt very strongly that students might
decide that field notes are a secondary, less informative, or less valuable component of
qualitative data. Orange (2016) explored the use of reflective journals among doctoral students
who are learning qualitative research and cited that many do not use journals in their reflective
practices. Orange stated that structured guidelines and prompts may improve details contained
in reflective journals, thereby increasing the level of engagement in research.
Cruz (2015) examined how she dealt with subjectivity of the qualitative research
process through self-reflexivity. Citing both explicit (conscious subjectivity) and implicit (not
so conscious), Cruz explains how the use of field notes facilitated a self-reflective process of
the implicit dimension that enabled the development of “a new sensitivity to be alert of my
own assumptions” (2015, p. 1725). Cruz urged researchers to be “constant and systematic”
(2015, p. 1733) throughout the reflective process. Like Cruz, we wanted our students to
embrace both their explicit and implicit subjectivity through a process that allowed them to
apply logic and reason to become more comfortable with qualitative data. It was our desire that
students as novice researchers embrace their positionality in qualitative research. We posit that
qualitative researchers’ ability to understand their positionality is connected to their
understanding and embracing of their explicit and implicit bias. Qualitative researchers should
be aware of their positionality throughout the design, data collection, data analysis and
reporting of the study. Awareness of one’s positionality enables the researcher to exercise
proper reflective practices to ensure accuracy of research findings.
Mruck and Breuer (2003) explained that reflection serves to disclose presuppositions,
choices, experiences, and actions during the research process. They posit that these discussions
are of necessity “because without such reflection, the outcomes of the research process are
regarded as ‘characteristics of objects,’ as ‘existing realities,’ despite their constructed nature
that originates in the various choices and decisions researchers undertake during the process of
researching” (Mruck & Breuer, 2003, p. 192). As qualitative researchers, we have often seen a
disconnect between the methods utilized and the novice researcher. Rather than a full
contextualized understanding of the phenomenon, some qualitative researchers—particularly
novice researchers—fail to understand the naturalistic components of the research
environment. The context of the setting is often misunderstood, misrepresented, or otherwise
distorted.
In their 2002 critical examination of reliability and validity techniques for qualitative
research, Morse, Barrett, Mayan, Olson, and Spiers (2002) called for paying attention to rigor
David M. Deggs & Frank Hernandez 2555
throughout the research process. They proclaimed that “the researcher’s creativity, sensitivity,
flexibility, and skill in using the verification strategies determine the reliability and validity of
the evolving study” (p. 20). We posit that integration of purposeful reflection, grounded in
robust researcher reflexivity, can help improve study validity and reliability—thereby
strengthening the quality of research produced. As more advanced methods to ensure validity
and reliability of qualitative data are called for and implemented, we argue that purposeful, in-
depth reflection can assist qualitative researchers in meeting standards of qualitative data
validity and reliability.
Maxwell (2012) argued that “observation can enable you to draw inferences about
someone’s perspective that you couldn’t obtain by relying exclusively on interview data” (p.
94). We were very careful to advise our students to not think of observations and the field notes
yielded as merely a required secondary data source for triangulation or other data validity
needs. We wanted our students to understand the value of field notes and understand as
Kawulich (2005) explained that field notes serve as a record of activities or ceremonies
observed and informal discussions from the field. We wanted our students to understand the
value of ceremonies and traditions in many qualitative research settings. We did not want them
to misinterpret the phenomenon under investigation or not have the proper context to analyze
interview data because they failed to understand the dynamics, activities, or ceremonies in the
natural research setting. This was a concern for us because we had both worked with novice
qualitative researchers in the past who struggled to understand the context in which research
was being conducted and missed valuable insights and understandings from the data collected.
We also wanted our students to be comfortable with the various types of field notes.
Neuman (2011) identified types of field notes used in qualitative data collection. Types of data
included jotted notes (short memory triggers), direct observation (written immediately after
leaving the field), inference (reflecting social relationships, emotions and meanings), analysis,
(methodological strategies and theoretical notes), interview notes (information about interview
location and interviewee), and personal journal (personal feelings and emotional reactions).
Our efforts to utilize reflection with our students was intended to help students develop field
notes that could be used for inference purposes. This process would in turn support the creation
of analytic notes and yield insightful qualitative data to ensure a complete understanding of the
research setting and phenomenon. Neuman (2011) cautioned researchers to ensure that
inference be separated from the other process so that the researcher keeps observations,
inferences and beliefs separated. We agree that such a separation must exist for data validity
and reliability purposes; however, we feel that this separation often creates a mental barrier
among researchers, particularly novice researchers who are unable to reconcile the objective
and subjective to ascertain the underlying meaning or context and not misinterpret data.
The reflective questions related to positionality that we present provide a method for
the researcher to examine positionality of the research subjects and themselves as observers.
We view that reflection about positionality between research subjects and researcher provides
a means for researchers to embrace both their explicit and implicit subjectivity about the
occurrences within the research setting and from within themselves. While we do not claim
that it is possible for researchers to fully understand the positionality of the research subjects
in their setting, we posit that our questions would enable researchers to manage the bias that
accompanies explicit and implicit subjectivity. We posit that the researcher can best understand
2556 The Qualitative Report 2018
the positionality of research subjects through careful and critical reflection of their own
positionality including taking ownership of their interpretive lenses and bias. While many of
our students are familiar with the types of research settings in which they would conduct their
research, we cautioned them to not make any assumptions based upon their experiences from
similar settings. To do so would discount the phenomena experienced by the subjects in which
they conduct their research. A closer understanding of positionality is a mechanism by which
to prevent that possibility.
To guide students through more rigorous critical reflection of what they observed, we
provided reflection questions that were meant to allow students to recall both the tactical (hands
on) experience of observing and explore the abstract (reflective) insights about what is
occurring in the research environment. We felt that our students needed to be prompted to
examine the research setting more critically. The work of Alvesson and Sköldberg (2017)
supports our strong disposition that our students as novice qualitative researchers needed to
examine their research more critically. Alvesson and Sköldberg (2017) identified four elements
of reflective research that are listed and explained below.
The following are the purposeful reflection questions that we provided to our doctoral
students. These questions were informed by both our personal experience as researchers and
the challenges we saw with other novice graduate student researchers in the past.
• Did you notice the composition of the people in the setting (e.g., gender,
race, social class, etc.)? What did it tell you about the dynamics within the
setting?
• What roles did people play? What actions were aligned to those roles?
• Were there groups? Subgroups? In group? Out Group?
• Were you a part of any group as the observer?
Positionality as an Observer
• How did you feel before, during, and after the observation?
• Did people realize you were observing others? Were you observed?
• What was your vantage point as an observer? Did you yield any vantage
points to your research subjects or co-researchers?
• How did your vantage point impact your insight and understanding about
what you observed?
Discussion
Access to research settings has been an issue that research methodologists have written
about extensively (e.g., Rossman & Rallis, 2016; Seidman, 2013). Our reflection questions
about this very common practice among qualitative researchers is meant to uncover bias by
examining expectations, reconciling struggles with access and rapport, and measures taken to
properly exit the setting. We posit that the researcher should pay careful attention to dynamics
how they negotiate and maintain access to fully understand and represent the phenomenon
within.
We have come to value the importance of cultural norms within qualitative research
settings, particularly those to which we are not members. With that in mind, we sought to help
our students understand the value of ceremonies and traditions in the naturalistic environments
in which they would conduct their research. Perhaps most importantly, from an ethical
perspective, we did not want our students to misinterpret the phenomenon under investigation.
To properly understand the phenomenon, we wanted our students to understand the dynamics,
activities, or ceremonies in the natural research setting to the best of their ability. The majority
of our students were novice researchers and practitioners in education leadership or higher
2558 The Qualitative Report 2018
education as well. We cautioned them to not make assumptions about any of the occurrences
within the settings in which they would conduct their research. We cautioned them to identify
the evidence that explains the phenomena that they will study. Our reflective questions
regarding norms, power, culture, and dynamics are intended to empower the researcher to look
beyond the surface by immersing themselves into the inter-workings of the research setting
thereby allowing them to embrace the contextualized meanings within.
Conclusion
While this article chronicled our efforts to help students enrolled in a doctoral level
qualitative research methods course embrace the observation process through the collection of
robust field notes and use reflexive practices to improve meaning of this form of qualitative
data, we assert that these techniques could be used by experienced researchers. The guided
questions we offer are not universally applicable to all research settings; however, they do
present a model that could be used in various settings. We assert that the categories of our
questions including (1) Research Setting Access, (2) Examining Norms and Cultures, (3)
Positionality of Research Subjects, and (4) Positionality as an Observer, provide a paradigm
that could help many qualitative researchers take a more systematic in-depth approach to the
collection, transcription, and analysis of field notes and other forms of qualitative data. We
encourage others to take adequate steps to ensure that the naturalistic research setting is not
misunderstood, misinterpreted, or misconstrued when presenting research results to expand
upon the body of knowledge. In an era where reflexivity of the researcher coupled with the
enhanced understanding positionality, it is important that qualitative researchers further
embrace and apply methods and techniques that further ensure the credibility of the researcher
and the value of the data that is collected through observations in the field. We feel that our
questions are designed to engage researchers in effective reflective practice so that they
challenge predispositions about research topics, research subjects, initial interpretations of
data, and recognize bias and further understand how their positionality informs their research
practice.
References
Alvesson, M., & Sköldberg, K. (2017). Reflexive methodology: New vistas for qualitative
research (2nd ed.). Thousand Oaks, CA: Sage.
Attia, M., & Edge, J. (2017). Be (com) ing a reflexive researcher: a developmental approach to
research methodology. Open Review of Educational Research, 4(1), 33-45.
Berger, R. (2015). Now I see it, now I don’t: Researcher’s position and reflexivity in qualitative
research. Qualitative Research, 15(2), 219-234.
Bourke, B. (2014). Positionality: Reflecting on the research process. The Qualitative
Report, 19(33), 1-9. Retrieved from https://nsuworks.nova.edu/tqr/vol19/iss33/3
Creswell, J. W., & Miller, D. L. (2000). Determining validity in qualitative inquiry. Theory
into Practice, 39(3), 124-130.
Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing
among five approaches (4th ed.). Thousand Oaks, CA: Sage.
Cruz, L. (2015). Self-reflexivity as an ethical instrument to give full play to our explicit and
implicit subjectivity as qualitative researchers. The Qualitative Report, 20(10), 1723-
1735. Retrieved from https://nsuworks.nova.edu/tqr/vol20/iss10/13
Finlay, L., & Gough, B. (Eds.). (2003). Reflexivity: A practical guide for researchers in health
and social sciences. Malden, MA: Blackwell Science.
Gabriel, Y. (2015). Reflexivity and beyond–a plea for imagination in qualitative research
David M. Deggs & Frank Hernandez 2559
Author Note
David Deggs is Executive Director of College Access Programs in the Simmons School
of Education and Human Development at Southern Methodist University. He also serves as an
Adjunct Professor in the Department of Education Policy and Leadership. Correspondence
regarding this article can be addressed directly to: [email protected].
Frank Hernandez holds the Annette and Harold Simmons Centennial Chair in
Education Policy and Leadership and serves as the Associate Dean in the Simmons School of
Education & Human Development at Southern Methodist University. Correspondence
regarding this article can also be addressed directly to: [email protected].
2560 The Qualitative Report 2018
Copyright 2018: David Deggs, Frank Hernandez, and Nova Southeastern University.
Article Citation
Deggs, D. M., & Hernadez, F. (2018). Enhancing the value of qualitative field notes through
purposeful reflection. The Qualitative Report, 23(10), 2552-2560. Retrieved from
https://nsuworks.nova.edu/tqr/vol23/iss10/18