Field Notes and Observation - Validity, Reliability and Triangulation

Download as pdf or txt
Download as pdf or txt
You are on page 1of 11

The Qualitative Report

Volume 23 Number 10 How To Article 18

10-27-2018

Enhancing the Value of Qualitative Field Notes Through


Purposeful Reflection
David Deggs
Southern Methodist University, [email protected]

Frank Hernandez
Southern Methodsist University, [email protected]

Follow this and additional works at: https://nsuworks.nova.edu/tqr

Part of the Quantitative, Qualitative, Comparative, and Historical Methodologies Commons

Recommended APA Citation


Deggs, D., & Hernandez, F. (2018). Enhancing the Value of Qualitative Field Notes Through Purposeful
Reflection. The Qualitative Report, 23(10), 2552-2560. https://doi.org/10.46743/2160-3715/2018.3569

This How To Article is brought to you for free and open access by the The Qualitative Report at NSUWorks. It has
been accepted for inclusion in The Qualitative Report by an authorized administrator of NSUWorks. For more
information, please contact [email protected].
Enhancing the Value of Qualitative Field Notes Through Purposeful Reflection

Abstract
This commentary outlines the efforts taken to provide doctoral students with purposeful reflection
questions to help them enhance the value and utility of qualitative data. It is based upon experiences
teaching a doctoral level qualitative research methods course for students enrolled in an executive format
doctoral program. Reflexivity of the researcher, reflection, and research design decisions are discussed.
Suggestions for purposeful reflection questions are also discussed and listed in four categories. The
categories of purposeful reflection questions include (1) Research Setting Access, (2) Examining Norms
and Cultures, (3) Positionality of Research Subjects, and (4) Positionality as an Observer. These four
categories of questions provide a paradigm that could help many qualitative researchers take a more
systematic in-depth approach to the collection, transcription, and analysis of field notes and other forms
of qualitative data.

Keywords
Field Notes, Reflection, Data Validation, Positionality

Creative Commons License

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International
License.

This how to article is available in The Qualitative Report: https://nsuworks.nova.edu/tqr/vol23/iss10/18


The Qualitative Report 2018 Volume 23, Number 10, How To Article 6, 2552-2560

Enhancing the Value of Qualitative Field Notes Through


Purposeful Reflection
David M. Deggs and Frank Hernandez
Southern Methodsist University, Dallas, Texas, USA

This commentary outlines the efforts taken to provide doctoral students with
purposeful reflection questions to help them enhance the value and utility of
qualitative data. It is based upon experiences teaching a doctoral level
qualitative research methods course for students enrolled in an executive format
doctoral program. Reflexivity of the researcher, reflection, and research design
decisions are discussed. Suggestions for purposeful reflection questions are
also discussed and listed in four categories. The categories of purposeful
reflection questions include (1) Research Setting Access, (2) Examining Norms
and Cultures, (3) Positionality of Research Subjects, and (4) Positionality as an
Observer. These four categories of questions provide a paradigm that could
help many qualitative researchers take a more systematic in-depth approach to
the collection, transcription, and analysis of field notes and other forms of
qualitative data. Keywords: Field Notes, Reflection, Data Validation,
Positionality

Professors of qualitative research methods seek new methods to help novice researchers
embrace their philosophical lens, personal epistemology, and bias as they embark on their first
efforts to collect, interpret, and analyze qualitative data. Students as novice researchers, seem
to be in conflict with their ideologies regarding data validity, trustworthiness, or credibility.
We recently co-taught a doctoral level qualitative research methods course for students
enrolled in an executive format doctoral program. The course included both K-12 and higher
education practitioners with various career experiences including instructional and school
leaders, student affairs professionals, and postsecondary faculty. Throughout the course, we
sought methods to help students think more deeply about their qualitative field notes thus
developing their critical thinking skills. Our efforts were motivated by the desire to help
doctoral students more truly understand the research phenomenon they would investigate,
develop a robust data source for triangulation purposes, and perhaps, most importantly,
improve qualitative data validity and reliability. This commentary outlines the efforts we took
to provide our students with purposeful reflection questions to help them enhance the value
and utility of qualitative data.

Reflective Practices in Qualitative Research

Reflexivity and the Researcher

The value of using reflective practices in qualitative data collection is documented in


the literature base (Oliver, Serovich, & Mason, 2005; Ortlipp, 2008) and in reflexivity, enabling
researchers to share their background and provide insight into how a researcher both interprets
data from the study and explains what was gained from the study (Creswell & Poth, 2017). It
is important to define what is meant by reflective practices in empirical research. Reflective
research is related to how the researcher interprets empirical data; no data source is in itself
neutral. All data is being interpreted by the researcher via measurement, observation,
David M. Deggs & Frank Hernandez 2553

interviews, or analysis of secondary data. A second element of reflective research is more


retrospective. It is focused on the effect of the research on the researcher (Attia & Edge, 2017).
There are many approaches to reflectivity in empirical research, and each pathway or approach
carries strengths and weaknesses. These are often determined by opportunities and costs. The
researcher must determine what type of reflective practice they will engage in so that they can
make a deliberate choice about the type of research in which they will engage (Finlay & Gough,
2003).
Guillemin and Gillam (2004) asserted that reflexivity in research impacts the kind of
knowledge produced and how that knowledge is generated. In speaking of ethical decisions in
qualitative research, Guillemin and Gillam argued that a reflexive researcher is aware of factors
that influence the research and can “step back and take a critical look at his or her own role in
the research process” (2004, p. 275). As professors of qualitative methods to a group of novice
researchers, we share this same concern. Although we had spent time in class helping students
to identify, articulate, and own their interpretive lenses and implicit bias, we struggled with the
process of getting many students to fully become aware of their positionality. Many of our
students were scholar practitioners and thus struggled with the process of managing the impact
of their positionality on data collection, analysis, and interpretation. We knew that our students
needed a means by which to think more critically about the data they had collected, specifically
through observations and field notes. Bourke (2014) offered advice about how to manage
positionality as a qualitative researcher, specifically the management of insider and outsider
roles. Bourke suggested that the researcher’s positionality should be addressed with all
participants and that the researcher cannot operate off assumptions related to physical
attributes. Bourke also suggested that qualitative researchers be clear about motivations for
data collection with themselves and their participants.
There have been increased emphasis on the role of reflexivity in qualitative research in
the published literature. For example, Jootun, McGhee, and Marland (2009) asserted that
reflexivity should occur throughout the study and thereby assist with making the qualitative
data collection process open and transparent. Gabriel (2015) further commented about the
consciously reflective researcher who is aware of the impact of their work in the field and how
the research portrays underlying values. Ultimately, the consciously reflective researcher will
emerge as different subjects. Berger (2015) argued that “the researcher must remain constantly
alert to avoid projecting his or her own experience and using it as the lens to view and
understand participants’ experience” (p. 330). Berger also advocated the use of three practical
measures including the use of a research log, repeated review, and peer consultation. The use
of repeated review after a lapse in time allows the researcher to “view the same material through
a ‘new lens’ and identify where one’s own experience interfered with accurately understanding
interviewee’s report” (2015, p. 330).
It was our concern that students would merely gloss over field notes and would fail to
take adequate time to allow the notes to become critical to the reflective process that should be
utilized by qualitative researchers throughout the research process. Valandra (2012) argued that
reflexivity can be useful to qualitative researchers while conceptualizing a study, while
implementing a study, and while analyzing and writing a study. We concur and posit that with
field notes in particular, reflective practices can prove to be useful to help qualitative
researchers make meaning out of the data they have collected through empirical processes. We
see greater reflective processes as an effort to examine positionality.
We were concerned that our students needed tools to make their field notes richer and
thus a robust data source. It is our concern as qualitative researchers that field notes are often
seen as a lesser form of qualitative data and are merely used to claim that data triangulation has
occurred for validity purposes. The need for acceptable methods to triangulate data was
documented by Mathison (1988) who explained that data may converge, be inconsistent, or be
2554 The Qualitative Report 2018

contradictory. Creswell and Miller (2000) asserted that triangulation is a validity method where
the researcher seeks convergence among multiple and difference sources of data. While only
the researcher’s lens is utilized, the method provides for corroborating evidence. The
practicality of triangulation as an analysis technique was emphasized by Leech and
Onwuegbuzie (2007) who called for at least types of data analysis tools to triangulate data.
Given the documented importance of triangulation as a data validation strategy, we felt it was
important to offer suggestions to improve the quality of field notes as a data source that could
be triangulated by researchers.

Research Journals

We concur with Ortlipp’s (2008) statement that using reflective journals allows the
researcher to make their experiences, opinions, thoughts, and feelings visible. It was our
concern that our students, like others in classes we have each taught before, would simply gloss
over qualitative data, particularly their field notes. We felt very strongly that students might
decide that field notes are a secondary, less informative, or less valuable component of
qualitative data. Orange (2016) explored the use of reflective journals among doctoral students
who are learning qualitative research and cited that many do not use journals in their reflective
practices. Orange stated that structured guidelines and prompts may improve details contained
in reflective journals, thereby increasing the level of engagement in research.
Cruz (2015) examined how she dealt with subjectivity of the qualitative research
process through self-reflexivity. Citing both explicit (conscious subjectivity) and implicit (not
so conscious), Cruz explains how the use of field notes facilitated a self-reflective process of
the implicit dimension that enabled the development of “a new sensitivity to be alert of my
own assumptions” (2015, p. 1725). Cruz urged researchers to be “constant and systematic”
(2015, p. 1733) throughout the reflective process. Like Cruz, we wanted our students to
embrace both their explicit and implicit subjectivity through a process that allowed them to
apply logic and reason to become more comfortable with qualitative data. It was our desire that
students as novice researchers embrace their positionality in qualitative research. We posit that
qualitative researchers’ ability to understand their positionality is connected to their
understanding and embracing of their explicit and implicit bias. Qualitative researchers should
be aware of their positionality throughout the design, data collection, data analysis and
reporting of the study. Awareness of one’s positionality enables the researcher to exercise
proper reflective practices to ensure accuracy of research findings.

Reflection and Research Design Decisions

Mruck and Breuer (2003) explained that reflection serves to disclose presuppositions,
choices, experiences, and actions during the research process. They posit that these discussions
are of necessity “because without such reflection, the outcomes of the research process are
regarded as ‘characteristics of objects,’ as ‘existing realities,’ despite their constructed nature
that originates in the various choices and decisions researchers undertake during the process of
researching” (Mruck & Breuer, 2003, p. 192). As qualitative researchers, we have often seen a
disconnect between the methods utilized and the novice researcher. Rather than a full
contextualized understanding of the phenomenon, some qualitative researchers—particularly
novice researchers—fail to understand the naturalistic components of the research
environment. The context of the setting is often misunderstood, misrepresented, or otherwise
distorted.
In their 2002 critical examination of reliability and validity techniques for qualitative
research, Morse, Barrett, Mayan, Olson, and Spiers (2002) called for paying attention to rigor
David M. Deggs & Frank Hernandez 2555

throughout the research process. They proclaimed that “the researcher’s creativity, sensitivity,
flexibility, and skill in using the verification strategies determine the reliability and validity of
the evolving study” (p. 20). We posit that integration of purposeful reflection, grounded in
robust researcher reflexivity, can help improve study validity and reliability—thereby
strengthening the quality of research produced. As more advanced methods to ensure validity
and reliability of qualitative data are called for and implemented, we argue that purposeful, in-
depth reflection can assist qualitative researchers in meeting standards of qualitative data
validity and reliability.

Observation and Field Notes

Maxwell (2012) argued that “observation can enable you to draw inferences about
someone’s perspective that you couldn’t obtain by relying exclusively on interview data” (p.
94). We were very careful to advise our students to not think of observations and the field notes
yielded as merely a required secondary data source for triangulation or other data validity
needs. We wanted our students to understand the value of field notes and understand as
Kawulich (2005) explained that field notes serve as a record of activities or ceremonies
observed and informal discussions from the field. We wanted our students to understand the
value of ceremonies and traditions in many qualitative research settings. We did not want them
to misinterpret the phenomenon under investigation or not have the proper context to analyze
interview data because they failed to understand the dynamics, activities, or ceremonies in the
natural research setting. This was a concern for us because we had both worked with novice
qualitative researchers in the past who struggled to understand the context in which research
was being conducted and missed valuable insights and understandings from the data collected.
We also wanted our students to be comfortable with the various types of field notes.
Neuman (2011) identified types of field notes used in qualitative data collection. Types of data
included jotted notes (short memory triggers), direct observation (written immediately after
leaving the field), inference (reflecting social relationships, emotions and meanings), analysis,
(methodological strategies and theoretical notes), interview notes (information about interview
location and interviewee), and personal journal (personal feelings and emotional reactions).
Our efforts to utilize reflection with our students was intended to help students develop field
notes that could be used for inference purposes. This process would in turn support the creation
of analytic notes and yield insightful qualitative data to ensure a complete understanding of the
research setting and phenomenon. Neuman (2011) cautioned researchers to ensure that
inference be separated from the other process so that the researcher keeps observations,
inferences and beliefs separated. We agree that such a separation must exist for data validity
and reliability purposes; however, we feel that this separation often creates a mental barrier
among researchers, particularly novice researchers who are unable to reconcile the objective
and subjective to ascertain the underlying meaning or context and not misinterpret data.

Positionality of Research Subjects and Positionality as an Observer

The reflective questions related to positionality that we present provide a method for
the researcher to examine positionality of the research subjects and themselves as observers.
We view that reflection about positionality between research subjects and researcher provides
a means for researchers to embrace both their explicit and implicit subjectivity about the
occurrences within the research setting and from within themselves. While we do not claim
that it is possible for researchers to fully understand the positionality of the research subjects
in their setting, we posit that our questions would enable researchers to manage the bias that
accompanies explicit and implicit subjectivity. We posit that the researcher can best understand
2556 The Qualitative Report 2018

the positionality of research subjects through careful and critical reflection of their own
positionality including taking ownership of their interpretive lenses and bias. While many of
our students are familiar with the types of research settings in which they would conduct their
research, we cautioned them to not make any assumptions based upon their experiences from
similar settings. To do so would discount the phenomena experienced by the subjects in which
they conduct their research. A closer understanding of positionality is a mechanism by which
to prevent that possibility.

Our Approach to Purposeful Reflection

To guide students through more rigorous critical reflection of what they observed, we
provided reflection questions that were meant to allow students to recall both the tactical (hands
on) experience of observing and explore the abstract (reflective) insights about what is
occurring in the research environment. We felt that our students needed to be prompted to
examine the research setting more critically. The work of Alvesson and Sköldberg (2017)
supports our strong disposition that our students as novice qualitative researchers needed to
examine their research more critically. Alvesson and Sköldberg (2017) identified four elements
of reflective research that are listed and explained below.

1. Systematics and techniques in research procedures that ensure well-


reasoned logic for interacting with empirical evidence.
2. Clarification of the primary of interpretation as a means to connect with
theory and pre-understandings.
3. Awareness of the political-ideological character of research which are
usually not neutral, but highlight the dimensions of construct, political and
ideological conditions.
4. Reflection in relation to the problem of representation and authority so that
researcher’s claim to authority and texts claims to reproduce some extrinsic
reality are not undermined.

Purposeful Reflection Questions

The following are the purposeful reflection questions that we provided to our doctoral
students. These questions were informed by both our personal experience as researchers and
the challenges we saw with other novice graduate student researchers in the past.

Research Setting Access

• Did you enter the research setting with any expectations?


• How easy was it to enter the research setting?
• Did you have to become a member of the group to enter?
• How did you exit the research setting?

Examining Culture and Norms

• What were the norms in the setting?


• How would you describe the culture of the setting?
• Was there a display of power in the setting? Who controlled it?
David M. Deggs & Frank Hernandez 2557

• Did you notice the composition of the people in the setting (e.g., gender,
race, social class, etc.)? What did it tell you about the dynamics within the
setting?

Positionality of Research Subjects

• What roles did people play? What actions were aligned to those roles?
• Were there groups? Subgroups? In group? Out Group?
• Were you a part of any group as the observer?

Positionality as an Observer

• How did you feel before, during, and after the observation?
• Did people realize you were observing others? Were you observed?
• What was your vantage point as an observer? Did you yield any vantage
points to your research subjects or co-researchers?
• How did your vantage point impact your insight and understanding about
what you observed?

Discussion

We present our reflection questions in four categories as means by which to reframe


how researchers view field work in qualitative data. While we do not claim that our questions
are comprehensive, we feel that they are succinct enough to allow researchers, including novice
and experienced alike, to begin to examine research environments more critically to deduce
meaning and properly represent the phenomenon experienced by research subjects. We assert
that these questions can help researchers to manage bias, both implicit and explicit, and
moreover ensure that the research setting is not misunderstood, misinterpreted, or
misconstrued.

Research Setting Access

Access to research settings has been an issue that research methodologists have written
about extensively (e.g., Rossman & Rallis, 2016; Seidman, 2013). Our reflection questions
about this very common practice among qualitative researchers is meant to uncover bias by
examining expectations, reconciling struggles with access and rapport, and measures taken to
properly exit the setting. We posit that the researcher should pay careful attention to dynamics
how they negotiate and maintain access to fully understand and represent the phenomenon
within.

Examining Norms and Cultures

We have come to value the importance of cultural norms within qualitative research
settings, particularly those to which we are not members. With that in mind, we sought to help
our students understand the value of ceremonies and traditions in the naturalistic environments
in which they would conduct their research. Perhaps most importantly, from an ethical
perspective, we did not want our students to misinterpret the phenomenon under investigation.
To properly understand the phenomenon, we wanted our students to understand the dynamics,
activities, or ceremonies in the natural research setting to the best of their ability. The majority
of our students were novice researchers and practitioners in education leadership or higher
2558 The Qualitative Report 2018

education as well. We cautioned them to not make assumptions about any of the occurrences
within the settings in which they would conduct their research. We cautioned them to identify
the evidence that explains the phenomena that they will study. Our reflective questions
regarding norms, power, culture, and dynamics are intended to empower the researcher to look
beyond the surface by immersing themselves into the inter-workings of the research setting
thereby allowing them to embrace the contextualized meanings within.

Conclusion

While this article chronicled our efforts to help students enrolled in a doctoral level
qualitative research methods course embrace the observation process through the collection of
robust field notes and use reflexive practices to improve meaning of this form of qualitative
data, we assert that these techniques could be used by experienced researchers. The guided
questions we offer are not universally applicable to all research settings; however, they do
present a model that could be used in various settings. We assert that the categories of our
questions including (1) Research Setting Access, (2) Examining Norms and Cultures, (3)
Positionality of Research Subjects, and (4) Positionality as an Observer, provide a paradigm
that could help many qualitative researchers take a more systematic in-depth approach to the
collection, transcription, and analysis of field notes and other forms of qualitative data. We
encourage others to take adequate steps to ensure that the naturalistic research setting is not
misunderstood, misinterpreted, or misconstrued when presenting research results to expand
upon the body of knowledge. In an era where reflexivity of the researcher coupled with the
enhanced understanding positionality, it is important that qualitative researchers further
embrace and apply methods and techniques that further ensure the credibility of the researcher
and the value of the data that is collected through observations in the field. We feel that our
questions are designed to engage researchers in effective reflective practice so that they
challenge predispositions about research topics, research subjects, initial interpretations of
data, and recognize bias and further understand how their positionality informs their research
practice.

References

Alvesson, M., & Sköldberg, K. (2017). Reflexive methodology: New vistas for qualitative
research (2nd ed.). Thousand Oaks, CA: Sage.
Attia, M., & Edge, J. (2017). Be (com) ing a reflexive researcher: a developmental approach to
research methodology. Open Review of Educational Research, 4(1), 33-45.
Berger, R. (2015). Now I see it, now I don’t: Researcher’s position and reflexivity in qualitative
research. Qualitative Research, 15(2), 219-234.
Bourke, B. (2014). Positionality: Reflecting on the research process. The Qualitative
Report, 19(33), 1-9. Retrieved from https://nsuworks.nova.edu/tqr/vol19/iss33/3
Creswell, J. W., & Miller, D. L. (2000). Determining validity in qualitative inquiry. Theory
into Practice, 39(3), 124-130.
Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing
among five approaches (4th ed.). Thousand Oaks, CA: Sage.
Cruz, L. (2015). Self-reflexivity as an ethical instrument to give full play to our explicit and
implicit subjectivity as qualitative researchers. The Qualitative Report, 20(10), 1723-
1735. Retrieved from https://nsuworks.nova.edu/tqr/vol20/iss10/13
Finlay, L., & Gough, B. (Eds.). (2003). Reflexivity: A practical guide for researchers in health
and social sciences. Malden, MA: Blackwell Science.
Gabriel, Y. (2015). Reflexivity and beyond–a plea for imagination in qualitative research
David M. Deggs & Frank Hernandez 2559

methodology. Qualitative Research in Organizations and Management: An


International Journal, 10(4), 332-336.
Guillemin, M., & Gillam, L. (2004). Ethics, reflexivity, and “ethically important moments” in
research. Qualitative Inquiry, 10(2), 261-280.
Jootun, D., McGhee, G., & Marland, G. R. (2009). Reflexivity: Promoting rigour in qualitative
research. Nursing Standard, 23(23), 42-46.
Kawulich, B. B. (2005). Participant observation as a data collection method [81
paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social
Research, 6(2), Art. 43, http://nbn-resolving.de/urn:nbn:de:0114-fqs0502430.
Leech, N. L., & Onwuegbuzie, A. J. (2007). An array of qualitative data analysis tools: A call
for data analysis triangulation. School Psychology Quarterly, 22(4), 557-584.
Mathison, S. (1988). Why triangulate? Educational researcher, 17(2), 13-17.
Maxwell, J. A. (2012). Qualitative research design: An interactive approach (Vol. 41).
Thousand Oaks, CA: Sage.
Morse, J. M., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification strategies
for establishing reliability and validity in qualitative research. International Journal of
Qualitative Methods, 1(2), 13-22.
Mruck, K., & Breuer, F. (2003). Subjectivity and reflexivity in qualitative research—The FQS
issues. In Forum Qualitative Sozialforschung / Forum: Qualitative Social Research,
4(2). Retrieved from http://www.qualitative-
research.net/index.php/fqs/article/view/696
Neuman, L. W. (2011). Social research methods: Qualitative and quantitative approaches (7th
ed.). Boston, MA: Allyn and Bacon.
Oliver, D. G., Serovich, J. M., & Mason, T. L. (2005). Constraints and opportunities with
interview transcription: Towards reflection in qualitative research. Social Forces,
84(2), 1273-1289.
Orange, A. (2016). Encouraging reflective practices in doctoral students through research
journals. The Qualitative Report, 21(12), 2176-2190. Retrieved from
http://nsuworks.nova.edu/tqr/vol21/iss12/2
Ortlipp, M. (2008). Keeping and using reflective journals in the qualitative research process.
The Qualitative Report, 13(4), 695-705. Retrieved from
http://nsuworks.nova.edu/tqr/vol13/iss4/8
Rossman, G. B., & Rallis, S. F. (2016). An introduction to qualitative research: Learning in
the field. Thousand Oaks, CA: Sage Publications.
Seidman, I. (2013). Interviewing as qualitative research: A guide for researchers in education
and the social sciences. New York, NY: Teachers College Press.
Valandra, V. (2012). Reflexivity and professional use of self in research: A doctoral student's
journey. Journal of Ethnographic & Qualitative Research, 6(4), 204-220.

Author Note

David Deggs is Executive Director of College Access Programs in the Simmons School
of Education and Human Development at Southern Methodist University. He also serves as an
Adjunct Professor in the Department of Education Policy and Leadership. Correspondence
regarding this article can be addressed directly to: [email protected].
Frank Hernandez holds the Annette and Harold Simmons Centennial Chair in
Education Policy and Leadership and serves as the Associate Dean in the Simmons School of
Education & Human Development at Southern Methodist University. Correspondence
regarding this article can also be addressed directly to: [email protected].
2560 The Qualitative Report 2018

Copyright 2018: David Deggs, Frank Hernandez, and Nova Southeastern University.

Article Citation

Deggs, D. M., & Hernadez, F. (2018). Enhancing the value of qualitative field notes through
purposeful reflection. The Qualitative Report, 23(10), 2552-2560. Retrieved from
https://nsuworks.nova.edu/tqr/vol23/iss10/18

You might also like