THE INTERNAL AUDIT AS TOOL

TO ENHANCE THE QUALITY OF

QUALITATIVE RESEARCH
W. Schurink
Sanlam Centre for Public Management and Governance
University of Johannesburg

INTRODUCTION
n general, researchers will agree that social science research is a systematic and
organised process whereby enough knowledge (evidence) is gained to provide an
accurate or truthful representation of a phenomenon under study. However, agreeing on
this does not mean that they will agree on theoretical or methodological issues. In fact,
there is a vast difference of opinion between qualitative and quantitative researchers as
to how the social world should be studied. (http://bgpinqmr.group.shef.ac.uk/workshop/
Facilitators_Guide_2.pdf).
The root of the difference lies in the worldview or ontology of the researcher. The one
will believe that human behaviour can be explained from the outside (epic) by means of
objectivistic observation and the use of general scientific laws (erklaren). The other will
assume that human behaviour can only be understood from an insider’s point of view
(emic) by gaining insight into the meaning (verstehen) that the participant gives to his/her
life world.
Qualitative and quantitative researchers therefore use different epistemological
assumptions stemming from different theoretical contexts to understand human behaviour.
In the context of this article it is important to comprehend that qualitative research aims at
understanding actors’ subjective meanings and interpretations to explain their behaviour
(Schwandt 2007), and that by accepting subjectivity as a legitimate domain for social
scientific research, a break is established between the natural and social sciences. (http://
bgpinqmr.group.shef.ac.uk/workshop/ Facilitators_Guide_2.pdf).
This is also true for the criteria by which qualitative research is measured. In other
words, a qualitative study resting on the premise of verstehen cannot be judged as truthful
or not by using the criteria meant to measure a study based on the premise of erklaren.
Therefore, when the quality of a piece of research is validated, the ontological and
epistemological position of the researcher should be the frame of the “magnifying glass”
used to establish the truthfulness of a study.

788 Journal of Public Administration • Vol 44 no 4.2 • November 2009

DESIGNING TRUSTWORTHY RESEARCH
he most important criterion that qualitative research should meet is that of credibility.
An important decision one needs to make when designing a qualitative research study
is to start thinking on how, and according to what principles and logic your claims
and analysis will be formulated and substantiated. Differently put, you should consider how
you are going to ensure that you do quality research. It is crucial that measures of quality
are considered during the research design phase, since they need to guide you in selecting
appropriate methods, sources and combinations of these as well as in other decisions
that need to be taken. Since these measures form part of the decision making process
they cannot be reserved for a later phase of the research process. Thus, it is of crucial
importance that these issues are addressed during the research design phase. “If they are
not, the researcher risks assembling an untidy bag of methods with little logic, and with little
hope of sensibly integrating the products into a coherent analysis” (Mason, 1996:26).
When conducting qualitative research, researchers face many ethical dilemmas and
must constantly decide on the proper conduct. Of course, ethical issues are the concerns
and dilemmas that arise over the proper way to execute research, more specifically
not to create harmful conditions for the participants in the research process. According
to Neuman (1997) these issues involve trade-offs between competing values and are
typically situational, that is they depend on the researcher, topic and research participants.
The following account by Jones (1996:33) illuminates these conflicts quite well: “Although
researchers may agree that knowledge and understanding are ideals worthy of pursuit,
there are other ideals that they may, at times, value even more highly. When such a
conflict of ideals occurs, the issue of whether or not one should conduct the research in
question may be said to pose an ethical dilemma. Quite literally, the researcher must ask
him or her, ‘Can this research be ethically justified’?
Ethically responsible research depends on the integrity of the individual researcher
and his or her values. Neumann (1997:443) puts this as follows: “Ethics begins and ends
with you, the researcher. A researcher’s personal moral code is the strongest defense
against unethical behavior. Before, during, and after conducting a study, a researcher
has opportunities to, and should, reflect on research actions and consult his or her
conscience… Ethical behavior arises from sensitivity to ethical concerns that researchers
internalise during their professional training, from a professional role, and from personal
contact with other researchers. Moreover, the norms of the scientific community reinforce
ethical behavior with an emphasis on honesty and openness. Researchers who are
orientated toward their professional role, who are committed to the scientific ethos, and
who interact regularly with other researchers are likely to act ethically”.
In the light of the preceding remarks, Mason (1996) is correct when she points out that
qualitative researchers should be as concerned to construct an ethical research design as they
are to construct an intellectually coherent and compelling one. “This means attempting not
only to carry out our data generation and analysis in an ethical manner, but also to begin by
framing ethical research questions. Of course this is easier said than done, because however
the research questions are framed, any project is likely to involve a range of interests, some of

W. Schurink 789
Article
which may be competing. Therefore the idea that there is one ethical or moral route which is
equally fair to all concerned may sound good in theory, but be elusive in practice”.
Since research ethics is a complex issue, and it is very unlikely that there will ever be
one clear solution, it is particularly appropriate to follow a practical approach in which
you ask questions and push yourself hard to answer them. You need to be honest about
the purpose of your research. Your study is likely to include not only the advancement of
knowledge or understanding of some aspect of the social world, but also factors involving
personal gain, such as the achievement of a personal qualification, a promotion, some
standing in a discipline (amongst colleagues, friends, rivals, relatives, etc.), and/or some
research funding. “It is part of the politics of research that you should engage with this
wider context in which your research is being done. Your research may have explicitly
moral or political purposes. You may wish to advance the interest of a particular group
through it, or to gain acceptance for some particular form of social organization, or to
expose some form of immoral organization or activity, and so on. This does not necessarily
make the ethics of your research more straightforward however, not least because ‘the
interests of a particular group’ may be diverse or contested. The notion of one moral route
may therefore still be elusive” (Mason, 1996:29-30).
Mason (1996) suggests that researchers ask a number of questions to themselves like:
(i) which parties, bodies, practices, or whatever, are potentially interested or involved in or
affected by a particular study? (ii) What are the implications of framing particular research
questions in a study for these parties, bodies, practices, and so on?
Naturally researchers’ answers to questions like these will not inform them whether
their research questions are ethical or not, but will guide them towards identifying the
potentially complex range of interests touched upon by their studies. If researchers are
explicit about these interests, they can begin to work out which courses of action seem
the most reasonable and ethical, and which do not and are better left alone.

ASSESSING THE TRUSTWORTHINESS OF QUALITATVE RESEARCH

he conventional criteria for good research are: (i) internal validity, (ii) external validity
or representativeness, (iii) reliability and (iv) objectivity. Qualitative researchers
generally regard these constructs as inappropriate in establishing the “truth value”
of a qualitative research project. Two prominent qualitative researchers, Lincoln and
Guba (1985), propose the following four alternative constructs they believe reflect the
assumptions of the qualitative paradigm more accurately:
• Credibility/authenticity
Is there a match between research participants’ views and researchers’ reconstruction
and representation of it?
• Transferability
Can the findings of the research be transferred from a specific situation or case to
another?
• Dependability
Is the research process logical, well documented and audited?

790 Journal of Public Administration • Vol 44 no 4.2 • November 2009

 • Confirmability
Does the researcher provide evidence that corroborates the findings and
interpretations by means of auditing?

The first of these criteria namely credibility is considered as the most important one.
Lincoln and Guba (1985) outline various strategies to increase the credibility of qualitative
research:
• Prolonged engagement and persistent observation in the field;
• Triangulation of different methods;
• Peer debriefing;
• Member checks; and
• Formalised qualitative methods such as grounded theory and analytic induction.

However, researchers (Schwandt, 1996) became increasingly critical of using research

criteria based on the quantitative paradigm. They argued that these criteria are based
on positivism and is therefore incompatible with qualitative research. How indeed
could criteria based on erklaren be used to evaluate research based on the premise of
verstehen? Researchers, especially post-modernists, challenged with renewed intensity the
use of any evaluation criteria to measure the quality of qualitative research. They argued
that these criteria are incompatible with the features of qualitative research and that it is
especially unfair to use criteria based on positivism to evaluate qualitative (interpretive and
constructivist) research. Therefore they argued that a separate set of evaluation criteria is
needed (Bryman, 2007).
Many other attempts were made to develop criteria for “good” qualitative research
based on the qualitative paradigm (Giorgi, 1988; Kavale, 1996 and Stiles, 1993). From the
many formulations we will discuss only a few prominent ones that could help to further
the discussion on the use of the internal audit as a possible means to enhance the quality
of qualitative research. Morse (1994) focuses her criteria for qualitative research on the
analysis of qualitative data by stressing the following:
• Comprehension (learning about a setting);
• Synthesising (identifying patterns in the data);
• Theorising (explanations that fit the data) and;
• Recontextualising (abstracting emergent theory to new setting and relating it to
• established knowledge).

Hammersley (1989; 1990 and 1992) adds to these criteria by developing the concept
internal reflexivity that leads to the development of the following criteria: (Kincheloe and
McLaren (1998) elaborate on this concept).
• Reflexive interrogation by the researchers of the epistemological baggage they bring
with them.
• Through a critical ethnography, researchers attempt to sensitise themselves and
participants to how hegemonic regimes of truth impact upon the subjectivities of the
disadvantaged.

W. Schurink 791
 • Positivist conception of validity rejected in favour of the credibility of socially constructed
realities to those who have democratically participated in their development.
• Ability to generalise rejected in favour of accommodation – where researchers use their
knowledge of a range of comparable contexts to assess similarities and differences.
• Catalytic validity - extent to which research changes those they study so that they
understand the world in new ways and use this knowledge to change it - link
to pragmatist criterion of practical adequacy (http://bgpinqmr.group.shef.ac.uk /
workshop/Facilitators_Guide_7.pdf).

Elliot, Fisher and Rennie (1999) developed what they term evolving guidelines for
critiquing qualitative research, namely:
• Owning one’s perspective, meaning that researchers describe their theoretical,
methodological and personal assumptions.
• Situating the sample, where the researcher describes the research participants and their
life circumstances.
• Grounding in examples. Examples of the data are presented to illustrate analytic
procedures used in the study to show how the researcher’s understanding developed
from it.
• Providing credibility checks. For example, checking understandings with participants,
using triangulation and comparing varied qualitative perspectives (analytic(al)
induction).
• Accomplishing general vs. specific research tasks. General understanding of the
phenomenon should be based on an appropriate range of participants and situations.
The findings are not intended to be extended to other contexts or participants. In
the case of specific understanding, participants and situations are systematically and
comprehensively described to provide the reader with a basis of understanding and it
also addresses limitations of transferability.
• Resonating with readers, meaning that the material is presented in such a way that the
reader judges it as credible.

Stating that the traditional criteria for qualitative research are inappropriate, Yardley
(2000) proposes the following criteria that could, according to her, lead to flexible
interpretation:
• Sensitivity to context. Here, the following dimensions are important: theory building,
use of relevant literature, awareness of the socio-cultural setting of the study, the
relationship between researcher and participant, and discussion of ethical issues.
• Commitment and rigour. This refers to an in-depth discussion of the topic/argument
and methodological competence through data collection and in-depth analysis of the
data.
• Transparency and coherence. The arguments must be clearly described, there must be
a fit between theory and research strategy and the methods used, and the researcher
should disclose and critically reflect on all his/her decisions taken during the entire
research process.

792 Journal of Public Administration • Vol 44 no 4.2 • November 2009

 • Impact and importance. Theoretical, methodological and practical contributions
should be made.

Charmaz (2007) presents the following premises, principles and practices for quality
qualitative research:

Premises
• A deep understanding of a studied life means entering it and thus taking an insider
perspective, and presenting an incisive analytical interpretation.
• To understand what is happening in a setting we need to know what things mean to
participants. Grasping the meaning participants have of their innermost experiences
and feelings, unlock our understanding of their actions and intentions.
• To learn participants’ meaning we need to be reflexive upon our own.
• Relationships between meanings and actions are dynamic and reciprocal. Actions
impact on meaning and meaning shapes actions.
• Flowing from this, the questions we ask and how we ask it, shape the answers we
obtain. For example, our way of gathering data impacts on what participants tell
us.
• Truths are relative, multiple and subject to redefining. For example, feminist-orientated
inquiry questioned traditional viewpoints on minority groups and women.

Principles
• A deep understanding of a studied life is based on the principle of intimate familiarity
with the phenomenon.
• Show a deep respect for our participants as human beings and treat them with
dignity.
• Using established theoretical perspectives as starting points, not ending points, can
help us to generate new theoretical insights.
• Accuracy means collecting sufficient data in order to have a full range of observations
of the phenomenon, not just theoretical saturation.
• Studying meanings and processes at both the subjective and social levels illuminates
the context.

Practices
Avoid using qualitative research methods in a mechanical way by:
• opening yourself up to the experience by allowing space for the unexpected to
happen;
• gathering sufficient data to make your study credible;
• paying attention to language; and
• looking beneath the surface.

W. Schurink 793
From the above, it is clear that many differently orientated qualitative research scholars
attempted to adapt or re-write the criteria for qualitative research as originally proposed by
Lincoln and Guba (1985). Apart from the attempts discussed above, there are many other
appraisal criteria lists for qualitative research, each mirroring the different ontological and
epistemological viewpoints of those who constructed it (Attee and Milton 2007:262). The
positivist will emphasise internal validity, construct validity, external validity, reliability
and objectivity. The realist stresses internal reflexive audit trails to demonstrate credibility,
dependability, confirmability, transferability and ecological validity or naturalism, while
subjectivists or constructionists would mostly agree with Schwandt that all measures
to assess qualitative research should be abandoned (http://bgpinqmr.group.shef.ac.uk/
workshop/Facilitators_Guide_7.pdf).
The important point is that despite the many attempts, there is still a lack of consensus
among researchers as to what criteria would be acceptable to measure the quality of
qualitative research. “A lack of consensus among researchers means that standards for
good qualitative enquiry remain elusive” (Attee and Milton, 2007:262). Finding the
golden standard for the evaluation of qualitative research is therefore difficult if not
impossible since qualitative researchers differ vastly in their approach, each having
their own theoretical background, methodological principles and aims (Flick, 2007:6).
Just as there is no consensus between the different qualitative approaches there can
never be a list of criteria for the appraisal of qualitative research that would be accepted
by all qualitative researchers. This means that when judging the quality of qualitative
research, the truthfulness of the study can only be justly validated if the reader/
reviewer is informed and understands the epistemological and ontological position
the researcher adopted. Differently stated, when evaluating qualitative research, the
researcher’s theoretical approach, research strategy and methodology followed in the
study should be seen as the launching pad from where the trustworthiness of the study
could be critiqued.
From a scan of recent literature, it appears that presently at least three distinct
perspectives on assessing the quality of qualitative research can be found amongst
scholars. These are: (i) qualitative and quantitative research should be evaluated by the
same measures, (ii) qualitative research should be evaluated by standards that have been
particularly developed for it, and (iii) what Holloway and Wheeler (2002) call criteriology,
that is, abandoning measures to asses qualitative work. “The belief that qualitative research
would achieve paradigm takeoff by imitating the methods of natural sciences is thus to
a greater extent discarded by qualitative researchers and in particular by postmodernists
(Schwandt, 2007:40). In the process qualitative researchers came up with more and more
checklists and frameworks for the development of alternative criteria based on qualitative
research principles” (Flick, 2007).
Existing criteria for trustworthy or credible research can never meet the approval of all
qualitative researchers. It is therefore not surprising that the notion of developing criteria of
soundness to meet the approval of all qualitative researchers has to date been discarded
to a large extent, or as Schwandt (2007) states that qualitative researchers have gone
beyond it. This however, does not mean that qualitative research has become unscientific

794 Journal of Public Administration • Vol 44 no 4.2 • November 2009

and that anything now goes. In fact, qualitative researchers have now turned to the use of
specific strategies to ensure rigour and quality in the research process.

STRATEGIES TO ENHANCE THE QUALITY OF QUALITATIVE RESEARCH

ince a clear and universal solution to the quality question, or criteria for good
qualitative research has to date not been attainable, researchers started to apply
strategies to promote quality in the research process. Thus, the notion of management
of the research process and transparency with specific reference to the auditing trail1 and
reflexivity became important strategies to ensure quality research.
The auditing trail displays the interaction between the researcher and his/her research
participant(s) in such a way that the research can be understood not only in terms of
what was discovered but also how it was discovered. The advantage of this is that our
interpretations can be better understood and validated by readers who are informed
about the position we adopt in relation to the study, and by our explicit questioning
of our own involvement. By revealing aspects of themselves and the research process
as a traceable audit trail, qualitative researchers establish the credibility, dependability
and confirmability of findings, and thereby show that the research was rigorously done
Seale (1999).
For qualitative researchers engaging themselves with difficult questions and issues
in the reasoning process through which they arrive at their answers, it is today not only
important to properly plan their studies, but also to provide a reflexive account of how
it was done, e.g. constructing natural histories, research stories, or internal audits, and/or
defending the logic of their decisions. Therefore, in order to be able to refer to particular
issues you were confronted with during various phases of the development of your
research proposal, as well as the decisions you took and your reasons therefore, you will
be well-advised to keep a research diary or journal from the onset of the research.
The problem of reflexivity has been discussed extensively by many researchers
(Denzin, 1997, 1995; Hammersley and Atkinson, 1995 and Etherington, 2006) to name
but a few. It is important to note that the post-modern trend in qualitative research
encourages researchers to undertake and record a continuous critical analysis of their
thinking and feelings concerning their conceptual framework, research questions,
methods, values and biases. They must also have an awareness of their presence in the
very situations they want to study, as well as of the inter-subjective dynamics between
themselves as researchers and the research they are conducting. Such a practice forms
an important tool that could be used to enhance the quality of qualitative research (see
Figure 1).
This argument could be taken a bit further by reasoning that researchers, relying on
interpretive methods, could only make sense of their data if they continuously reflect on
their decisions taken during the research process and having a heightened awareness
of their epistemological, theoretical, and ontological conceptions and how this impact
on their research practices (Mauthner and Doucet, 2003). The suggestion is made by
(Mauthner and Doucet, 2003) that the more researchers can be self-conscious about,

W. Schurink 795
Figure 1: Elements of the auditing trail

PROCESS OF OBSERVATION
AND REFLEXIVITY
Reconstruction of data and results of synthesis: categories,
findings and integration of it with the literature

DISTORTED BY:

Personal notes and expectations of the participants

Data reduction and results of syntheses: summarising and theoretical notes. SOCIO-CULTURAL

EMOTIONAL
Process notes: methodological notes and decisions regarding
trustworthiness and credibility of findings THEORY-LADEN

PHYSIOLOGICAL
Observational notes
BACKGROUND

The raw data, the collection and recording

FACTS OF REALITY

and articulate their role in research processes and products, the more readers can engage
in symbolic dialogues with the author(s), and the more their confidence in the work will
increase. More specifically, it could help the researcher to be reflexively interconnected in
the research process, particularly when analysing his/her data. The analysis of qualitative
data is mostly regarded as a process of sorting, organising, and indexing the information
obtained. This is often wrongly seen as a mechanical process that takes place in a social
vacuum. In reality (as is clear from Figure 1), researchers are all but invisible since they
have their own ontological, epistemological and theoretical assumptions as well as their
own backgrounds, personal, emotional and interpersonal influences that they bring with
them to the research process (Mauthner and Doucet, 2003).
It has therefore become more important for qualitative researchers to reflect on the
research process in terms of: What happened? Who was involved? Where did the activities
occur? What circumstances or issues impacted on the data? What are the major issues
emerging? What issues need to be followed up on? It also involves exploring the meaning
of the data further by asking “so what” and “what if” questions. The reflexive process is
very helpful to relating occurrences, for example words, expressions, interactions and
social processes to people, events, other occurrences, values and norms of particular

796 Journal of Public Administration • Vol 44 no 4.2 • November 2009

groups of people. Discovering such linkages are important in selecting further theoretical
incidents, persons, behaviours, etc. and establishing and verifying evolving ideas, themes
and typologies.
The practice of reflexivity questions the assumption that research findings naturally
emerge from the process of data collection and analysis. Researchers supporting the
practice of reflexivity feel that findings are shaped by the choices made by researchers
and their critical reflection of these choices. (http://bgpinqmr.group.shef.ac.uk/workshop/
Facilitators_Guide_3.pdf). In this regard, reflexivity can be described as an effort to
understand how the process of doing research shapes its outcomes. Many attempts
are made to define reflexivity (Hardy, et al., 2001; Alvesson and Skoldberg, 2000; and
Woolgar, 1988). Perhaps the best definition is that of Etherington (2006): “This means
interpreting our own interpretations, looking at one’s own perspectives, and turning a
self-critical eye into one’s own authority as interpreter and author … this enhances the
trustworthiness of the findings and outcomes of research”.
This definition reflects the essence of reflexivity namely that it entails a critical appraisal
of the knowledge produced and how that knowledge was generated (Guillemin and
Gillam, 2004). “In essence, reflexivity entails engaging in critical appraisal of own practice...
Reflecting on why we frame issues in particular ways, investigate them in particular ways,
and how such approaches lead us to particular kinds of solutions and theories and not
others” (http://bgpinqmr.group.shef.ac.uk/workshop/ Facilitators_Guide_3.pdf).
Although self-critical, a clear distinction should be drawn between being critical and
being judgmental. In fact, Davies, Browne, Gannon, Honan, Laws, Mueller-Rockstroh
and Bendix Petersen (2007) warn us that “the moral gaze restricts, binds one’s thoughts,
defers and refers constantly to the discourses of respectability, of morality, of discipline
of method… The reflexive eye/I in contrast, ideally gazes without judgement, like a good
novelist, finds the unexpected, the surprising – the contradictions, the “good” and the “bad”
in all its detail – not with a mind to censor, but to say with fascination “oh so that’s how
it is!” As Charmaz (2007:13) states: “… enter the phenomenon and open yourself to the
research experience. Face the inevitable ambiguities. Flow with the existential dislocation of
bewilderment. Bring passion, curiosity and care to your work. In the end, you will transform
our images of studied life, and your research journey will transform you”.
On a deeper level, reflexivity also means continuously questioning your understanding
of the phenomenon, synthesising the data by identifying patterns, theorising by
developing explanations that fit the data and recontextualising by relating emergent theory
to established knowledge. Therefore the process of continuous reflexivity should help us
and others to understand why we frame questions in particular ways, investigate them in
a certain manner, how we gain an understanding of the meanings research participants
give to their life-world, gather sufficient data to make our research credible and how such
approaches lead us to interpret the data the way we do (http://bgpinqmr.group.shef.ac.uk/
workshop /Facilitators_Guide_3.pdf).
Just as in the case of criteria lists, various frameworks for reflexivity have been
offered by differently orientated researchers. For example Johnson and Duberley (2003)
equate different kinds of reflexivity with different kinds of theoretical commitments (e.g.

W. Schurink 797
positivism, postmodernism and critical theory), Holland (1999) talks about four different
‘levels’ and Finlay (2002) suggests five different ‘maps’ (http://bgpinqmr.group.shef.
ac.uk/ workshop /Facilitators_Guide_3.pdf). In order to gain a deeper understanding of
the concept reflexivity it is necessary to place it within the framework of methodology,
epistemological commitments and disciplinary maps (http://bgpinqmr.group.shef.ac.uk/
workshop /Facilitators_Guide_3.pdf).
Methodologically, we are centrally concerned with practices and procedures for
research and how these impacted on in the conclusions reached, i.e. how and why
the research was designed, conducted and analysed, and how this led to particular
conclusions. Questions asked in this regard include:
• How should the research be designed or conducted in order to provide a convincing
account?
• What alternative interpretations are there?
• What role do researchers play in producing results?
• What choices were made and what were the reasons for them?

Reflexivity on the methodology of the research is important because it renders the

research process transparent and accountable (http://bgpinqmr.group.shef.ac.uk/ workshop
/Facilitators_Guide_3.pdf).
Ontological and epistemological reflexivity concerns our assumptions about what we
can know and how we can claim to know it. ‘An important function of reflexive analysis
is to expose the underlying assumptions on which arguments and stances are built
(Holland, 1999).
We become more consciously reflexive by thinking about our own thinking, by
noticing and criticising our own epistemological pre-understandings and their effect on
research, and by exploring possible alternative commitments (Johnson and Cassell, 2001).
Epistemological assumptions reflect researcher commitments to particular philosophical
beliefs. But all ontological and epistemological stances have weaknesses. Consequently
awareness, debate and reflection are crucial if we want to enhance the quality of
qualitative research. In this regard the following questions should be asked.
• What can our measures actually tell us about the nature of the world and human
action?
• What are our aims in conducting the research?
• What assumptions are implicated in the theories that drive our research and are
produced as a result of our research?

It is necessary for us to reflect on how our disciplinary assumptions and background may
structure a research project and how such assumptions may favour the production of
certain types of knowledge, and how participants may be affected by this. For example,
in psychology, the ‘psychologising’ of the individual (examination of ‘internal’ cognitions
etc.) can have the effect of individualising or putting responsibility on the individual and
ignoring the structural/policy aspects of a situation (http://bgpinqmr.group.shef.ac.uk/
workshop/Facilitators_Guide_3.pdf).

798 Journal of Public Administration • Vol 44 no 4.2 • November 2009

 However, despite the growing interest in reflexivity as a method to enhance quality
research, the theoretical and philosophical discussions about reflexivity and its relationship
with the construction of knowledge remain weak (Davies, Browne, Gannon, Honan,
Laws, Mueller-Rockstroh and Bendix Petersen, 2007). Researchers are offered little
guidance as to understand how to turn a self-critical eye on themselves and decisions that
they took during the research process and how these decisions impacted on their study
(Mauthner and Doucet, 2003) 2. Researchers are also scared to reflect by putting words on
paper for everyone to see. As Davies, Browne, Gannon, Honan, Laws, Mueller-Rockstroh
and Bendix Petersen (2007) warn: “Reflexive writing can be passionate and emotional, it
can be writing in which the mind, heart and body are all engaged. Yet once those words
are out there in the world, objects themselves of reflection by others as well as ourselves,
they can become weapons to turn against us….The self is both subject and object of the
reflexive eye…”
Yet no researcher can deny that he/she is present in the research setting, in field
note writing, in analysing and interpreting the data, and in the writing up of the findings
(Davies, Browne, Gannon, Honan, Laws, Mueller-Rockstroh and Bendix Petersen, 2007).
This is true even when he/she takes an objectivist stance. Then the question is how can
the researcher not reflect on the research process? Denzin (1997) refers to the reflexive
individual in the world. To take this argument further, everyone reflects upon life in order
to make meaning of it. Reflexivity could thus be seen as the researcher’s attempts to make
meaning of his/her research journey.
The construction of knowledge in research is therefore a reflexive process that could
bring rigour to the study (Guillemin and Gillam, 2004). This ensures as Flick (2007:64)
states that “qualitative research, including research with a post-modernistic paradigm should
be developed and produced in the tensional field of (theoretical, conceptual, practical and
methodological) creativity and (methodological) rigour in studying phenomena. Therefore
quality should be located in the tensional field – between being rigorous and being
flexible”. The goal of reflexivity in this sense has to do with improving the quality of the
research (Guillemin and Gillam, 2004).
Seeing that reflexivity plays such an important role in qualitative research, the question
is how can the practice of reflexivity be enhanced and clarified? Schatzman and Strauss
(1973) developed a model or scheme, which considers the prerequisites of reflexive
field-note taking. In many ways this scheme provides the specific guidelines lacking in
the frameworks proposed by Johnson and Duberley (2003), Holland (1999), and Finlay
(2002). This scheme has been utilised by various qualitative researchers. This model
originally consisted of three elements, namely observational notes, theoretical notes and
methodological notes. Richardson (2004) later added a fourth category, namely personal
notes. The elements of this suggested model as depicted in Figure 1 may be described
as follows:
• Observational notes (ON). Observational notes are detailed notes on what happened,
what was heard, seen and experienced. “Each ON represents an event deemed
important enough to include in the fund of recorded experience, as a piece of evidence
for some proposition yet unborn or as a property of context or situation. An ON is the

W. Schurink 799
 Who, What, When, Where and How of human activity. It tells who said or did what,
under stated circumstances” (Schatzman and Strauss, 1973:100).
• Methodological notes (MN). Methodological notes are mainly reminders, instructions
and critical comments to the recorder or researcher on how to collect the data, how to
improve the quality of the interviews, whom to talk to next, etc. “Methodological notes
might be thought of as observational notes on the researcher himself and upon the
methodological process itself; as complete a chronicle as the recorder finds necessary
or fruitful “(Schatzman and Strauss, 1973:101).
• Theoretical notes (TN). Theoretical notes are self-conscious, systematic attempts by
the researcher to critically reflect on what took place (ON’s), what he/she thought and
experienced. Schatzman and Strauss (1973: 101) write: “The observer as recorder thinks
about what he has experienced, and makes whatever private declaration of meaning
he feels will bear conceptual fruit. He interprets, infers, hypothesises, conjectures; he
develops new concepts, links these to older ones, or relates any observation to any
other…”
• Personal notes (PN). These are the researcher’s critical reflection about his/her feelings
about the research.

CONCLUSION
he above guidelines could help us to reflect critically on how we construct
knowledge from the research process and how the quality of that knowledge could
be enhanced. More specifically it could help us to answer the following essential
questions about the research process:
• What kind of factors influenced the researcher’s construction of knowledge?
• How are these influences accounted for in the research process?

These guidelines could also help us to focus on improving the quality of qualitative research
by not only critically evaluating the research methods and the data but also the researcher,
the participants and the research context. Since these proposed guidelines also focus on the
interpersonal aspects of the research and the interaction between researcher and participant,
it also reflects on the very important ethical dimensions of research. Therefore, the use of
the above guidelines could help researchers to do both rigorous and ethical research. By
using it to critically reflect on ourselves and our participant(s) within the framework of our
research design, we could enhance the quality of our research. As Etherington (2006:32)
states: “If we can be aware of how our own thoughts, feelings, culture, environment and
social and personal background inform us as we dialogue with participants, transcribe their
conversations with us and write our representations of the work, then perhaps we can come
close to the rigour that is required of good qualitative research”.

NOTES
1 That is, a systematically maintained documentation process of the researcher’s continuous critical analysis of all deci-

800 Journal of Public Administration • Vol 44 no 4.2 • November 2009

 sions and actions taken during the entire research process, often also referred to as research story (Schwand 2007).
2 The only exception is interviewing and the researcher-participant relationship (Denzin 2001).2.

Bibliography
Alvesson, M. and Skoldberg, K. 2000. Reflexive Methodology: New Vistas for Qualitative Research.
London: Sage Publications.
Attee, P. and Milton, B. 2007. Critically appraising qualitative research for systematic reviews:
defusing the methodological cluster bombs. In Bryman, A. (Ed). Qualitative research 2 volume
II – Quality issues in qualitative research. 261–280. Thousand Oaks: Sage.
Bryman, A. 2007. Editor’s introduction. In: Bryman, Alan (Ed.). Qualitative research 2. Volume 1.
Collecting data for qualitative research (pp. xxiii-xlv). London: Sage.
Charmaz, K. 2007. Premises, Principles, and practices in qualitative research: Revisiting the
foundations. In Bryman, A. (Ed.) Qualitative research 2 volume II – Quality issues in qualitative
research, pp.112-134. Thousand Oaks: Sage.
Davies, B., Browne, J., Gannon, S., Honan, E., Laws, C., Mueller-Rockstroh, B., and Bendix Petersen,
E. 2007. The ambivalent practices of reflexivity. In Bryman, A. (Ed.) Qualitative research 2
volume III –Issues of representation and reflexivity, pp. 136–167. Thousand Oaks: Sage.
Denzin, N.K. 1997. Interpretive ethnography: Ethnographic practices for the 21st century. Thousand
Oaks: Sage.
Denzin, N.K. 2001. The reflexive interview and performative social science. Qualitative research,
1(1), pp. 23–46.
Elliot, R., Fisher C.T. and Rennie, D.L. 1999. Evolving guidelines for publication of qualitative
research studies in psychology and related fields. British Journal of Clinical Psychology.
Flick, U. 2007. (Eds.) Designing qualitative research. The SAGE qualitative Research Kit. Thousand
Oaks, CA: Sage.
Guba, E.G. and Lincoln, Y.S. 1989. Fourth generation evaluation. Newbury Park, CA: Sage.
Giogi, A. 1988. Validity and reliability from a phenomenological perspective. In Baer, Moss, L.,
Rappard, H. and Stam, H. (Eds.), Recent trends in theoretical psychology, New York: Springer-
Verlag.
Guillemin, M. and Gillam, L. 2004. “Ethics and reflexivity, and Ethically important moments” in
research. Qualitative Inquiry, 10(2), pp. 261–280.
Hammersley, M. 1990. Reading Ethnographic Research: A Critical Guide, Longman: London.
Hammersley, M. 1992. What’s Wrong with Ethnography? Routledge: London.
Hammersley, M. 1995. The Politics of Social Research. Sage: London.
Hammersley, M. and Atkinson, P. 1995. Ethnography: Principles in practice. London: Routledge.
Hardy, C., Phillips, N. and Clegg, S. 2001. Reflexivity in organization and management theory: A
study of the production of the research ‘subject’. Human Relations, 54, pp. 531–560.
Holland, R. 1999. Reflexivity. Human Relations, 52, pp. 463–483.

W. Schurink 801
Holloway, I. and Wheeler, S. 2002. Qualitative research in nursing. (2nd ed) Oxford, UK: Blackwell.
http://bgpinqmr.group.shef.ac.uk/workshop/Facilitators_Guide_2.pdf.
Johnson, P. and Cassell, C. 2001. Epistemology and work psychology: New agendas. Journal of
Occupational and Organizational Psychology, 74, pp. 125–141.
Johnson, P. and Duberley, J. 2003. Reflexivity in management research. Journal of Management
Studies, 40, pp. 1279–1303.
Jones, M.O. 1996. Studying Organizational Symbolism: What, How, Why?, Qualitative research
methods series, Vol. 39(1).
Kavale, S, 1996. Interviews: An introduction to qualitative research interviewing. Thousand Oaks:
Sage.
Kinchloe, J.L. and McLaren, P.L. 1998. Rethinking critical theory and qualitative research. In Denzin,
N. and Lincoln, Y. (eds.) Handbook of Qualitative Research. London: Sage.
Lincoln, Y.S. and Guba, E. 1985. Naturalistic Enquiry. Beverly Hill CA: Sage.
Mason, J. 1996. Qualitative researching. Thousand Oaks: Sage.
Mauthner, N.S. and Doucet, A. 2003. Reflexive accounts and accounts of reflexivity in qualitative
data analysis. Sociology, 37(3), pp. 413–431.
Morse, J.M. 1994. Emerging from the data: the cognitive process of analysis in qualitative enquiry.
In Morse, J.M. Critical Issues in Qualitative Research Methods. London: Sage.
Neuman, W.L. 1997. Social research methods: Qualitative and quantitative approaches. Boston:
Allyn and Bacon.
Richardson, L. 2004. Writing a method of inquiry. In Hesse-Biber, S. and Leavy, P. (Eds.), Approaches
to qualitative research. A reader on theory and practice. New York: Oxford University Press,
pp. 473–495.
Schatzman, L. and Strauss, A.L. 1973. Field research; strategies for a natural sociology. New York:
Prentice Hall.
Schwandt, T.A. 1996. Farewell to criteriology. Qualitative inquiry, 2(1), pp. 58–72.
Schwandt, T.A. 2007. The Sage dictionary of qualitative inquiry. Thousand Oaks: Sage.
Seale, C. 1999. Quality in Qualitative Research. Qualitative Inquiry, 5(4), pp. 465–478.
Stiles, W.B. 1993. Quality control in qualitative research. Clinical Psychology Review, 13, pp. 593–618.
Woolgar, S. (ed.) 1988. Knowledge and Reflexivity. London: Sage.
Yardley, L. 2000. Dilemmas in qualitative health research. Psychology and health, 15(2), pp. 215–228.

802 Journal of Public Administration • Vol 44 no 4.2 • November 2009