Observation and Document Analysis
Observation and Document Analysis
Introduction
The last Element introduced you to a range of methods of collecting
qualitative data. This Element will continue that theme by looking at two
more ways of collecting data and at what you can do with that data once
you have collected it. This Element is divided into 3 sections.
• Section 1: observation
• Section 2: making use of documents from the research site.
• Section 3: analysing the resultant data.
5.1
ELEMENT 5 Observation and document analysis
Section 1: Observation
The first major point to note is that schools and colleges are busy places.
A great deal can be observed there. The first thing you must decide is,
'What am I going to observe?'. As always, such decisions will be
determined by the underlying research purpose which requires that some
form of observation needs to be undertaken. The sort of research
questions where some form of observation would be appropriate are:
• Do children in mixed classes behave differently from those in single
sex classes?
• What sorts of interaction takes place between teachers and pupils?
• What proportion of teacher time is spent on task?
• At what time in the day/week are teachers most busy and what do
they do at these times?
See your answers None of these questions can be sensibly explored without observing the
to activity 1 page situation. Remember that observation is particularly important because
2.6 it has the potential to give us insights into what people actually do rather
than what they say they do or what they think they do. In Element 2 you
determined your research topic and went on to decide your research
question(s) and the issues that your study would explore. Look back at
these now and identify what particular aspects could usefully be
explored using observation. Remember that observation is only one of a
number of techniques that may be used to collect data. Remember too
that it would be unusual, indeed unwise, to rely on only one method of
data collection and that observational work should be used to
complement other methods.
R Reading
Have a look at Chapter 7 of Hopkin’s book – methods of observation in
classroom research
T Task 1
Make a brief list here of those aspects of your research
question(s) that you feel are most appropriate for observation.
F Feedback on Task 1
5.2
ELEMENT 5 Observation and document analysis
Recording Data
See Element 3 Given that the research purpose defines the focus, the next question is,
'How will I record my observational data?' There are, essentially, four
main methods of collecting information.
• via memory
• via written notes
• via pre-determined schedules
• via mechanical devices – audio/video tape, camera
R Reading
Read page 188 of Cohen, Manion and Morrison’s book for some good
tips on recording observational data.
5.3
ELEMENT 5 Observation and document analysis
Participant observation
Element 6, Section Whatever form of recording you use, any visit to a teaching environment
3 goes into validity will disrupt the very activity you wish to observe. This creates problems
in greater detail of validity. In other words, does the situation you are observing describe
or reflect what it is supposed to describe or reflect? This effect is likely
to be strongest (and your data least reliable) when you first start your
observation. It is best to make your observation a regular event, so that
whatever you observe can settle down to being 'normal'. So remember
not to rely on initial data. Wait until your participants have, at least to a
degree, accepted the situation.
5.4
ELEMENT 5 Observation and document analysis
within their normal social context. Of course this can go too far.
Anthropologists have a term, 'going native’, to describe a researcher who
gets so involved and active with subjects that their original intentions get
lost.
Think about your own research project and consider carefully where on
the participant-observer continuum you wish to be when you are doing
your own observation. You could, for example, carry on your normal
duties whilst making notes occasionally in a hard-backed book
(emphasizing the participant role). Alternatively, you could take a ‘fly-
on-the-wall’ approach and using a structured observation record what is
happening on a clipboard for a designated period of time (emphasizing
the observer role).
T Task 2
5.5
ELEMENT 5 Observation and document analysis
F Feedback on Task 2
5.6
ELEMENT 5 Observation and document analysis
Observation schedules
You have made some very important decisions by this stage but it is
worth remembering that they are not written on tablets of stone. You can
alter them when further information is taken into consideration. Before
you carry out your observation:
• use your research question to decide the focus of your observations
• give some thought to the sorts of observations that will be useful to
your research
• before you collect any data, make sampling decisions, i.e. who, what,
where, and when to collect data. Think carefully about these since in
your final report you will need to justify why, for example, you have
selected those particular people to observe at that particular time.
Finally, you should decide how you intend to record the event you wish
to observe and you should have your memory, notebook, schedule, tape
recorder, camera and/or camcorder at the ready.
5.7
ELEMENT 5 Observation and document analysis
Observation schedules are not easy to use and some practice is required
in order to understand their strengths and weaknesses, and how best to
apply them. The next task will give you some practice in this.
T Task 3
We want you to collect observational data for this study using the
schedule on the next page. But before you start, think about how
you are actually going to do this in practice. If you choose to tell
the administrators exactly what you intend to observe then you
would probably find that they all work diligently on task for the
period of observation. Your data may not be valid or reliable. But
suppose you choose not to tell them what you are doing. If you
then stand over them with pen poised over clipboard, they are
going to wonder what you are doing and you will be in an ethical
Have they gone quandary.
yet?
One way out of this impasse would be to ask for their permission
to observe them for a short amount of time, for example 10
minutes. You could (correctly) say it is for a course you are doing
and you could state in general terms what you will be observing,
for example ' I want to get a picture of the sort of things you do
during an ordinary day’. The process of obtaining permission to
collect data is called accessing and the first practical step in
obtaining your data is always to gain access to people, places or
events, to get permission to observe or interview people.
For this particular task try to find a friendly administrator who will
be willing to act as a '
guinea pig’ for you - but if this is impossible
look for anyone who is carrying out a specific task. It does not
matter much what the task is. Anyone carrying out any task is
likely to spend some time off-task.
All you have to do is scan one person (or more if you feel
confident) over a period of ten minutes noting down what they are
doing in the categories which follow:
5.8
ELEMENT 5 Observation and document analysis
As an example of the sort of data you will obtain the schedule for
administrator Joan is given:
Administrator:
Time (minutes) 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th
Off-task activity
We can see that Joan was on task for 4 of the 10 minutes (during
the 3rd, 4th, 8th and 10th minutes) and that for the remaining six
minutes Joan was off-task. Remember to include an ‘other’
category (here it was no. 8) in case some observations cannot be
placed in the designated category list. It is worth noting that some
categories are not easy to apply, for example, during the second
minute how can we be sure that Joan is daydreaming since she
may have been thinking and therefore on task.
Now you try. Carefully select someone and use the schedule to
collect data on one subject. Use the space below to record your
findings.
F Feedback on Task 3
T Task 4
When you have absorbed this material, select and observe some
event related to your research topic. You may wish to carry out
5.9
ELEMENT 5 Observation and document analysis
more than one observation which will provide you with insights or
answers to your research question. Record your findings here.
F Feedback on Task 3
Section 2: documents
Documents broadly include any papers, especially official ones, which
provide more or less direct evidence of decisions, transactions, status,
thoughts, debates or actions, which are directly or indirectly related to
the purpose of a research inquiry. Documents may be contemporary or
historical in nature and include both text and visual data. Documents
which are considered primary sources of data would include, for
example, annual reports, minutes of meetings, personal diaries,
memoranda, records, letters, files, institutional prospectuses, videos,
photographs, diagrams, or catalogues.
Documents are a useful source of evidence but they have their
limitations. One problem with them is that we may not know how they
came into being in the first place or who wrote them. Another problem is
pointed out by Andrew (1985) who makes an important point with
regard to ‘complete’ records:
T Task 5
About 40 minutes Think about your research question again. Make a list of the sorts
work of document you could find within your working environment which
5.10
ELEMENT 5 Observation and document analysis
The next step is to ask a range of questions of the documents before us.
We could begin by making the following kinds of quantitative analysis:
• approximately how many words are there per page?
• how many paragraphs are there per page?
5.11
ELEMENT 5 Observation and document analysis
Observational data, like other forms of qualitative data, are not looked
upon favorably by all researchers, particularly when viewed from a
scientific perspective. Cohen, Manion and Morrison (2000) say:
5.12
ELEMENT 5 Observation and document analysis
5.13
ELEMENT 5 Observation and document analysis
on this particular issue. So, as you can see, researchers make sense of
their data soon after they have collected it. This enables them to refine
the focus of their study
The ongoing analysis of data gathered through observations and field-
notes is used to help progressive focussing and in generating grounded
theory which explains the phenomena observed. Progressive focussing
is the continual funnelling process which limits and clarifies the scope,
subject and structure of a research project. In this process it is not
uncommon for the initial research question to be transformed almost
beyond recognition. The fieldwork may increasingly reveal that this
question simply did not address the issue that was fundamental to the
problem under review. Grounded theorising is the ongoing process of
developing hunches, ideas and interpretations of the data. These serve to
guide data collection and are refuted, or substantiated and developed, in
the light of the new data collected. Theory building and data collection
can therefore be viewed as interrelated.
Remember that it is important in doing this kind of analysis to be alert to
the danger of 'going native'. This involves getting so caught up in the
events that you are supposed to be researching that you fail to examine
them critically and take for granted the very phenomena that should be
regarded as problematic. Teachers and others working in the education
system carry out research into aspects of their own work and this has
helped to improve practice. However, practitioners frequently take their
everyday practice for granted and they may be blinkered by prior
assumptions and personal values. This is why it is vital to use a
structured approach (a research approach) to making sense of data. A
way of avoiding the charge of being blinkered by prior assumptions is by
searching for exceptions or negative cases which disprove previously
held or emerging hypotheses - a process referred to as analytical
induction in the methodology literature. It is a useful exercise to try to
find ways of making the familiar strange or providing yourself with
different frameworks for thinking about your data.
It is important not to put off analysing your data until after you have left
the field. There are two good reasons for this. First of all, you will be
collecting quite a lot of data and if you leave analysis until the last
moment you will find it a daunting, if not psychologically damaging,
process. The second reason is that analysis in the field helps focus
further data collection and refine emerging ideas and concepts.
Therefore, as your data come in, try to make sense of them as soon as
possible. You will probably find it helpful to adopt the following
approach.
• Force yourself to make decisions about the type of study you want to
accomplish.
• Force yourself to make decisions that narrow the study.
• Develop analytic questions.
5.14
ELEMENT 5 Observation and document analysis
• Plan data collection sessions in the light of what you have found in
previous data collection.
• Bring together in a field diary what you are learning from the various
sorts of data you collect.
• Try out your ideas on key informants.
• Begin exploring the literature while you are in the field.
Now you can go on to analyse your data. You will find this easier if you
follow the steps outlined below. But remember that you have already
done some preliminary analysis. You have already made decisions about
what to observe, what questions to ask, and whom to interview.
5.15
ELEMENT 5 Observation and document analysis
2. The second stage is to look in more detail at the emergent pattern. Are
there any sub-patterns? For the above example, what forms of
anxiety have been expressed and how is each one constituted? Are
some of these forms of anxiety more widespread than others?
5.16
ELEMENT 5 Observation and document analysis
T Task 6
S Summary
In this Element we have considered three important issues. The first two
sections were concerned with data collection techniques – observation
and documents – and both approaches are capable of yielding good
quality evidence. We also looked at how to record data, in particular the
limitations and strengths of each of the approaches. Finally, we
discussed analysis, initially in general terms and later in a more
pragmatic way.
Analysis and making sense of data collected are fundamental to the
research enterprise. They are also perhaps the most difficult aspects of
the process. Many researchers, even experienced ones, struggle with
interpreting the data. It is natural that you too may at first find this aspect
difficult and frustrating. Don’t be scared of ‘wrestling with the data’. The
most important thing to remember is that you must immerse yourself in
the data as soon as possible. Do not put off analysing them.
A good way of forcing yourself to analyse is to start writing. The act of
writing, of putting your ideas down on paper, forces you to organise
them – to analyse them.
R References
5.17
ELEMENT 5 Observation and document analysis
5.18