Also available from Continuum:

Richard Pring: The Philosophy of Educational Research

David Scott and Robin Usher: Researching Education
Bill Gillham: The Research Interview
Educational Research
Bill Gillham: Case Study Research Methods
Bill Gillham: Developing a Questionnaire
Marilyn Nickson: Teaching and Learning Mathematics: A Teacher's
Guide to Recent Research and its Application Contemporary Issues
and Practical Approaches
Jerry Wellington

Continuum
London and New York
14 Educational Research

It remains a mystery to me why those who work in education

should attempt to aspire towards science when scientific methods,
processes and codes of conduct at best are unclear and at worst lack
the objectivity, certainty, logicality and predictability which are
falsely ascribed to them. Surely educational research would do
better to aspire to being systematic, credible, verifiable, justifiable,
useful, valuable and 'trustworthy' (Lincoln and Guba, 1985). Approaches to research in education

There are many different approaches, types or paradigms in

educational research, with labels implying opposite poles, such as:
positivist/ interpretive; interventionist/ non-interventionist; experi-
mental/ naturalistic; case-study I survey; and qualitative I quantita-
tive. In actual research, however, there may well be a mixture or
overlap of the two approaches, e.g. survey and case-study work;
collection of qualitative and quantitative data. In addition to these
supposed contrasts, we often hear the terms 'action research' and
'practitioner research' used to describe a project or even a paradigm.
This section explores briefly some of these terms, but for a full
discussion of paradigms and approaches readers will need to dig
into some of the many references given at the end of the book.

Interpretive versus Positivist

This is one of the most common contrasts made. Positivists (and I
have yet to meet one, even amongst physicists) are said to believe in
objective knowledge of an external reality which is rational and
independent of the observer. The aim of the positivist researcher is
to seek generalizations and 'hard' quantitative data. Positivism is
often (wrongly, in my view) perceived as synonymous with 'scien-
tific'.
The concept of positivism is usually traced back to the French
philosopher August Comte (with his Cours de Philosophie Positive,
1832-42) and the ideas of J. S. Mill (A System of Logic, 1843). One of its
ideas is that true knowledge is based on the sense-perception of an
objective, detached, value-free knower. Positivist knowledge is there-
fore deemed to be objective, value-free, generalizable and replicable
(terms which are discussed later).
The sciences, it is alleged, do generate knowledge of this kind, i.e.
16 Educational Research Approaches to research in education 17

objective, value-free and independent of the knower. Thus scientific Table 2.1: Contrasting research 'approaches'
method is believed to be based on positivist principles.
Life history, biography, case Traditional, positivist approach
The positivists in the social sciences and in educational research study approach
have argued that sociology, and research in education, should follow
the methods of the natural sciences. They therefore advocate a Main emphasis Importance of the observer/ Detached, 'objective', invisible
positivist approach. I am not sure whether such people exist any author author; removal of 'the self'
The researcher Subjectivity acknowledged, Subjectivity denounced,
more, at least in the research community. However, as discussed in a
researchers put their own eschewed
later chapter, they may be lurking in the media (see their reporting of 'cards on the table'
educational research), and positivist traits may still linger among Writing Personal, collaborative writing, Anonymous, passive tone,
certain critics of educational research. account, story impersonal style
Positivists may be entitled to their opinion but they are certainly Model Eschews traditional, Attempts to mimic the natural
wrong on one count. The view that modern science is positivist (even mechanistic models of the sciences
sciences
if older science was) is totally false. Modern science cannot always Aim Search for 'personal Search for 'objective',
clearly identify and control its variables; it is not always, if ever, knowledge' generalizable knowledge;
successful at determining clear cause-effect relationships, i.e. agent X separation of 'facts' from
causes phenomenon Y, and it is rarely objective and value-free. We 'values'
Researcher's status Democratic (involves Autocratic - higher status,
only have to follow the recent debates on cold fusion, BSE and
informants and stakeholders), privilege of researcher
genetically modified (GM) foods to see this. participative, equal status of all
The interpretive researcher, however, accepts that the observer
makes a difference to the observed and that reality is a human
construct. The researcher's aim is to explore perspectives and shared collection and presentation of quantitative data is seen as a positivist
meanings and to develop insights into situations, e.g. schools, class- tendency and, therefore, not to be pursued. This results from the false
rooms. Data will generally be qualitative and based on fieldwork, polarization of the two approaches in which all the mistaken
not~s_and transcripts of conversations/interviews. A crude summary attributes of positivism are alleged to be present in any quantitative
of the two contrasting approaches is shown in Table 2.1 (there is a approach. Table 2.2 presents the resulting split between quantitative
growing literature on the more specific 'life history' approach within and qualitative approaches.
the interpretivist paradigm: two good starting points are Faraday Table 2.2 is, of course, a caricature of reality. Quantitative methods
and Plummer (1979) and Sparkes (1994)). are not always theory-laden or hypothesis-driven, and certainly
never (because they are employed by people) value-free. Similarly,
Qualitative and Quantitative Approaches qualitative research can never be complete fiction; it must depend on
Unfortunately, the critics of positivism have sometimes succeeded in some inter-subjective (if not 'objective') reality. The two approaches
throwing the baby out with the bath water. As Hammersley (1995) can complement each other. Background statistics, or just a few
very succinctly puts it: figures from available records, can set the scene for an in-depth
We must recognise that absolute certainty is not available about qualitative study. When it comes to data collection, most methods in
anything, and that attempts to produce absolutely certain educational research will yield both qualitative and quantitative data
knowledge by appeal to serve data, or to serve anything else (discussed further in, for example, Layder, 1993, p. 112). Interviews
are doomed to failure. However, accepting this does not mean can produce quantitative data; questionnaires can collect qualitative
concluding that any view is likely to be as true as any other, or data, e.g. in open response questions; case studies can involve
that anything can be true in some other frameworks if not in systematic, semi-quantitative observations.
ours. (pp. 17-18) This book is based on the premiss that methods can and should be
mixed. To use a simple analogy, if I read a report on a soccer (or
One aspect of the view which Hammersley criticizes is that the cricket, netball or hockey) match, I seek both qualitative (descriptive)
18 Educational Research Approaches to research in education 19

Table 2.2: An exaggerated polarization LIVERPOOL 2 ARSENAL 1

(Scorers: Owen, 20 mins; (Scorers: Bergkamp, 46 mins;
Quantitative Qualitative
Fowler, 89 mins) sent off: Viera, 32 mins)
Guiding principles Guiding principles Attendance: 38,411
Theory-laden (theory determines Grounded theory
practice) This may be a crude analogy, but it does demonstrate how qualita-
Hypothetico-deductive Inductive tive data gives richness and colour; quantitative data provides
Hypothesis testing Research is descriptive structure.
Replication 'Subjective'
Search for generalization Value-laden
'Objective' Naturalistic Research
Value-free Another approach, often aligned with the interpretive, and therefore
Neutral in contrast to positivism, is often labelled 'naturalistic'. Table 2.3
Data Collection/Methods Data Collection/Methods shows the main features of a naturalistic approach to educational
Numerical evidence ('hard data') Textual evidence (or image-based) research.
Observations are atemporal, asituational Researcher is the key instrument, situated In some ways this is best contrasted with the experimental
Researcher is detached from the situation in the world being studied
Researchers are part of the situation
approach because the former involves research conducted in a
Outcomes are central
Social world is like the natural world Processes of research are central natural setting or context as opposed to a controlled, clinical labora-
tory experiment. In the traditional experimental study a control
Data Analysis Data Analysis
Independent of the researcher/analyser Dependent on the researcher group is set up with features supposedly identical in all relevant
Inductive respects (an impossible goal) to an experimental group. Things are
Interpretative done to the experimental group but not the control group, e.g. they
·view of the World View of the World are taught with an item of ·n ew technology; they use a different
Reality is objective Reality is subjective, constructed teaching or learning approach; as a fictional example (I hope): the
Facts are external Researcher is central experimental group is injected on a daily basis with a wonder drug
Researcher is neutral/objective Reflexivity is vital which makes them learn more efficiently while the control group are
Findings are independent of the
given sugar lumps.
researcher
Methods Associated with ... Methods Associated with ... Table 2.3: Some features of 'naturalistic' research
Questionnaires Case studies
Surveys Observation 1 Setting Research is carried out in the natural setting or
Experimental (pre-testing, post-testing) Participant observation context, e.g. school, home, classroom,
If interviews, structure totally determined Interviews - path and structure partly playground
by researcher (interviewer totally 'in determined by participants 2 Primary data gathering instrument The researcher
charge') 3 Background knowledge Personal, tacit, intuitive knowledge is a valuable
Sampling addition to other types of knowledge
Sampling
Opportunistic 4 Methods Qualitative rather than quantitative methods will
Probabilistic
Purposive be used but not exclusively
5 Sampling Purposive sampling is likely to be preferred over
representative or random sampling
6 Design The research design tends to unfold/emerge as
the study progresses and data is collected
and quantitative (numerical) information. The reporter can wax 7 Theory Theory tends to emerge from (be grounded in)
lyrical about what a great game it was, who played well, how the the data
crowd reacted, who eventually triumphed, and whether the referee Source: Lincoln and Guba (1985); Robson (1993)
survived the ordeal. But I also require the following:
20 Educational Research Approaches to research in education 21

Practitioner Research institutions in which these practices are carried out. (Carr and
One idea which has received wide recognition in education is the Kemmis, 1986)
notion of 'practitioner research'. This is research conducted by a
practitioner/professional in any field (be they a doctor, a nurse, a This is obviously linked to the idea of practitioner research in that it
policeman, a solicitor or a teacher) into their own practice. Terms and may well involve a teacher studying, researching into or intervening
notions expressing a similar idea or research philosophy are 'the in his or her own practice, setting or system. But the key aim of action
reflective practitioner' (Schon, 1983) and 'the teacher as researcher' research is to bring about critical awareness, improvement and
(Stenhouse, 1975). Practitioner research has a number of advantages, change in a practice, setting or system. It therefore involves reflection,
some of which relate to the earlier summary of naturalistic research, planning and action as key elements.
e.g. being able to carry out research in a 'natural' setting such as one's There is quite a long history of action research in education dating
own school or classroom. It may also pose certain problems. Table 2.4 back to perhaps Lewin (1946) and Corey (1953). Some of the large
sums up the potential benefits and difficulties of practitioner amount of literature in this area published between then and now is
research. listed at the end of this book. Each discussion seems to attempt its
own diagram to show the process of action research, and many are, in
Table 2.4: Practitioner/insider research: potential advantages and problems my view, far too complicated to be of real value. The essence of the
process seems to be a spiral of cycles involving
Potential advantages Possible problems
PLANNING -ACTING - OBSERVING/EVALUATING -
Prior knowledge and experience of the Preconceptions, prejudices REFLECTING - RE-PLANNING
setting/context (insider knowledge)
Improved insight into the situation and Not as 'open-minded' as an 'outsider' and so on. My own attempt to present this as a diagram is shown in
people involved researcher Figure 2.1.
Easier access Lack of time (if working inside the Research students sometimes ask the question: 'Is what I'm doing
organization) and distractions/constraints Action Research~'?' The answer probably lies most clearly in the
II

due to being 'known' intention behind the research. If the research is conducted with a view
Better personal relationships, e.g. with 'Prophet in own country' difficulty when to changing or improving a situation, e.g. a policy, a curriculum, a
teachers, pupils reporting or feeding back management system, then it probably merits the label of action
Practitioner insight may help with the Researcher's status in the organization, research. But for some advocates of action research this description
design, ethics and reporting of the e.g. a school would probably be too broad and would include too much. A less
research inclusive definition would be to say that action research involves
Familiarity Familiarity intervening in a situation and later evaluating that intervention. This
would be part of a cycle:

Action Research identify the issue or problem ~ research it ~ suggest action ~

Another notion linked to practitioner research is the now well- implement action ~ evaluate ~ revisit the issue/problem
established concept of 'action research'. In one of the classics in this Research of a contrasting kind (though it would be derogatory to
field it was defined as call it 'non-action' research) would have the purpose of studying,
exploring or illuminating a situation - it might not be driven by the
a form of self-reflective enquiry undertaken by participants ~tention to change it. Such research would therefore not intervene in
(teachers, students or principals for example) in social (including the situation (except that every researcher or observer has an effect on
educational) situations in order to improve the rationality and the situation being studied), and would probably not manufacture or
justice of (a) their own social or educational practices (b) their create a situation, i.e. it would explore events or situations occurring
understanding of these practices and (c) the situations and naturally. (One of the best summaries of the ideas of action research
22 Educational Research Approaches to research in education 23

and resources, clarifying their suppositions and consequences,

relating their potentialities to the twilight zone at the frontiers of
knowledge' (Kaplan, 1973). The research process itself therefore
involves a scrutiny or an evaluation of methods: 'the methods we
choose are there to be tested, just as much as the substantive
hypothesis' (Walker, R., 1985a).
Although most of this book discusses methods, often in a very
practical vein, it should not be forgotten that methodology, i.e.
reflection on those methods, is a vital part of any research project,
small or large.
Box 2.1 summarizes some of the key aspects of methodology in
educational research.

Box 2.1 : Methodology

... thinking about methods, reflecting on them, evaluating them, assessing your
data ...

• Why did you use them?

• What was the quality of the data they gave you?
• Can you learn lessons, or perhaps 'generalize' from the data?
• How could your sample have been better?
• Could, or should, other methods have been' used? Why?
• How did you (the researcher) affect the data you collected?

Mixing Methods: Triangulation

Figure 2.1: The action research spiral Even in a small-scale study, a mixture of methods can often be
adopted. Schatzman and Strauss (1973) refer to such an approach as
was written by a teacher who carried out her own action research 'methodological pragmatism':
project based in Bath: see McNiff, 1992.)
The field researcher1 is a methodological pragmatist. He sees
any method of inquiry as a system of strategies and operations
METHODS AND METHODOLOGY designed- at any time -for getting answers to certain questions
about events which interest him.
Methodology?
Methodology is defined by the Shorter Oxford English Dictionary as Such a view therefore implies that qualitative and quantitative
the 'science of method' or, more historically, as 'treatise on method'. methods can exist side by side in an enquiry: 'there is no fundamental
My own interpretation of methodology is: the activity or business of clash between the purposes and capacities of qualitative and quanti-
choosing, reflecting upon, evaluating and justifying the methods you tative methods or data' (Glaser and Strauss, 1967, p. 17).
use. Indeed, the latter is an essential feature of any written report or The concept of using a multi-method approach in collecting data,
research thesis, i.e. justifying the decisions we have made on
methods. No one can assess or judge the value of a piece of research 1
Most commentators on the choice of method in carrying out an empirical study in
without knowing its methodology. Thus, the aim of methodology is social science use terms such as the field, field research, and field researcher (e.g.
'to describe and analyse methods, throwing light on their limitations Burgess, 1984 and 1985a; Fettermen, 1984). These terms will be followed here.
24 Educational Research Approaches to research in education 25

information or evidence (these terms are discussed on page 83) is be achieved by checking with the individuals that your interpretation
discussed in full by Cohen and Manion (1994) in the chapter entitled matches, and accurately reflects, their views and attitudes.
Triangulation. They discuss types of triangulation and their advan-
tages. They define triangulation as 'the use of two or more methods THE PLACE OF 'THEORY' IN ER
of data collection in the study of some aspect of human behaviour' (p.
254). The origin of the term lies in the use by navigators, surveyors, There is nothing so practical as a good theory. (Lewin, 1946,
military strategists or others involved in physical measurement, of p. 169)
several locational markers to pinpoint a certain position or objective.
In research, triangular techniques can be used to 'map out, or explain One of the perennial debates in educational research over the years
more fully, the richness and complexity of human behaviour by has concerned the status, the purpose and the function of theory. The
studying it from more than one standpoint and in so doing by matter is complicated, of course, by lack of agreement over what
making use of both quantitative and qualitative data' (Ibid., p. 233). educational theory is. The issue is complex but it is an important one
A typology.of triangulation has been suggested by Denzin (1970) for anyone involved in' educational research. The discussion of
who lists the principal types of triangulation which might be used in 'theory' is more than a theoretical matter - students, writers and
research. The types can be summarized, briefly, as follows: researchers are often accused of lacking a theoretical framework or a
'theory base' to their work.
1. Data triangulation which is subdivided into: Practical outcomes of this accusation could be the non-award of a
(a) time triangulation: the researcher attempts to consider the higher degree by thesis, the rejection by a referee of an article
influence of time using cross-sectional and longitudinal submitted for publication or the refusal of a funding body to hand
research designs; over thousands of pounds. In short, being accused of lacking a
(b) space triangulation: researchers engage in some form of theoretical base or, even worse, of being 'a-theoretical' can be,
comparative study, e.g. of different regions, different coun- practically, very serious.
tries;
(c) person triangulation at the following levels of analysis: What Is 'Theory'?
(i) the individual level; Like most problematic words, 'theory' does not lend itself to easy
(ii) the interactive level among groups; definition - and, worse, we cannot (unlike the proverbial undefinable
(iii) the collective level. elephant) always recognize one when we see one. The Oxford English
2. Investigator triangulation: more than one person examines the Dictionary shows that the word originates from the ancient Greek
same situation. idea of a 'theor', a person who acts as a spectator or an envoy,
3. Theory triangulation: alternative or competing theories are used perhaps sent on behalf of a state to consult an oracle. More recently,
in any one situation. the word theory was taken to mean a mental view or a conception; or
4. Methodological triangulation, which involves 'within method' a system of ideas used or explanation of a group of facts or
triangulation, that is the same method used on different occasions, phenomena (dated 1638 in the Oxford English Dictionary).
and 'between method' triangulation, when different methods are In the physical sciences, the distinction between phenomena/
used in relation to the same object of study. events (i.e. things which happen), laws and theories is relatively
(after Denzin, 1970) clear. A law is a statement telling us what happens in terms of a
general pattern or rule. If a metal rod is heated it expands; if pressure
This book relates mainly to research involving methodological is exerted on a gas in a container, its volume decreases (Boyle's Law);
triangulation of the latter kind, i.e. where a variety of methods are every action has an equal and opposite reaction (Newton's Third
used to study the same issue. However, another important kind of Law). Laws are simply statements of patterns or connections. For this
triangulation can be used when analysing and reporting on indivi- reason they are less tentative an'd more long-lasting than theories. I
duals' views and attitudes gleaned from surveys. Triangulation can would wager my entire life-savings that the law 'When a gas is
26 Educational Research Approaches to research in education 27

heated it expands' (Charles' Law) will be true in two centuries from in science but more debatably in educational research) can shape or
now. But the theories used to encompass or support laws are more determine the way we subsequently 'see' things. In short, observa-
tentative. tion in science is often theory-laden. The theory determines the
Theories are used to explain why specific events and patterns of observation. In the context of educational research we return to this
events occur as they do. As such, they are explanations constructed debate later - does theory determine observation and data-collection,
by human beings, and therefore subject to improvement, refinement or does theory 'emerge' from our observations or data?
and sometimes rejection, i.e. they are tentative. Finally, it needs to be noted that an established theory can predict as
Take a concrete example: if some air is trapped in a tin can and well as explain, i.e. theories may be predictive as well as explanatory.
heated, its pressure increases. This event or phenomenon is one The particle theory of matter can be used not only to explain what
instance of a general law which says that 'Gases trapped in a happens to matter, e.g. phenomena like melting or boiling, but also
container and heated will increase in pressure' (the Pressure Law). what will happen in new situations, e.g. if impurities are added, how
But why does this happen? The current theory (the Particle Theory of will boiling be affected? Similarly, the theory that the Earth's surface
Matter) tells us that everything (including a gas) is made up of tiny is rather like a jigsaw puzzle or a collection of plates (the theory of
little bits, called particles, which jiggle around all the time, get faster plate tectonics) can be used to explain not only why earthquakes
and faster when heated and bang against the wall of their container occur but also to predict future occurrences'.
harder and harder. This theory is good enough to explain why
heating gases makes them expand if they are allowed to, or just Theories in Educational Research
increases their pressure if they are trapped. It is just a theory, but it is The role of theory in educational research, just like the physical
a very good one, and has its roots in the time of Democritus a couple sciences, is to help us to understand events and to see them in a new
of millennia ago. or a different way. A theory may be a metaphor, a model or a
But Democritus' idea that matter is made up of tiny, indivisible framework for understanding or making sense of things which
particles, like billiard balls, is just not good enough to explain other happen in education. Other elements in educational research which
events and phenomena, e.g. electricity or radioactivity. These phe- are sometimes (often unjustifiably) given the name 'theory' are little
nomena required new theories at the end of the nineteenth century more than generalizations, alleged patterns, ideas or even mere
and the beginning of the last. The atomic model of that era portrayed labels.
Democritus' 'atmos' as being 'rather like' the solar system with a My own view is that a theory in educational research is only
nucleus in the middle and electrons orbiting round the outside. This worthy of the name if it helps us to explain phenomena, and thereby
model, or theory, lasted well, and still works in explaining many aid our understanding of it. It provides a new way of 'seeing' things.
events. But it has since been superseded by the quantum theory of A theory may also have predictive power as well as explanatory value,
matter and the introduction of new subatomic particles, such as although this may be expecting too much in educational research.
quarks and leptons, to explain new, observed phenomena. Similarly, Metaphors and models often fulfil at least the first criterion. To take
the theories of Newton, which work perfectly well in everyday life, examples from learning theory, we can see Vygotsky's idea of a 'zone
have been complemented by Einstein's Theory of Relativity, which is of proximal development' and Bruner' s notion of 'scaffolding' as
broader and capable of explaining at a more 'universal' level. useful metaphors. Metaphors are like bridges (the word metaphor
So what has this to do with educational research? First, theories are literally means 'carry over' or 'carry across') which link the unknown
used to explain why things happen. They are tentative, but not that or the unfamiliar to the known or familiar.
tentative (Newton was born over 300 years ago and his theories still Models are similar in that they provide highly simplified repre-
have widespread applicability and practical value, e.g. building sentations of very complex events or realities. A classic case is the
bridges; getting to the moon and back). Secondly, theories are a way world-renowned map of the London Underground: a simplification
of seeing things. They often involve models, or metaphors, which help or idealization of a messy, complicated system. But the model or map
us to visualize or understand events, e.g. the atom is 'rather like' the we use serves its purpose. Similarly, models of teaching, education or
solar system. Thirdly, the existence of an established theory (certainly the learning process are simplifications of reality. But, like meta-
28 Educational Research Approaches to research in education 29

phors, they help in making complex situations clearer, more intelli- Whether or not these criticisms are justified is discussed later. The
gible and, therefore, better understood. Piaget's model of stages of point here is: how can educational research become 'cumulative'?
development is one example: it is a simplification of reality, espe- Should theory be brought in prior to the research in order to guide it
cially if taken too literally (and wrongly) as a series of discrete, and make observation theory-laden, i.e. a priori? Or should theory
concrete steps with definite ages attached to them. But it has great 'emerge' from data collection and observation and be developed
value in explaining conceptual progress and children's development, from it, i.e. inductively, a posteriori?
especially when it is related to curriculum demands (see for example, On the one hand, Anderson (1990) urges that 'in your study and
Shayer and Adey, 1981, and subsequent work by Adey). My guess is prior knowledge you should attempt to identify appropriate theore-
that many teachers apply Piaget's ideas unwittingly in their own tical and conceptual frameworks which bare [sic] relation to your
practice and staffroom discussion. problem' (p. 47). He counsels researchers to ground their research in
One final point in ending this sub-section concerns the use of labels antecedent work which has 'generated contemporary constructs
which have emerged in educational research. For example, Shulman guiding subsequent investigation', i.e. data collection will be theory-
(1987) has identified and labelled different categories of 'teacher guided or theory-laden to use the term from science.
knowledge' which teachers draw upon in their practical teaching. An apparently opposite approach is to generate theory (induc-
These are often labelled (though Shulman's actual categorization is tively) from the data. Theory 'emerges' as the data collection
more refined) as 'subject knowledge' (SK) and 'pedagogical content progresses and is firmly 'grounded' in it, and derived from it, i.e. a
knowledge' (PCK). The latter includes teachers' knowledge of posteriori. This approach is often called 'grounded theory' (after
explaining, putting things across, pedagogy, breaking down complex Glaser and Strauss, 1967).
ideas into simpler steps, and so on. Generally, it relates to the art, So the crucial questions are: Should categories, patterns or theories
craft and wisdom of teaching. Shulman's ideas have great application be generated from the data, or should they be imposed upon it? How
in considering initial teacher education, mentoring, professional can research be 'cumulative' if it does not use previously determined
development and other areas. But are they theories? My own view categories? Do researchers have to recreate theory every time they
is that they do help us to understand the above areas, and underlying collect and analyse data?
the labels are valuable conceptualizations or categorizations. They These are complex and important questions. But the simple answer
have some explanatory value and, perhaps, even predictive power. is that it depends on the nature of the research, its purpose and the
A similar discussion could be held over Schon' s idea of the area being investigated. In some fields there are ample theories,
'reflective practitioner' (Schon, 1983), Willis' notion of 'the lads' sufficiently well developed, and it would be wrong not to use them in
(see Appendix 1), the label 'vocationalism' (applied by many authors shaping research design and data collection. In others there may be a
to the growing links between schooling and industry I employment shortage of suitable theory, or it may be extremely tentative, thus
in the 1980s), or the notion of the 'hidden curriculum'. Perhaps in implying a different approach. Similarly, with the purpose of a
the end it is a semantic debate over whether they are theories or research project, a key aim of a project may be to replicate previous
not. research in order to lend support to a theory, or perhaps to attempt to
refine it. In others, the aim may be to develop new, tentative theories
When Does Theory Come in: a priori or a posteriori? which, perhaps, subsequent researchers might build upon.
The key question for those engaged in, or about to embark on, These are all issues which we return to later in considering research
educational research is not whether theory should make its entry but approaches, 'paradigms', methodology and methods.
when. One of the recent criticisms of educational research (Chapter
12) is that new research is not always based on previous work, i.e. it is TWO CONTENTIOUS TERMS: VALIDITY AND RELIABILITY
'non-cumulative'. It is argued that, in turn, this has led to the failure
of educational research to create a sound, reliable body of knowledge These are two terms which have been widely used in discussing
which can inform practitioners and ultimately improve education research, not least educational research. They also tend to be abused,
(as, allegedly, medical research has done with medical practice). partly because they are difficult to define and to understand. For
30 Educational Research Approaches to research in education 31

example, the terms are often, especially in conversation, used to know reality by observing it or measuring it; and how can we know
signal approval. Thus people may say, in meetings, for example: that our measurement or observation corresponds to reality?
'That's a valid point', meaning no more than that they agree with it. There is no higher court of appeal to which we can turn. We only
Similarly, people (including the media) may describe a piece of have our perceptions - the only way we can judge whether our
research as 'reliable', meaning that they approve of it and/ or trust perceptions match reality is to appeal to our perceptions. We are
the person or team who conducted it. The two words do have caught in a circular trap - the only saving grace is that we are all
technical meanings, however, and I will attempt to define them and caught in the same trap. Hence the importance of sharing, commun-
also to give a loose intuitive meaning for them here. ication, inter-subjectivity and mutual control.
The problem of external validity is no more soluble - this is an
Validity assessment of the degree to which our observations or measurements
Validity refers to the degree to which a method, a test or a research can be generalized from, i.e. extended to other 'external' groups or
tQ_ol actually measures what it is supposed to measure. For example, domains which have not been observed or measured.
in the old debate on IQ tests the main issue was whether the tests Le Compte and Preissle (1984) explain this clearly, although their
actually did measure what they claimed to measure, i.e. intelligence. use of the word 'scientific' is unnecessary:
Does our ability to do an IQ test measure our intelligence, or does it
Distinctions are commonly drawn between internal and external
simply measure our ability to do an IQ test?
validity. Internal validity is the extent to which scientific obser-
There are three important points here which apply across the
vations and measurements are authentic representations of
whole of education and educational research.
some reality. External validity is the degree to which such
First, we can never be 100 per cent sure of validity. We can only lay
representations may be compared legitimately across groups.
some sort of claim that our test or method is valid. The only claim
(p. 323)
that we can make with certainty is the circular or tautologous one
that, for example, a person's ability to do a test, whether it be of We can never claim to be sure of either. To do so would be to commit
numeracy, literacy, intelligence or spatial awareness, measures their what the philosopher David Hume called the 'fallacy of induction'.
ability to do that test on that day at that time under those conditions.
Hence the issue of reliability. Reliability
Secondly, any discussion of validity rests squarely on the founda- The term 'reli9 bility' is equally contentious. This is a judgement of the
tion of how the characteristic being measured is defined. Thus extent to which a test, a method or a tool gives consistent results
'intelligence' may be defined in a certain way and this may then across a range of settings, and if used by a range of researchers. It is
increase the validity of something which sets out to measure it. We linked to the idea of 'replicability', i.e. the extent to which a piece of
could even complete the circle the other way by defining intelligence research can be copied or replicated in order to give the same results
as: 'the ability to succeed in an IQ test'. It would then follow that in a different context with different researchers.
those tests will have 100 per cent validity. However, this would seem Le Compte and Preissle (1984, p. 332) define reliability and claim
a somewhat vacuous way to proceed. But if we treat 'intelligence' as that no researcher studying the social world can achieve total
a highly problematic term (e.g. by adopting the model that people reliability. They describe it as
have a range of intelligent abilities or 'multiple intelligences') then a
the extent to which studies can be replicated. It assumes that a
traditional IQ test becomes invalid. In other words, any assessment of
researcher using the same methods can obtain the same results
validity depends heavily on the definition or meaning of the term
as those of a prior study. This poses an impossible task for any
underlying it, and many of these terms in education are extremely
researcher studying naturalistic behaviour or unique phenom-
problematic: understanding, ability, achievement, numeracy, lit-
ena.
eracy, learning, development, knowledge, IT literacy, and so on.
Finally, there is an essentially insoluble problem of internal I would concur with this view - but the consolation is that current
validity in research and in all our knowledge generally. We can only philosophers and sociologists of science are increasingly sceptical
32 Educational Research Approaches to research in education 33

about the possibility of total reliability and replicability in modem involved in a given study, making data collection, interpretation and
scientific research (see, for example, Collins, 1985 and Woolgar, 1988 therefore extrapolation, problematic.
as early examples of this scepticism). Schon' s discussion of problems in public learning draws examples
As for an intuitive 'feel' for reliability and validity, the best analogy mainly from areas of public planning and policy, but his conclusions
I know is the situation in which a group of people attempt to measure are relevant to the attempt to gather data in the fields of education
the depth of an empty swimming-pool with an elastic, stretchy tape and training (i.e. the difficulties of access, perception and datedness
measure. They do not realize that the swimming-pool has a deep end in gathering data; the problem of interpretation; the impossibility of
and a shallow end. They each take measurements at different points designing an 'experiment'; and the danger of extrapolation). These
along the pool believing that this is the average depth, so their inherent difficulties are all taken as starting points for the discussion
measurements are invalid (they are not measuring what they think that follows.
they are measuring). In addition, some measurers stretch the measur-
ing tape more than others - the elasticated ruler is unreliable. The
researchers are unreliable in that they cannot all be relied upon to
hold the ruler at exactly the same tension (understandably).

A FEW CAUTIONARY NOTES

Some of the more general problems inherent in any study of human
organizations and societal patterns were discussed at length by
Schon (1971), and are worth considering in this context before
embarking on a discussion of method. In a chapter entitled, aptly,
'What can we know about social change?', Schon pointed to four
problems 'inherent in public learning': gaining knowledge, the status
of data, designing an experiment and extrapolating from results.
Schon argues that one of the problems in gaining knowledge of
social change is that 'data gathering is a political process' (p. 207).
This raises problems of access (in Schon's example, of his entry into a
black neighbourhood) and of the perceptions of those doing research.
The second problem is that 'data may not endure' - and may
become out-of-date almost as soon as it is collected. Schon goes on to
discuss the problem of diagnosing and interpreting data. People will
interpret data differently according to their own personal perspec-
tive.
The next problem in 'public learning', as Schon calls it, is the great
difficulty of designing any scientific or even systematic experiment in
studying society: 'it is almost never possible to hold some variables
constant while manipulating others' (Ibid.).
Finally, Schon points to the fourth main difficulty in studying
social change: the problem of extrapolating or generalizing from the
results of any study. This is linked to the problems inherent both in
gathering data and interpreting it. A number of perspectives will be