Teaching Health Statistics Lesson and Se

Download as pdf or txt
Download as pdf or txt
You are on page 1of 114

Please see Table of Contents for access to the entire publication.

Teaching
health statistics
LESSON AND SEMINAR OUTLINES
Second edition

Edited by
S.I{. Lwanga Cho-Yook Tye o. Ayeni
Department of Formerly Department of Special Programme of
Health Systems, Social Medicine and Research, Development and
World Health Organization, Public Health, Research Training in Human
Geneva, National University of Singapore, Reproduction, Department of
Switzerland Singapore Reproductive Health
and Research, World Health
Organization, Geneva,
Switzerland

World Health Organization


Geneva
1999
Dedicated to the late C. R. Lowe, C.B.E., Emeritus Professor of Community Medicine,
Welsh National School of Medicine, Cardiff Wales
Please click inside the blue box below to access the chapters within.

Contents

Preface vii
Introduction ix

Part I Statistical principles and methods 1


Outline 1 Introduction to the role of statistics in health sciences and health care delivery 3
Outline 2 Health data: sources, levels and quality of measurement 11
Outline 3 Health information systems 23
Outline 4 Organization and presentation of data 30
Outline 5 Measures of central tendency and location 43
Outline 6 Measures of variability 51
Outline 7 Introduction to probability and probability distributions 58
Outline 8 Sampling and estimating population values 66
Outline 9 Tests of statistical significance 79
Outline 10 Association, correlation and regression 91

Part 11 Health statistics, including demography and vital statistics 103


Outline 11 Censuses and vital registration 105
Outline 12 Measurement of morbidity 114
Outline 13 Measurement of mortality 122
Outline 14 Measurement of fertility 133
Outline 15 Population dynamics 138
Outline 16 Indicators of levels of health 145

Part III Statistics in medicine, including medical records 155


Outline 17 Medical records and health facility statistics 157
Outline 18 International Classification of Diseases (ICD) and certification of causes of death 169
Outline 19 Design of health investigations: health surveys and clinical trials 178
Outline 20 Use of computers in health sciences 191
Outline 21 Rapid methods for interim assessment 198
Outline 22* Statistical and medical ethics 203
Outline 23* Critique of a scientific paper 211

Annex A Supplementary data sets 215


Annex B Statistical tables 220
Annex C Random numbers 223
Index 225

* Seminar.

v
Preface

The need for a statistical approach is now well recognized in epidemiology and
public health, since these fields are concerned with communities or populations
where the laws of large numbers and random fluctuations clearly apply.
Teachers of health workers and students, however, have been slow to recognize
the need for a knowledge of statistics, even though all aspects of diagnosis and
prognosis are affected by rules of probability.
This book is intended to contribute to the long-term reorientation of the health
information systems of Member States, by bringing about improved data
generation, handling, processing and use, in order to meet future health
requirements.
The extent of statistical knowledge and skills that students need to acquire var-
ies from country to country, according to such factors as the common health
problems and methods of delivering health care in the country, and the career
prospects of the students on graduation. Nevertheless, there is a core of statisti-
cal knowledge that all students need to have, irrespective of their country of
training.
The present set of outlines is a revised version of Teaching health statistics: twenty
lesson and seminar outlines (Lwanga SI< & Tye C-Y, eds. Geneva, World Health
Organization, 1986). The topics covered form an internationally acceptable stand-
ard basic curriculum for teaching health statistics. While based on those of the
first edition, the lesson and seminar outlines have been revised and updated in
both content and orientation. They cover not only the conventional topics of
data collection, presentation and analysis, probability and vital statistics, but also
such topics as health indicators, use of computers and rapid methods of interim
assessment. The concepts highlighted by the outlines should be useful to all
students in the health field, and are meant to be used selectively by teachers of
statistics in preparing their courses.
This new edition is a result of close collaboration between a number of
eminent teachers of statistics, and has been coordinated and edited by
Mr S. K. Lwanga, Statistician, Department of Health Systems, World Health
Organization, with valuable assistance from Dr o. Ayeni, Biostatistician, Special
Programme of Research, Development and Research Training in Human
Reproduction, Department of Reproductive Health and Research, World Health
Organization.
The preparation of the first edition of this book was conceived by Dr Boga
Skrinjar-Nerima while she was Chief Medical Officer at the World Health Or-
ganization in charge of the development of health statistical services. Her contri-
bution is still highly appreciated.

vii
PREFACE

viii

The World Health Organization wishes to thank the following eminent teachers
who made invaluable contributions to this edition of lesson and seminar out-
lines: Professor E. Bamgboye, Department of Family and Community Medicine,
College of Medicine, King Saud University, Riyadh, Saudi Arabia; Professor R.
Biritwum, Department of Community Health, University of Ghana Medical
School, Accra, Ghana; Professor A. Indrayan, Division of Biostatistics and Medi-
cal Informatics, University College of Medical Sciences, Delhi, India; and Profes-
sor 1(. Surnbuloglu. Department of Biostatistics, Hacettepe University, Faculty of
Medicine, Ankara, Turkey. Thanks are also due to all the teachers and colleagues
who contributed to the first edition of the book or reviewed the various drafts of
this revised version.
This publication is specially dedicated to the memory of the late Ron Lowe, C.B.E.,
Emeritus Professor of Community Medicine, Welsh National School of Medi-
cine, Cardiff, Wales, who helped and guided the Organization's efforts towards
the improvement of the teaching of statistics and their use in epidemiology and
public health.
PART I

Statistical principles
and methods
OUTLINE 1 Introduction to the role of statistics in
health sciences and health care delivery

Introduction to the lesson


Statistical methods are consciously or subconsciously applied in health care delivery at the com-
munity and individual patient levels. At the community level, they are used to monitor and
assess the health situation and trends, or to predict the likely outcome of an intervention pro-
gramme. At the patient level, they are used to arrive at the most likely diagnosis, to predict the
prognostic course and to evaluate the relative efficacy of various modes of treatment. Knowl-
edge of statistics is also essential for a critical understanding of the medical literature. Statistical
principles are essential for planning, conducting and interpreting biomedical, clinical and com-
munity health research.

Objective of the lesson


The objectives of this lesson are to introduce the students to the role of statistics in the health
sciences, health care delivery, the study of human populations, and the management of uncer-
tainty. The lesson also aims to create an awareness of the need to acquire an understanding of
statistical principles and methods.

Enabling objectives
At the end of the lesson the students should be able to:

(a) Explain various meanings of the term statistics.

(b) Indicate, through examples (without necessarily going into great detail), how statistical
principles and concepts are relevant in the following situations:

• handling of variation in characteristics (for example, physiological or chemical) encoun-


tered in the field of health care;

• diagnosis of patients' ailments and health problems of communities;

• prediction of likely outcomes of disease intervention programmes in communities or of


diseases in individual patients;

• selection of appropriate forms of treatment for individual patients;

• public health administration and planning;

• planning, conducting, analysing, interpreting and reporting of medical research.

(c) List sources of uncertainty in health sciences and health care delivery.

(d) Describe the role of statistics in the management of uncertainty in health sciences and health
care delivery.

3
TEACHING HEALTH STATISTICS

(a) As an introduction, discuss the general and specific objectives of the course as a whole,
making it clear that it is not intended to produce health statisticians, but health workers who
will be able to make rational decisions in their work. Emphasize the use of statistics as a tool
rather than as an end. Give an overview of the course, its structure, organization, teaching
methods and timetable.
(b) Explain the meaning of "statistics" and "statistical methods", giving examples of their appli-
cation in health care. Explain the need for data in decision-making. Hence, explain the
importance of the study of survey design, instrument calibration, and data collection, process-
ing, analysis, presentation, interpretation and communication.
(c) Discuss the problems posed by variation and uncertainty in: the study of disease etiology,
causation or risk factors; the evaluation of response to treatment; the determination of "nor-
mal", "usual" and "ideal" values of characteristics; and hence the methods needed to handle
them.
(d) Explain the essential role of statistics in the field of health (for example, in acquiring and
using medical knowledge, and in medical practice). Use examples to show how decisions are
made by health workers in the course of their duties (for example, in making a diagnosis,
assessing prognosis and deciding on the correct treatment for a patient), and by health ad-
ministrators, planners and evaluators.
(e) Point out the widespread use of statistical methods in medical journals. Progressive health
workers depend, to a considerable extent, on literature to update their knowledge. Some-
times the handouts distributed by pharmaceutical firms also contain statistical results.
Readers, therefore, need to have the ability to evaluate the validity and reliability of the
information in these reports. They also need to be familiar with the basic technical language
of the statistical and epidemiological methods which are commonly used in the medical
literature. Health workers themselves may have to use this language in the reports of their
work.

Lesson exercises
Lesson exercises should test the students' ability to describe the importance and uses of statisti-
cal methods in the field of health, and should give as many examples as possible. The exercises
should, therefore, be of such a nature as to elicit from the students examples of the need for
statistics and their use in solving health problems.

• What statistics would be required to decide whether to build a clinic for a


village in a district ?

• Give four areas in health care delivery where the science of statistics is
applied.

• Describe the importance of including somebody with knowledge of statistics


in a District Health Management Team (DHMT) which has the responsibility of
health services development in a district.

References
Bland M. An introduction to medical statistics. Oxford, Oxford University Press, 1987.
OUTLINE 1 ROLE OFSTATISTICS IN HEALTH SCIENCES AND HEALTH CARE DELIVERY

Dixon RA. Medical statistics: content and objectives of a core course for medical students. Medi-
cal education, 1994, 28:59-67.
Feinstein AR. On teaching statistics to medical students. Clinical pharmacology and therapeutics,
1975, 18:121-126.

Feinstein AR. Clinicalbiostatistics. St Louis, Mosby, 1977.


Gore 5M, Altman DG. Statistics in practice. London, British Medical Association, 1982.
Indrayan A. Changes needed in style and content of teaching statistics to medical undergradu-
ates. Internationaljournal of mathematical education, science and technology, 1986, 17:95-102.
Knapp RG, Miller RC. Clinical relevance: an issue in biostatistical training for medical students.
Medical education, 1987,21:32-37.
HANDOUT 1.1

Sources of uncertainty in health and medicine

(a) Uncertainty is caused by variations in:

• biological factors (for example, age, sex, birth order, heredity);


• environmental factors (for example, nutrition, addictions, stresses, water supply, sanitation, socioeconomic
status, availability and use of health facilities);
• methodological factors (forexample, relating to observers, instruments, laboratory techniques, chemicals and
reagents, questionnaires or record forms, diagnostic tools such as X-ray);
• chance and unknown factors (forexample, difference in birth weight of identical twins, or varying results from
repeated samples of blood, urine and tissue).
(b) Other sources of uncertainty include:

• incomplete information on the person or patient (patient in coma, lack of facilities for medical investigations,
illiteracy, recall failure, etc.);
• an imperfect tool (false positive and false negative results of laboratory and radiological investigations, clinical
signs and symptoms are sometimes not specific, lack of accepted measure of important concepts such as
community health, etc.);
• poor compliance with the prescribed regimen (non-compliance with treatment schedule, imperfect post-
surgical care, breakdown of a vaccine cold chain, non-acceptance of family planning advice, etc.);
• inadequate medical knowledge (lack of treatment for AIDS, unknown causes of many cancers, inability to
restore severely malignant tissues, lack of a universally applicable cheap and effective method to break the
parasite-vector-host cycle in malaria transmission, unknown specific factors causing women to live longer
than men, unknown relationship between the mind and physiological and biochemical mechanisms, etc.).

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 10


OUTLINE 2 Health data: sources, levels and quality
of measurement

Introduction to the lesson


The systematic and continuous process of health policy formulation, planning, program-
ming, budgeting, implementation, and general integration of different programmes within the
overall health system depends on good information support. The types of data gathered and
the analysis applied to them depend on the potential users and the kind of information they
are likely to need. The quality of the information given depends on the sources of the data,
how they are collected, the instruments (equipment, recording forms, etc.) used for data
collection, and the statistical methods used for analysis. In addition, the amount of information
that can be obtained from the data and the choice of statistical methods for the analysis
depend in part on the scale or level (nominal, ordinal or interval) on which the data have been
measured.

Objective of the lesson


The objective of this lesson is to enable the students to understand the nature, sources, types
and collection of data needed for the planning and management of health programmes and
activities.

Enabling objectives
At the end of the lesson the students should be able to:

(a) Describe the possible sources of health data.

(b) Distinguish between regular and ad hoc health data collection systems.

(c) Describe the procedures for health data collection.

(d) Discuss the major differences between the three data measuring procedures:

• instrumental- measurement done technically, without human intervention in the deci-


sion on the value of the measurement;

• human - measurement done by persons who decide on the measurement value;

• by interaction between humans and equipment.

(e) Explain the concepts of reliability and validity with regard to measurement, and discuss
their implications for the use of health data.

(f) Distinguish between the four principal scales of measurement (nominal, ordinal, interval
and ratio), indicating their respective application for health data collection.

(9) Distinguish between quantitative and categorical data.

(h) Distinguish between random errors and fluctuations.

11
OUTLINE 2 HEALTH DATA

17

Examples:

• Lack of proper food for nursing mothers and children in a drought-stricken


area may be classified as critical, severe, moderate or slight.
• Social status of patients may be measured as upper, middle or lower class.
An interval scale is characterized by a numerical unit of measurement, such
that the difference between any two measurements is explicitly known in
terms of an interval between the two measured points. The unit of measure-
ment and the zero point (the origin or starting point) of the scale interval are
arbitrary and only fixed by convention. Example: body temperature is usually
measured on an interval scale, of which the unit may be, for example, degrees
Celsius (QC).

Although the ordinal scale can be transformed into a pseudo-interval scale


by assigning scores to its measurement categories, it retains the qualities of an
ordinal scale.
The ratio scale has all the characteristics of the interval scale, as well as a true or
absolute zero, so that the ratio between two values on the scale is a meaningful
measure of the relative magnitude of the two measurements. Examples: height
in metres; weight in kg.
Certain arithmetical operations are permissible on each scale.
Nominal scale. The arithmetical operation of "equivalence" is permissible on this
scale. For example, one "woman" is equivalent to another "woman". Equivalent
measurements can be aggregated into a particular category and counted. Pro-
portions belonging to each measurement category out of the total number meas-
ured can be calculated.
Ordinal scale. On this scale one measurement can be equal to another (equiva-
lent) or described as greater (higher) than or less (lower) than the other. The
difference between one measurement and another is not explicit, and differ-
ences between adjacent measurements are not equivalent. Again, equivalent
measurements can be aggregated into a measurement category and counted,
and the proportion in each category calculated.
Interval scale. Arithmetical operations permissible on this scale include all those
allowed on the ordinal scale; in addition, measurements can be added, subtracted,
divided and multiplied by a constant, to yield interpretable results. Comparison
between intervals on this scale is meaningful, and is independent of the unit of
measurement or the system of assigning scores.
Ratio scale. All arithmetical operations are permissible, and the ratio of any two
measurements is meaningful and independent of the unit of measurement.

Quantitative and categorical data


Data can be divided into two broad categories according to the strength of the
scale of measurement: categorical data and quantitative data.
Categorical data are measurements in which the notion of magnitude is absent
or implicit. Such variables are measured either on a nominal or an ordinal scale.
These data are also referred to as attributive or qualitative.
OUTLINE 3 Health information systems

Introduction to the lesson


A health information system (HIS) provides information for the management of a health pro-
gramme or system and for monitoring health activities. A HIS is made up of mechanisms and
procedures for acquiring and analysing data and providing information (such as management
information, health statistics and health literature) needed by:
• all levels of health planners and managers for the planning, programming, budgeting, moni-
taring, control, evaluation and coordination of health programmes;
• health care personnel, health research workers and educators in support of their respective
activities;
• national policy makers, socioeconomic planners and the general public outside the health
sector.

Objective of the lesson


The objective of this lesson is to provide the students with an understanding of the importance
of information-based health services management and a health care delivery system.

Enabling objectives
At the end of the lesson the students should be able to:
(a) Explain the importance of information-based health services management.

(b) Describe the role of health personnel in the health data generation process.

(c) Identify the relevant sources of data for a HIS.

(d) Describe the various types of HIS (for example, public, hospital, private sector).

(e) Describe the various levels of a HIS.

(f) Describe the uses of a HIS in decision-making.

Required previous knowledge


Sources of health data, the structure of the health care delivery system, International
Classification of Diseases.

Lesson content
Definition and description of a health information system
A HIS is made up of mechanisms and procedures for acquiring, analysing, using
and disseminating health data for health management.

23
OUTLINE 3 HEALTH INFORMATION SYSTEMS

25

on feedback procedures continuously to improve the quality of data and


information.

Health information system data management


The data generated by a HIS has to be managed correctly and efficiently to yield
the desired information. This management is carried out at the micro (collection
point) level and macro (district, national, etc.) level. Data management involves
collation, checking on accuracy and completeness, storage, processing, analysis,
report generation and information communication. Computers are useful tools
for data management at all levels of a HIS.

Desirable characteristics of a health information system


The key desirable characteristics of a HIS are that it should be:

• used by and cover all levels of the health system;


• affordable and manageable;
• flexible, functional, useful, reliable and relevant.

Decision-making; health information system; health policy; health programmes; health system
management; feedback; disease surveillance; vital registration.

Structure of the lesson


When presenting this lesson, the teacher should:
(a) Clearly define what a HIS is, explaining the roles played by the users of the health system,
the health care providers and the managers.
(b) Discuss the need for information-based decision-making, and the importance of an informa-
tion system and its subsystems in this process.
(c) Build up the description of a HIS from the simple information system of a health unit
in a small community (for example, the village health worker's information system),
through other referral health facilities in a district, to the region, and finally to the national
level.
(d) Describe the information systems of departments of a health facility, through the system of
the whole facility, to the complex country system.
(e) Prepare a package of the forms used by the national HIS to show to the students. Prepare a
handout listing the forms (see Handout 3.2).
(f) Draw examples from the students' environment to illustrate:

• the structures (from the peripheral units, medical records departments of hospitals, up to
the headquarters);

• management (dates and frequency of reporting, local analysis and feedback mechanisms);
• use (development of indicators and setting of priorities) of a HIS.
TEACHING HEALTH STATISTICS

26

(9) Explain the types and training of the personnel of the system, with particular reference to
the country of interest to the students. For example, the personnel of the HIS may consist of
the following:
• Medical records department: records officer, assistant records officers, records assistants,
statistical assistants.
• National health statistics unit: medical statistician, records officer, biostatistics assistants,
computer system analysts, computer programmers.
(h) Describe the health data collecting forms in use in the HIS (see Handout 3.2). If possible, the
forms should be reviewed for any improvements that may be warranted.
(i) Explain the system of reporting and all the legislation regarding health information
reporting.
(j) Explain the use of computers in a HIS, for storage, retrieval and processing of health
data.

Lesson exercises
The teacher should set exercises that test the students' knowledge of the various components of
a HIS, the relevant forms for data collection, the factors that affect the quality of the data, and
the usefulness of the data.

• List six important forms in use in one health information subsystem of your
country. For each form, describe the information to be derived and how it is
used.

• Give five factors that can affect the quality and timely reporting of informa-
tion from the HIS.

• Select one of the forms in use in the HIS and describe its use, in covering:
• frequency of reporting;
• latest date for submission of forms;
• channel of reporting;
• required local analysis;
• feedback.

• Describe the characteristics, advantages and disadvantages of a sentinel re-


porting system in your country.

• Describe the contribution of the nongovernmental sector to the national HIS.

References
Ferrinho PD et al. Developing a health information system for a primary health care centre in
Alexandra, Johannesburg. South African medicaljournal, 1991,80(8):400-403.
Kleczkowski BM, Elling RH, Smith DL. Health systemsupportfor primary health care. Geneva, World
Health Organization, 1984 (Public Health Papers, No. 80).
OUTLINE 3 HEALTH INFORMATION SYSTEMS

27

Last JM, ed. A dictionary of epidemiology, 3rd ed. New York, Oxford University Press, 1995.
McLachlan G, ed. Information systems for health services. Copenhagen, World Health Organization
Regional Office for Europe, 1980 (Public Health in Europe, No. 13).
The roleof research and information systems in decision-making for the development of human resources for
health. Reportof a WHO Study Group. Geneva, World Health Organization, 1990 (WHO Techni-
cal Report Series, No. 802).
WHO cooperation in strengthening national health information systems: a briefing note for WHO country
representatives and ministries of health. Geneva, World Health Organization, 1997 (unpublished
document WHOjHST/97.2; available on request from Department of Health Systems, WHO,
1211 Geneva 27, Switzerland).
HANDOUT 3.1

Definitions of new terms and concepts

Disease surveillance: The continuing scrutiny of all aspects of occurrence and spread of diseases to detect changes
in trends or distribution, as a basis for instigating control measures.

Feedback: The process by which information is passed back to the people providing the data. To be effective the
information should have useful analytical comments.

Health information system (HIS): The mechanisms and procedures for acquiring and analysing data, and provid-
ing information (for example, management information, health statistics, health literature) for the manage-
ment of a health programme or system, and for monitoring health activities.

Health policy: A set of statements and decisions defining health priorities and main directions for attaining health
goals.

Health system management: The management of the interrelated component parts, both sectoral and intersectoral,
as well as within the community itself, which produce a combined effect on the health of a population.

Vital registration: The formal recording of events of human life, such as births, deaths, marriages and divorces.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 28


TEACHING HEALTH STATISTICS

34

(g) Explain how distribution patterns of data are more readily discerned by using diagrams in-
stead of tabulated data. Describe how the following diagrams should be constructed, either
manually or using a computer, from data given in frequency tables: frequency histogram,
frequency polygon, cumulative frequency polygon and cumulative frequency chart (ogive),
bar chart, pie chart.

Lesson exercises
The teacher should organize a set of raw data suitable for tabular and graphic presentation. The
exercises should give emphasis mostly to the process of data reduction, tabulation and graphic
presentation, including the uses and interpretation of graphs and other diagrams.

• Using data on intra-ocular pressure (in Example 1 in Handout 4.2) indicate


the type(s) of diagram that would be appropriate to present the data set on each
of the variables.

• Using the data on intra-ocular pressure in Annex A (A. 1), draw a frequency
polygon for one of the variables and produce cross-tabulations for any two
variables.

• List four different graphic methods for presenting data from a survey on
family planning in a village. Illustrate with two variables for each graphic method.

References
Campbell MJ, Machin D. Medical statistics. Chichester, John Wiley, 1990.
Colton T. Statistics in medicine. Boston, Little, Brown, 1995.
Daly LE, Bourke GJ, McGilvray J. Interpretation and uses of medical statistics. Oxford, Blackwell,
1985.
Dawson-Saunders B, Trapp RG. Basic and clinical biostatistics. Norwalk, Appleton & Lange, Prentice-
Hall International, 1990.
Gore SM, Altman DG. Statistics in practice. London, British Medical Association, 1982.
Huff D. How to lie with statistics. New York, Norton, 1954.
Kirkwood BR. Essentials of medicalstatistics. Oxford, Blackwell, 1988.
HANDOUT 4.1

Definitions of new terms and concepts

Bar chart: Diagrammatic presentation of frequency data for nominal classes by bars whose length is proportional to
the class frequencies.
Class: One of the intervals into which the entire range of the variable has been divided (for example, each of the
intervals 3.0-3.3, 3.4-3.7, ... , 5.0-5.3 is a class).
Class frequency: The number of observations in each class, also known as the absolute class frequency.
Class limits: The true values at the beginning and end of each class, which depend on the accuracy of measurement
(for example, if measurement is accurate to the nearest tenth, then the class limits for the class 3.0-3.3 are
2.95 and 3.34).
Class marks: The variable values thatdemarcate each class (forexample, 3.0 and 3.3 are, respectively, the lower and
upper class marks of the class 3.0-3.3).
Classification: The process of subdividing the range of values of a variable into classes or groups.
Cross-tabulation: A frequency table involving at least two variables that have been cross-classified (tabulated
against each other).
Cumulative class frequency: The number of observations up to the end of the particular class. It is obtained by
cumulating the frequencies of previous classes, including the class in question.
Frequency polygon: Diagrammatic presentation of the frequency distribution of a quantitative variable, with class
frequencies plotted against class midpoint marks, the points being joined by straight lines.
Frequency table or distribution: A tabular arrangement showing the number of times that data with particular
characteristics occur within a data set.
Histogram: Diagrammatic presentation of the frequency distribution of a quantitative variable, with areas of rec-
tangles proportional to the class frequency.
Ogive: Graph of the cumulative relative frequency distribution.
Ordered array: Simple rearrangement of the individual observations in order of magnitude.
Pie chart: Sectors of a circle, with areas proportional to class frequencies, used to present data in nominal classes.
Relative class frequency: The absolute class frequency expressed as a fraction of the total frequency.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 35


HANDOUT 4.2

Illustrative examples of data presentation

Example 1
Extract of data on intra-ocular pressure measurements of 135 adults (forthe full data set see Annex A). The data are
given in mmHg, but may also be expressed in kPa (1 mmHg == 0.133 kPa).

Age (yrs) Sex Rta tt'' Diff' Potential for glaucoma

24 M 20 27 -7 high
52 M 18 12 6 high
26 M 16 13 3 low
71 F 14 14 0 normal
49 M 13 14 -1 normal

39 M 21 16 5 high
71 M 14 12 2 low
32 F 13 12 1 normal
38 F 13 12 1 normal
33 F 9 8 1 normal
a Right eye (mmHg).
b Left eye (mmHg).
C Difference between the right and left eye measurements.

Using seven equal intervals, data on intra-ocular pressure in theright eye may be presented in a frequency distribution
table as inTable 4.1.

Table 4.1 Frequency distribution table of intra-ocular pressure (right eye)


Intra-ocular pressure Number of Relative
(mmHg) observations frequency

0-3 0 0
4-7 1 0.7
8-11 16 11.9
12-15 63 46.7
16-19 40 29.6
20-23 13 9.6
24-27 2 1.5
Total 135

Evident features of the distribution of right eye intra-ocular pressure values, among the 135 subjects studied,
include their variation from 4 to 27 and the factthat an appreciable number of persons have values between 12 and
15.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 36


HANDOUT 4.2 (continued)

Figure 4.7 Equal intervals on the horizontal axis for unequal data intervals

1000

900

800

700

600
en
...co
.t:

..
Cl)
'0
0
500
d
z
400

300

200

r-,
100 セ
セ0)
0

0
0 5 15 25 35 45 55 65 75 80

Age (years)

Figure 4.8 Crowded graph

90......------------------------,

Infectious and parasitic diseases


80 Malignant neoplasm
Diseases of the circulatory system
70 Diseases of pulmonary circulation and other forms of
heart disease
Accidents and adverse effects •
60
v>-
c
セC" 50

cv
>
Nセ 40
Qj
cc::

30

20

10

o 10 20 30 40 50 60 70 80
Age(years)

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 41


OUTLINE 5 Measures of central tendency and location

Introduction to the lesson


Sets of measurements cannot be meaningfully and adequately described by the values of all
the individual measurements. Appropriate summary indices must therefore be obtained. One
type of index describes the "central" point (for example, an average of the values), or the most
characteristic value, of the measurements. These are the measures of central tendency and
location.

Objective of the lesson


The objective of the lesson is to define, and discuss, the indices of central tendency and other
locations (the mean, median, mode and quartiles), their use, interpretation and limitations.

Enabling objectives
At the end of the lesson the students should be able to:
(a) Explain why summary indices are needed in medicine.

(b) Compute the mean, median and mode of a given set of data (grouped and ungrouped).

(c) Compute percentages for categorical data.

(d) Discuss, with examples, the uses and limitations of the mean, median and mode, and their
relative advantages and disadvantages as summary indices of health data.
(e) Explain the use of quartiles and percentiles to summarize health data.

(f) Select an appropriate measure of central tendency and location for a given application.

(g) Differentiate between "average", "normal" and "ideal" values, with reference to health data.

Required previous knowledge


Contents of Outlines 1 to 4 and, if computers are to be used, basic knowledge of their use.

Lesson content
The lesson should cover the definitions, calculation, relative advantages and dis-
advantages, and appropriate data situations for use of the following:

Measures of central tendency


• Arithmetic mean
• Geometric mean
• Weighted mean

43
TEACHING HEALTH STATISTICS

44

• Median
• Mode

Othermeasures of location
• Quartiles
• Percentiles
• Proportions
The teacher should be able to construct an outline of this lesson content with
reference to the material in the proposed handouts. The following are illustra-
tive examples of the computations of some of these descriptive statistics.
Examples (such as those given below or in Handout 5.2) should be used through-
out the lesson. Examples based on real data, and on topics familiar to the
students, would be preferable.

Example 1:Arithmetic mean, median, mode, quartiles and 50th percentile


Prior to an extensive health survey of a region in a certain country, the sampling
procedure required some knowledge of the household sizes of the survey re-
gion. A (random) sample of 31 households was therefore selected and details of
the number of residents in each household were as follows:
5 5 6 3 8 2 3 1 6 2
3 9 9 8 6 6 8 8 10
5 4 9 7 8 5 1 1 11
4
Calculate the arithmetic mean, median, mode, quartiles and 50th percentile of
the number of residents per household.

Arithmetic mean
To calculate the arithmetic mean, there are two steps:
First step: add all values to obtain the total number of people in the
households:
5 + 5 + 6 + 3 + ... + 4 == 165.
Second step: divide this total (165 people) by the number of households (31).
Thus the arithmetic mean is 165/31 == 5.32 persons per household.

Median
To determine the median household size, there are also two steps:
First step: arrange all values in order of their magnitude (this arrangement is
called an array):
1,1,1,1,1,2,2, ... 9,9,9,10,11.
Second step: select the value which divides this distribution into two halves (for
example, the middle observation if the number of observations is
HANDOUT 5.1

Definitions of new terms and concepts

Arithmetic mean: The sum of all values of a set of observations divided by the number of observations.
Geometric mean: A mean derived by multiplying together the n individual values in a series of observations and
calculating the nth root. The logarithm of the geometric mean isthus the arithmetic mean of the logarithm of
individual values.
Measures of central tendency and location: Summary indices describing the central 11 11 point, or the most
characteristic value, of a set of measurements.
Median: Value that divides a distribution into two equal halves; central or middle value of a series of observations
when the observed values are listed in order of magnitude.
Mode: The most frequently occurring value in a series of observations.
Multimodal distributions: Data distributions with more than one mode. Distributions with two modes are
bimodal.
Percentiles: Those values in a series of observations, arranged in ascending order of magnitude, which divide the
distribution into 100 equal parts (thus the median isthe 50th percentile).
Quartiles: The values which divide a series of observations, arranged in ascending order, intofour equal parts. Thus
the second quartile is the median.
Summary indices: Values summarizing a set of observations.
Weighted mean: A mean for which individual values in the set are weighted, very often by their respective
frequencies.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 49


HANDOUT 5.2

Examples of computation of mean and median for


grouped data

Table 5.3 Systolic blood pressure for 240 mett


Systolic blood Mid-value of
pressure in mmHg Frequency the class Product Cumulative
(class interval) (f) (x) (fx) frequency
Under 100 4 95b 380 4
100- 16 105 1680 20
110- 18 115 2070 38
120- 40 125 5000 78
130- 66 135 8910 144
140- 56 145 8120 200
150- 34 155 5270 234
160- and over 6 165 b 990 240
Total 240 32420
a Data are given in mmHg, but may also beexpressed in kPa (1 mmHg = 0.133 kPa).
b These are assumed mid-values, with the lowest value being 90 and the highest value 170.

Mean
The approximate mean isthe weighted average of the class mid-values:
i.e. 32 420/240 == 135.1 mmHg.

Median
The median blood pressure lies in the interval between 130 and 140 mmHg. It isthe average of the 120th and 121 st
observations. Their estimated values are respectively:

130 + (120 -78) x 10/66 == 136.36


and 130 + (121 - 78) x 10/66 == 136.52
Therefore the median is 136.4 mmHg.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 50


OUTLINE 6 Measures of variability

Introduction to the lesson


Knowledge of a single summary figure (such as any of the measures of central tendency dis-
cussed in Outline 5), for describing the characteristics of a population, is not enough without a
measure of the extent of variability or spread of the measurements around this summary index.
Health workers often have to decide whether to classify an individual as healthy or sick, suffer-
ing from a particular disease or not, needing treatment or not, etc. For this task, the so-called
"normal" values of certain clinical, laboratory or radiological measurements provide the neces-
sary yardstick. But the word "normal" value is a statistical concept and depends, to a great
extent, on the distribution of the classifying attribute in the population. Measures of spread
or dispersion or variability are, therefore, essential for understanding, using and interpreting
this concept of "normal" values, and for a complete description of a given health data set.
No description of any health data by summary indices is complete without the measures of
variability.

Objective of the lesson


The objective of this lesson is to define and discuss the sources of variation in health data, and
the various measures of variability, their use, interpretation and limitations.

Enabling objectives
At the end of this lesson, the students should be able to:
(a) Explain the meaning of a measure of variability or dispersion and its place in descriptive
statistics.
(b) Explain the uses of the terms: range, inter-quartile range, variance, standard deviation and
coefficient of variation, as measures of variability of health data.
(c) Compute the following, given either grouped or non-grouped data, with the aid of reference
material:

• range;
• inter-quartile range;
• variance;
• standard deviation;
• coefficient of variation.
(d) Describe the relative advantages and disadvantages of the five indices listed above.

(e) Select an appropriate measure of variability for a given data situation.

(f) Discuss the concept of normality of health data in terms of mean, standard deviation and
percentiles.

51
OUTLINE 6 MEASURES OFVARIABILITY

53

Coefficient of variation
- Used for the comparison of relative variability of two distributions.

- Measures level of variability in the data relative to the average value.

- It is independent of any unit of measurement, and thus useful for compari-


son of variability in two distributions having variables expressed in different
units (for example, height expressed in centimetres for one distribution and
weight in kilograms for the other).

- Takes into account each value of the distribution.

Establishing "normal" values for health data


The establishment of "normal" values permits the selection of appropriate ac-
tions in medical practice.

Variability is inherent in all biomedical measurements upon which decisions on


individual patient care or community health programmes are based. It is there-
fore necessary to have established standards on which decisions can be based.
These standards are often referred to as "normal values", and are generally based
on measurements made on population groups categorized as "healthy". In
statistical reasoning, what occurs most frequently is considered as "normal"
and the problem is often where to draw a cut-off line between "normal" and
"abnormal" .

Two types of "normal" values are usually required for medical decisions: the
"point normal" values and the "normal ranges". Point normal values are esti-
mated by measures of central tendency (refer to Handout 5.1 for definitions of
measures of central tendency and location). Normal ranges give the general level
(in terms of an interval) of a characteristic for healthy population groups. Some
people in the population will have exceptionally high or low values of a particu-
lar characteristic and yet apparently be perfectly healthy. These are called
"outliers". Such exceptional values cannot be regarded as typical of the popula-
tion group. Hence sometimes a few very extreme measurements are excluded
from the computation of normal values.

Most biomedical normal ranges have been adopted to ensure that 95 % of


randomly selected healthy people would fall within the limits. Where a vari-
able follows a unimodal and symmetrical distribution, it is easy to compute
the normal range in terms of the mean and standard deviation (SD), by -using
the properties of the theoretical normal distribution. For example, for the nor-
mal distribution, the range, mean plus or minus 1 SD, covers approximately
68 % , and mean plus or minus 1.96 SD includes approximately 95 % , of the
sampled population. For multimodal or asymmetrical distributions, the compu-
tation of the normal range can be quite involved, although the same principles
apply.

Very often, normal values differ between geographical areas or between sexes
or age groups. For example, "normal" blood pressure differs between sexes,
and also varies with age, and its pattern is not the same in all human populations.
A statement of normal values must therefore indicate the population referred
to.
TEACHING HEALTH STATISTICS

54

Coefficient of variation; dispersion; normal values; range; standard deviation; standard error; theo-
retical normal (Gaussian) distribution; variance.

Structure of the lesson


The lesson may proceed in the following sequence.

(a) Recapitulate the various sources of variation as presented in Outline 1, and illustrate their
cumulative effect on the validity and reliability of measurements in health data. Distinguish
between random and systematic variations.

(b) Describe the nature of measures of variability or dispersion and their place in descriptive
statistics. Differentiate between a summary index of central tendency and a summary index
of dispersion, and explain their complementary roles in the study of any characteristic among
a group of subjects (for example, as indicators of homogeneity and heterogeneity), and for
comparison between different groups of subjects. Explain how variability mayor may not be
related to the magnitude of the variable, and hence differentiate between indices of absolute
dispersion and of relative dispersion.

(c) Give the definitions and methods of computation of the different summary indices of abso-
lute dispersion commonly encountered in the literature. These should include:

• index based on distributional positions (range);

• indices summarizing the squares of differences of individual values from the mean (sum
of squares, variance or mean square, standard deviation);

• handling open-ended intervals for grouped data.

(d) Draw attention to the concept of a range of "normal" values, often determined arbitrarily as
the interval spanning the central 95% of values in the frequency distribution (that is, the
range from the 2.5 percentile to the 97.5 percentile), and explain how the standard devia-
tion is often used to estimate this normal range in the form x ± 1.96 SD.

(e) Give special attention to this use of エィ・Gセ。ョ、イ deviation, which derives from the proper-
ties of the theoretical normal distribution or normal curve. Explain the concept of the stand-
ard normal deviate z (distance from the mean expressed in standard deviation units), and
illustrate how percentiles of the normal distribution are related to values of z. Mention how
the proportions of the normal distribution that lie within or outside various multiples of z
below or above the mean (for example, x ± SE, x ± 1.96 SE) can be used to determine the
"normal" range of values. Discuss when and why the standard deviation mayor may not be
used in this way for empirical data (observed frequency distributions).

(f) Summarize the uses and limitations of the different measures of variability or dispersion.

Lesson exercises
The teacher should obtain data that can demonstrate variation in an attribute, such as a continu-
ous variable, and ask the students to calculate the various measures of variation and to describe
how they can compare variation in variables measured in different units.
OUTLINE 7 INTRODUCTION TO PROBABILITY AND PROBABILITY DISTRIBUTIONS

59

Lesson content
Concept of probability
• Definition of probability (subjective definition and the frequency concept).
• Definitions of technical terms (trials, experiments, outcomes, events, chance,
odds; see Handout 7.1).
• Scale of measurement of probability and its interpretations.

Laws of probability
• Explanations of simple and compound events.
• Mutually exclusive and independent events.
• The addition and multiplication rules.
• Dependent events and definition of conditional probability.

Probability distributions
• Discrete probability distributions (binomial).
• Continuous probability distributions (normal).
• Properties and uses of the distributions.

Additive law of probability; binomial coefficients; binomial distribution; conditional probability;


dichotomous independent events; inductive statistics; multiplication law of probability; mutually
exclusive events; population; probability; probability distribution; relative frequency; statistical
inference.

Structure of the lesson


The students do not need to know probability theory in detail, but should be acquainted with
some of its basic concepts, principles, rules and applications.
The lesson content may be presented in the following sequence.
(a) Introduce the idea of subjective probability in everyday life, which is not usually quantified.

(b) Explain the range of values for probabilities (0-1) and the interchangeable use of the terms
"chance" and "probability".
(c) Briefly review the uses of the descriptive statistical methods already learned, and introduce
the concept and meaning of inductive statistics, illustrating with medical data (for example,
how criteria of abnormality used in diagnosis are based on descriptive information but
applied to new patients).
(d) Explain the meaning of such terms as trials, outcomes, events, experiments.

(e) Explain the relationship between probability and observed proportions in data on a dicho-
tomous attribute.
Example: what is the chance of finding a person with Type A blood, or the chance of an
unborn child being male?
OUTLINE 7 INTRODUCTION TO PROBABILITY AND PROBABILITY DISTRIBUTIONS

61

laws of probability and the binomial probability distribution with particular reference to health
problems.

To demonstrate and reinforce the concept of a sampling distribution and to show how it is
governed by laws of probability, let the students generate an empirical (observed) sampling
distribution and compare it with the theoretical (expected binomial) distribution. One way of
doing this is to use coloured beads to represent persons with different attributes in a population.
Give examples of dichotomous medical attributes that can be represented by beads of two
colours, say black and white, for example, genetic traits (sickle cell anaemia, blood grouping,
etc.).

• From a box containing a large number of beads of two colours (for example,
black and white), let each student take a random sample of a given size n (for
example 5); tabulate the number of black (or white) beads seen in each sample.
This gives an observed sampling distribution.
Given the actual proportion of black (or white) beads in the box, calculate the
binomial probability distribution for sample size n. Hence calculate the expected
sampling distribution for the observed number of samples.
Compare and comment on the observed and expected sampling distributions.

The goodness-of-fit will be tested later when the students have learned about the chi-squared
test.

The following exercise is designed to demonstrate the application of the laws of probability and
the binomial probability distribution.

• Give data on prevalence of various attributes (for example, diseases) in a


population, and ask questions on the probability of a person having or not
having various combinations, or all, or none, of these attributes.
Ask questions about the expected number of 2- or 3-child families that have
various numbers of sons or daughters.
Give an exercise to show the role of probability statements in the context of a
diagnostic test, for example, concerning its specificity, sensitivity or predictive
value.

• Supposing that a midwife delivering babies at a maternity home does


not know the sex of the baby until it is delivered, how can one determine
the probability of delivering a male baby, using data from the maternity
home?

• For a couple desiring to have a baby boy following three female births, if it is
known that the chance of a pregnancy resulting in a male baby is 0.5, what is
the chance that the fourth pregnancy will result in a male birth?

• If in a hospital, over a period of three years, 27 out of 30 babies with neonatal


tetanus have died, what is the probability that a neonate with tetanus will sur-
vive if there has not been any change in how tetanus is managed?
HANDOUT 7.2

Worked example of a binomial


probability distribution

The binomial equation for the probability of specified outcomes of an event


The probability of observing r successes in n independent repeated trials, given that the probability of a success
in each trial (p) and the probability of a failure (1-p) is the same from each trial, can be obtained from the equation:

P(r successes) = [ny((n - r)! r!)] . pr . (1 - prr

where n! is n factorial == n(n - 1)(n - 2) ... (2)( 1)

and O! == 1 by definition.

At the height of the drought in a given region, it was estimated that 70% of the children under 10 years old were
severely malnourished. If five children, under 10 years old, were selected at random from the region, what is the
probability that: all, 4, 3, 2, 1, 0, are severely malnourished?

Table 7. 1 Binomial probabilities


Number Terms of binomial
malnourished expansion Probability
5 (all) pS 0.16807
4 5p4 q 0.36015
3 10p3 q2 0.30870
2 10p2 q3 0.13230
1 5pq4 0.02835
o(none) qS 0.00243

Inthis table we have: n == 5; P == 0.7; q == 0.3.


Applying the binomial equation:
• When all children are malnourished, then r == 5;

hence P(r= 5) = [5V(5 - 5)! 5!] x 0.75 x (1 - 0.7)5-5


== 0.75
== 0.16807.

• When 3 children are malnourished, then r == 3;

3
hence P(r= 3) = [5V(5 - 3)! 3!] x 0.7 3
x (1 - 0.7t
== 5 x 2 X 0.7 3 X 0.3 2
== 0.30870.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 65


OUTLINE 8 Sampling and estimating
population values

Introduction to the lesson


Whenever we infer the characteristics of other persons in the population at large from those of
a finite group which we have studied, we are making use of information about samples to draw
conclusions or make inductive inferences. Such information has some limitations regarding re-
liability, precision and validity. Information based on limited samples constitutes most, if not all,
of the medical knowledge that we have of human populations.

Objective of the lesson


The objective of this lesson is to enable the students to understand the concepts of population,
samples, sampling methods, sampling errors and estimation problems, and drawing inferences
on the basis of probability.

Enabling objectives
At the end of the lesson the students should be able to:
(a) State the reasons for sampling with the different sampling methods.

(b) Distinguish between probability and non-probability sampling.

(c) Differentiate between sampling and non-sampling errors.

(d) Differentiate between statistics and parameters.

(e) List possible advantages and disadvantages of collecting health information through
samples.
(f) Discuss the relative advantages and disadvantages of each of the following sampling meth-
ods, as applied to the design of a health survey:
• probability (random) sample;
• simple random sample;
• stratified random sample;
• systematic sample;
• cluster sample;
• multistage sample.
(g) Calculate the standard error of the sample mean or proportion, given the relevant data and
formulae.
(h) Differentiate between point and interval estimates of health indices.

(i) Explain the concept of the central limit theorem.

(j) Explain the meaning and application of confidence limits of an estimate of health indices.

66
OUTLINE 8 SAMPLING AND ESTIMATING POPULATION VALUES

67

(k) Explain how sampling error is related to sample size and to variability of the characteristic
under study.
(I) State the information needed to estimate the minimum sample size for a health survey.

Required previous knowledge


Concept of probability, measures of central tendency and variability.

Lesson content
The concept of sampling
• Population (universe)
• Sample
• Sampling
• Reasons for sampling
• Sampling unit
• Sampling frame
• Sampling fraction
• Unit of inquiry
• Probability and non-probability sampling

Sampling and non-sampling errors


• Sampling variation
• Concept of bias
• Methods for minimizing sampling errors
• Methods for controlling non-sampling errors

Sampling distributions
• Meaning of parameters and statistics
• The central limit theorem

Advantages and disadvantages of using sampling methods to collect health data


Sampling methods and their advantages and disadvantages
• Simple random
• Stratified random
• Systematic
• Cluster
• Multistage

Estimation
• Concept of standard error
• Point and interval estimation (mean and proportion)
TEACHING HEALTH STATISTICS

68

• Precision
• Determination of minimum sample size

Bias and selection in sampling; cluster sampling; confidence limits; confidence range; difference
between sampling and non-sampling error; dummy tables; estimation of a population mean; esti-
mation ofapopulation proportion; health survey; level ofconfidence; method ofsampling; multistage
sampling; point and interval estimates; population (universe); population parameter; precision
of estimates; pre-coded data; probability sampling; quality of sample; representativeness of a
sample; sample statistic; sampling error; sampling fraction; sampling frame; sampling unit; self-
coding record forms; self-selected or natural samples; standard error; statistical estimation; strati-
fied random sampling; survey questionnaire; systematic sampling; validity of estimates; unit of
inquiry.

Structure of the lesson


The lesson content may be presented in the following sequence:

Concept of sampling
Explain the concept of population, sample and sampling, giving the reasons for sampling: lim-
ited resources available for estimation, lack of access to total population, or sampling may be the
only feasible method of collecting the information.
Also explain the following terms: sampling frame, sampling unit, sampling fraction.
Explain the characteristics of a good sample (the sample must be selected at random to reduce
bias, be representative to improve validity, and be large enough to increase precision).
Distinguish between random sampling and non-random (purposive) sampling.

Sampling and non-sampling errors


Explain sample statistics and population parameters.
Describe the use of sample statistics as estimates of population parameters. Owing to chance,
different samples give different results, a phenomenon called sampling variation. Sampling er-
rors and the concept of bias must be explained.
Explain the concept of sampling error: the unavoidable difference between the value of a sample
statistic and the corresponding population parameter. An increase in sample size results in a
reduction in the sampling error. Non-sampling errors are systematic errors during estimation.
Give examples of non-sampling errors and how they can be reduced.

Sampling distributions
Explain the concept of a sampling distribution using simple examples, without invoking mathe-
matical statistics. Explain the differences between a parameter and a statistic, and that every
sample statistic belongs to a sampling distribution.
Describe the principles and applications of the central limit theorem which states that, for all
variables, whether normally distributed or not, the sample mean will tend to be normally
distributed.
OUTLINE 8 SAMPLING AND ESTIMATING POPULATION VALUES

69

Advantages and disadvantages of sampling


Give examples to demonstrate that, by using a well chosen and reasonably large sample, the
estimates will be close to the expected values and the sample will cover the study population
adequately. The advantages and disadvantages set out in Handout 8.2 should also be discussed.

Methods of sampling, their advantages and disadvantages


The various methods of sampling must be explained and their relative advantages and disadvan-
tages should be discussed (see Handout 8.2).
Simple random sampling is useful when the sampling frame is well described and not too large.
Systematic sampling requires a list of the sampling units or that sampling units are in an ordered
sequence. Cluster sampling is useful when sampling units form logical groupings and when the
sample frame is difficult to obtain.

Estimation
Explain the concepts of statistical estimation. The following should be covered:
• The context of statistical estimation: the need to estimate population parameters from sample
statistics; the problem posed by sampling error for making reliable estimates; the concepts of
point and interval estimation; validity and precision of a statistical estimate.
• The concepts of confidence limits and level of confidence: the connection between sampling
distributions, confidence limits and levels of confidence. Interval estimation: estimation of a
population parameter in terms of an interval that has a specified probability of containing the
true value. The interval is the confidence interval, and the limits are the confidence limits.
• The estimation of normal anthropometric values for a population, with examples; estimation
of mean birth weight from hospital births; and estimation of disease prevalence in morbidity
surveys.

Determination of minimum sample size'


Discuss the various factors that determine minimum sample size. Describe the use of the equa-
tions given in Handout 8.3 to estimate the sample size required to achieve a desired precision for
an estimate, expressed in terms of a confidence range. The following should also be covered:
• The relation between confidence limits and standard error: the meaning of standard error; the
basis for derivation and distribution of confidence limits in terms of the standard error.
• Computation of the standard error of the mean and proportion, with examples from the
literature to illustrate their computation and use in statistical estimation.

Lesson exercises
The exercises for this lesson should focus on helping the students crystallize the concepts of
sampling and estimation of population values covered in the lesson. The emphasis should not be
on correct memorization of formulae but on their appropriate use and interpretation of the
results. The exercises should, in particular, cover all the major points indicated in the enabling
objectives of the lesson (reasons for sampling, the advantages and disadvantages of the different
sampling methods, interpretation of confidence interval, etc.).

1 See Lwanga SK, Lemeshow S. Sample size determination in health studies: a practical manual, Geneva, World Health
Organization, 1991.
TEACHING HEALTH STATISTICS

70

• What sampling method would you recommend in the following instances?


• Determining the proportion of undernourished five-year-olds in a village.
• Investigating nutritional status of preschool children.
• Selecting maternity records for the study of previous abortions or duration of
postnatal stay.

• In the estimation of immunization coverage in a province, data on seven


children aged 12-23 months in 30 clusters are used to determine the proportion
of fully immunized children in the province.
• Give three reasons why the cluster sampling method is used in such a survey.
• Give two sources of systematic error and two sources of random error that
may be associated with an immunization coverage survey.
• In the immunization coverage survey, the 30 villages are selected by system-
atic sampling. If the investigator uses 30 clusters that are easy to reach, what
is the type of sampling method in this case, and what possible sampling errors
would be associated with the method?

• If the results obtained from an immunization survey indicate that 48 % of


the children were fully immunized (standard error 3 % ), calculate the 95 °10 in-
terval estimate of fully immunized children in the study.

• In a family planning clinic, there are 2500 clients. Suppose the anticipated
prevalence of HIV infection is 3 0/0 and the investigator is willing to accept an
absolute error of 1 0/0. What is the minimum sample size required to estimate the
prevalence of HIV with 95 0/0 confidence?

References
Bland M. An introduction to medicalstatistics. Oxford, Oxford University Press, 1987.
Colton T. Statistics in medicine. Boston, Little, Brown, 1995.
Daly LE, Bourke GJ, McGilvray J. Interpretation and uses of medical statistics. Oxford, Blackwell,
1985.

Dawson-Saunders B, Trapp RG. Basic and clinicalbiostatistics. Norwalk, Appleton & Lange, Prentice-
Hall International, 1990.
Dixon RA. Medical statistics: content and objectives of a core course for medical students.
Medical education, 1994,28:59-67.
Feinstein AR. Clinicalbiostatistics. St Louis, Mosby, 1977.
Huff D. How to lie with statistics. New York, Norton, 1954.
Kirkwood BR. Essentials of medicalstatistics. Oxford, Blackwell, 1988.
Lemeshow S et al. Adequacy of sample size in health studies. London, John WHey, 1990.
Lwanga SI<, Lemeshow S. Sample size determination in health studies: a practical manual. Geneva,
World Health Organization, 1991.
HANDOUT 8.1

Definitions of new terms and concepts

Confidence limits: The upper and lower limits of the interval in interval estimation. The interval itself is called the
confidence interval or confidence range. Confidence limits are so-called because they are determined in ac-
cordance with a specified orconventional level of confidence or probability that these limits will infact include
the population parameter being estimated. Thus, 95% confidence limits are values between which we are
95% confident that the population parameter being estimated will lie. Confidence limits can often be derived
from the standard error.
Interval estimation: Providing an estimate of a population parameter in terms of an interval or range of values
within which it is likely to lie.
Level of confidence: Conventionally 95% or 0.95, but may be set higher or lower as desired.
Point estimation: Providing an estimate of a population parameter in terms of a single value that it ismost likely to
have. A point estimate is usually provided by a sample statistic. By itself, point estimation ignores sampling
error.
Population: Any specified group (usually large) of persons, things, or measurement values.
Population parameter: A descriptive index whose value refers to the population at large, as opposed to a sample
of the population (for example, a population mean or population proportion).
Precision of an estimate: The inverse of thestandard error of theestimate. The less the sampling error that is likely
to occur, the greater the precision; that is, the smaller the confidence range, the greater the precision. Hence,
precision can be specified in terms of the confidence range or the standard error.
Sample: A subset of a population, whose properties have been, or are to be, generalized to the population.
Sample statistic: A descriptive index, the value of which is obtained from observations in a sample (forexample, a
sample mean or a sample proportion).
Sampling: The process of selecting a sample from a population.

Sampling distribution: The distribution of probabilities with which sampling error of different magnitudes can
occur purely by chance for a particular sample statistic and sample size. It can be demonstrated experimentally
bytabulating the values of the same sample statistic obtained from repeated samples of the same size taken
randomly from the same population. It can also be calculated theoretically (forexample, using the binomial or
the normal sampling distribution). Every sample statistic is a member of a sampling distribution, that is, the
distribution ofvalues of thatstatistic that can be expected to occur in different samples of the same size drawn
randomly from the same universe.
Sampling error: A difference that occurs purely by chance between the value of a sample statistic and that of the
corresponding population parameter (forexample, the difference between the value of the mean of a random
sample and that of the universe). Sampling error cannot be avoided or totally eliminated, and must always be
allowed for when making inferences or drawing conclusions from sample statistics. It can be reduced by
increasing sample size or using a more appropriate sampling method.
Sampling fraction: The proportion of sampling units to be selected from a specified sampling frame for inclusion in
the sample.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 71


I HANDOUT 8.1 (continued)

Sampling frame: The set of sampling units from which a sample is to be selected. For example, a list of names, or
places, or other items to be used as sampling units.
Sampling unit: The unit of selection in the sampling process. For example, a person, a household or a district. It is
not necessarily the unit of observation or study.
Standard error (SE): The standard deviation of a statistic.
Unit of inquiry: Smallest unit on which data are collected.
Universe (of a sample): The population of values, of which thevalues observed inthesample are a random sample,
and to which the properties of the sample can validly be generalized. The universe of a sample may be an
abstract or a real population of values, and it may be finite or infinite, depending on the type of sample and the
nature of the information under study.
Validity of an estimate: The extent to which an estimate corresponds to the parameter it is estimating. It depends,
not on the size of the sample, but on the representativeness of the sample. Hence it depends on the type or
nature of the sample, how it was selected, and on the accuracy of the information from which it was calcu-
lated and of the calculation itself.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 72


HANDOUT 8.2

Methods of sampling, and their advantages


and disadvantages

Sampling
Advantages Disadvantages

• Sampling reduces demands on resources such as finance, • There is always a sampling error.
personnel and materials.
• Sampling may create a feeling of
• Results are obtained more quickly. discrimination within the population.

• Sampling may lead to better accuracy of collected data; a • Sampling may be inadvisable where every unit in the
smaller sample allows more effortto be made to reduce population is legally required to have a record.
non-sampling errors and non-response biases.
• For rare events, small samples may not yield sufficient
• Precise allowance can be made for sampling error (which cases for study.
can befound bycalculation), although notfor non-sampling
errors.

Probability sampling
• All individuals (elements) in the population have a known chance (probability) of selection. The chance of selection
need not be the same for each individual or element.

• The knowledge ofthe selection probability isin contrast with thesituation for non-probability sampling techniques,
such as quota and chunk sampling.

• There must be an identified sampling frame, whether of individual elements or clusters of elements, from which the
sample isto be drawn.

Simple random sampling


• Every sample of the same size has the same chance of being selected.
• Every sampling unitin the sampling frame has the same chance of being selected.
• Random selection from thesampling frame can be done byballoting, using a table ofrandom numbers, or employ-
ing a computer.

Advantages Disadvantages

• Because every unit in the population has an equal chance of • If the sampling frame is large, this method may be
being included in thesample, the sample is assured of being impracticable because of the difficulty and expense of
representative and subject only to sampling error. constructing or updating it in large-scale surveys.

• Estimates are easy to calculate. • Minority subgroups of interest in the population may not
be present in the sample in sufficient numbers for study.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 73


HANDOUT 8.2 (continued)

Stratified random sampling


• The population is first divided into groups or strata according to a characteristic of interest (for example, sex, age,
geographical location).

• A simple random sample is then selected from each stratum using the same sampling fraction, unless otherwise
prescribed for special reasons.

Advantages Disadvantages

• Every unit in a stratum has the same chance of being • The sampling frame of the entire population has to be
selected. prepared separately for each stratum.

• Using the same sampling fraction for all strata ensures • Varying the sampling fraction between strata, to ensure
proportionate representation in the sample of the selection of sufficient numbers in minority subgroups for
characteristic being stratified. study, affects the proportiona I representativeness of the
subgroups in the sample as a whole.
• Adequate representation of minority subgroups of interest
can be ensured by stratification and by varying the sampling
fraction between strata as required.

Systematic sampling
• Involves the selection of every j(h unit in the population or the sampling frame, where 1/ k is the sampling fraction.

• The first unitto be selected is selected at random from among the first k units.

Advantages Disadvantages

• The sample is easy to select. • The sample may be biased if a hidden periodicity in the
population coincides with that of the selection.
• A suitable sampling frame can be identified more easily.
• It is difficult to assess the precision of the estimate from
• The sample is evenly spread over the entire reference
one survey.
population.

Cluster sampling
• The population is first divided into clusters of homogeneous units, usually based on geographical contiguity.

• A sample of such clusters isthen selected.

• All the units in the selected clusters are then examined or studied.

Advantages Disadvantages

• Cuts down on the cost of preparing a sampling frame. • Sampling error is usually higher than for a simple random
sample of the same size.
• Cuts down on the cost of travelling between selected units.

• Eliminates the problem of packing (in health surveys,


11 11

especially those involving case finding and treatment, it is


not unusual for neighbouring houses not included in the
sample to transfer their households temporarily to a
selected house).

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 74


HANDOUT 8.2 (continued)

Multistage sampling
• Selection is done in stages until the final sampling units (for example, households orpersons) are arrived at.
• In the first stage, a list of large-sized sampling units is prepared. These may be towns, or villages or schools.
• A sample of these is selected at random, with probability of selection proportional to size.
• For each of the selected first-stage units, a listof smaller sampling units is prepared. (For example, if the first-stage
units are towns, then second-stage units may be houses or households.)
• A sample of these second-stage units is then randomly selected from each of the selected first-stage units. These
are then studied.

• The procedure may contain three or more stages.

Advantage Disadvantage

• Cuts down the cost of preparing a sampling frame. • Sampling error is increased compared with a simple
random sample of the same size.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 75


HANDOUT 8.3

Examples of sample size determination

Determination of minimum sample size


The minimum sample size (n) depends on the:

• objective;

• design of the study;

• plan for statistical analysis;

• accuracy of the measurements to be made (d);

• degree of precision required for generalization;

• degree of confidence with which to conclude.

With simple random sampling, for a given magnitude of confidence interval, the precision (z) can be measured by:

Z == d/SE.
If we want a 95% confidence interval, z must be 1.96 (see Table B.1, Annex B). Since the SE depends on n, we can
calculate the value of n required to achieve the chosen level of confidence.

If 5 is the sample estimate of the population standard deviation (see Outline 6), then the standard error (SE) of the
mean, for a sample of size n, is siJn.

For estimating a population mean, with SE == 51 Jn, the minimum required sample size, in general, is:
n == s2/SE 2 == Z2 52 / d'.
For a population of size n, involving a binomial distribution with probability p (see Outline 7), let a individuals be
observed with the relevant characteristics. Then the standard error of the estimate of p (that is, aln) is J(pqln), where
q == 1 - p.

Since SE == J(pql n),

n == pq/SE 2 == Z2 pq/ d'.


These results apply only if sampling is from a very large (theoretically infinite) population, where the ratio of the
sample size to the population size is very small.

If sampling isfrom a finite population of size N, then the minimum sample sizes are:

n = z2s / {d2+ Z2S2/N),

for estimating the mean, and:

n = z2 pq/{d2+ z2 pQ/N),

for estimating p in the binomial distribution.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 76


HANDOUT 8.3 (continued)

If no is the sample from an infinite population, the finite population sample size is:

Sampling for a quantitative characteristic


When sampling for a quantitative characteristic (for example, the mean level of haemoglobin in a population), one
needs to state:

- how precisely one wishes to estimate this mean level; that is, the amount of sampling error that can be tolerated
(d), in either absolute or relative terms;

- the standard deviation (5) of the distribution of haemoglobin in the population;


- the chance the experimenter is willing to take to get an unlucky sample giving a sampling error greater than d;
a 5% chance of error (that is, a 95% confidence interval) is conventional.
This means that x ± d are the required 95% confidence limits, so that d == 1.96 SE, where SE == si)n. Hence,
d11.96 == si)n, and therefore the required (minimum) sample size for a very large population is given by:

Example 1
A health officer wishes to estimate the mean haemoglobin level in a defined community. Preliminary information is
that this mean is about 150 mg/I with a standard deviation of 32mg/1. If a sampling error of up to 5mg/I in the
estimate is to be tolerated, how many subjects should be included in the study?

Here, 5 == 32mg/I, and d == 5 mg/1.


If the population is assumed to be very large, the required minimum sample size would be:

Thus, the study needs at least 158 persons.


If the community to be sampled has 1000 people, the required minimum sample size would be:

Therefore at least 136 people would have to be studied. For a larger community with, for example, N == 3000 people,
the required sample size would be:

At least approximately 150 people would have to be studied.

Sampling for an attribute


When sampling for an attribute (to estimate the proportion of persons with a certain characteristic in a population)
one needs to state:
- a rough approximation to the proportion (p);

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 77


HANDOUT 8.3 (continued)

- the sampling error that can be tolerated (d) in either absolute or relative terms;
- the acceptable chance of an unlucky sample (conventionally 50/0).
The minimum sample required, for a very large population, is then:

Example 2
If P == 0.26, and d == 0.03, then, for a very large population:

n = (1.96f x 0.26 x 0.74f(o.03f = 821.2.


Thus, the study should include at least 822 persons.
If the sample were from a relatively small population of, for example, 3000 people, the required minimum sample
could be obtained from the above estimate byadjustment as:

821.2/(1 + 821.2/3000) == 644.7.

Thus the study should include at least 645 people.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 78


OUTLINE 9 Tests of statistical significance

Introduction to the lesson


Tests of significance are standard statistical procedures for drawing inferences from sample esti-
mates about unknown population parameters. Sample estimates are never exact, being subject
to sampling errors. In the design of any medical research, attempts are made to reduce these
sampling errors. Tests of significance allow us to decide whether the sample estimates, or the
differences between estimates, are within their normal biological variation, commonly called
variability due to chance.
Chance variation can give rise to differences between samples being studied, and so every time
a difference is observed the question arises as to its statistical significance, that is, whether the
difference is unlikely to have occurred purely by chance alone.

Objective of the lesson


The objective of this lesson is to enable the students to understand the meaning and application
of tests of significance and their role in statistical inference. Emphasis is placed on their uses and
interpretation rather than on the theory and methodology of the tests.

Enabling objectives
At the end of the lesson the students should be able to:
(a) Explain the context and meaning of statistical hypothesis.
(b) Explain when, and why, a test of significance needs to be carried out.
(c) Explain the procedures for carrying out tests of significance.
(d) Differentiate between type 1 and type 2 errors in hypothesis testing.
(e) Explain the possible outcomes of a test of statistical significance and their respective inter-
pretations in relation to the context of the test.
(f) Differentiate between statistical and medical significance.
(9) Select an appropriate test statistic for the comparison of two means, for independent and
dependent samples.
(h) Select an appropriate test statistic for the comparison of two proportions.
(i) Carry out an appropriate test of statistical significance for the difference between two means,
for independent and dependent samples.
(j) Carry out an appropriate test of statistical significance for the difference between two
proportions.

Required previous knowledge


The students should have covered the material in all preceding lessons. It is desirable, at the
beginning of this lesson, to stress and check that the students have attained the enabling

79
TEACHING HEALTH STATISTICS

80

objectives of Outlines 6 and 7 and, in particular, that they have understood the concepts of
sampling error and sampling distributions (Outline 8).

Lesson content
Construct an outline of the lesson with reference to the definitions and explana-
tions of new terms and concepts in Handout 9.1, with the following content.

Nature of statistical hypothesis


• The null and alternative hypothesis
• Need to test the null hypothesis
• The meaning of a test of significance

Situations for tests of significance


• Comparison of sample estimates against a specified standard
• Comparison between two sample estimates

Procedure for testing statistical hypothesis


• Sta te the null hypothesis
• State the alternative hypothesis (indicate I-tail or 2-tail)
• State the level of significance (explain type 1 and type 2 errors)
• Choose the test statistic (explain parametric and non-parametric tests)
• Compute the numerical value of the test statistic from the observed data
• Compare the calculated value of test statistic with tabulated values in ap-
propriate standard distribution tables at a specified probability level of
significance
• Decide whether or not to reject the null hypothesis according to the p-value

Interpretation of results of p-values


• Statistical significance versus medical importance or significance
• Role of sample size in determining statistical significance

Comparison of two mean values


• Independent versus dependent samples (give practical examples)
• The t-test for each statistical design
• The standard errors of the difference between two independent sample means
for data situations when equal or unequal variances are assumed

Comparison of two sample proportions


• Use of z for large sample sizes
• Use of t-test for small sample sizes
OUTll NE 9 TESTS OF STATISTICAL SIGNIFICANCE

81

• Pooled or non-pooled estimate of variance for standard error of a difference


• The use of the X2 (chi-squared) test to test for association between two cat-
egorical variables when data are presented in 2 X 2 contingency tables
• Yates' correction for continuity
• Fisher's Exact Probability Test

Alternative hypothesis; degrees of freedom; hypothesis testing; level of significance; null hypothe-
sis; 1-tailed and 2-tailed tests; p-value; probability of a difference occurring purely by chance;
rejection of a hypothesis; statistical significance; test statistic (z, t, X2) ; type 1 and type 2 errors.

Structure of the lesson


During the lesson, liberal use should be made of examples taken from the literature to illustrate
the role of statistical significance in the interpretation of data and drawn conclusions.
Worked examples of how a test of significance is actually carried out should be given to the
students. Examples of the z-test, the X2 test and the t-test are suggested in Handouts 9.2 and 9.3.
The lesson content may be presented in the following sequence.
(a) Outline the context and concept of a statistical test of significance. Make reference to:

• differences, for example, between the means of certain biochemical, physiological, demo-
graphic or any health measurements in different samples, or between the proportions
with certain attributes in different samples, or between the observed and expected number
of occurrences of certain events;
• formulation and testing of the null hypothesis;
• the probability that a difference of a given magnitude or greater magnitude can occur
purely by chance; illustrate this in relation to the theoretical sampling distribution;
• the direction of difference and the implications for I-tailed or 2-tailed tests;
• the probability of being wrong in rejecting or not rejecting a hypothesis; type 1 and type 2
errors.
(b) Introduce the concept of level of significance

• The lowest value p (the probability) must have for an event to be considered "unlikely",
and hence for the null hypothesis to be rejected and the difference to be described as
being statistically significant.
• Describe the conventional levels of significance, Le. "significant" for p < 0.05; "highly
significant" for p < 0.01; "not significant" for p > 0.05 or p == 0.05.
(c) Describe the role of significance testing and the implications of the outcome

• Discuss the possible causes of the observed difference:


- chance (the null hypothesis);
- the factor under study;
- other "real" factors;
- "spurious" factors, such as bias and non-comparability.
TEACHING HEALTH STATISTICS

82

• The test of significance only takes care of the factor of chance; discuss how the other
possible causes of observed difference are dealt with. Emphasize the difference between
statistical and medical significance. For example:
- a statistically significant difference but of no clinical importance;
- a non-statistically significant observation but with the results pointing to a possible
clinical or medical importance.
• Discuss possible follow-up as a result of a statistical test of significance, for example, re-
peat of study with an enlarged sample size.
(d) Outline the methodology of the various tests of significance

There are many types of tests of significance, catering to different types of data and differences
being dealt with. The most commonly encountered are the z-test, the r-test and the X2 test.
Mention only the usefulness of the X2 test and indicate that detailed treatment of this test statistic
will follow in the next lesson. At least one type of test should be carried out by the students to
learn the concepts and principles involved; that is, how to:
• select the appropriate test to be used;
• differentiate between parametric and non-parametric tests;
• calculate the test statistic;
• evaluate its magnitude in relation to its theoretical sampling distribution, in terms of the
probability that this magnitude could have arisen purely by chance (if the null hypothesis
were true);
• decide whether the difference is significant and, if so, at what level of significance.
Refer to the worked examples given in Handouts 9.2 and 9.3.

Lesson exercises
The class exercises should emphasize the proper selection of the test to be used in each specific
situation and how to interpret the results obtained. The teacher should obtain a data set which
has both categorical and continuous variables that can be used for the various tests on means in
dependent and independent situations and in the case of proportions for small and large data
sets.
Class exercises are given to provide practice in carrying out tests of significance, and interpreting
the results in the context of the study objectives.

• For each of the following comparisons, name the appropriate test of


significance:
• mean weight for preschool boys and girls;
• mean family size for urban and rural families;
• serum albumin values for women using an intrauterine contraceptive device
and for women not using such a device;
• number of sexual partners of HIV-positive men before and after two years of
counselling;
• temperature of children with fever taken before treatment and one hour after
treatment.
OUTLINE 9 TESTS OF STATISTICAL SIGNIFICANCE

83

• The average clinic utilization rate for 1152 infants who reported to Kasangati
Health Clinic from 1961 and 1979 is provided in Table 9.1. (Kasangati Health
Clinic is the field station for Makerere Medical School, Institute of Public Health,
Kampala, Uganda.) The study was reported in the East African medical journal
(March, 1994).

Table 9.1 Average clinic utilization rate, Kasangati Health


Clinic, Uganda
Year Number of infants Mean utilization rate
1961 3 1.7
1962 9 2.7
1963 55 2.5
1964 88 1.9
1965 102 3.1
1966 164 3.1
1967 147 2.8
1968 67 2.0
1969 60 1.6
Mean = 2.6
SD = 1.9
Year Number of infants Mean utilization rate
1970 16 1.6
1971 90 3.5
1972 80 4.0
1973 71 3.9
1974 65 3.4
1975 43 3.0
1976 42 3.2
1977 33 2.9
1978 11 2.1
1979 6 1.2
Mean = 3.4
SD = 2.3
Source: Biritwum RB. Record keeping onearly childhood diseases in two decades,
at the health centre level in Uganda. East African medical journal, 1994, 71:
199-203. Reproduced bypermission.

• Determine whether the average utilization rate per child in the 1960s is sta-
tistically different from the rate in the 1970s.
• Comment on the distribution of the data for the test selected.

• Table 9.2 gives the summary of the data on immunization of children in Yemen,
as reported in the Demographic and Maternal and Child Health Survey, 1991/
1992 (source: Demographic and Health Surveys, 1991-92, Macro International
Inc., Calverton, MD, USA).
TEACHING HEALTH STATISTICS

84

Table 9.2 Summary of data on immunization of children, Yemen


Percentage of children who received
Number
OPT Polio
of
Characteristics BCG 2 3+ 2 3+ Measles Alia None children

Child's age (months)


<6 29.2 28.9 18.6 9.6 28.9 18.6 9.6 13.5 7.6 68.0 718
6-11 47.3 48.6 42.8 30.9 48.6 42.8 30.9 34.8 25.1 49.6 802
12-17 58.8 60.3 56.3 48.7 60.3 56.3 48.7 51.4 45.9 37.3 627
18-23 61.9 62.4 54.6 45.8 62.4 54.6 45.8 51.6 44.0 35.8 628
24-59 66.6 65.8 61.3 53.3 65.9 61.3 53.3 58.5 50.8 30.4 3939
Sexof child
Male 61.3 61.1 55.8 46.7 61.1 55.8 46.7 51.8 43.7 35.6 3427
Female 56.8 56.7 50.9 42.9 56.8 51.0 42.9 47.2 40.3 40.2 3288
Residence
Urban 81.3 81.1 76.5 68.1 81.2 76.6 68.1 70.3 63.0 14.9 1113
Rural 54.7 54.6 48.8 40.2 54.6 48.8 40.2 45.4 37.9 42.4 5602
Region
North-west 55.8 56.4 50.6 42.0 56.4 50.6 42.0 46.9 39.3 41.0 5793
South-east 79.6 75.4 71.1 62.4 75.4 71.1 62.4 66.5 59.4 18.3 922
Mother's education
No education 56.5 56.6 51.0 42.5 56.6 51.0 42.5 47.1 39.8 40.4 5836
Primary 84.2 81.8 77.8 69.2 81.8 77.8 69.2 73.4 65.1 14.6 383
More than 89.1 87.3 83.8 75.2 87.3 83.8 75.2 80.5 72.2 7.9 211
primary
Information not 59.3 57.3 49.8 37.1 57.3 49.8 37.1 47.6 35.0 36.6 202
collected

Total 59.1 59.0 53.4 44.8 59.0 53.4 44.8 49.6 42.0 37.9 6715b

BCG, bacille Calrnette-Guerin: DPT, diphtheria-pertussis-tetanus.


Source: Macro International Ine. Demographic andhealth surveys, 1991-92. Reproduced bypermission.
a Children who are fully vaccinated (that is, those who have received BCG, measles and three doses of DPT and polio

vaccines).
Note: The DPT coverage rate for children without a written record is assumed to be the same as that for polio vaccine,
since mothers were specifically asked whether the child had received polio vaccine. For children whose information was
based on the mother's report, the proportion of vaccinations given during the first year of life was assumed to be the
same as for children with a written record of vaccination.
b Editors' note: Differences between the total numbers of children accounted for under the different Characteristics"
11

headings are not explained in the original source.

• Which tests should be used to determine whether the proportion of males


who are fully vaccinated is different from the proportion of females who are
fully vaccinated?
• Identify which of the variables show significant differences in the proportions
of fully vaccinated children.
• Give three reasons why your conclusions may not be correct either medically
or statistically.

• Explain the different errors that can be made in a statistical test of a


hypothesis.
OUTLINE 9 TESTS OF STATISTICAL SIGNIFICANCE

85

References
Bland M. An introduction to medicalstatistics. Oxford, Oxford University Press, 1987.
Colton T. Statistics in medicine. Boston, Little, Brown, 1995.
Daly LE, Bourke GJ, McGilvray J. Interpretation and uses of medical statistics. Oxford, Blackwell,
1985.

Dawson-Saunders B, Trapp RG. Basic and clinical biostatistics. Norwalk, Appleton & Lange, Prentice-
Hall International, 1990.
Dixon RA. Medical statistics: content and objectives of a core course for medical students. Medi-
cal education, 1994,28:59-67.
Feinstein AR. Clinical biostatistics. St Louis, Mosby, 1977.
Huff D. How to lie with statistics. New York, Norton, 1954.
Kirkwood BR. Essentials of medicalstatistics. Oxford, Blackwell, 1988.
Warwick DP, Lininger CA. The sample survey: theory and practice. New York, McGraw Hill, 1975.
HANDOUT 9.1

Definitions of new terms and concepts

1-tailed and 2-tailed tests: When the difference being tested for significance is not specified in direction (that is,
takes noaccount of whether Xl < X2 or Xl > X2) , then theprobabilities inboth tails of the sampling distribu-
tion are used in the test: a 2-tailed test is required. When the difference being tested isdirectionally specified
beforehand (when Xl < X2, but not Xl > X2, is being tested against the null hypothesis Xl == X2), then a 1-
tailed test is appropriate because we are only concerned with the probability P(Xl < X2) and not P(Xl > X2) .
Level of significance: The probability of a difference arising purely by chance, below which it is considered suffi-
ciently "unlikely" for the difference to be considered statistically significant (conventionally 0.05). The prob-
ability of wrongfully rejecting the null hypothesis.
Null hypothesis: The hypothesis of "no difference" or, more correctly, the hypothesis that the observed difference is
entirely due to sampling error, that is, that it occurred purely by chance. In a test of significance, the" null
hypothesis" is postulated to establish the basis for calculating the probability that the difference occurred
purely by chance. When the difference is not significant, the null hypothesis is not rejected; when the differ-
ence is significant, the null hypothesis is rejected in favour of other hypotheses about the causes of the
difference. Note thatthe null hypothesis is never proved completely right orwrong, or true or false, but is only
rejected or not rejected at the probability level of significance concerned, for example, 0.05 or 0.01.
p-value: The probability of obtaining the results or more extreme results than those observed in the study under the
null hypothesis.
Statistical significance: The concept by which results are judged as due to chance or not.
Type 1 and 2 errors: Type 1 error isthe risk of erroneously rejecting a null hypothesis that is really true. Type 2 error
is the chance of erroneously failing to reject a null hypothesis that is, in fact, false.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 86


HANDOUT 9.2

Worked example of the z-test for comparing


two proportions

Data situation
A rural health survey investigated 124 households in avillage and recorded their sources of water supply. By reviewing
thevillage's health centre morbidity records for a period of three months prior to the survey, it was possible to identify
household members with a history of diarrhoeal episodes. A total of 88 used the river forwater supply and 49 of them
had episodes of diarrhoea, as against 10 from the 36 households using the well. There was no piped water in this
village. Is there a statistically significant difference in the proportions with episodes of diarrhoea between the house-
holds using river and well water supplies?

Solution
Null hypothesis: there is no difference in the proportion with episodes of diarrhoea between household members
using river or well water supplies.
Alternative hypothesis: there is a difference in the proportion of diarrhoea episodes as a result of different sources of
water supply. (Note that this is a 2-tailed test as no direction is indicated for thedifference in episodes of diarrhoea.)
Level of significance: 0.05.
Test statistic: The z-test for proportion is chosen as appropriate here:

SE(P1 - P2 ) == standard error of difference in proportion

or SE(P1 - P2) = セサpHQ - p)(l/ n, + 1/n2)}


where P == セH + r2 ) / (n 1 + n2 )

r, and r2 are the numbers with attributes (in this case episodes of diarrhoea) in each group;
n, and n2 are the sample sizes in each group.
In our data situation,
t, == 49
n, == 88
r2 == 10
n2 == 36

P1 == 49/88 == 0.5568

P2 == 10/36 == 0.2778

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 87


HANDOUT 9.2 (continued)

1 - P1 == 39/88 == 0,4432
1 - P2 == 26/36 == 0.7222

Z= (0.5568 - 0.2778)/ セサHPNUVX x 0.4432)/88 + (0.2778 x 0.7222)j36}

z == 3.044,

Conclusion
Checking with the table of the normal distribution shows that the value of z at the 50/0 level is 1.96; therefore we
reject the null hypothesis that the proportion with diarrhoeal episodes is the same in the two groups of households
using the different sources ofwater supply. The difference intheproportion with episodes of diarrhoea is unlikely to be
due to chance, p < 0.05. In fact, it appears that the household members using the river have statistically significantly
more episodes of diarrhoea than those using the well. However, to establish a causal relationship, further investiga-
tions would have to be done.
Note: These data can also be tested by the X2 test, but the results have to be presented in a 2 x 2 contingency table,
as shown below.

Table 9.3 Three-month history of diarrhoeal episodes


Number of
households according
to water supply

Status River Well Total

Nodiarrhoea 39 26 65
Diarrhoea 49 10 59
Total 88 36 124
Percentage with diarrhoea 56,7 27,8 47,6

The hypothesis to be tested will now be that of no association between diarrhoeal episodes and source of water
supply. The data in fact indicate that an association exists between diarrhoeal episodes and source of water in the
village (with diarrhoeal episodes in 56.7 % and 27.8 % of households using river and well water, respectively).

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 88


HANDOUT 9.3

Worked example of the t-test

The following data are from a study to compare the mean concentration of lead (in mg/1 00g)in the blood of a group
of workers in a battery plant (exposed) with that of a group of workers in a textile factory (not exposed).

Table 9.4 Mean concentration of lead (in mg/100g) in the blood of


workers in a battery plant and a textile factory
Battery workers Textile factory workers
(X1) (X2 )

0.082 0.040
0.080 0.035
0.079 0.036
0.069 0.039
0.085 0.040
0.090 0.046
0.086 0.040

Battery workers Textile factory workers

LXl == 0.571 LX2 == 0.276


セxl == 0.046847 セxl == 0.010957
セxl == 0.0002697143 セxl == 0.0000757143
セU == 0.0000449524 5; == 0.0000126190
51 == 0.0067047 52 == 0.0035523
Xl == 0.08157 X2 == 0.03943
n, == 7 n2 == 7

where Xl == Xl - Xl
and X2 == X2 - X2
We find

:!- (pooled) == 0.000028786


and SEd == sJ(1/ n, + 1/n2) == 0.002868

where the suffixes 1 and 2 refer to battery workers and textile factory workers, respectively, and SEd is the standard
error of the difference in mean lead concentrations between the two groups.

The null hypothesis (Ho) isthat there is nodifference in the mean lead concentration inthe blood of theworkers ofthe
two industries. This implies a 2-tailed test. We have

d == Xl - X2 == 0.042414.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 89


HANDOUT 9.3 (continued)

The difference is tested against zero, with


t == d/SE d, with tn, + n2 - 2) degrees of freedom
== 14.7, with 12 degrees of freedom; p < 0.001 (see Table B.2 of Annex B).
The null hypothesis istherefore rejected. There isevidence of a significant difference in the mean lead concentration in
the blood of the workers of the two industries.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 90


outliセe 10 Association, correlation and regression

Introduction to the lesson


The idea of causal relationships lies behind much medical decision-making, in both the preven-
tive and therapeutic fields. As much of the evidence for relationships in medical science is
of a statistical nature, students need to understand the statistical basis of such information or
knowledge about relationships, in order to be able not only to appreciate the limitations of
conclusions that they read about in the literature, but also to evaluate their own experiences
more rationally, quantitatively and objectively.

Objective of the lesson


The objective of this lesson is to give the students an understanding of the nature of statistical
evidence for relationships between different characteristics or events in a population, and to
enable them to use and interpret the statistical methods and indices employed to describe and
measure such relationships.

Enabling objectives
At the end of the lesson the students should be able to:

(a) Give examples of types of questions concerning health or medicine that are answered by
analysis of statistical association or correlation.

(b) Explain the concept of association between two categorical variables.

(c) Describe a contingency table.


2
(d) Carry out the X test when required, with the help of reference material.

(e) Explain the concept of relationship between two quantitative variables presented in a scatter
diagram.

(f) Distinguish between linear and non-linear relationships.

(g) Interpret the value of a coefficient of correlation.

(h) Assess the statistical significance of the sample correlation coefficient.

(i) Explain the concept and application of linear regression.

(j) Plot a regression line when the equation is given.

(k) Use linear regression for interpolation and prediction.

(I) Differentiate between statistical and causal relationships.

Required previous knowledge


The contents of all previous lessons.

91
TEACHING HEALTH STATISTICS

92

Lesson content
Situations in which analysis of statistical association or correlation can provide
answers
• Studies of two or more variables measured on the same subject (or unit of
inquiry) where the interest is in their relationship

Association between two categorical variables


• Cross-tabulation and contingency tables
• Distinction between contingency tables and other statistical tables
• Calculation of expected cell frequencies under the null hypothesis
• Importance of testing statistical significance of the association

The X2 test
Procedure
• The null hypothesis
• Calculation of expected frequencies (E) for each cell under the null
hypothesis
• The concept of degrees of freedom
• Calculation of the X2 statistic
• Correction for continuity, for 2 X 2 tables

Li mitations
• Effect of small expected frequencies
• Applicability only to categorical data

Interpretati 0 n
• Use of the table of the theoretical distribution of X2 to determine significance

Relationship between two quantitative variables


(refer to Figures 10.1-10.7 in Handout 10.2)
Pearson coefficient of correlation (r)
• Scatter diagram and its use in indicating the nature and strength of the
relationship
• Linear (Figures 10.1, 10.3, 10.4 and 10.7) and non-linear (Figures 10.2 and
10.6) relationships
• Difference in scatter in the case of strong (Figures "10.1 and 10.3) and weak
(Figures 10.4 and 10.7) relationships
• Positive (Figures 10.3 and 10.4) and negative (Figures 10.1 and 10.7)
• Pearson correlation coefficient as a measure of strength and direction of
linear relationship
OUTLINE 10 ASSOCIATION, CORRELATION AND REGRESSION

93

Properties
• Unit free (the coefficient r is an absolute number)
• Independent of change of origin and scale
• Lies between - 1 and +1

Magn itude, sign and interpretation


• Meaning of a particular value of r
• Interpretation of the magnitude and sign of r for a linear relationship

Uses
• Measure of the strength of association between two quantitative variables

Misuses
• Concluding no relationship from zero correlation, while in fact a strong non-
linear relationship may exist
• Unwarranted conclusion from spurious correlation
• Concluding a cause-effect relationship from a correlation, while it might just
be an indirect relationship
• Concluding an agreement between pairs of measurements, while they may
not have the same values at all points

Computation of the correlation coefficient


Using the formula available in text books or using programmable calculators or
computers. Different books may give different versions of the formula. The teacher
should decide which version would be the easiest for the students to use.

Assessing the statistical significance of a coefficient of correlation


• Procedure
• Limitations
• Degrees of freedom
• Interpretation

Linear regression
A regression estimates the nature of the relationship. The concept and applica-
tions of linear regression should be covered, with explanation of the terms de-
pendent and independent variables. A description of the regression line should
be given.

Definition and characteristics of the linear regression


• The equation Y == hX + c
• The regression coefficient or slope b, given by L(X - X) (Y - Y) jL(X - X)2
• The intercept c, given by Y - hX
TEACHING HEALTH STATISTICS

94

Computation of the linear regression


• The computation of band c
• Plotting the line on the scatter diagram

Uses
• Measure of linear association
• Interpolation
• Prediction

Misuses
• Extrapolation without assurance that the trend remains the same
• Using a regression relationship whose slope has been shown to be not signifi-
cantly different from zero
• Forgetting that the predicted values are subject to sampling error
• Concluding that a cause-effect relationship exists, whereas the relationship
may just be statistical
• Applying a relationship established in one group of subjects to another group,
without the assurance that it is applicable to all groups

Association; bivariate relationship; multi-factorial relationship; neteffect; cross-tabulation; X2; con-


tingency table; cell frequency; rowtotal; column total; grand total; expected frequency; independ-
entand dependent variables; linear and non-linear relationships; spurious correlation; agreement;
slope and intercept; regression coefficient; extrapolation and interpolation.

Structure of the lesson


(a) Introduce the topic by describing the general purpose of correlation and regression analyses.
Give examples from the current literature on topics of local interest. Illustrate the statistical
nature of the relationships and the importance of studying those relationships.
For example:
• smoking and lung cancer;
• intake of iron and folic acid in diet or as supplement and haemoglobin level;
• mother's education and size of family;
• quality of drinking-water and diarrhoea;
• height of a person and height of his or her father;
• energy intake by a woman during pregnancy and birth weight of her child;
• severity of disease and cure rate.
(b) Differentiate between the nature of relationships among categorical and quantitative
variables. Explain the need to have different procedures for the two types of variables.
OUTLINE 10 ASSOCIATION, CORRELATION AND REGRESSION

95

2
(c) Recapitulate the basic principles of hypothesis testing. For the X statistic as a means for
testing the statistical significance of the association between two categorical variables, give a
heuristic explanation of the formula of X2 so that the students realize that (0 - E)2/ E is a
measure of deviation from independence. Emphasize that E is the cell frequency expected
under the null hypothesis of independence (that is, of no association).
(d) Explain the concept of degrees of freedom by giving actual examples of, say, 2 X 2 and
2 X 3 tables illustrating the "freedom" to choose frequencies in one and two cells, respec-
tively, under the constraint of fixed marginal totals. Stress the interpretation of a significant
X2 as mere presence of association, with no implication for the strength of the association.
Point out that the magnitude of X2 is severely affected by n. Indicate what further calcula-
tions are required to measure the strength of the association.
(e) Distinguish between linear and non-linear relationships between two quantitative variables
by giving examples, as shown in Handout 10.2. Emphasize that the coefficient of correlation
measures only the linear component of the relationship, which may not exist in some cases,
despite the presence of a strong non-linear relationship. Make liberal use of diagrams to
illustrate the magnitude and direction of correlation coefficients.
(f) Use scatter plots to explain the fluctuations around a line, even in the case of a linear rela-
tionship. In the case of high fluctuations, the predictive value of the relationship can be
reduced substantially.
(g) Briefly explain the statistic [r J(n - 2)] / [J(1 - r)] following Student's t distribution with
n - 2 degrees of freedom, subject to the normality of either X or Y. This is just to test the
hypothesis that the correlation is zero in the population. For testing other values of correla-
tion, tests based on Fisher's z transformation are required. Explain the role of the sample size
in placing confidence on the value of a coefficient of correlation.
(h) Briefly explain the use of the t-test for the hypothesis of no correlation. This test does not
provide any clue to the magnitude of the correlation. A better indication of the magnitude of
correlation can be obtained by computing 100 X (1 - r), as the percentage of the variation
in the dependent variable is explained by its association with the independent variable. Also
mention that the t-test requires normality of Y, particularly for small samples, and that this
test should not be used indiscriminately.
(i) Discuss the need to obtain the nature of the relationship in the form of an equation. Restrict
the lesson to linear relationships only.
(j) Come back to the scatter plots used earlier in the context of correlation and illustrate various
types of regression lines. Use the illustrations in Handout 10.2 to explain the meaning of the
slope measured by the regression coefficient and of the intercept. Show the equivalence of
the use of rand b to indicate association. Give examples of situations for preferring one over
the other.
(k) Superimpose scatters of different variability and explain how variability affects the reliability
of predictions - whether extrapolated or interpolated.
(I) Discuss the uses and misuses of regression lines on the basis of the examples chosen from
published literature.

Lesson exercises
The teacher should give two different kinds of health-related data to the students: one set for
2 X 2 cross-tabulated data of two discrete variables, and the other for two quantitative variables
measured on the same individuals. The exercise should focus on enabling the students to pro-
duce a scatter diagram and to carry out appropriate procedures to test association between the
TEACHING HEALTH STATISTICS

96

variables in each of the two situations. The students should also be tested on their ability to
interpret the results of the tests .

• Give students a 2 X 2 table with relatively small frequencies, so that the X2 is


not significant. Multiply each frequency by 10 so that the proportion remains
the same. Compute X2 again and see how dramatically the value and, conse-
quently, the significance change. An example could be as follows:

Favouring sex education


Yes No Total
Male 8 5 13
Female 5 7 12

Total 13 12 25

x ==
2
0.987, df == 1, P > 0.25. When each frequency is multiplied by 10, then
X2 == 9.87, df== 1 but now p < 0.001.

• Give some scatter plots of known data and ask the students to make an edu-
cated guess of the magnitude and direction of the coefficient of correlation. In-
clude among the scatter plots at least one random plot (no correlation) as well as
at least one with a non-linear relationship. Let the students calculate r to check
how good their guesses were.

• Give some regression lines with different slopes and ask the students to inter-
pret each of them. Superimpose scatter plots on them with different variability
and let the students describe the impact of variability on the reliability of the
conclusions based on the regression equation.

• Use the data on age, height and weight of a male preschool child, followed up
from the age of six months, to draw a scatter diagram and find the best regres-
sion line for age and weight.

Age (months) Height (cm) Weight (kg)

6 66.9 7.1
7 68.5 7.2
12 72.0 7.8
16 77.0 8.3
18 79.0 8.9
22 82.1 9.2
24 82.7 9.5
26 84.2 10.4
30 86.0 11.0
32 86.5 10.8
34 89.5 11.4
35 89.7 11.8
43 95.0 13.0
OUTLINE 10 ASSOCIATION, CORRELATION AND REGRESSION

97

References
Armitage P, Berry G. Statistical methods in medical research. Oxford, Blackwell, 1987.
Campbell MJ, Machin D. Medical statistics. Chichester, John WHey, 1990.
Colton T. Statistics in medicine. Boston, Little, Brown, 1995.
Edwards AL. An introduction to linear regression and correlation, 2nd ed. New York, WH Freeman,
1984 (first six chapters only).
Kirkwood BR. Essentials of medical statistics. Oxford, Blackwell, 1988.
Last JM, ed. A dictionaryof epidemiology, 3rd ed. New York, Oxford University Press, 1995.
Wang C. Sense and nonsense of statistical inference: controversy, misuse and subtlety. New York, Marcel
Dekker, 1993.
HANDOUT 10.1

Definitions of new terms and concepts

Association: 1 The degree of statistical dependence between two or more events or variables.
Bivariate relationship: Association (or relationship) between two variables.
Cell frequency: The number of observations in a cell of a contingency table.
Column total: The total number of observations in a column of a contingency table.
Contingency table:1 A tabular cross-classification of data such thatsubcategories of one characteristic are indicated
horizontally (in rows) and subcategories of another characteristic are indicated vertically (in columns).
Dependent variable: In a regression analysis, this is the variable of which the value is thought to be predictable
from another variable.
Expected frequency: The number of observations to be expected in a class or cell if the null hypothesis is true.
Extrapolation: The use of the regression line to predict a value of the dependent variable from that of the indepen-
dent variable outside the range of values actually observed.
Grand total: The total number of observations cross-classified in a contingency table.
Independent variable: The variable, in a regression analysis, of which the value is thought to be predictive of
another variable.
Interpolation: The use of a regression line to estimate a value of thedependent variable from that of the independ-
entvariable within the range of values actually observed.
Linear relationship: In a regression analysis, when the mathematical model describing the dependent variable in
terms of the independent variable is in the form of a straight line.
Multi-factorial relationship: Association (or relationship) between several factors or variables.
Non-linear relationship: When the form of the model describing y in terms of x is not a straight line.
Regression analysis: 1 Given data on a dependent variable yand an independent variable x, regression analysis
involves finding the" best" mathematical model (within some restricted form) to describe y as a function of x
or to predict yfrom x.
Regression coefficient(s): For a linear regression, these are the estimated slope and the intercept of the straight
line describing the dependent variable as a function of the independent variable.
Row total: The total number of observations in a row of a contingency table.
Spurious correlation: 1 An association between two variables that may be artefactual, fortuitous, false ordueto all
kinds of non-causal associations resulting from chance or bias.

1 From Last JM (ed.) A dictionary of epidemiology, (3rd ed.) New York, Oxford University Press, 1995.

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 98


HANDOUT 10.2

Diagrammatic presentation of different


types of correlations

Figure 10.1 Strong negative correlation (r = -1)


0 10 20 30 40 50 60 70 80 90
0
•••

-10

... •
-20 ...... .
-30
WHO 97533

Figure 10.2 Non-linear

100

80 •
..........


60
••
40 •

20


oMKセイNL
o 20 40 60 80 100

Figure 10.3 Strong positive correlation (r = 1)


30

.-' -
20


.- •

10
••

•• •
0
0 10 20 30 40 50 60 70 80 90 WHO 97530

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 99


HANDOUT 10.2 (continued)

Figure 10.4 Positive correlation (0 < r< 1)


40

30
• ••
• • •
• • ••
20 •
••
••
10 •
• • •
0
0 10 20 30 40 50 60 70 80 90
WHO 9753 1

Figure 10.5 No correlation (r = 0)


100
• •
• •
• • • •
80 •
• •
60
• • •
• • •
40

• • •

20
• •
• •

0
0 20 40 60 80 100
WHO 97529

Figure 10.6 Non-linear

100

•••
80 • •
• •
60 • •
• •
40 • •
• •
20 • •
• •
••
0
0 20 40 60 80 100 120 140

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 100


HANDOUT 10.2 (continued)

Figure 10.7 Negative correlation (-1 < r < 0)


10
• 10 • 20 30 40 50 60 70 80 90
0
• •
-10 • •

• • •• • --.
-20
••
-30
WHO 97532

TEACHING HEALTH STATISTICS © WORLD HEALTH ORGANIZATION 1999 101

You might also like