Validity and Reliability of Partial Examination To Assess Severe Periodontitis
Validity and Reliability of Partial Examination To Assess Severe Periodontitis
Validity and Reliability of Partial Examination To Assess Severe Periodontitis
periodontitis
Health Research, Portland, OR, USA and
3
Facultad de Odontologı́a, Universidad de
Campeche, Campeche, Mexico
Abstract
Objectives: To compare the extent and severity index (ESI) with a gold standard
represented by actual readings of loss of attachment on six sites around all teeth
present (excluding third molars).
Methods: Five standardized dentists (k 5 0.6) examined 712 subjects X20 years of
age at a dental school (1993–1995). Sensitivity, specificity, positive and negative
predictive values, and true and apparent prevalence were established.
Results: True severe periodontitis prevalence was 95.8%. ESI underestimated the
severity (0.1 mm), extent (4%), and prevalence (16%) of periodontitis. The severity, as
established by ESI, coincided 23.4% with the gold standard. ESI failed to identify
16.7% of subjects with severe periodontal disease, but specificity and positive
predictive value were very high.
Conclusions: The underestimation of severe periodontitis through ESI may lead to
inadequate recommendations for further treatment. Accurately identifying subjects
with severe periodontitis requires a full-mouth examination. Because the ESI relies on
measurements taken on only 28 periodontal sites to estimate the periodontitis status of
Key words: epidemiology; extent and severity;
the entire mouth, the validity and reliability of ESI may be modified by the prevalence periodontal disease; reliability; validity
of severe periodontal disease and the distribution of disease according to age and
operational definitions. Accepted for publication 10 April 2003
As in the case of other clinical entities tative certain sites and certain teeth are ing only a few sites rather than every
assessed from an epidemiological per- of the status of the entire mouth. Partial periodontal site.
spective, signs of periodontal disease measurements, as opposed to full-mouth The question remains as to whether
have been used as indicators of period- measurements, have traditionally been certain periodontal sites – e.g., those
ontitis. The many indices developed to employed in periodontal epidemiology more accessible to examination such as
measure such signs (for example, Ramf- to make the assessment effort less time- mid-buccal and mesio-buccal, which
jord 1956, Greene 1974, Löe 1974, consuming and less costly. Some of have been used in large surveys (Miller
Massler 1974, O’Leary 1974, Russell these approaches have targeted one et al. 1987) – accurately reflect pre-
1974, Barmes et al. 1986) have diverse upper and one lower quadrant (Carlos valence and severity of periodontitis at
shortcomings. Most importantly, an et al. 1986, Brown et al. 1990), the population level. Various attempts
ideal index should be reproducible randomly selected, either contralateral have been made to quantify the detri-
and valid, in that the measurements or on the same side of the mouth. The mental effect of partial measurements
ought to reflect adequately the variable underlying assumption is that period- (Alexander 1970, Downer 1972, Mills
the index was designed to describe. The ontal disease affects dentition in a et al. 1975, Ainamo & Ainamo 1985,
many indices designed over the years symmetrical manner across the midline Haffajee et al. 1985, Carlos et al. 1986,
have focused on different analysis units – (Hirschfeld & Wasserman 1978, Löe et Hunt 1987, Kingman et al. 1988, Hujoel
periodontal site, tooth, person. Conse- al. 1978a, b, Burmeister et al. 1984, & Loesche 1990, Hunt & Fann 1991,
quently, there have been various inter- Listgarten et al. 1989). If this premise is Diamanti-Kipioti et al. 1993, Papapanou
pretations of diverse features of perio- true, a periodontal assessment may be et al. 1993). No unequivocal answer has
dontal disease, including how represen- accurate and more efficient by examin- been found, however.
Extent and severity of periodontitis 113
One of the more recent indices, the Five examiners recruited from the proportions. True and apparent pre-
extent and severity index (ESI) (Carlos resident program in periodontology valence of periodontitis were calcu-
et al. 1986), has enjoyed substantial were trained and standardized to con- lated by age group. The extent and
popularity, because it not only estab- duct the exams in a fully equipped the severity of periodontitis were
lishes the severity of periodontitis as the dental clinic. Examiners used a dental determined by measuring all 168 sites,
mean attachment loss in diseased sites, chair, an air/water syringe, a dental 140 sites, or the 28 ESI sites and
but also incorporates the proportions of mirror, Michigan periodontal probes, contrasted by sex and age group. The
all sites that are affected by period- and Gracey periodontal curettes. A pilot 28 sites comprised by ESI were elimi-
ontitis (extent). Because the ESI mea- study in 15 patients (who were not nated from this contrast to remove the
sures attachment loss in only 28 sites in included in the final study group) was bias resulting from using those sites
contralateral quadrants, it has the same undertaken to familiarize examiners both in the full-mouth assessment and in
potential for inaccuracy as other partial with the data collection process, refine ESI (Fleiss et al. 1987). The reliability
measurements. The present investiga- data audit procedures, and evaluate of the diagnosis of periodontitis was
tion compares the sensitivity, specifi- intra- and inter-examiner agreement. obtained by comparing the severity
city, positive and negative predictive Intra- and inter-examiner agreement from ESI with that from the 140 sites
values, and true and apparent preva- (k) each were 0.6, including 1.0 mm. using Cohen’s k. The effects of sensi-
lence calculated by ESI with those Data were first collected on paper tivity, specificity, and true prevalence
found using a ‘‘gold standard.’’ Our forms and subsequently entered in a on apparent prevalence and predictive
study also measured the strength of computer database. After data quality values were estimated.
correlation between the two components audits were implemented, the period- This study design adhered to the recom-
of the index and the attachment loss ontal measurements were analyzed mendations laid out by the Institutional
readings derived from the gold standard. using NCSSs (Kaysville, UT, USA) Review Board of the National Univer-
and WIN EPISCOPE 1.0as (Programa sity Dental School.
epidemiológico de dominio público.
Financiado con el proyecto CONS I+D
Material and Methods P50/98 del Gobierno de Aragón). Data
We examined adult patients referred to analyses initially used student’s t-test Results
the Periodontology Specialty Clinic at and one-way ANOVA. The mean loss of A total of 712 participants X20 years of
the National University Dental School attachment was obtained for each site; age were examined. The mean age and
in Mexico City between January 1993 the extent of periodontitis (proportion of SD among women was 45.2 13.5
and November 1995. Data were ob- sites affected by periodontitis) was years, and among men 47.1 14.4
tained following a standardized protocol calculated using the criteria listed in years ( p40.05). Table 2 presents the
that included operational definitions laid Table 1. distribution by age and gender. As a
out for examiners (Table 1). Loss of ESI validity was obtained by calcu- whole, 15,160 teeth were examined; the
attachment was determined in the six lating sensitivity, specificity, and posi- mean number of teeth present was 21.3
periodontal sites (disto-buccal, mid- tive and negative predictive values (21.0 in women and 21.7 in men). The
buccal, mesio-buccal, mesio-lingual, overall and by age group. Confidence mean number of teeth present declined
mid-lingual, and disto-lingual) of each intervals were established using a nor- from 25.0 in the 20–29-year-old group
tooth except third molars. mal approximation to standard error for to 16.8 in the 601 group ( po0.001)
Table 2. Distribution of the study population by age group and gender would identify 568, missing 16.7%.
Specificity values indicated that ESI
Age (years) Gender
would correctly identify 27 (90%) of 30
female male total persons without severe periodontitis.
The positive predictive value was
n % n % n % 99.4%, as only three persons identified
through ESI as affected by severe
20–29 55 12.3 34 12.8 89 12.5
30–39 111 24.9 51 19.2 162 22.8 periodontitis (n 5 571) would not have
40–49 127 28.5 71 26.7 198 27.8 that level of disease. The negative
50–59 80 17.9 63 23.7 143 20.1 predictive value was only 19.1%, how-
X60 73 16.4 47 17.6 120 16.9 ever, as 114 (80.9%) persons would be
total 446 100 266 100 712 100 incorrectly considered not to have
severe periodontitis (Table 6).
ESI measurements have significant
Table 3. Mean number of teeth by age group and gender correspondence with actual incre-
ments in severity (r 5 0.82, p 5 0.01)
Age (years) Gender n Mean SD Sum Min Max Range
and extent (r 5 0.88, p 5 0.01) values
20–29 female 55 24.76 3.17 1362 11 28 17 (Figs 1 and 2).
male 34 25.38 3.66 863 7 28 21 In Fig. 3, we present variations in
total 89 25.00 3.36 2225 7 28 21 apparent prevalence and predictive va-
30–39 female 111 22.94 3.83 2546 10 28 18 lues derived from ESI measurements
male 51 24.90 2.57 1270 18 28 10 when true prevalence is modified. When
total 162 23.56 3.59 3816 10 28 18
the latter is 100%, apparent prevalence
40–49 female 127 21.50 4.49 2730 5 28 23
male 71 21.04 5.29 1494 9 28 19 is only 83%; the positive predictive
total 198 21.33 4.78 4224 5 28 23 value tends to increase, and the negative
50–59 female 80 19.02 5.79 1521 6 28 22 predictive value tends to decrease.
male 63 21.59 4.46 1360 11 28 17 Generally speaking, ESI underestimates
total 143 20.15 5.38 2881 6 28 22 true prevalence.
X60 female 73 16.64 6.89 1214 2 28 26 At the individual site level, we found
male 47 17.00 6.84 799 3 28 25 that the locations with higher mean loss
total 120 16.78 6.84 2014 2 28 26
of attachment were the disto-buccal and
total female 446 21.02 5.55 9374 2 28 26
male 266 21.75 5.55 5786 3 28 25 disto-lingual sites of the upper first
total 712 21.29 5.56 15,160 2 28 26 molars and the mesio-buccal and me-
sio-lingual sites of the lower central
ANOVA, F 5 47.9, po 0.001 for age group. incisors. Table 7 shows the proportion
ANOVA, F 5 2.9, p 5 0.08 for gender. of times individual sites had attachment
loss X2 mm; highest values were found
in the disto-buccal and disto-lingual
(Table 3). Furthermore, extent and (3.7 mm) (p 5 0.001) using the gold sites of the upper first molars. The
severity values increased with older standard; ESI measurements also con- least-affected sites were the mid-buccal
age, regardless of the number of sites cluded that differences were significant. sites from the upper central incisor to
being evaluated. Using the gold standard, the extent of the upper first premolar.
For the overall population, ESI found periodontitis was found to be higher in
a mean loss of attachment of 3.7 mm, men (70.5%) than in women (66.7%)
while the gold standard reported 3.8 mm (p 5 0.04); ESI measurements were not
( p 5 0.03). ESI found that 64.4% of the significantly different (Table 5). Discussion
sites were affected, compared with Table 6 includes reliability values The present study is not the first
68.3% for the gold standard (p 5 contrasting the number of participants evaluation to compare the validity or
0.001). Mean percent extent differences deemed to have no severe periodontitis reliability of partial measurements com-
were identified between the gold stan- and severe periodontitis according to pared with full-mouth assessments. This
dard and the 28-site assessment for the ESI (28 sites), and according to the gold investigation, however, incorporated a
30–39-year-old group (t 5 2.6, p 5 standard (140 sites). The ESI classified carefully designed gold standard with
0.009) and in the 40–49-year-old group 27 persons with non-severe periodonti- six periodontal sites measured per tooth
(t 5 2.5, p 5 0.012). Differences were tis as non-severe. And 568 participants in a very large number of participants
also identified in the mean severity were considered to have severe period- with established, advanced periodontal
reading obtained from the 28-site as- ontitis by both ESI and the gold problems, objectively screened and
sessment and the gold standard for the standard. The agreement between diag- categorized. Furthermore, estimates for
50–59-year-old group (t 5 2.9, p 5 noses was 0.836, but when agreement true prevalence of severe periodontitis
o.001). In participants older than 60 by chance was controlled through k, this have not been included in the previous
years of age, the ESI and the gold coefficient was reduced to only 0.265. reports.
standard did not differ significantly In terms of ESI validity, it was found In the present study, we determined
(Table 4). Periodontitis was more severe that sensitivity was 83.3%. Out of 682 how close the estimates of apparent and
in men (4.0 mm) than in women persons with severe periodontitis, ESI true prevalence were when ESI was
Extent and severity of periodontitis 115
Table 4. Extent and severity of periodontitis by age group according to the different evaluations can be ascribed to the measurements
undertaken. An instrument with poor
Age (years) n Severity (mean SD, mm) Extent % (mean SD)
reliability can be expected to misclassi-
168 sites n
140 sites nn
28 sites nnn
168 sitesw 140 sitesww 28 siteswww fy subjects, leading to erroneous con-
clusions about the epidemiological
20–29 89 3.0 0.6 3.1 0.6 3.0 0.7 51.8 25.6 52.4 68.9 48.9 26.5 profile. High reliability means less
30–39 162 3.6 1.0 3.6 1.0 3.6 1.1 64.9 23.6 65.9 25.8 59.6 25.5 chance of realizing certain types of bia-
40–49 198 3.8 1.1 3.8 1.2 3.7 1.2 68.8 23.2 69.8 23.8 64.1 25.9
ses that undermine information quality.
50–59 143 4.0 1.2 4.1 1.3 3.7 1.2 72.1 21.9 72.6 23.0 69.4 22.9
X60 120 4.3 1.6 4.3 1.6 4.2 1.8 78.9 20.6 79.4 20.8 76.6 23.7 The contrasting results that have been
total 712 3.8 1.2 3.8 1.2 3.7 1.3z 68.3 24.1 68.9 18.3 64.4 26.2zz reported in studies aiming to identify
the presence and prevalence of severe
Differences among age groups. periodontitis stem, to a considerable
n
F 5 15.1, po0.001. extent, from the indices and the opera-
nn
F 5 15.0, po0.001.
nnn tional definitions of disease used
F 5 11.1, po0.001.
w (Moore et al. 1982, Burmeister et al.
F 5 16.9, po0.001.
ww
F 5 16.6, po0.001.
1984, Brown et al. 1996). Such defini-
www
F 5 15.4, po0.001. tions have included, for example, four
Differences between ESI and the gold standard. or more sites with a periodontal pocket
z
t 5 2.18, p 5 o.001. X5 mm (Robertson et al. 1987); eight
zz
t 5 3.85, p 5 .029. or more teeth with loss of attachment
X5 mm (Moore et al. 1982, Burmeister
et al. 1984); ‘‘at least eight or more
Table 5. Extent and severity of periodontitis by gender according to the different evaluations teeth with at least 5 mm of attachment
loss, where at least three of them are not
Gender N Severity (mean SD, mm) Extent % (mean SD)
first molars or incisors’’ (Vrahopoulos
168 sites 140 sites 28 sites 168 sites 140 sites 28 sites 1995); and 4 or more sites with attach-
ment loss X5 mm, at least one of them
female 446 3.7 1.2 3.7 1.2 3.6 1.2 66.7 23.9 67.4 19.4 63.4 26.1 with a periodontal pocket greater than
male 266 4.0 1.2 4.0 1.3 3.8 1.3 70.5 24.3 71.4 16.8 66.0 26.3 4 mm (Beck et al. 1990). This glimpse
total 712 3.8 1.2 3.8 1.2 3.7 1.3 68.2 24.1 68.9 18.3 64.4 26.2
at the diversity of definitions and
Comparison by gender. thresholds suggests that a direct com-
Severity: 168 sites, t 5 13.1, p 5 o0.001. parison across studies is fraught with
Severity: 28 sites, t 5 5.7, p 5 0.017. difficulties. We need to better under-
Severity: 140 sites, t 5 13.9, po0.001. stand the criteria to diagnose period-
Extent: 168 sites, t 5 4.2, p 5 0.041. ontitis objectively for population-based
Extent: 28 sites, t 5 1.7, p 5 0.188. research.
Extent: 140 sites, t 5 4.5, p 5 0.034. Studies that purport to assess the
validity of partial measurements to
determine the presence of periodontitis
Table 6. Reliability and validity of the extent and severity index; distribution of periodontitis by suggest some general trends based on
severity, 28 versus 140 sites (gold standard) similarities in the findings. Using ESI,
28 sites, severe PD 140 sites, severe PD Brown et al. (1996) found higher
severity and extent of periodontitis
yes no total associated with older age. Just as we
established through the present study,
yes 568 3 571 they also found that men had worse
no 114 27 141
periodontitis than women and suggested
total 682 30 712
that the partial evaluation underesti-
Confidence interval 95% mated the prevalence of periodontitis.
k 0.265 0.212–0.317 Using only four sites per tooth, King-
sensitivity 83.3% 80.4–86.1 man et al. (1988) concluded that partial
specificity 90.0% 79.3–100 measurements systematically underesti-
positive PV 99.4% 98.9–100 mate disease prevalence. Our findings in
negative PV 19.1% 12.7–25.6 a study population with substantial
true prevalence 95.8% 94.3–97.3 periodontal involvement (as in the de-
apparent prevalence 80.2% 77.3–83.1
sign by Kingman & Albandar (1997))
PD, periodontitis; PV, predictive value. support this conclusion. Again using
only four sites per tooth, Diamanti-
Kipioti et al. (1993) compared period-
administered to identify severe period- estimated with ESI or with the gold ontal conditions by means of various
ontitis. While severity and extent values standard. Generally speaking, these indices (including ESI). They concluded
tended to increase with older age, the characteristics in an index are of that partial measures offered acceptable
magnitude of the disease differed when importance to determine the weight that estimates of the individual means of
116 Borges-Yáñez et al.
Fleiss, J., Park, M., Chilton, N., Alman, J., valence and severity of periodontal disease. Moore, W. E. C., Holdeman, L. V., Smibert, R.
Feldman, S. & Chauncey, H. (1987) Repre- Journal of Periodontology 59, 707–713. M., Hash, D. E., Burmeister, J. A. & Ranney,
sentativeness of the ‘‘Ramfjord teeth’’ for Listgarten, M. A., Slots, J., Rosenberg, J., R. R. (1982) Bacteriology of severe period-
epidemiologic studies of gingivitis and per- Nitkin, L., Sullivan, P. & Oler, J. (1989) ontitis in young adult humans. Infection and
iodontitis. Community Dentistry and Oral Clinical and microbiological characteristics Immunity 1137–1148.
Epidemiology 15, 221–224. of treated periodontitis patients on mainte- O’Leary, T. (1974) The periodontal screening
Grbic, J. T. & Lamster, I. B. (1992) Risk nance care. Journal of Periodontology 60, examination. Journal of Periodontology 45,
indicators for future clinical attachment 452–459. 617–624.
loss in adult periodontitis. Tooth and site Löe, H. (1974) The gingival index, the plaque Papapanou, P., Wennstrom, J. & Johnsson, T.
variables. Journal of Periodontology 63, index and the retention index systems. (1993) Extent and severity of periodontal
262–269. Journal of Periodontology 45, 610–616. destruction based on partial clinical assess-
Greene, J. (1974) The oral hygiene index – Löe, H., Anerud, A., Boysen, H. & Smith, M. ments. Community Dentistry and Oral Epi-
development and uses. Journal of Period- (1978a) The natural history of periodontal demiology 21, 181–184.
ontology 45, 625–635. disease in man. Study design and baseline Ramfjord, S. (1956) Indices for prevalence and
Haffajee, A., Socransky, S., Goodson, J. & data. Journal of Periodontal Research 13, incidence of periodontal disease. Journal of
Lindhe, J. (1985) Intraclass correlation of 550–562. Periodontology 30, 51–59.
periodontal measurements. Journal of Clin- Löe, H., Anerud, A., Boysen, H. & Smith, M. Robertson, P. B., Buchanan, S. A., Armitage, G.
ical Periodontology 12, 216–224. C., Newbrun, E., Taggart, E. J. & Hoover, C.
(1978b) The natural history of periodontal
Hirschfeld, L. & Wasserman, B. (1978) A long- I. (1987) Evaluation of clinical and micro-
disease in man. The role of periodontal
term survey of tooth loss in 600 treated biological measures to predict treatment
destruction before 40 years of age. Journal
periodontal patients. Journal of Periodontol- response in severe periodontitis. Journal of
of Periodontology 49, 607–620.
ogy 49, 225–237. Periodontal Research 22, 230–232.
Massler, M. (1974) The P-M-A index for the
Hujoel, P. & Loesche, W. (1990) Efficiency of Russell, A. (1974) The periodontal index.
assessment of gingivitis. Journal of Period-
split-mouth designs. Journal of Clinical Journal of Periodontology 45, 585–591.
ontology 45, 592–601. Vrahopoulos, T. (1995) The apical border
Periodontology 17, 722–728.
Hunt, R. (1987) The efficiency of half-mouth Miller, A., Brunelle, J., Carlos, J., Brown, L. & plaque in severe periodontitis. An ultrastruc-
examinations in estimating the prevalence of Loe, H. (1987) Oral health of United States tural study. Journal of Periodontology 66,
periodontal disease. Journal of Dental Re- adults. The National Survey of Oral Health in 113–124.
search 66, 1044–1048. U.S. employed adults and seniors: 1985–
Hunt, R. & Fann, S. (1991) Effect of examining 1986, National findings. Epidemiology and
half the teeth in a partial periodontal record- Oral Disease Prevention Program. National Address:
ing of older adults. Journal of Dental Institute of Dental Research, U.S. Depart- S. Aı´da Borges-Yáñez
Research 70, 1380–1385. ment of Health and Human Services, Public División de Estudios de Posgrado e
Kingman, A. & Albandar, J. M. (1997) Validity Health Service, National Institutes of Health, Investigación
of partial recording of attachment loss in NIH Publication No. 87-2868. pp. 9–11, Facultad de Odontologı´a, UNAM
early onset periodontitis. Journal of Dental 164–165. Circuito Exterior S/N
Research 76, (Spec. Issue): 229 (abstract Mills, W., Thompson, G. & Beagrie, G. (1975) Ciudad Universitaria CP 04510
1722). Partial mouth recording of plaque and Distrito Federal
Kingman, A., Morrison, E., Loe, E. & Smith, J. periodontal pockets. Journal of Periodontal Me´xico
(1988) Systematic errors in estimating pre- Research 10, 36–43. E-mail: aborges@servidor.unam.mx