9-l

POD METHODS AND THE LINK OF AVAILABLE DATA TO FIELD PROCESSES

Ward D. Rummel
D&W Enterprises, LTD
8776 W. Mountainview Lane,
Littleton, CO 80125-9406.
United States of America
I. SUMMARY
An experimental procedure is described for transferring
nondestructive evaluation (NDE) procedure performance
(probability of detection - POD) capabilities, that have been
validated on simple specimens, to complex conligurations found in
field applications. Methodologies and logic are discussed.
Requirements and cautions in use of the method are discussed.
2. INTRODUCTION
Increasing materials knowledge, demand for more efficient
structures and systems, and demand for life-extension of aging
structures and systems have prompted increasing use of damage
tolerance requirements in engineering design, maintenance, rework
and life-cycle management. Implementation of damage tolerance
methods requires knowledge and supporting data on: (I) materials
properties; (2) loads and load distribution; (3) functional operation /
service cycles; (4) environment; and (5) inherent flaw sizes,
locations orientations and distributions. The requirement for flaw
knowledge and data is a signilicant addition to prior practices / art.
Flaw detection, flaw sizing, flaw location and orientation must
necessarily be nondestructive in nature. The added requirement for
quantification of nondestructive evaluation (NDE) measurements
presents a significant challenge to the engineering community.
Although some NDE capabilities have been assumed in prior
designs, the assumptions were often faulty and have been shown to
be inadequate by systematic measurement and quantification.
Erroneous assumptions have included: (1) no flaws assumed; (2)
an incorrect detectable flaw size assumed; (3) assumption that NDE
detects all significant flaws; (4) detection assumed to be the
calibration flaw size; and (5) detection assumed to be the
smallest flaw previously detected. Characterization of specific
NDE procedures, NDE technicians and NDE facilities was and is
required.
The metric that has been developed to quantify NDE capabilities
and to provide a method of data exchange is the probability of
detection (POD). Generation of a characteristic POD curve (Figure
1) requires: (1) passing a statistically significant number of
representative flaws through and NDE procedure; (2) the flaw
distribution must be near the expected NDE detection threshold; (3)
flaws located in representative materials, geometries and surface
conditions; (4) systematic control of NDE procedures; and (5)
documentation of the results of application2~3~4.
Fatigue cracks in simple test specimens are frequently used as the
test artifacts. Fatigue cracks have been determined to be
representative of severe detection conditions and are relatively
inexpensive to produce. A large data base has been generated for
NDE capabilities of relatively simple specimens. Simple test
specimen geometries may not be representative of the NDE
challenges in a complex structure or system and methodologies for
transfer of the measured capability to complex shapes are required.
This paper describes such methodology and the rationale used in
application. Transfer of measurements is focused on those NDE
procedures which produces a quantified, scalar output such as eddy
current and ultrasonic methods. The discussions are therefore
intended primarily for those methods and applications
Figure 1. Typical POD curve
3. PROBABILITY OF DETECTION RATIONALE
The capability of an NDE procedure is a direct function of its
signal response output from small flaws and its relationship to the
background application response that is generated by unflawed
areas adjacent to flaw?. The background response is conveniently
termed the NOISE response and must not be conmsed with
electronic noise that is familiar in electronic instrument analyses.
When repetitive measurements of a single flaw are made by an NDE
procedure, a distribution of response values from the flaw are
generated that are similar to those produced in classical mechanical
measurement methods. Simultaneously, a lower level signal
(background) response is generated that is characteristic of the
surface condition, surface texture, grain structure, stress state, etc. of
the test object. This background response is termed NOISE. A
typical response from experimental measurements from a single
flaw is shown in Figure 2.
g
A
-=
3
L
Noise
Signal
3
P
Signal Response Level
Figure 2. Repetitive response from a single flaw
Paper presented at the RTO AVT Workshop on Airframe Inspection Reliability under Field/Depot
Conditions, held in Brussels, Belgium, 13-14 May 1998, and published in RTO MP-10.
9-2
Repetitive response from multiple flaws of equal size results in
broadening of the response distribution as shown in Figure 3. This
broadening is the results of flaw to flaw variations and are
accounted for by using multiple flaws in the generation of a typical
POD curve. The spread between the upper limit of the noise and the
lower limit (signal and noise) of the flaw response enables repetitive
detection and discrimination / identification of flaws of that size
without false calls (Type II errors). For small flaws. the signal and
noise responses overlap and detection / discrimination are not
Noise
Signal
. .
Signal Response Level
Figure 3. Repetitive response from multiple flaws of equal size
Slots generally used for purposes of set-up and calibration of an
NDE system are more readily detected due to their higher signal
response. Figure 4 show a typical response when similar
measurements of a calibration slot (artifact) are added to the
response data from a crack of equal size.
c 4
b
.cn I
Noise
Crack Slot
Signal Response Level
Figure 4. Comparative responses from a crack and a slot of equal
size.
4. TRANSFER OF ARTIFACT RESPONSE
Most classical measurements are made with the aid of a reference
calibration artifact or standard. Calibration standard artifacts
are measured by reference to a master standard that is traditionally
retained as a as a national resource and commonality is achieved by
international agreements to provide a common basis for exchange in
commerce. It was therefore logical that a reference slot has evolved
as a calibration artifact for most NDE measurements and physical
measurement of slot size may be traceable to a national master
standard. Slots are economical to produce with available
technology and are commonly specified in establishing and
applying NDE procedures. Traceability of reference calibration
artifacts (slots) are assumed when they are used in validated NDE
procedures. Unfortunately, a single slot is often used for reference
and set-up and linearity of response of the NDE procedure is
assumed. Modem electronic instruments are produced with linear
response and periodic validation of the response linearity is
performed. Since the electronic instrument constitutes only a part of
the NDE system, system capability validation for each specilic
application is recommended. Figure 5. illustrates a typical causal
model for response to cracks and slots of varying size. For larger
flaws, the response is linear. As the size of the slot / crack
approaches the size of the transducer / probe element. the response
function changes. It is therefore important to validate the functional
resoonse for an NDE procedure. particularly when addressing small
Noise
Actual Artifact Size
b
Figure 5. Typical causal response from slots and cracks
Once a relationship between responses to slots and cracks is
established, a continuous function may be plotted in the form shown
in Figure 5. This is the same response required in use of the a / &
method used in POD generation (Response and actual crack size are
plotted as logarithmic function - In/In).
After an experimental relationship between the response of cracks
of varying size and slots of equivalent varying size are established
from test specimens in simple configurations, the capability of an
NDE procedure for application to a complex configuration may be
linked to the performance on a simple configuration using slots as
the transfer artifacts. Since slots of equal physical size and shape
can be economically produced in both simple and complex
specimen configurations, they may be used as duplicate and
traceable artifacts. A quantitative NDE response relationship ma)
then be experimentally generated using equivalent size slots in both
simple and complex specimen configurations. Care in making
measurements must be exercised to link NDE performance
capability (POD) based on equivalent signal response (termed
equivalent reflectivity by some experimentalists8.9). Rigid control
and measurement of both test specimens and data recording are
required. Primary considerations include: (I) cracks used for
measurements in simple specimens must be representative of the
population of cracks that must be detected / measured: (2) slots used
for measurements must be geometrically equivalent (size. shape.
width, radius sharpness, etc.): (3) signal and noise response
distributions measured must be representative of the distributions
anticipated in an application; and (4) response measurements must
be recorded and included in the validation data for an NDE
procedure. The same slots in the complex configuration may then be
incorporated into the NDE process control history by periodically
determining that the response distributions for slot measurements
are repeatable.
Figure 6 illustrates typical response distributions for repetitive
measurements of two slots of equivalent size and the corresponding
noise responses in both simple and complex specimen
configurations. I f the process is repeated using two slots of a
different size, the same proportional relationship is obtained if the
response is linear and continuous. The response relationship may
then be assumed to be a constant within the bounds used in the
original crack and slot measurements. The predicted causal response
for slots in the complex configuration may then be calculated over
the range of crack sizes used in development of data for the simple
specimen configuration. The noise data is overlaid as an upper
bound limit from actual measurements made on the complex test
specimen(s).
The relationship may thus be expressed as:
14
Slot
In Slot a(C)
= Response (C
(
Slot
In Slot a(F)
r
Response (Fb
In
( R~~nse(CO+n>=K In ( Reflse (Fo+n>
Figure 7 shows a calculated, continuous response for slots of
varying sizes over the test range of the initial data. The
corresponding noise response level is shown as an overlay as
measured at the upper bound of the measured noise distribution in
the complex (shape) specimen.
L
Signal
Slot Slot
.!4
n
B
ii
K
1
.F
v)
. . . .
Signal Response Level
Figure 6. NDE response distributions for two eouivalent size slots
in a flat plate and shape (complex configuration)
Actual Artifact Size
Figure 7. Calculated slot response for the complex (shape)
specimen over the range or slot sizes previously
quantitied on flat specimens.
In like manner, response of a single crack size may be as shown in
Figure 8 and a continuous crack response may be calculated from
fhe flat plate crack data and the established slot i slot transfer
constant. This relationship may be expressed as:
b
:
. . . .
Signal Respor&e [evei l
Noise
Signal
Cracks Slots
Figure 8. Calculated NDE response distributions for based on crack
and slot equivalency in flat plate and complex (shape)
configurations.
The response of a cracks of varying sizes in complex specimen
configurations may be calculated over the same size range that was
used for the flat specimens to produce a continuous response curve.
The extrapolated continuous crack response is shown in Figure 9.
Extrapolated
Crack Cymplex
Slot Flat
. . . ..~=**~
_.*** -m-*
Actual Artifact Size
Figure 9. Extrapolated continuous crack response based on slot
artifact response transfer
The probability of detection (POD) threshold crack size may be
adjusted to that crack size which produces an equivalent response in
the complex (shape) configuration as shown in Figure IO. A.
This method provides and equivalent POD threshold. but does not
account for the change in noise, thus the false call rate would be
increased. Adjustment to provide an equal false call rate and thus
account for the increased noise requires setting the threshold at a
point where the signal and noise margin is equal to that provided by
the original flat plate data (Figure 10, B value). A new POD curves
based on the extrapolated crack responses may be calculated by
either the a / 9 or hit / miss methods and plotted as shown in
Figure 11.
ACWPlL CRACK l&NOT ,,cil,
Figure 10. Adjusted POD threshold
Figure 11. Recalculated and adjusted POD curve
5. CAUTIONS
Rigor is application of the method described is required and
documentation of each data acquisition and calculation step is
necessary for both process control and for hrture re-validation.
Further: (1) cracks and slots must be reproducible and must be
representative of the conditions under which the measurement and
evaluations are to be applied; (2) physical measurements of slots
and cracks must be traceable to established measurement standards;
(3) all measurements must be made using the same procedure that is
intended for the application; (4) crack to crack variance is not
transferred and is assumed to be equal to the variance found in the
flat test specimens; and (5) variances in part stress state and crack
orientation are not transferred and must be addressed by the mode
of application of the NDE procedure.
THE METHOD DESCRIBED DOES NOT TAKE INTO
ACCOUNT ANY HUMAN FACTORS VARIATIONS IN
APPLICATION OR EVALUATION. HUMAN FACTORS
HAVE LESS IMPACT ON DISCRIMINATION LEVEL
WHEN AUTOMATED ALARMS AND RECORDING ARE
USED. HUMAN FACTORS FOR HAND SCANNING MUST
BE ADDRESSED SEPARATELY AND INTEGRATED IN
THE PROCEDURE QUALIFKATION.
6. SUMMARY
Modem .design and life-cycle management require the use of
damage tolerance methods and disciplines. Nondestructive
detection, measurement and evaluation of both surface connected
and internal anomalies is and essential part of damage tolerance
methods. Nondestructive evaluation procedures must therefoi be
capable, reliable and quantitative in order to support damage
tolerance design, acceptance and life-cycle management.
Prior to the introduction of damage tolerance methods,
nondestructive evaluation procedures had not generally been
rigorously characterized to establish their capability and reliability.
Assumptions of capabilities were often faulty. The metric developed
to quantify NDE capabilities and to provide a method of data
exchange is the probability of detection (POD). POD data can be
readily developed using flawed test specimens in simple
configurations - often flat plates. Flawed test specimens in complex
shapes and configurations are, however, difficult to obtain or may
not be available or producible for new designs. A method of linking
data from simple specimens to more complex applications is
required.
The logic and methodologies described in this paper provide and
approach to transferring nondestructive evaluation (NDE) procedure
performance (probability of detection - POD) capabilities from
simple test specimens to more complex applications. The methods
cannot be applied in a cook book manner. but require a thorough
understanding of NDE procedures, procedure characteristics,
limitations and boundary conditions for application. The transfer
method must therefor be considered to be a tool for use by qualified
NDE engineers as a part of damage tolerance design and life-cycle
management technology applications.
REFERENCES:
1.
2.
3.
4.
5.
6.
7.
8.
9.
D&W Enterprises, LTD., 8776 W. Mountainview Lane,
Littleton, CO 80125-9406, USA; TEL: (303) 701-1940, FAX:
791-1940 (Automatic switch)
A. P. Berens, NDE Reliability Data Analysis. in I&,&&
Handbook. 91h Edition, Vol.17. p 689,ASM International.
1989.
W.D. Rummel et al, Recommended Practice for a
Demonstration of Nondestructive Evaluation Rrliability on
Aircraft Production Part, Materials Evaluation. 40. p 922.
1982.
W.D. Rummel, G.L. Hardy & T.D. Cooper. Applications of
NDE Reliability to Systems, in Metals Handbook. gth Edition.
Vol. 17, p 674, ASM International, 1989.
NDE Capabilities Data Book. 3d Edition. DB-2. 1997.
available through NTIAC, (5 12) 263-2106.
Ward D. Rummel, Considerations for Quantiative NDE and
NDE Reliability Improvement. Review of Progress in
Quantitative Nondestructive Evaluation, Vol., 2A. pl9. 1983,
Plenum Press, New York.
A.P. Berens, op cit.
R.H. Burkel, D.J. Sturges, R.S. Gilmore and W.T. Tucker.
Effective Reflectivity: POD Methodology for Ultrasonic
Inspection, Paper presented to the 1995 Fall Conference of the
American Society for Nondestructive Testing. Dallas, Texas.
Olav Forli, et al, Guidelines for replacing NDE techniques
with one another, NT Report 300, NORTEST. P.O. Box 116,
FIN-02 15 I ESPOO, Finland. 1995.