RESEARCH ARTICLE | APRIL 20 2018

Advanced methods in NDE using machine learning

approaches 
Christian Wunderlich  ; Constanze Tschöpe; Frank Duckhorn
AIP Conf. Proc. 1949, 020022 (2018)
https://doi.org/10.1063/1.5031519


View Export
Online Citation

Articles You May Be Interested In

Shear horizontal ultrasonic waves excited by piezoelectric fiber patches

AIP Conference Proceedings (February 2016)

Multi-scale radiographic applications in microelectronic industry

AIP Conference Proceedings (February 2016)

Image feature detection and extraction techniques performance evaluation for development of panorama
under different light conditions
AIP Conference Proceedings (April 2018)

03 October 2024 10:23:02

 Advanced Methods in NDE using Machine Learning
Approaches
Christian Wunderlich1, a) and Constanze Tschöpe1, b) and Frank Duckhorn1

1
Fraunhofer IKTS, Maria-Reiche. Str. 2, 01109 Dresden, Germany.
a)
Corresponding author: [email protected]
b)
[email protected]

Abstract. Machine learning (ML) methods and algorithms have been applied recently with great success in quality control
and predictive maintenance. Its goal to build new and/or leverage existing algorithms to learn from training data and give
accurate predictions, or to find patterns, particularly with new and unseen similar data, fits perfectly to Non-Destructive
Evaluation. The advantages of ML in NDE are obvious in such tasks as pattern recognition in acoustic signals or automated
processing of images from X-ray, Ultrasonics or optical methods. Fraunhofer IKTS is using machine learning algorithms
in acoustic signal analysis. The approach had been applied to such a variety of tasks in quality assessment. The principal

03 October 2024 10:23:02

approach is based on acoustic signal processing with a primary and secondary analysis step followed by a cognitive system
to create model data. Already in the second analysis steps unsupervised learning algorithms as principal component analysis
are used to simplify data structures. In the cognitive part of the software further unsupervised and supervised learning
algorithms will be trained. Later the sensor signals from unknown samples can be recognized and classified automatically
by the algorithms trained before. Recently the IKTS team was able to transfer the software for signal processing and pattern
recognition to a small printed circuit board (PCB). Still, algorithms will be trained on an ordinary PC; however, trained
algorithms run on the Digital Signal Processor and the FPGA chip. The identical approach will be used for pattern
recognition in image analysis of OCT pictures. Some key requirements have to be fulfilled, however. A sufficiently large
set of training data, a high signal-to-noise ratio, and an optimized and exact fixation of components are required. The
automated testing can be done subsequently by the machine. By integrating the test data of many components along the
value chain further optimization including lifetime and durability prediction based on big data becomes possible, even if
components are used in different versions or configurations. This is the promise behind German Industry 4.0.

INTRODUCTION
While the Digital transformation is widely discussed in all sectors of the economy under the keyword “Industry
4.0” or “Industrial Internet of Things” (IIoT), this discussion has had so far surprisingly limited impact on the academia
and service industry using Non-Destructive Evaluation (NDE) methods. As Fraunhofer IKTS is focusing on
developments along the value chains of structural and functional ceramic components on one hand as well as value
chains for NDE systems on the other hand, we feel well positioned to discuss this issue in a broader context.
For the Material Diagnostics branch of Fraunhofer IKTS (until 2014 Fraunhofer IZFP-D) we have defined four
core areas for digital transformation in the field of NDE in Fig. 1.

44th Annual Review of Progress in Quantitative Nondestructive Evaluation, Volume 37

AIP Conf. Proc. 1949, 020022-1–020022-7; https://doi.org/10.1063/1.5031519
Published by AIP Publishing. 978-0-7354-1644-4/$30.00

020022-1
 a b c d

FIGURE 1. Key areas for digital transformation in the field of NDE as defined for Fraunhofer IKTS

The four areas in Fig. 1 require and enable various innovation elements:
a) Remote monitoring and evaluation using augmented reality tools needs the development of a new level of
portable inspection devices with a modern human machine interface (HMI), IP enabled and interacting e.g.
with data lenses.
b) Automated failure detection by sensor fusion and/or by machine learning is offering from our perspective
huge potentials for different methods and will be discussed more in detail below.
c) As Industry 4.0 is envisioning cyber-physical systems, communicating with each other in process, quality and
logistical aspects, we see that NDE and process (and environmental) monitoring will be applied seamlessly.
Again, machine learning methods will be key to analyze the data and learn from them.
d) If the data gained in various steps along the value chain can be integrated in a “Digital Twin” of a component

03 October 2024 10:23:02

or system, these data can be used later as reference for condition or structural health monitoring. Machine
learning methods must be applied here for predictive analytics to calculated preventive maintenance or
remaining lifetime.

In summary machine learning has potential impact in at least 3 of the 4 areas of the digital transformation defined
here. It should become a key method in NDE.

MACHINE LEARNING FOR NDE

Machine learning is a subset of computer science that has evolved over the last couple of years dramatically. In
principle it means to learn from data without an a priori knowledge about underlying physical laws and relationships
between parameters. Machine Learning models are generated by established (but carefully selected) algorithms, (and
training data) to make later predictions with unknown data or see patterns in data. Other than with established statistics,
it works for high dimensional data sets and requires large training data sets.
First, machine learning has to be split into methods of supervised learning and unsupervised learning. Supervised
learning means, that there is a large set of labeled data available, where the label represents the answer assumed to be
correct. The algorithm will be trained by those data, correct training is validated, and once the algorithm is trained,
unknown data sets can be processed. Supervised learning covers both data with real valued labels, called regression
problems, and cases with discrete labels called classification problems. Typical tools are Logistic Regression, Support
Vector Machines, or Neural Networks.
Unsupervised learning means the discovery of hidden regularities (e.g., clusters) or anomalies in the data (for
instance, some unusual machine function or a network intrusion). So unsupervised learning doesn’t require any
annotated data. Typical tools are Ensemble Methods, Clustering Algorithms or Principal Component Analysis.
Both methods can be applied for a batch of data already available (batch learning) or an incoming data stream (online
learning).
If a Neural Network has a high number of hidden layers or several methods are combined in a hierarchical way in
one algorithm, like Convolutional Neural Network for image analysis, the machine learning is called Deep Learning.
Deep learning is performed in an end-to-end approach, which means the initial data (like pictures) are processed as

020022-2
they are. Data preparation steps like segmentation and feature separation are included in the algorithm. These methods
require large training data and huge processing power as provided by specific hardware (e.g. GPU processing). Then
they may become extremely powerful.

The obvious advantage of machine learning becomes clear if we look at a fingerprint signal of welding current and
acoustic emission data as shown in Fig. 2. Nobody needs to study and understand why the correct welding process
parameters, the noise emitted and the quality of a weld line are related to each other. We like to decide however from
experience, looking at these data, if a weld line is correct. This is a perfect task for machine learning.

03 October 2024 10:23:02

FIGURE 2. Fingerprint signal of welding current and acoustic data.

The application of machine learning to the field of NDE requires a systematic and careful approach using the
knowledge of experts from the very beginning. A potential systematic process is shown in Fig. 3. Developing business
understanding and data understanding is an iterative process typically starting again after first data evaluation results
are available.

FIGURE 3. cross-industry standard process for data mining – reference model.

020022-3
 The field of machine learning is evolving with high speed resulting in impressive results in some areas. This trend
is driven by four elements:
a. broader availability of ready to use algorithms, some of them covering end-to-end solutions;
b. availability of large training sets of pictures and data due to the amount of data stored on the internet or
in private cloud-based infrastructures;
c. transfer of time consuming computational steps to massive parallel processing units like graphic chips or
even specialized tensor processing units;
d. strong interest in technologies like autonomous driving, automated diagnostics in radiology or automated
data processing in online commerce.

Machine learning will sustain and should be applied to Non-Destructive Testing and Evaluation.

MACHINE LEARNING AND ACOUSTIC ANALYSIS AT FRAUNHOFER IKTS

Fraunhofer IKTS has been working in machine learning and acoustic pattern recognition for many years [1]-
[15]. Applications range from quality assessment of gears in automotive industry, lifetime analysis of wearing parts
(such as magnetic valves or rollers), detection of cracks and impacts in aircraft materials, determination of softness
of tissue paper to evaluation of nontechnical signals (speech, music, bio signals). Figure 1 shows the principal
structure of a system of artificial intelligence for automatic interpretation of sensor signals. With the help of signal
analysis, pattern recognition, and machine learning, sensor signals can be automatically interpreted, e.g., “Object is
good“ or “Component residual life at 20%.”
Apart from the actual AI components, Fraunhofer IKTS uses established signal analysis technology. Primary
analysis is essentially based on signal processing methods, including time-frequency transformations (STFT, WT,
etc.), digital filters and filter banks, signal modeling (e.g., with optimal filters), and AI-based signal analysis (e.g.,
CNNs). Secondary analysis is performed to extract relevant information for pattern recognition from the primary

03 October 2024 10:23:02

features, e.g., by Principal Component Analysis (PCA) or Linear Discriminant Analysis (LDA).

FIGURE 4. AI-System for automated interpretation of sensor signals

Pattern recognition is based on, e.g., Deep Neural Networks (DNNs), Gaussian mixture models (GMMs),
Hidden Markov Models (HMMs), or support vector machines (SVMs) and includes interpretation of results. The
class models required for this are built through machine learning processes, such as deep learning (DNN), the EM
algorithm (GMM, HMM), and convex optimization (SVM). In a training phase, the system is supplied with training
examples, or sensor signals with known meanings (e.g., “good“ or “bad“). Models can then be taught further
(adapted) during operation for improving the AI system or adapting it to changed tasks. In certain DNN
configurations, the pattern recognizer assumes the task of secondary analysis, as well as primary analysis in part.

020022-4
Recently the IKTS team was able to transfer the software for signal processing and pattern recognition on an
embedded system [14][15]. In that way, a mobile, modular, miniaturized hardware is available executing
applications like those described above.
Still the algorithms will be trained on an ordinary PC; however, recognition algorithms will run on the hardware
comprised of a Digital Signal Processor and a FPGA chip. The FPGA (Field Programmable Gate Array) chip here is
equivalent to GPU based data processing in a larger stationary system.

OPTICAL COHERENCE TOMOGRAPHY

Another application for Machine learning is the analysis of transparent and translucent ceramic materials using a
method called Optical Coherence Tomography (OCT). In the fourier-domain OCT, the light coming from a shortwave
coherent light source is split in a sample- and reference arm (Fig. 5). The signal reflected by disturbances or interfaces
in the sample interfers with the reference signal and results after Fast-Fourier-Transformation (FFT) in a depth
reflection signal also called A-scan. By moving the optical signal with a mirror system, a sequence of A-scans will
result in a cross sectional view into the material (B-scan). By moving the sample on an X-Y-table, a 3D tomogram
can be created.

03 October 2024 10:23:02

a b
FIGURE 5. Optical Coherence Tomography: a) principle b) x-y scanning system for a lab.

Automated failure detection in an OCT test station that is setup to classify ceramic components by failures
(according to a defined failure catalogue) is a reasonable approach. In practice, a number of open questions arise. First,
a single B-scan results in a huge data set of 42GB. By cutting the B-Scan and data compression techniques, the amount
of data can be reduced to 400MB. A second step of filtering and pre-classification will reduce the amount of data to
about 0.4 MB. So, handling huge amounts of data is an issue here, as B-scans will be used in machine learning
algorithms to learn from thousands of samples.
Another open question is whether the chosen path from raw data to 2D-visualization and subsequent application
of machine learning tools for pattern recognition in pictures is more robust and more quickly implemented than the
more exotic online learning by pattern recognition in the primary data stream of the FFT processed raw data. It is also
important that, for image processing, large libraries like Tensor Flow are more readily available.
This example illustrates how important it is to combine 1) the domain knowledge of the NDT expert and specialists
familiar with both the method and the devices with 2) the experiences of machine learning specialists. The NDT expert
may gradually evolve into a data scientist who understands the technical and business needs in a test or inspection
case and has a good understanding of data management and machine learning techniques.

020022-5
 Optical coherence tomography offers an interesting path to analyze 3D printed structures of polymeric and ceramic
materials in operando during the printing process (Fig. 6).

FIGURE 6. OCT 3D-data of additiv manufactured ceramics

Because the OCT scan can look into a certain depth (2-5 mm) of the material and see beyond the top surface layer,
it can deliver information about 1) flaws under the surface (those that might be created during cooling down while a
layer 3 elements above is generated) and 2) the material integrity into the depth of the printed process. By building a
voxel-by-voxel representation of any single component, as it is printed, the exact digital representation of this
component is generated. This is called a Digital Twin (on the component level). The idealistic CAD model once used
to generate the .STL or .AMF file can be replaced by the corrected model as built. By this approach a quality
management process for components manufactured in lot size = 1 can be realized.

03 October 2024 10:23:02

FROM NDE TO DIGITAL TWINS
A Digital Twin is a dynamic digital model of components, processes or systems that can simulate their behavior
over the product’s life cycle. This simulation learns from data, constantly updates and recalibrates itself. As shown
above, an NDE method like OCT can update information in CAD models to get a better representation of the actual
component after the production phase. Subsequently, the CAE calculation or assembly simulation can be updated as
well. This is the new paradigm of quality management in Industry4.0: massive environmental material and process
data gathered in operando, linked to the individual component as “Digital Twin,” and managed over the product life
cycle.
In the same way, other NDE methods such as Structural Health Monitoring or Condition Monitoring can update
the Digital Twin during the use or operation of a component or system. In general, it should be stated that NDE
methods are enabling technologies to make what may appear today as a visionary idea of Digital Twins into a reality.
By taking the data from the digital twins as they evolve and modify over the value chain and life cycle of the
product, machine learning approaches can be applied back to design, material selection and manufacturing
technologies, thereby optimizing them. It is assumed that new business models will emerge by this wide-ranging
availability of data. Manufacturers or platform operators can provide extended support and optimized asset
management to decrease liability and reduce cost.

SUMMARY
Two cases of how machine learning is applied today have been discussed. While machine learning in acoustic
emission monitoring has been used within Fraunhofer IKTS for many years, the application in the field of Optical
Coherence Tomography is just starting. In any case, machine learning will be a highly relevant tool set to improve
NDT methods further, combine them, and deliver reliable results in a fast and cost efficient way.

020022-6
 Even if many decisions in NDT testing will be taken by automated systems in the near future, NDT experts will
still be needed. However, their role is changing. NDT experts will become data scientists, understanding the technical
and business needs test or inspection cases and having a good understanding of data management and machine learning
techniques.

Although the widespread availability of “Digital Twins” in industry is today still more vision than reality, the unique
role of Nondestructive Evaluation and Structural Health Monitoring methods for the Digital Twin must be underlined.
These methods offer unique opportunities to extract, understand and create data while processing, testing or using a
component or system. So, NDT isn’t any longer a cost causing end-of-pipe technology but an enabling technology
creating valuable data for the digital transformation in the manufacturing world known as Industry 4.0.

REFERENCES
1. C. Tschöpe, D. Hentschel, M. Wolff, M. Eichner, R. Hoffmann: Classification of Non-Speech Acoustic Signals
using Structure Models, IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP
2004), Montreal, 17-21 May 2004, Proc. 5, pp. 653-656.
2. M. Wolff, U. Kordon, H. Hussein, M. Eichner, R. Hoffmann, C. Tschöpe: Auscultatory Blood Pressure
Measurement using HMMs. IEEE International Conference on Acoustics, Speech, and Signal Processing
(ICASSP 2007), Hononlulu, Hawaii, (15-20 April 2007), Proc. 1, pp. 405-408.
3. C. Tschöpe, M. Wolff: Automatic Decision Making in SHM using Hidden Markov Models
18th International Conference on Database and Expert Systems Applications (DEXA 2007), Regensburg, 3-7
(Sept. 2007), pp 307-311.
4. C. Tschöpe, E. Schulze, H. Neunübel, M. Wolff, R. Schubert: Experiments in Acoustic Structural Health
Monitoring of Airplane Parts, IEEE International Conference on Acoustics, Speech, and Signal Processing

03 October 2024 10:23:02

(ICASSP 2008), Las Vegas, (March 30 - April 04 2008), Proc. 2037-2040.
5. T. Pusch, C. Cherif, A. Farooq, S. Wittenberg, R. Hoffmann, C. Tschöpe: Early fault detection at textile machines
with the help of structure-borne sound analysis, Melliand English (11-12/2008), S. E 144 - E 145.
6. G. Strecha, M. Wolff, F. Duckhorn, S. Wittenberg, C. Tschöpe: The HMM Synthesis Algorithm of an Embedded
Unified Speech Recognizer and Synthesizer, Proc. Interspeech 2009, 10th Annual Conference of the
International Speech Communication Association, 1763-1766, Brighton 2009, (06.-10.9.2009).
7. M. Wolff, C. Tschöpe: Pattern Recognition for Sensor Signals, Proc. IEEE SENSORS 2009 Conference, 665-
668, Christchurch, Neuseeland, (25.-28.10.2009).
8. C. Tschöpe, M. Wolff: Statistical Classifiers for Structural Health Monitoring, IEEE Sensors Journal, 9, No. 11,
(Nov. 2009), pp 1567 - 1576.
9. C. Tschöpe, M. Wolff: Processing and evaluation of gear data using statistical classifiers
CD-ROM Proceedings of the 6th European Congress on Computational Methods in Applied Sciences and
Engineering (ECCOMAS 2012), Wien, (Sept. 2012).
10. C. Tschöpe, D. Joneit, F. Duckhorn, G. Strecha, R. Hoffmann, M. Wolff: Voice control for measurement devices,
AIA-DAGA 2013 Conference on Acoustics, Merano, (18.-21.03.2013).
11. C. Tschöpe, F. Duckhorn, M. Wolff, G. Saeltzer: Estimating Blood Sugar from Voice Samples - A Preliminary
Study, CSCI’15: The 2015 International Conference on Computational Science and Computational Intelligence,
Las Vegas, (07.-09.12.2015).
12. M. Barth, F. Duckhorn, K. Tschöke, C. Tschöpe, B. Köhler: Testing of Ceramics by Ultrasound Microscopy and
Vibration Analysis, 19th World Conference on Non-Destructive Testing (WCNDT 2016), Munich, (13.-
17.06.2016).
13. F. Duckhorn, M. Huber, W. Meyer, O. Jokisch, C. Tschöpe, M. Wolff: Towards an Autarkic Embedded
Cognitive User Interface, Interspeech, Stockholm, Sweden, 21.-25.08.(2017).
14. C. Tschöpe, F. Duckhorn, C. Richter, P. Blüthgen, M. Wolff: An Embedded System for Acoustic Pattern
Recognition, IEEE Sensors 2017 Conference, Glasgow, UK, 29.10.-1.11.(2017).
15. C. Tschöpe, F. Duckhorn, C. Richter, P. Blüthgen, M. Wolff: Intelligent Signal Processing on a Miniaturized
Hardware Module, IEEE Sensors 2017 Conference, Glasgow, UK, 29.10.-1.11.(2017).
16. Opitz, Jörg et al, Materials diagnostics and integrated testing technology for ceramic parts; Keramische
Zeitschrift 68 (2016), Nr.4/5, S.249-254.

020022-7