Fault classification for Photovoltaic Modules Using

Thermography and Machine Learning Techniques

V S Bharath. Kurukuru, Ahteshamul Haque, Mohammed Ali Khan, Arun Kumar Tripathy,
Department of Electrical Department of Electrical Department of Electrical National Institute of Solar Energy
Engineering, Faculty of Engineering, Faculty of Engineering Haryana, India
Engineering & Technology Engineering & Technology Faculty of Engineering & [email protected]
Jamia Millia Islamia (Central Jamia Millia Islamia (Central Technology
University) University) Jamia Millia Islamia (Central
Delhi, India Delhi, India University)
[email protected]. [email protected] Delhi, India
[email protected]

Abstract— Fault classification and detection are necessary fault detection method for PV modules by measuring the
for safety, efficiency and reliability of photovoltaic systems. series resistance and comparing them with reference values
Regardless of the fact that PV system requires low or datasheet. John et al. reviewed the use of thermography in
maintenance due to the absence of moving parts, they are still fault detection of PV modules and stated that infrared
vulnerable to many faults. Particularly for PV modules, it’s thermography is a well-established and competitive fault
problematic to shut them down completely during faults as it diagnostic method. But it is observed that most of the
will affect their performance and result in early degradation. research involving thermography also uses flash test to
Additionally, the inability of conventional fault detection
methods in early detection of faults leads to higher risk of
extract the VI characteristics of the PV module. This process
failure. In this paper thermography and machine learning has multiple drawbacks like, observing the thermal images
based PV module fault classification is developed. Texture with human eyes for fault identification, and performing
feature analysis is adapted to study the features of different flash tests to observe the VI curves to confirm the fault. This
faulty panel thermal images. The extracted features are trained involves a lot of equipment and time delay. Apart from
by implementing Artificial Neural network classifier to develop these, there are many methods like IV curve monitoring,
the fault classifier. The developed algorithm depicted 93.4% Dark current measurement, and resistance extraction
training efficiency and 91.7% testing efficiency which is better methods. The common property of the above-mentioned
when compared with the conventional classification techniques. measurement methods is that, they can be commonly used
Keywords— Photovoltaic Modules, Thermography, Grey on-site with little adjustments but lack in efficiency and
Level Co-occurrence Matrix (GLCM), Artificial Neural Network. detection time. Remote monitoring is also considered as a
major issue in most of the methods. In order to over come
I. INTRODUCTION these drawbacks, new classification technique needs to be
Fault diagnosis is necessary for safety, efficiency and developed.
reliability of photovoltaic systems [1]. Regardless of the fact On a parallel track, the field of image processing and
that PV system requires low maintenance due to the absence analysis has greatly advanced in the recent past. Primary
of moving parts, they are still vulnerable to many faults. works mainly addressed the enhancement, denoising, and
Particularly for PV modules, it’s problematic to shut them compression of grayscale images in satellite and medical
down completely during faults as it will affect their applications. However, as sensor technology progressed,
performance and result in early degradation. Major faults color images with mega-pixel resolutions required not only
that can affect the operation and efficiency of PV modules more sophisticated algorithms, but also new solutions for the
are frame defects [2], glass defects [3], encapsulant defects detection, classification, and comparison of objects in
[4], back sheet defects [4] and other cell defects like snail multiple images. Information retrieval methods became
trail [5], hot spot [6], cracked cell [7], broken cell [8] and relevant as datasets surpassed the billionth image boundary.
faulty interconnections [9]. In addition, conventional series- In fact, research in image analysis spans a wide variety of
parallel PV configurations increase voltage and current disciplines including feature extraction, pattern recognition,
ratings, leading to higher risk of large fault currents or dc classification, and artificial intelligence.
arcs. In general, defects on the modules are usually not In this paper thermography, image processing and
visible by the naked eye and special methods are needed to machine learning techniques are adapted together for
detect them. developing a new fault classification technique as proposed
Conventional detection methods deal with diagnostics in figure 1.
using radiation, static characteristics measurement, and Thermography and
Grey image
dynamic parameters measurement (Impedance Data Preparation
Spectroscopy). Serigu et al. [3] developed a fault Grey Level Contrast Homogenity
Feature Dataset

classification method for PV module cells using Co-occurrence

electroluminescence images. Places, which are affected by Matrix Entropy Correlation
some damage, and with high density of defects, are seen as Texture Features
extraction Energy
dark places in the electroluminescence images. Raghavi et
al. [10] observed photoluminescence imaging of PV
modules with sunlight excitation. In this process a strong Classifier
radioactive recombination is stimulated which can help in Figure.1 Block representation of proposed classification algorithm
the process of PV cell diagnostics. Dezso [11] presented a

978-1-5386-8125-1/19/$31.00 ©2019 IEEE

 The developed technique can overcome the drawbacks features are computed from the statistical distribution of
of conventional techniques by providing advantages with observed combinations of intensities at specified positions
remote monitoring, least detection time and high relative to each other in the image [12], usually in a square
classification efficiency. Further sections of the paper local window of odd size. According to the number of
provide a detailed overview of every aspect of the proposed intensity points (pixels) in each combination, statistics are
technique. Section 2 provides an overview on thermography classified into first-order, second order and higher-order
and advantages with thermal images. Section 3 describes the
statistics. First order statistics are, for example, mean,
feature extraction method for thermal images using gray
variance and higher order moments. They are easy to
level Co-occurrence matrix. Section 4 details the machine
learning techniques and classification requirements for compute and variance. However, they only take into account
developing the algorithm. The experimental analysis, results global texture parameters, and not mutual intensity
and conclusion were given in section 5 and section 6 distribution between pixels. Second order statistical texture
respectively. features are called grey level co-occurrence matrices
(GLCMs) and were first presented by Haralick et al. [12].
II. THERMOGRAPHY
Thermal imager registers thermal energy naturally B. Grey level co-occurrence matrices
radiated from objects and produce images called A GLCM is a matrix where the number of rows and
thermograms. These instruments detect thermal radiation columns is equal to the number of grey levels, G, in the
without involving direct contact with the targeted object or image.
scene. In fact, this is an advantageous feature of infrared There are many features that can be extracted from the
thermography, not only for the convenience of reading GLCM. Usually they measure the homogeneity or the
precise temperature measurements remotely, but most contrast in the local window. Some of them were presented
importantly, for meeting security, quality, and non- in the original paper [12], while others have been invented
intervention requirements of many applications. A color- later [13]. Some features were discovered based on the
coded thermal image provides a means for the visual Mahalanobis distance between average GLCMs for two
assessment of temperature distributions of a targeted scene classes. In this research standard GLCM features were
in real-time. This is a relevant qualitative feature. extracted by converting the acquired thermal image in to
Nonetheless, the pivotal aspect to be examined is the grey image. Features such as Contrast, Correlation, Energy,
quantitative use of thermography. Homogeneity, and Entropy were extracted using equations
In general, the quantitative analysis of a thermal image (1)-(7).
involves three main steps: (i) the analysis of temperature (1)
data, (ii) the identification of temperature irregularities, and
(iii) the comparison to a standard or different state. The (2)
analysis of temperature data relies on the extraction of
(3)
statistical values, namely the minimum, maximum, mean,
and standard deviation, from the selected region of interest (4)
within an image. The selection of the region of interest is (5)
generally performed manually by using simple geometric
(6)
shapes. Such manual process is time-consuming, error-
prone, and expert-dependent. By using simple-shaped (7)
regions, except for simple-shaped objects, the statistical Where is elements of GLCM with two different
values become inaccurate from either including background instances i and j, is the mean of GLCM, and
or excluding marginal data into calculations. In infrared
contribute to the variance of instances.
thermography, the identification of temperature irregularities
Once the features are extracted, they are tabulated for
is based on the assessment of hot/cold spots and isotherms.
classification process. Corresponding classes for each
Current practice reports the application of global methods
feature set were defined for efficient functionality of the
for the identification of salient spots and regions. In this
classifier.
case, relevant localized anomalies may not be recognized.
Finally, the comparison to a standard or different state IV. CLASSIFICATION PROCESS
requires the analysis of regions within particular Classifier or classification process is defined as the
thermograms for change detection. collection of specific machine learning algorithms for
extracting patterns from data. These algorithms are primarily
III. FEATURE EXTRACTION
Feature extraction is the primitive step in pattern focused on automatic learning to recognize complex patterns
and make intelligent decisions based on data [14].
recognition process. This step extracts some features such as
Nowadays, there are multiple machine learning (ML)
color, shape, and texture. These features comprise the
algorithms, often inspired in nature around us. In this paper
relevant information of an image and can be used in multiple
pattern recognition and classification aspects of Artificial
applications. In this paper texture features of images are
extracted as they provide a higher-order description of the Neural Network (ANN) technique is used. The adapted
image and includes information about the spatial distribution Machine Learning technique is described in detail in further
sections. A general overview of other machine learning
of tonal variations or gray tones [12].
techniques is described in detail in [15].
A. Texture description
A. Artificial Neural Networks
One of the most widely used texture description
methods is statistical texture analysis. In this method, texture
 Artificial neural networks are inspired by biological (9)
brains. They can be used to solve a variety of tasks, such as
computer vision, medical diagnosis or speech recognition. Where, is the number of outputs, is the actual value and
Neural networks can improve their performance on certain is the predicted value. MSE punishes large mistakes much
task by taking examples into consideration. Such process more than small mistakes and its output is always positive.
can be simply called learning. In general, MSE measures the average squared error of our
A neural network (NN) consists of artificial neurons. predictions. For each point, it calculates square difference
An artificial neuron (simply neuron) is inspired by between the predictions and the target and then average
biological neuron. Artificial neurons are connected together. those values. This metrics is highly useful while dealing
Each connection is used to transmit a signal to another with unexpected values that needs to be taken care. With
neuron. The receiving neuron can process it and signal other that said, it is commonly used as a cost function for
neurons connected to it. Each neuron has set of weights. The regression tasks where the goal is to predict n real values.
weights are edges, connected to each neuron in the previous Considering all the above-mentioned aspects, the
layer. Neuron has one output value, which is weighted sum feature data is subjected for classification process. The
of input values plus a bias passed to an activation function. simulation and experimental analysis are explained in the
The Output value of neuron is defined by the following further sections.
formula:
V. EXPERIMENTAL ANALYSIS
(8) The Experimental analysis is carried out on grid
Where are input values, are weights, tends to connected roof top solar PV system of capacity 169.92kW
number of input variables, is bias, is activation (320W X 531 modules arranges in 27 strings) at Faculty of
function and is output of the neuron. Engineering and Technology, Jamia Millia Islamia, New
The simplest type of neural network is called Delhi, India. After careful Inspection, thermal images of
perceptron. It consists of one neuron and it is able to decide normal and potential faulty PV modules are captured in
whether an input defined by a vector of numbers belongs to multiple irradiance and temperature conditions to develop
some specific class or not. Single-layer perceptron’s are the algorithm. Fluke TiS45 thermal imager is used to capture
capable of solving linearly separable problems [16]. The the thermal images as shown in figure 2.
neuron in single-layer perceptron uses an activation function
which maps the weighted sum of inputs plus bias to values 0
or 1. Typically, neurons are organized into groups called
layers. A neural network consists of at least two layers, that
is input and output layer. It may also contain hidden layers.
Multi-layer perceptron (MLP) consists of at least three
layers. The output and hidden layers use non-linear
activation function. In contrary to single-layer perceptron,
multi-layer perceptron is capable of learning data that is not
linearly separable. MLP uses backpropagation as a learning
algorithm which is used to calculate adjustments to the
weights during supervised learning. The goal of supervised
learning is to map set of inputs to their correct output. A
scaled conjugate backpropagation is used to train neural
network with multiple layers by tuning the weights Figure. 2 Thermal image captured using Fluke TiS45.
according to loss function which represents the difference A total of 7 faulty modules and 1 normal operation
between the network’s output and the expected output. module are considered for experimental analysis with the
While training with neural network, activation and loss proposed approach. The size of each image captured is
function play a major role in classification process. An 280pixels by 500 pixels. In order to increase the sample size
activation function defines the output of a neuron for any for each class, the original image is divided into 14 X 14
given input. In general, there are large number of activation subregions as shown in figure 3.
functions like binary step, sigmoid function, hyperbolic
tangent and rectified linear unit. For the purpose of pattern
recognition and classification sigmoid activation function is
adapted. A sigmoid function maps the resulting values in
between 0 and 1and is used in the output layer for tasks
where the output is probability.
Loss function or cost function is used to represent the
error of the neural network. It can be used to compare the
performance of multiple models. The loss function is chosen
based on the type of task being solved. The cross entropy Figure. 3 Process of capturing Thermal Image and splitting into sub-
windows
and mean square error are two frequently used loss
functions. Cross entropy is used to quantify the difference Each sub-window has 1/14 size of total surface and is
between two probability distributions while solving 20 pixels by 35 pixels. This resolution provides all the
multiclass classification tasks. Whereas, mean squared error useful features required for developing the classification
(MSE) is used to measure the error of a neural network and algorithm. Any division above the 14 X 14 sub regions with
it is defined by: each region corresponding to 20 X 35 pixels will tamper
with the features available, and any division less than it will
increase the processing and detection time. A total of 196
sub-windows are obtained for each thermal image. The
texture analysis methods considered in the previous sections
is tested on the thermal image set of PV module surface.
Each sub-window region is transformed to a gray scale
image and the red, green, blue channels are extracted. To
obtain the texture features the grey level co-occurrence
matrix is calculated as explained in section in 2. In this
research the normalization of co-occurrence matrix is
ignored as the features are used for comparison purposes
where normalization is not vital. The contrast, correlation,
homogeneity, energy and entropy of all the sub-windows are
extracted using equations and tabulated along with
corresponding classes for providing input to the classifier.
Once the feature data is obtained, neural network
classifier is adapted for developing the classification
algorithm. All the 1568 samples are subjected to their Figure.5 Training state of neural network
corresponding classes in order to prepare input and target Figure 5 depicts the training state of the neural network
data. Random division is applied to divide the data for by depicting the gradient and validation fail iterations. The
training, validation and testing purposes. With 8 classes and developed algorithm adapted scaled conjugate gradient back
5 features, the neural network is structured as 5-12-8 with 5 propagation algorithm whose value is observed to be
input layers, 12 hidden layers and 8 output layers. Scaled 0.0025671 in logarithmic scale for each epoch. This value
conjugate gradient back propagation is used to adjust the depicts the least position of local minimum for the goal
weights and bias of the network. Cross entropy and error function. The validation fails are iteration that represent the
percentage are used to quantify the performance of the increase in mean square error of validation data. Multiple
classifier. The classifier is trained for 203 epochs to obtain validation fails indicate that the data is overfitting with
the best results with Cross Entropy corresponding to training data.
4.09945e-0 for training, 11.5387e-0 for validation and
11.50579e-0 for testing. The error percentage corresponds to
6.6% for training, 8.7 % for validation and 8.3 % for testing.
The performance results are depicted in figure 4 to 8.

Figure.4 Performance Plot for Neural Network Training

Figure.6 Error histogram for Neural Network training
Figure 4 depicts the performance plot of the training Figure 6 depicts the error histogram for the training
record for training testing and validation performance. It can process of the classifier. There are 20 bins (vertical bars)
be observed that as the number of epochs increase, the error ranging from -0.8994 to 0.9513 which depicts the total error
keeps on reducing. But at epoch 197 the error on the range of neural network. As there were 20 bins, each bin
validation dataset increases as the neural network overfits corresponds to a width of 0.092535. These bins represent the
the training data. In order to avoid this, the classifier is number of samples from the validation data set which have
adjusted to stop the training process after six consecutive error.
validation check fails. Hence the best performance can be
depicted by observing the epoch at least validation check
fail.
 Validation & Validation & Validation &
Testing Testing Testing
No. of samples 1568 1568 1568
Network 5 Input, 12 5 Predictors, 5 Predictors,
Structures Hidden, 8 8 Responses. 8 Responses.
Output layers
Epochs 203 -- --
Performance Cross Entropy Cross Cross
Criteria and mean square Validation Validation
error
Training Scaled Quadratic Euclidean
algorithm conjugate Kernel Kernel
gradient
Classification 92.8% 80.3% 56.8%
Accuracy

From the table it can be learnt that the classification

accuracy of the developed algorithm using NN techniques is
much better when compared to conventional classification
techniques like Support Vector Machine, and K- Nearest
Neighbor. Also, the method has been found to perform the
classification in the testing phase within fractions of a
second. From these results, the approach confirms
significant improvements on the prior works in the field.
Figure.7 All confusion plot for Neural network classifier.
Figure 7 depicts the confusion matrix for the
classification process of neural network. The green diagonal
represents the truly classified samples and all the samples on
the left and right side of the diagonal are the falsely VI. CONCLUSION
classified samples. The overall classification accuracy is Thermography based fault classification algorithm is
observed to be 92.8%. The training efficiency is 93.4%, developed in this paper. Thermal Images of 8 different
validation efficiency is 91.3% and the testing efficiency is panels (7 faulty and 1 normal) were captured from
91.7%. 169.92kW PV system installed at Faculty of Engineering
and Technology, Jamia Millia Islamia, New Delhi, India, for
experimental purpose. The thermal images were split into
sub-windows to improve the efficiency of the developed
algorithm. The sub-windows are subjected for GLCM
feature extraction to observe the texture features of the
image. The extracted texture features are tabulated for
providing input to the neural network classifier and the
corresponding classes are tabulated for providing target data.
Scaled conjugate gradient back propagation algorithm is
adapted to adjust the weights and bias of the classifier
during training process. The final results of the classifier
depicted 93.4% training and 91.7% testing accuracy which
is better when compared to other classification methods.
These results validate the efficiency and reliability of the
developed algorithm.
REFERENCES
Figure.8 Receiver Operating Characteristics
[1] A. Hooshyar, E. F. El-Saadany, and M. Sanaye-Pasand, “Fault
Figure 8 visualizes the performance of the neural Type Classification in Microgrids Including Photovoltaic DGs,”
IEEE Trans. Smart Grid, vol. 7, no. 5, pp. 2218–2229, 2016.
network by plotting the receiver operating characteristics [2] G. M. Lonkar, P. Klinkhachorn, U. B. Halabe, and H. V. S.
between true positive rate and false positive rate. It is Gangarao, “Automatic Detection of Subsurface Defects Using
observed that most of the data is classified towards true Infrared Thermography,” Evaluation, vol. 24, pp. 1469–1477.
positive rate depicting the effectiveness of the classifier. [3] S. Deitsch et al., “Automatic Classification of Defective
Photovoltaic Module Cells in Electroluminescence Images,”
From the results its observed that the developed 2018.
classification process is very effective in classifying the [4] B. V. Dobaria, V. Sharma, and A. Adeshara, “Investigation of
module faults. A mere comparison with other classification failure and degradation types of solar PV plants in a composite
techniques is given in table 1. climate: Abstract after 4–6, years of field operation,” Lect. Notes
Electr. Eng., vol. 435, pp. 227–235, 2018.
[5] S. Meyer et al., “Snail trails: Root cause analysis and test
Table 1. Comparison between classification accuracy of procedures,” Energy Procedia, vol. 38, pp. 498–505, 2013.
different classifiers [6] S. G. We, M. One, P. I. D. E. Pid, and P. I. Degradation, “The
five most common problems with solar panels,” pp. 1–2, 2017.
Parameters ANN SVM KNN [7] S. Kajari-Schröder, I. Kunze, and M. Köntges, “Criticality of
(Artificial (Support (K-Nearest cracks in PV modules,” Energy Procedia, vol. 27, pp. 658–663,
Neural Vector Neighbour) 2012.
Network) Machine) [8] A. Bouraiou et al., “Experimental investigation of observed
Training, Training, Training, defects in crystalline silicon PV modules under outdoor hot dry
 climatic conditions in Algeria,” Sol. Energy, vol. 159, no. [12] R. M. Haralick and K. Shanmugam, “Textural Features for Image
December 2017, pp. 475–487, 2018. Classification,” 1973.
[9] M. H. Ali, A. Rabhi, A. El Hajjaji, and G. M. Tina, “Real Time [13] C. Calvin, Gotlieb, and Herbert., “Texture Descriptors Based on
Fault Detection in Photovoltaic Systems,” Energy Procedia, vol. Co-occurrence Matrices CALVIN,” vol. 86, p. 92741R, 2014.
111, no. September 2016, pp. 914–923, 2017. [14] C. Bishop and N. Nasrabadi, “Pattern recognition and machine
[10] R. Bhoopathy, O. Kunz, M. Juhl, T. Trupke, and Z. Hameiri, learning,” Pattern Recognit., vol. 4, no. 4, p. 738, 2006.
“Outdoor photoluminescence imaging of photovoltaic modules [15] R. Malhotra, “A systematic review of machine learning
with sunlight excitation,” Prog. Photovoltaics Res. Appl., vol. 26, techniques for software fault prediction,” Appl. Soft Comput. J.,
no. 1, pp. 69–73, 2018. vol. 27, pp. 504–518, 2015.
[11] S. Spataru, D. Sera, T. Kerekes, and R. Teodorescu, “Diagnostic [16] J. J. Shynk, “Performance Surfaces of a Single-Layer
method for photovoltaic systems based on light I-V Perceptron,” IEEE Trans. Neural Networks, vol. 1, no. 3, pp.
measurements,” Sol. Energy, vol. 119, pp. 29–44, 2015. 268–274, 1990.