International Journal of Advanced Engineering Research and Science (IJAERS) [Vol-7, Issue-4, Apr- 2020]

https://dx.doi.org/10.22161/ijaers.74.38 ISSN: 2349-6495(P) | 2456-1908(O)

Use of two inpainting Techniques to Restore

Partially detected Cartographic Features
Ana Luisa Chaves Figueira1, Erivaldo Antonio da Silva2, Maurício Araújo
Dias1, Breno Strogueia Maia da Cruz1, Guilherme Pina Cardim2

1Dept. of Mathematics and Computing, UNESP, Presidente Prudente, São Paulo, Brazil
2Dept. of Cartography, UNESP, Presidente Prudente, São Paulo, Brazil

Abstract— The continuous use of methodologies to extract cartographic features of digital images have been
of great importance in the area of cartography. Many techniques can be used by features extraction processes,
however, the results obtained by these techniques usually have partially detected features, culminating in loss
of quality of the extraction process. To keep searching for better results, it is possible to use techniques based
on inpainting, that has as its main purpose image restoration and removal of occlusions. Therefore, the main
objective of this article is to show a methodology of reconstruction of partially detected features using two
inpainting techniques proposed by [1] and [2], aiming to improve the quality of results in the process of
extraction of cartographic features and digital images. Observing the final analysis of the results obtained
with the techniques in three entry images, the technique of [1] showed an improvement of 0.61% compared to
the extracted feature. While the technique of [2], an improvement of 6.82%. The good results obtained
regarding the improvement of the quality of the process of extraction of partially detected cartographic
features will be of great use in the area of cartography.
Keywords— Remote Sensing, Inpainting, Digital Image Processing, Cartography, Partially Detected
Features.

I. INTRODUCTION images. Both techniques are compared, in order to

The continuous use of methodologies to extract understand which one provided better results.
cartographic features of digital images have been of great A. Bertalmio et al. [1] Inpainting Algorithm
importance in the area of cartography. These processes This algorithm is mainly based on nonlinear partial
have as focus the identification of existing targets in the differential equations and the imitation of techniques of
terrestrial surface and its changes, which are required to artists specialized in restoration of museum paintings.
update cartographic products. Many techniques can be
used by features extraction processes, however, the results Being A the region in which the inpainting process will
obtained by these techniques usually have partially be carried out and bA its boundary, the isophote lines that
detected features, culminating in loss of quality of the focus on bA will be prolonged, maintaining its incidence
extraction process. To keep searching for better results, it is angles. After this procedure, it is defined the contour of the
possible to use techniques based on inpainting, that has as area that will be inpainted. This area will be filled from the
its main purpose image restoration and removal of extent of the regions around A. The different regions
occlusions. It operates by gathering information around the contained in A, determined by the contour lines, will be
damaged area and making a subtle junction of this completed by the colours that match the colours of bA.
information with the area of interest. Thus, based on what The general equation of the algorithm is show in
has been said, the main objective of this article is to show a equation (1).
methodology of reconstruction of partially detected 𝐼 (𝑛+1) (𝑖, 𝑗) = 𝐼 𝑛 (𝑖, 𝑗) + ∆𝑡𝐼𝑡𝑛 (𝑖, 𝑗), ∀(𝑖, 𝑗) ∈ 𝐴 (1)
features using two inpainting techniques proposed by [1]
and [2], aiming to improve the quality of results in the where n is the inpainting time, i and j are the pixel
process of extraction of cartographic features and digital coordinates, ∆tis the rate of improvement, 𝐼 𝑛 (𝑖, 𝑗) is the

www.ijaers.com Page | 299

International Journal of Advanced Engineering Research and Science (IJAERS) [Vol-7, Issue-4, Apr- 2020]
https://dx.doi.org/10.22161/ijaers.74.38 ISSN: 2349-6495(P) | 2456-1908(O)

entry image and 𝐼𝑡𝑛 (𝑖, 𝑗) is the improved version of the of the source region around a pixel p, the confidence term
entry image. of p will get a higher value.
Equation (1) shows that 𝐼 (𝑛+1) (𝑖, 𝑗), which is originated Equations (3) and (4) define the priority of a patch, so
from 𝐼𝑡𝑛 (𝑖, 𝑗), will be an improved version of the entry we select the one with the highest priority, and fill the
image. As n grows, the algorithm tends to have better target region with the patch from the source region that is
results. most similar to it.
To ensure the correct definition of the direction field, the 𝐶(𝑝) = 0, ∀𝑝 ∈ Ω
{ (3)
diffusion process is intertwined with the inpainting process 𝐶(𝑝) = 1, ∀𝑝 ∈ 𝜔
described, that is, the next step is the application of few 𝐷(𝑝) = −0.1, ∀𝑝 ∈ Ω ∪ 𝜔 (4)
iterations of image diffusion. This diffusion prevents the
where 𝐶(𝑝) and 𝐷(𝑝) is the confidence term and data
lines from crossing each other, resulting in a smoothing
term of a pixel, respectively, Ω is the area of interest and ω
effect. [1] uses the anisotropic diffusion, determined by the
is the region that doesn’t belong to the area of interest.
following:
𝜕𝐼
The similarity between two patches is measured by the
(𝑥, 𝑦, 𝑡) = 𝑔𝜀 (𝑥, 𝑦)𝜅(𝑥, 𝑦, 𝑦)(∇𝐼(𝑥, 𝑦, 𝑡)), ∀(𝑥, 𝑦) ∈ following equation:
𝜕𝑡
𝐴𝜀 (2) 𝑎𝑟𝑔𝑚𝑖𝑛
𝛾𝑝 = 𝛾𝑞 ∈𝜃𝑑(𝛾𝑝 , 𝛾𝑞 ) (5)
𝜀
where 𝐴 is the dilation of A with a ball of radius ε, κ is
the Euclidean curvature of the isophotes of Iand𝑔𝜀 (𝑥, 𝑦) is Each pixel p’is filled with the corresponding pixel in 𝛾𝑞 ,
the smooth function in 𝐴𝜀 . by using equation (6):

The only input parameters of the algorithm are the image 𝑝′ ∈ 𝛾𝑝 ∩ Ω (6)
to be restored and the mask that delimits the portion to be Then, the confidence term is updated to:
inpainted of the input image. The algorithm performs a
𝐶(𝑞) = 𝐶(𝑝), ∀𝑞 ∈ 𝛾𝑝 ∩ Ω (7)
pre-processing step where the entire original image goes
through the smoothness process of anisotropic diffusion. All of these processes are repeated iteratively until the
After that, the image enters an inpainting loop, where only target region is completely filled.What differentiates the
some values within A are modified. At each iteration, an technique proposed by [2] from the common exemplar-
anisotropic diffusion step is applied. This process is based algorithms is a new definition of the priority of the
repeated until a stable state is reached. patches taken and the similarity equation. The new priority
definition is described in equation (8).
In the restoration loop X inpainting steps occurs using
equation (1), then Y diffusion steps with equation (2), and 𝐷(𝑝), 𝑓𝑜𝑟 𝑡ℎ𝑒 𝑓𝑖𝑟𝑠𝑡 𝑝ℎ𝑎𝑠𝑒
𝑃(𝑝) = { (8)
again X steps of equation (1), and so on. The total number 𝐶(𝑝), 𝑓𝑜𝑟 𝑡ℎ𝑒 𝑠𝑒𝑐𝑜𝑛𝑑 𝑝ℎ𝑎𝑠𝑒
of steps is T. This number may be pre-determined or the The first phase concentrates the geometric propagation
algorithm may stop when image changes are below the of the target region, and the second, the propagation of the
given limit. The value of T depends on the size of A. texture. The algorithm automatically estimates the number
B. Deng et al. [2] Inpainting Algorithm of iterations required for the execution of the first phase.

The algorithm proposed by Liang-Jian Deng Ting-Zhu As for the similarity equation, it was changed to
Huang, Xi-Zhao is not based on partial differential equation (9).
equations (EDPs). It fills regions of interest by copying and 𝑎𝑟𝑔𝑚𝑖𝑛
𝛾𝑝 = 𝛾𝑞 ∈𝛾′𝑞 𝑑(𝛾𝑞 , 𝛾𝑝 ) (9)
pasting the portions of the source regions, so that the
texture of the image remains the same. The type of where 𝛾𝑝 and 𝛾𝑞 are patches being compared, 𝛾′𝑞 is the
technique exploited by this algorithm is called exemplar- largest patch with it’s center being 𝛾𝑞 ’s center and
based. 𝑑(𝛾𝑞 , 𝛾𝑝 ) is the sum of the quadratic differences of the
Originally, exemplar-based algorithms are based on two pixels that already filled the two patches.
attributes: a confidence term and a data term. The data term C. Quantitative Metrics
propagates the target region geometrically, and the term of
In [3], [4], [5] the lack of quantitative metrics to evaluate
confidence describes the dependence of the area of the
the results of an inpainting process is addressed. The
patch to be copied and pasted in relation to the
reason why this happens is that there is usually no
neighbouring pixels of the source region, that is, the texture
reference image, and because the content of the area to be
propagation of the original image. If there are more pixels

www.ijaers.com Page | 300

International Journal of Advanced Engineering Research and Science (IJAERS) [Vol-7, Issue-4, Apr- 2020]
https://dx.doi.org/10.22161/ijaers.74.38 ISSN: 2349-6495(P) | 2456-1908(O)

rebuilt is unknown. Therefore, in most cases, a visual

evaluation is used, where it is verified if the result is the
appropriate one. However, visually analysed results are
complex and unpredictable due to human factors that are
difficult to control. Thus, an alternative is to use known
quality metrics in the area of digital image processing,
among them the most used ones are: MSE, PSNR and
SSIM.
The MSE is the mean square error of an estimator, its
value is always positive and the results close to zero are
better.
The PSNR is a term for the relation between the
maximum signal value and the maximum noise value that
affects the fidelity of a representation. To calculate it, the 1. Test image 1.
MSE is needed.
SSIM is an index that predicts the quality of images and
videos, when measuring the structural similarity between
two images. SSIM was created as an enhancement of MSE
and PSNR comparing methods.
The main difference between SSIM and its predecessors
is that SSIM is a method based on visual perception [6], [7]
and [11] reiterate that SSIM is more efficient when
compared to MSE and PSNR methods. This is due to the
latter not detecting distortions perceptible by the human 2. Test image 2.
visual system. The reason why both work that way is that
they only consider the individual state of each pixel and
not its structural information, contrary to how SSIM
operates.
Also in [7], [8], [9] and [10] it is argued that MSE and
PSNR are not suitable for binary images. In this case, the
MSE represents the number of differences between two
images, and the large number of different pixels does not 3. Test image 3.
always result in a large structural difference, because
binary images do not have many texture details and their B. Softwares Used
pixel distribution is simpler. The software used for the implementation was the
Thus, this work included the manual construction of a Matlab R2017a, 64-bit version. The processing tests were
reference image, based on the original unprocessed image made on an Intel Core i7 processor computer with an
and applied the SSIM metric, to evaluate the quality of the Nvidia GeForce 940MX 2GB graphics card.
results obtained. The metrics were applied in the entry C. Results
images and in the inpainted results. When compared with
Figure 4 shows the test image 1, figure 5, the reference
the SSIM of the entry image, the results obtained after the
image, and in figures 6 and 7, the results obtained with the
application of the technique evidences the improvement of
implementation of the inpainting algorithm of Bertalmio et
the quality of the process.
al. [1] and Deng et al.[2] respectively.

II. METHODOLOGY AND RESULTS

A. Images Used In The Tests
Three images containing partially detected features were
used, those are presented in Figures 1, 2 and 3.

www.ijaers.com Page | 301

International Journal of Advanced Engineering Research and Science (IJAERS) [Vol-7, Issue-4, Apr- 2020]
https://dx.doi.org/10.22161/ijaers.74.38 ISSN: 2349-6495(P) | 2456-1908(O)

4. Test image 4. 7. Result of the inpainting technique of [2].

Figure 8 shows the test image 2, figure 9, the reference
image, and in figures 10 and 11, the results obtained with
the implementation of the inpainting algorithm of
Bertalmio et al. [1] and Deng et al.[2] respectively.

5. Reference image. 8. Test image 2.

9. Reference image.

6. Result of the inpainting technique of Bertalmio et al. [1].

10. Result of the inpainting technique of Bertalmio et al.

[1].

www.ijaers.com Page | 302

International Journal of Advanced Engineering Research and Science (IJAERS) [Vol-7, Issue-4, Apr- 2020]
https://dx.doi.org/10.22161/ijaers.74.38 ISSN: 2349-6495(P) | 2456-1908(O)

15. Result of the inpainting technique of [2].

11. Result of the inpainting technique of [2]. In order to facilitate the visualization of the results
obtained with the application of the technique of inpanting,
Figure 12 shows the test image 3, figure 13, the
table I presents all the results obtained, and Table II, the
reference image, and in figure 14 and 15, the results
processing time of each technique.
obtained with the implementation of the inpainting
algorithm of Bertalmio et al. [1] and Deng et al.[2] Table I. SSIM comparison between original images and
respectively. results of the inpainting techniques.
Test Image SSIM 1(%) SSIM 2 (%)

[1] [2]

1 77.21 78.94 96.11

2 88.77 88.91 98.67
3 93.02 93.75 97.30
Mean 86.33 87.20 97.36
12. Test image 3.

SSIM 1 refers to the comparison between the entry

image and the reference image, while SSIM 2, between the
reference image and the result images of the corresponding
techniques.
Table II. Processing time of each image for both
techniques
13. Reference image. Test Image Processing time(s) Processing time (s)
Bertalmio, 2000 [2]
1 0.82 3.87
2 1.53 4.58
3 1.22 3.56

III. CONCLUSIONS
14. Result of the inpainting technique of Bertalmio et al. Observing the final analysis of the results obtained with
[1]. the techniques proposed by [1] and [2] in the entry images,
we can understand that the results were highly satisfactory.
The technique of [1] showed an improvement of 0.87%
compared to the extracted feature. While the technique of
[2] obtained an improvement of 11.03%. This difference
may happen due to the first technique not fully allow
removal of occlusions and incorrectly detected features.
This type of removal is frequent in most study cases, as we

www.ijaers.com Page | 303

International Journal of Advanced Engineering Research and Science (IJAERS) [Vol-7, Issue-4, Apr- 2020]
https://dx.doi.org/10.22161/ijaers.74.38 ISSN: 2349-6495(P) | 2456-1908(O)

can see in the test images presented, which can be [11] W. Zhou,Objective Image Quality Assessment: Facing The
considered a weak point for [1] technique. Real-World Challenges. Electronic Imaging, 2016.

However, the processing time for each algorithm differs

significantly considering the amount of regions to be
inpainted, image’s dimensions, and other attributes that
may interfere in the restoration process. [1] technique
proved to be faster than [2]. Also, [1] algorithm seems to
be as useful as [2] to fill features with missing regions.
Therefore, we can say that the inpainting algorithm of
[1] is more appropriate to restore images as long as there
are few occlusions and incorrectly detected features,
otherwise [2] for the removal process.
At any rate, the good results obtained regarding the
improvement of the quality of the process of extraction of
partially detected cartographic features can be used in the
area of cartography, by supporting processes that update
cartographic products.

ACKNOWLEDGEMENTS
The authors thank the Foundation for Research Support
of the State of São Paulo - FAPESP (Process 2017/13029-
8) and the National Council of Scientific Research - CNPq.

REFERENCES
[1] M. Bertalmio, G. Sapiro, V. Caselles, C. Ballester,Image
inpainting. Proc. Siggraph Computer Graphics, 2000.
[2] L-J.Deng, T-Z.Huang, X-L.Zhao,Exemplar-Based Image
Inpainting Using a Modified Priority Definition. PLoS ONE,
2015.
[3] A. C.Kokaram,Detection of Missing Data in Image
Sequences. IEEE Transactions on Image Processing, 1995.
[4] O.Feier, A.Ioana, Digital Inpainting for Artwork
Restoration: Algorithms and Evaluation, 97f, 2012.

[5] O. L.Meur, Guillemot C,Image Inpainting: Overview and

Recent Advances. IEEE Signal Processing Magazine. 2014.
[6] T. T.Dang, A.Beghdadi, M. C.Larabi,Inpainted image
quality assessment. European Workshop on Visual
Information Processing (EUVIP), 2013.
[7] N.Peter, K. Hisakazu, Y. Masahiro, W. Hidenori, M.
Shogo,SSIM image quality metric for denoised images.
International Conference on Visualization, Imaging and
Simulation – Proceedings, 2010.
[8] W. Maged, E. Rasha,Quality Assessment of Visual
Cryptography Images. Australian Journal of Basic and
Applied Sciences, 2016.
[9] B. Furht et al.,Multimedia Encryption and Watermarking.
Springer US, 2005.
[10] C. Zhang, Z. Qiu,Simple quality assessment for binary
images. Journal of Electronics (China), 2007.

www.ijaers.com Page | 304