464 IEEE SIGNAL PROCESSING LETTERS, VOL.

28, 2021

CNN Prediction Based Reversible Data Hiding

Runwen Hu and Shijun Xiang

Abstract—How to predict images is an important issue in the we are noting that the multi receptive fields [18], [19] and global
reversible data hiding (RDH) community. In this letter, we propose optimization capabilities of convolutional neural networks
a novel CNN-based prediction approach by luminously dividing (CNNs), which could be served as a predictor for improvement
a grayscale image into two sets and applying one set to predict
the other set for data embedding. The proposed CNN predictor is a of the prediction accuracy for RDH. In [20], the authors proposed
lightweight and computation-efficient network with the capabilities a CNN-based RDH method for stereo images by using the right
of multi receptive fields and global optimization. This CNN predic- image and the left image to predict each other. Also, CNN-based
tor can be trained quickly and well by using 1000 images randomly data-driven methods have achieved satisfactory performance in
selected from ImageNet. Furthermore, we propose a two stages of global image analysis [21]–[24] and image coding [25], [26].
embedding scheme for this predictor. Experimental results show
that the CNN predictor can make full use of more surrounding In this letter, we propose a new and efficient predictor based on
pixels to promote the prediction performance. Furthermore, in CNN. An 8-bits grayscale image is first divided into two subset
the experimental way we have shown that the CNN predictor images, which can be used to predict each other by using the
with expansion embedding and histogram shifting techniques can proposed CNN-based predictor (CNNP). The proposed CNNP
provide better embedding performance in comparison with those is a lightweight and computation-efficient network with the
classical linear predictors.
capabilities of multi receptive fields and global optimization,
Index Terms—Convolutional neural network, reversible data which can be trained well by using 1000 images randomly
hiding, global optimization capability. selected from ImageNet [27]. Experimental results show that
the CNNP can promote prediction performance due to the use of
I. INTRODUCTION more surrounding pixels as the context. Experimentally we have
shown that the CNNP with classical expansion embedding and
EVERSIBLE data hiding (RDH) technologies are well
R known for the ability to recover the original image and the
information without any loss [1], [2]. Based on the characteristic
histogram shifting techniques can provide better performance by
comparing with several classical linear predictors. To the best
of our knowledge, this work is the first report in detail on how
mentioned above, RDH methods are widely used in military, to use CNN to predict grayscale images for RDH.
medical, and super-resolution processing fields [3]. The rest of this letter is organized as follows. The proposed
So far, many approaches have been developed in RDH. One RDH scheme is described in detail in Section II, and the
type of approaches focus on finding new embedding ways on experiments comparing with other predictors are reported in
how to deal with the prediction errors for reduction of embedding Section III. Finally, we conclude our work in Section IV.
distortion, such as the difference expansion [4], [5], histogram
shifting [6], [7], and prediction-error expansion [8]. The other
II. PROPOSED METHOD
type of approaches focus on how to design advanced predictors
to improve prediction accuracy, including difference predictor A. Pre-Processing Images
(DP) [4], median edge direction predictor (MEDP) [8], gradient At first, the original image I is divided into two subset images:
adaptive predictor (GAP) [9], [10], bilinear interpolation predic- the “Cross” set image IC and the “Dot” set image ID . The image
tor (BIP) [11], [12], and others by using multi-predictors [13] partition pattern is shown in Fig. 1. For the “Cross” set image, the
and adaptive strategies [14]–[17]. pixel values of the positions belong to the “Dot” set are assigned
In the literature, all of the predictors for grayscale images to 0, and so does the “Dot” set image. Such a partition pattern
share a weakness, that is to only apply one or a few adjacent ensures that the two subset images are independent but relevant.
pixels as the context for prediction. That is to say, there still After dividing the original image, the two subset images will be
exists room for RDH by making full use of more neighboring used to train and converge the proposed CNNP.
pixels as the context of a to-be-predicted pixel. To this direction,
B. Architecture Overview
Manuscript received December 9, 2020; revised January 22, 2021; accepted
February 6, 2021. Date of publication February 12, 2021; date of current version Based on the correlation of image pixels and the properties
March 10, 2021. This work was supported in part by the NSFC under Grant of CNN, the “Cross” set image and the “Dot” set image are
61772234. The associate editor coordinating the review of this manuscript, and
approving it for publication was Prof. Mylene Q. Farias. (Corresponding author: designed to predict each other in this letter. Fig. 2(a) illustrates
Shijun Xiang.) how to use the “Cross” set image to predict the “Dot” set image,
The authors are with the College of Information Science and Technol- including two main steps: the feature extraction step (for the
ogy/College of Cyber Security, Jinan University, Guangzhou 510632, China
(e-mail: [email protected]; [email protected]). use of the multi receptive fields) and the image prediction step
Digital Object Identifier 10.1109/LSP.2021.3059202 (for the use of global optimization). After the feature extraction
1070-9908 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 08,2022 at 07:05:22 UTC from IEEE Xplore. Restrictions apply.
HU AND XIANG: CNN PREDICTION BASED REVERSIBLE DATA HIDING 465

Fig. 1. Illustration to divide an image to “Dot” and “Cross” set images.

Fig. 3. Illustration on the use of convolution layers in Fig. 2(a).

By connecting the convolution blocks with different kernel sizes

in parallel in the feature extraction step and the fine-tuning
convolution blocks in series in the image prediction step, the
proposed CNNP can make full use of the global optimization
characteristics of CNN to better predict the “Dot” set image.
Considering the lightweight and computational effectiveness
Fig. 2. Overview of the proposed CNNP.
of the proposed predictor, the kernel sizes in the feature extrac-
tion step are set to 3 × 3, 5 × 5, and 7 × 7, and the number of
step, we get the intermediate predicted “Dot” set image IˆD , channels in the proposed CNNP is set to 32. Such a network
which further is used for the image prediction step. Finally, we predictor can be compressed to around 0.7 MB.
get the predicted “Dot” set image I˜D . The feature extraction
step is composed of several convolution blocks arranged in C. Training
parallel to exploit multi receptive fields. The kernel sizes of The proposed CNNP is trained by using ImageNet by ran-
these convolution blocks are K × K, where K is an odd number domly selecting 1000 images. In the training, all the images are
greater than 0 but less than the image size. The structure of the converted to 8-bits grayscale images with a size of 512 × 512.
convolution block is shown in Fig. 2(b), where a leaky rectified Suppose the input is the “Cross” set image IC , the target is the
linear unit (LeakyReLU) activation locates in the middle of two “Dot” set image ID and the output of the proposed CNNP is the
convolution layers. predicted “Dot” set image I˜D . Based on back-propagation [28],
Examples of the convolution block with K = 3 and K = 5 the Adam optimizer [29] with a batch size of 4 is used to
for feature extraction are demonstrated in Fig. 3(a). When the minimize the following loss function:
“Cross” set image is used to predict the “Dot” set image, the
P
K × K kernel indicates to use the surrounding K 2 /2 cross 1  ˜ 2
loss = (ID − ID ) + λ ω22 , (2)
set pixels to predict the central dot set pixel, where · is the P i=1
floor function. With the increase of K, more surrounding cross
set pixels are adopted to predict the central dot set pixel, which where P is the number of training data, λ is the weight decay, and
can be formulated as, ω denotes all the weights in the network. Aiming to effectively
suppress over-fitting and accelerate network convergence, λ is
K
 set to 10−3 . In this letter, the proposed CNNP is trained on an
I˜D (x, y) = IC (x + i, y + j)·w(i, j) + b, (1) Intel Core i9 CPU (3.6 GHz) with 32 GB RAM and an NVIDIA
i,j=1
Titan V GPU.
where I˜D is the predicted “Dot” set image, IC is the “Cross” set
image, w is the weight of convolution kernel, and b is the bias. D. CNNP Based RDH Method
In the image prediction step in Fig. 2(a), the features extracted The flowchart of the proposed embedding scheme with the
from different convolution blocks are added together and fed into trained CNNP is shown in Fig. 4. The “Cross” set image IC is
two convolution blocks with kernel size of K = 3. The pixel firstly used to generate the predicted “Dot” set image I˜D by using
values of the intermediate predicted “Dot” set image IˆD are the proposed CNNP (as illustrated in Fig. 2). Subsequently, ID
fine-tuned by five predicted dot set pixels as shown in Fig. 3(b). and I˜D are used to reversibly hide part of the information W1

Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 08,2022 at 07:05:22 UTC from IEEE Xplore. Restrictions apply.
466 IEEE SIGNAL PROCESSING LETTERS, VOL. 28, 2021

Fig. 4. The proposed reversible data embedding scheme.

Fig. 5. The proposed reversible data extracting scheme.

and generate the data hidden “Dot” set image IDW . Similarly, TABLE I
AVERAGE MSE, ABSOLUTE MEAN AND VARIANCE OF THE PREDICTION
IDW is fed into the proposed CNNP to generated the predicted ERRORS IN 100 IMAGES FOR FIVE DIFFERENT PREDICTORS
“Cross” set image I˜C , which is in cooperated with IC to embed
the other part of the information W2 and generate the data hidden
“Cross” set image ICW . After embedding the whole information
W (W = W1 + W2 ), ICW is added with IDW to generate the
data hidden image IW .
The flowchart of extracting the hidden information and re-
covering the original image is shown in Fig. 5. The data hidden
image IW is divided into two subset images (i.e., ICW and of the training image set, 100 images are randomly selected from
IDW ) with the same partition pattern in Fig. 1. After that, ImageNet for statistical experiments. Without special mention,
the data hidden “Dot” set image IDW is firstly fed into the the results in this section are the average of 100 images.
proposed CNNP to generate the predicted “Cross” set image
I˜C , which is in cooperated with ICW to extract the information
W2 and recover the original “Cross” set image IC . The recovered A. Prediction Accuracy
“Cross” set image IC is fed into the proposed CNNP to generate Table I lists the MSE, the absolute mean, and the variance of
the predicted “Dot” set image I˜D , which is used to recover the prediction errors for the proposed CNNP and four classical
the original “Dot” set image ID and extract the information predictors. The MSE of the CNNP is 99.4, which is lower than
W1 . Subsequently, the recovered images ID and IC are added BIP (154.8), MEDP (234.2), GAP (231.9), and DP (230.8).
together in the spatial domain to recover the original image I. Besides, the absolute mean and the variance in the proposed
Meanwhile, W1 and W2 are combined to recover the information CNNP are 4.77 and 66.9, respectively, which are more con-
bits W . centrated and lower than BIP (6.25 and 100.5), MEDP (7.37
and 161.3), GAP (9.86 and 167.6), and DP (5.13 and 196.6).
From Table I, we can conclude that the prediction accuracy of
III. EXPERIMENTAL RESULTS the proposed CNNP is better than several classical predictors.
The performance of the proposed CNNP is evaluated by The basic reason is that the proposed predictor can exploit multi
comparing it with several classical predictors, including the receptive field prediction and global optimization capabilities
BIP, MEDP, GAP, and DP. The detailed description of these of CNN by optimizing the network parameters with the loss
predictors can be seen in [30]. function. Furthermore, we have plotted the histograms of the
Aiming to assess the prediction performance of the proposed prediction error for these five different predictors on the image
CNNP, the mean square error (MSE) is adopted since it can Lena, as shown in Fig. 6, in which the abscissa range is set
reflect the difference between the predicted images and the target to [−30, 30] for better display. We can see that the histogram
images well. For a fair comparison, the predictors mentioned of the prediction errors generated from the proposed predictor
above are combined with the same embedding schemes for RDH, is higher and sharper than the histograms of the other four
in which the performance of these predictors is measured by predictors, and the number of the prediction error values in
computing the peak signal-to-noise ratio (PSNR) values of the the region [−3, 3] of the proposed CNNP is larger than that
watermarked images at the same embedding rates. Independent of other predictors. From these experimental results, we can

Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 08,2022 at 07:05:22 UTC from IEEE Xplore. Restrictions apply.
HU AND XIANG: CNN PREDICTION BASED REVERSIBLE DATA HIDING 467

TABLE II
AVERAGE PSNR (IN dB) OF 100 IMAGES FOR FIVE DIFFERENT PREDICTORS BY
USING EXPANSION EMBEDDING TECHNIQUE IN [8]

Fig. 6. Histograms of the image Lena under five different predictors. TABLE III
AVERAGE PSNR (IN dB) OF 100 IMAGES OF THE PROPOSED CNNP-BASED
METHOD AND THE METHOD [11]

CNNP so as to compare the proposed CNNP-based RDH scheme

with the classical BIP-based RDH method [11]. Table III lists
Fig. 7. Comparison of five RDH methods by using four benchmark images.
the average PSNR values of these two RDH methods. From
this table, we can see that the average PSNR values of the
proposed CNNP based RDH method is around 1 dB higher than
conclude that the proposed CNNP has achieved better prediction the BIP-based RDH method. The basic reason is that the BIP can
performance in comparison with several classical predictors. be regarded as a convolution operation with K = 3, while the
proposed CNNP contains three parallel convolution operations
B. Embedding Performance with K = 3, K = 5, and K = 7. In other words, the CNNP has
exploited more adjacent pixels for prediction due to its multi
There are two main embedding techniques for RDH: expan-
receptive field and global optimization capacities.
sion embedding [8] and histogram shifting [11]. For a better
comparison, we first adopted the classical error expansion em-
IV. CONCLUSION
bedding technique for these five predictors (DP, MEDP, GAP,
BIP, and the proposed CNNP). Fig. 7(a)–(d) show the PSNR In this letter, we design a CNN-based predictor for grayscale
values of four benchmark images (Lena, Yacht, Peppers and images, which can make full use of the capabilities of the multi
Barbara) with embedding capacities from 10 000 to 120 000 receptive fields and global optimization of CNN for RDH. The
bits. From this figure, we can see that the PSNR values of the basic idea is to divide an image into two sets and use one set to
RDH method with the CNNP are larger. Statistically, we have predict the other set. The proposed CNNP is a lightweight and
tested 100 different images and computed their average PSNR computation-efficient network and can be trained well by using
values for different embedding capacities in Table II. When the 1000 different images. Experimental results have shown that the
data to be hidden is 10 000 bits, the average PSNR value in prediction accuracy of the CNNP is higher than those classical
the proposed CNNP based RDH method is 47.9 dB, which is linear predictors. Also, we have used the proposed CNNP with
higher than BIP (46.4 dB), MEDP (44.2 dB), GAP (41.7 dB), two embedding ways to show that the proposed CNN-based
and DP (41.3 dB) based RDH methods. As the data to be hidden RDH scheme can achieve satisfactory performance by testing
increases from 10 000 to 120 000 bits, the average PSNR values 100 different images. To the best of our knowledge, there is no
of the proposed RDH method are still higher. detailed report on how to use CNN to predict a grayscale image
To better evaluate the performance of the proposed CNNP, for RDH. We consider this work makes a room for the RDH
the histogram shifting technique is combined with the proposed community.

Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 08,2022 at 07:05:22 UTC from IEEE Xplore. Restrictions apply.
468 IEEE SIGNAL PROCESSING LETTERS, VOL. 28, 2021

REFERENCES [16] I.-C. Dragoi and D. Coltuc, “Local-prediction-based difference expansion

reversible watermarking,” IEEE Trans. Image Process., vol. 23, no. 4,
[1] J. M. Barton, “Method and apparatus for embedding authentication infor- pp. 1779–1790, Apr. 2014.
mation within digital data,” US Patent 5 646 997, Jul. 1997. [17] H. Chen, J. Ni, W. Hong, and T.-S. Chen, “High-fidelity reversible data
[2] C. W. Honsinger, P. W. Jones, M. Rabbani, and J. C. Stoffel, “Lossless hiding using directionally enclosed prediction,” IEEE Signal Process.
recovery of an original image containing embedded data,” US Patent 6 Lett., vol. 24, no. 5, pp. 574–578, May 2017.
278 791, Aug. 2001. [18] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Object
[3] Y.-Q. Shi, X. Li, X. Zhang, H.-T. Wu, and B. Ma, “Reversible data hiding: detectors emerge in deep scene CNNs,” 2014, arXiv:1412.6856.
Advances in the past two decades,” IEEE Access, vol. 4, pp. 3210–3237, [19] W. Luo, Y. Li, R. Urtasun, and R. Zemel, “Understanding the effective
2016. receptive field in deep convolutional neural networks,” Adv. Neural Inf.
[4] J. Tian, “Reversible data embedding using a difference expansion,” IEEE Process. Syst., vol. 29, pp. 4898–4906, 2016.
Trans. Circuits Syst. Video Technol., vol. 13, no. 8, pp. 890–896, Aug. 2003. [20] T. Luo, G. Jiang, M. Yu, C. Zhong, H. Xu, and Z. Pan, “Convolutional
[5] A. M. Alattar, “Reversible watermark using the difference expansion of a neural networks-based stereo image reversible data hiding method,” J.
generalized integer transform,” IEEE Trans. Image Process., vol. 13, no. 8, Vis. Commun. Image Representation, vol. 61, pp. 61–73, 2019.
pp. 1147–1156, Aug. 2004. [21] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image
[6] Z. Ni, Y.-Q. Shi, N. Ansari, and W. Su, “Reversible data hiding,” IEEE recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016,
Trans. Circuits Syst. Video Technol., vol. 16, no. 3, pp. 354–362, Mar. 2006. pp. 770–778.
[7] X. Li, W. Zhang, X. Gui, and B. Yang, “Efficient reversible data hiding [22] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely
based on multiple histograms modification,” IEEE Trans. Inf. Forensics connected convolutional networks,” in Proc. IEEE Conf. Comput. Vis.
Secur., vol. 10, no. 9, pp. 2016–2027, Sep. 2015. Pattern Recognit., 2017, pp. 4700–4708.
[8] D. M. Thodi and J. J. Rodríguez, “Expansion embedding techniques for [23] X. Zhang, X. Zhou, M. Lin, and J. Sun, “ShuffleNet: An extremely efficient
reversible watermarking,” IEEE Trans. Image Process., vol. 16, no. 3, convolutional neural network for mobile devices,” in Proc. IEEE Conf.
pp. 721–730, Mar. 2007. Comput. Vis. Pattern Recognit., 2018, pp. 6848–6856.
[9] D. Coltuc, “Improved embedding for prediction-based reversible water- [24] M. Tan and Q. V. Le, “EfficientNet: Rethinking model scaling for convo-
marking,” IEEE Trans. Inf. Forensics Secur., vol. 6, no. 3, pp. 873–882, lutional neural networks,” 2019, arXiv:1905.11946.
Sep. 2011. [25] I. Schiopu, Y. Liu, and A. Munteanu, “CNN-based prediction for lossless
[10] D. Coltuc, “Low distortion transform for reversible watermarking,” IEEE coding of photographic images,” presented at the 2018 Picture Coding
Trans. Image Process., vol. 21, no. 1, pp. 412–417, Jan. 2011. Symp. (PCS), San Francisco, CA, USA, Jun. 24–27, 2018.
[11] V. Sachnev, H. J. Kim, J. Nam, S. Suresh, and Y. Q. Shi, “Reversible wa- [26] I. Schiopu and A. Munteanu, “Deep-learning based lossless image coding,”
termarking algorithm using sorting and prediction,” IEEE Trans. Circuits IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 7, pp. 1829–1842,
Syst. Video Technol., vol. 19, no. 7, pp. 989–999, Jul. 2009. Jul. 2020.
[12] L. Luo, Z. Chen, M. Chen, X. Zeng, and Z. Xiong, “Reversible image [27] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet:
watermarking using interpolation technique,” IEEE Trans. Inf. Forensics A large-scale hierarchical image database,” in Proc. IEEE Conf. Comput.
Secur., vol. 5, no. 1, pp. 187–193, Mar. 2010. Vis. Pattern Recognit., 2009, pp. 248–255.
[13] I. F. Jafar, K. A. Darabkh, R. T. Al-Zubi, and R. A. Al Na’mneh, “Efficient [28] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learn-
reversible data hiding using multiple predictors,” Comput. J., vol. 59, no. 3, ing applied to document recognition,” Proc. IEEE, vol. 86, no. 11,
pp. 423–438, 2016. pp. 2278–2324, Nov. 1998.
[14] X. Li, B. Yang, and T. Zeng, “Efficient reversible watermarking based [29] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
on adaptive prediction-error expansion and pixel selection,” IEEE Trans. 2014, arXiv:1412.6980.
Image Process., vol. 20, no. 12, pp. 3524–3533, Dec. 2011. [30] S. Xiang and Y. Wang, “Non-integer expansion embedding techniques
[15] X. Li, J. Li, B. Li, and B. Yang, “High-fidelity reversible data hiding for reversible image watermarking,” EURASIP J. Adv. Signal Process.,
scheme based on pixel-value-ordering and prediction-error expansion,” vol. 2015, no. 1, 2015, Art. no. 56.
Signal Process., vol. 93, no. 1, pp. 198–205, 2013.

Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 08,2022 at 07:05:22 UTC from IEEE Xplore. Restrictions apply.