A method for embedding binary data into JPEG2000 bit streams based on the layer structure

Katsutoshi ANDO , Hiroyuki KOBAYASHI , and Hitoshi KIYA

Department of Electrical Engineering, Tokyo Metropolitan University 1-1 Minami-Osawa, Hachioji-shi, Tokyo, 192-0397 JAPAN Department of Electrical Engineering, Tokyo Metropolitan College of Technology Higashi-Oi 1-10-40, Shinagawa-ku, 140-0011 Japan [email protected] ABSTRACT features is that the length of the image data is not changed by embedding. It also results that the embedding process can be performed independently of the encoder and decoder of an image coding. The other feature is that the deterioration of the reconstructed image due to embedding is suppressed effectively. 2 JPEG2000 coding [912]

A method for embedding binary data into JPEG2000 bit streams was developed. Using the method, a decoder can perfectly extract embedded data as well as it can when using the conventional methods. Moreover, by exploiting the layer structure of a JPEG2000 bit stream, the method effectively suppress the quality deterioration of a reconstructed image due to embedding, and it can perform embedding and extracting independently of image coding. The length of data that can be embedded and the deterioration of the reconstructed image due to embedding were simulated. 1 Introduction

Watermarking techniques that embed information into digital images imperceptibly has been studied extensively recently [17]. One of the applications of such a watermarking techniques is copyright protection [6]. Illegal use of digital images is discouraged by means of embedded identication information. An other application is data embedding for multiplexing other contents in a digital image so that an image and other contents altogether. This paper focuses on a watermarking technique for the latter application. In this application, embedded data need not to be in hidden form and there are various objective contents for embedding, for example, index label [2] and audio data. In video coding, errors can be concealed by embedding motion vectors into bit streams [8]. As a method of data embedding for multiplexing other contents, it is essential to embed binary data into a digital image and extract them perfectly. It is also required that an image with embedded data can be decoded by a general decoder. The conventional methods, which targets JPEG images, satises these points [1, 2]. In the current work, a more advanced method for data embedding that targets JPEG2000 [912] coded images was developed. This method embeds data into a JPEG2000 bit stream by overwriting certain parts of the bit stream. The part to be overwritten is selected adequately by utilizing the layer function of the JPEG2000 coding system. While the developed method satises the requirements for data embedding as mentioned above, additional features, which were not available with the conventional methods, are provided. One of the

2.1 Review of coding procedure Figure 1 is a block diagram of a JPEG2000 encoder. An input image is decomposed into sub-bands by a discrete wavelet transform and the wavelet coefcients of each sub-band are quantized. After quantization, each sub-band is divided into rectangular blocks, called code-blocks, which form the independent input to coefcient bit modeling and arithmetic coding. A separate arithmetic code is generated for each code-block. The arithmetic code for each code block has a lot of truncation points so that it can be truncated into a various lengths. One truncation point is allocated for each code blocks subject to a target bit rate. After that, the layer structure is formed for scalability of a bit stream. Since the developed method exploits the layer structure, we explain it in detail. 2.2 Layer formation Figure 2 shows the structure of a JPEG2000 bit stream. A global bit stream consists of a header and a sequence of layers. Each layer includes a part of coded code blocks so that it successively improves the image quality. The most signicant layer comprises the most signicant contribution to image quality and the following layers increment image quality successively. Thus, the decoder can decode a bit stream with improving image quality progressively. Figure 3 shows a conceptual view of the layer structure. Figure 3(a) is a bit stream whose rate is 1.0 bits/pixel. It has two uniform layers, namely each layer has a rate of 0.5 bits/pixel. This bit stream can be decoded by the decoder with two steps of image quality improvement. Furthermore, the lower rate bit stream (Figure 3(b)) can be generated by truncating the latter layer of the longer bit stream (Figure 3(a)). The lower rate bit stream generated in such a way is nearly same as the bit stream encoded to have the lower rate

EBCOT
Header

Layers

Input Image

DWT

Quantization

Codeblock partition

Coefficient bit modeling

1 1
0

Formatting

Header

JPEG2000 Stream

Layer formation

Code truncation (Rate control)

Arithmetic coding

Truncation

Figure 1: JPEG2000 encoder.

0.5 Bit rate (bits/pixel)

1.0

Header

The most significant layer

... ...

The least significant layer

Code stream

Figure 3: Layer truncation. (The latter layer of a double-layered bit stream was truncated.)

Header

Header

Code Code block 0 block 1

Layers

Layer
Code stream

2
Replace

MSB

...

Arithmetic code (for codeblock 0)

Header

Binary data

Binary data

Figure 2: Structure of a JPEG2000 bit stream. Figure 4: Data embedding to the latter layer. (The shaded area is replaced with embedded data.)

originally. It should be noted that the number of layers and the length of each layer could be assigned arbitrarily in a JPEG2000 encoder. 3 New embedding method

The developed method for embedding data into a JPEG2000 bit stream is designed to satisfy the following conditions. Note that the conditions (a) and (b) were satised by the conventional method, yet (c) and (d) were not. Condition (c) states that embedding and extracting can be performed independent of image coding. (a) The embedded data can be extracted perfectly under a lossy coding scheme. (b) A general decoder can be used. (c) The length of the bit stream is not changed. (d) The deterioration of a reconstructed image due to embedding is suppressed. First, we explain the embedding process in the developed method. Then we explain the decoding of a bit stream containing embedded data. We show that the developed method satises the conditions. In addition, we mention the affair concerning marker codes that is important for the embedding and extracting. 3.1 Data embedding into a JPEG2000 bit stream In the new method, rst, an image is encoded into JPEG2000 bit stream with two layers by means of a general JPEG2000 encoder. Next, the encoded bit stream of the latter layer is scanned in backward order and replaced with the data that we want to embed (see Figure 4.). One byte of embedded data replaces one byte of the original bit stream. This means that the length of the bit stream is not changed. If the latter layer has surplus bytes to be replaced, they are left alone.

The maximum length of data that can be embedded depends on the length of the latter layer of a double-layered bit stream, so the length of the latter layer is made longer than that of the data to be embedded. 3.2 Decoding of a JPEG2000 bit stream containing embedded data

A JPEG2000 bit stream containing embedded data can be decoded by a general JPEG2000 decoder. Depending on knowledge of embedding, i.e., the fact that the developed method was applied, and on the necessity of extracting embedded data, there are three types of decoding as explained follows. (A) Decoding without knowledge of embedding Even if the decoder has no knowledge of embedding, a JPEG2000 bit stream containing embedded data can be decoded because the header information and the length of bit stream are not changed, namely, the bit stream is compatible with JPEG2000 (Figure 5). The embedded data are decoded as noise, so some quality deterioration of a reconstructed image occurs, yet the deterioration is not serious since the latter layer makes a less signicant contribution to image quality. (B) Decoding with the knowledge of embedding When the decoder has knowledge of embedding, the quality deterioration due to embedded data can be further reduced by truncating the latter layer. As shown in gure 6, the latter layer, which contains the embedded data, is truncated. A truncated bit stream that has a shorter length is thus generated. The reconstructed image from the truncated bit stream is usually better than that from the original bit stream which including the embedded data, because noise come from decoding the embedded data no longer appears.

Layers

Layers

Header

Header

Data embedded code stream

binary data

Data embedded code stream

binary data

binary data Extraction

Truncated code stream

Header

JPEG2000 decoder

Truncation

Figure 5: Decoding without knowledge of embedding. (Type A: the whole bit stream was decoded.)

JPEG2000 decoder

Layers

Data embedded code stream

Header

binary data

Figure 7: Extracting of embedded data. (Type C: The shaded area was extracted as the embedded data and the truncated bit stream was decoded.)

Truncated code stream

Header

Truncation

JPEG2000 decoder

sition/composition process based on two-channel lter banks with Daubechies 9/7 bi-orthogonal wavelet lter running under the JPEG2000 verication model 7.0 [13]. The target bit rate was 1.0 bits/pixel (32,768 bytes). 4.1 Available length of embedded data

Figure 6: Decoding with knowledge of embedding. (Type B: only the rst layer was decoded.)

(C) Extracting of embedded data If the decoder has knowledge of embedding, the embedded data can be extracted by scanning the bit stream of the latter layer in backward order as shown in Figure 7. The length of the embedded data should be known to the decoder for extracting the embedded data correctly unless the length of the latter layer is the same as that of the embedded data. In this case, the reconstruction of the image can be done in the same way as type B described above. 3.3 Avoidance of embedding fake marker codes

We determined how much data can be embedded into a JPEG2000 bit stream by the developed method. Figure 8 shows the relationship between the bit rate of the latter layer and the maximum length of the embedded data. As shown in the gure, the method can embed various lengths of data. This is because the length of each layer can be assigned arbitrarily within the whole length of a bit stream in a JPEG2000 encoder and because the maximum length of embedded data depends on the length of the latter layer of a double-layered bit stream. 4.2 Image deterioration due to embedding

When embedding data into a JPEG2000 bit stream, we should take into account that a JPEG2000 coder uses codes ranging from 0xFF90 to 0xFFFF, which do not arose in the arithmetic encoder used in JPEG2000, as the special maker codes that have special meaning (e.g., 0xFFD9 means End of code). If the data to be embedded contains a bit stream equal to such codes, (in other words if fake marker codes are included in the embedded data,) correct decoding of a bit stream with embedded data will be occasionally prevented. Therefore, the data must be transformed without including such codes before embedding. One possible way to do this kind of transform is to apply arithmetic encoding, same as that used in a JPEG2000 coder, to the data before embedding. In the extracting, the original data can be reconstructed by arithmetic decoding. 4 Simulation

We demonstrated the effectiveness of the developed embedding method by simulation. The method was applied to the standard image Lena (256 grayscale, , 262,144 bytes). The image coding was done by a ve-level decompo-

The image deterioration due to embedding in the developed method was measured in terms of peak signal-to-noise ratio (PSNR) between the original image and the reconstructed image. Higher PSNR means lower deterioration. First, the deterioration due to embedding by the new method is compared to that of the conventional methods based on JPEG coding [1] under the same condition. We assumed that the decoder has knowledge of the embedding, i.e., it corresponds to type B mentioned in 3.2. The result of the comparison is shown in Figure 10. It is clear that the new method entirely outperforms the conventional method in terms of PSNR. Next, the PSNR against the length of the embedded data when a bit stream containing embedded data is decoded with a general JPEG2000 decoder was measured. The two ways of decoding, types A and B described in 3.2, were compared. Type A decodes the whole bit stream without the knowledge of embedding. Type B truncates the latter layer of the doublelayered bit stream before decoding with utilizing the knowledge of embedding. The results are shown in the Figure 9. It is clear that type B demonstrates better performance than type A. This results shows that further suppression of the deterioration is possible by the new method if the decoder has the knowledge of embedding.

5000

40.5 40 39.5 Proposed Conventional

Embedded data size [Bytes]

4500 4000 3500 3000 2500 2000 1500 1000 500 0 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 length of the latter layer [bits/pixel]

PSNR [dB]

39 38.5 38 37.5 37 36.5 36 0 1000 2000 3000 4000 Embedded data size [Bytes] 5000

Figure 8: Available size of embedded data vs. the length of the latter layer.

Figure 10: Comparison of the new method and the conventional method. (PSNR corresponding to the length of embedded data.)

40.5 40 39.5 39 38.5 38 37.5 0 1000 2000 3000 4000 5000 Embedded data size [Bytes] without truncation (Type A) with truncation (Type B)

labeling, Proc. IEEE Int. Conf. Image Proc., 28AP2, Oct. 1999. [3] H. Kiya, Y. Noguchi, A. Takagi, and H. Kobayashi, A method of inserting binary data into MPEG bitstreams, Proc. IEEE Int. Symp. Circuits and Systems, Orlando, May 1999. [4] Geert Depovere, Ton Kalker, etc.: The VIVA project: Digital watermarking for broadcast monitoring, Proc. IEEE Int. Conf. Image Proc., 26AP1, Oct. 1999. [5] Chrisine I. Podilchuk and Edward J.Delp: Digital watermarking: algorithms and applications, IEEE Signal Processing Magazine, pp.3346, July 2001. [6] F. Boland, J.O. Ruanaidh, and C. Dautzenberg, Watermarking digital images for copyright protection, Proc. IEEE Int. Conf. Image Proc., vcol.III, pp.219222. 1996. [7] A. H. Tewk and M. Swanson, Data hiding for multimedia; personalization, interaction, and protection, IEEE Signal Processing Magazine, vol.14, no.4, pp4144, 1997. [8] M. Kurosaki, H. Kiya, Error concealment using a data hiding technique for MPEG video, Proc. European Conference on Circuit Theory and Design, pp. 81-84, Aug. 2001. [9] B.E. Usevirch, A tutorial on modern lossy wavelet image compression: Foundation of JPEG2000, IEEE Signal Processing magazine, Vol.18, No.5, pp.2235, Sep. 2001. [10] A. Skodras, C. Christopoulos, and T. Ebrahimi, The JPEG2000 still image compression standard, IEEE Signal Processing magazine, Vol. 18, No.5, pp.3658, Sep. 2001. [11] ISO/IEC 15444-1: JPEG2000 Part1 nal draft international standard , Aug. 2000. [12] D. Taubman and M.W. Marcelin, JPEG2000: Image compression fundamentals, standards, and practice, Kluwer academic publishers, 2001. [13] ISO/IEC JTC 1/SC 29/WG 1 WG1N1684: JPEG2000 Verication Model 7.0 , Apr. 2000.

Figure 9: Comparison of type A and type B of decoding. (PSNR corresponding to the length of embedded data.)

The above results show that the new embedding method is capable of embedding various lengths of data and can suppress the deterioration due to embedding effectively. 5 Summary

We developed a method of embedding binary data into JPEG2000 bit streams. This method exploits various capacity of the JPEG2000 coding system, such as layer structure, so that it can effectively embed data into a digital image. Simulation results demonstrated the effectiveness of the new method. Acknowledgments
The authors would like to thank Mr. Takahiro Fukuhara and Mr. Seiji Kimura of Sony Corporation for their helpful advice to our research. This work was supported in part by Grant-in-Aid for Scientic Research No.11875088 from Japan Society for the Promotion of Science.

References
[1] Y. Noguchi, H. Kobayashi, and H. Kiya, A method of extracting embedded binary data from JPEG bitstreams using standard JPEG decoder, IEICE Trans. Fundamentals Vol.E82-A No.8, pp.14851492, Aug. 1999. [2] H. Kiya, Y. Noguchi, A. Takagi, and H. Kobayashi, A method of inserting binary data into JPEG bitstreams for video index

PSNR [dB]