2017 IEEE Information Theory Workshop - ITW 2017 Kaohsiung

An Iterative Algorithm to Construct Optimal Binary

AIFV-m Codes
Ken-ichi Iwata Hirosuke Yamamoto
Dept. of Information Science Dept. of Complexity Science and Engineering
University of Fukui The University of Tokyo
Fukui-shi, Fukui, 910-8507, Japan Kashiwa-shi, Chiba, 277-8561, Japan
E-mail: [email protected] E-mail: [email protected]

TABLE I: Average codeword lengths of AIFV-m codes and Huffman codes,

Abstract—We propose an algorithm to construct an optimal and Entropy (bits per source symbol) for source 1: PX = (0.900, 0.050, 0.049,
code that achieves the minimum average codeword length in the 0.001) and source 2: PX = (0.33, 0.3, 0.119, 0.1, 0.05, 0.04, 0.03, 0.02, 0.01,
class of binary AIFV-m codes with m code trees T0 , T1 , · · · , Tm−1 0.001).
for a given stationary memoryless source. The algorithm is an
iterative algorithm such that the optimal Tk for a given set of LHuff LAI−2 LAI−3 LAI−4 LAI−5 Entropy
source 1 1.1500 0.7515 0.6551 0.6356 0.6345 0.5761
costs is derived by dynamic programming (DP) and the costs are
source 2 2.5630 2.5035 2.5003 2.5002 2.5000 2.4894
updated from the set of code trees (T0 , T1 , · · · , Tm−1 ), iteratively.
The proposed DP works with polynomial time and space for internal nodes and leaves. Yamamoto, Tsuchihashi, and Honda
source alphabet size. We prove the AIFV-m code obtained by
the proposed algorithm is optimal for m = 2, 3, 4, 5 although the
[2, Sect. IV-C] showed that the optimal AIFV codes with
algorithm works for any m and we conjecture the optimality also two code trees can be constructed by combining an iterative
holds for m ≥ 6. Furthermore, we verify by some examples of algorithm to improve a parameter called cost and an integer
sources that the average codeword length of the optimal binary programming (IP) problem to derive the optimal code trees
AIFV-m codes can be decreased as m becomes large. for a given cost. Furthermore, we showed in [3] that the IP
problem can be replaced with a dynamic programming (DP)
I. I NTRODUCTION
problem.
We consider binary lossless source coding of fixed-to- In this paper, we extend the iterative algorithm with DP to
variable length (FV) codes. Huffman coding [1] achieves the obtain the optimal binary AIFV-m code trees. The proposed DP
minimum average code length in the class of uniquely decod- works with O(mn2m+1 ) time and O(mnm+1 ) space for n = |X|.
able FV codes with a single code tree for stationary memoryless We prove that the binary AIFV-m code obtained by the proposed
sources. But, the binary AIFV (almost instantaneous FV) codes algorithm is optimal for m = 2, 3, 4, 5.
with two code trees [2] can attain a better average code length As examples, in Tab. I, the average code length LAI−m of
than the Huffman coding. Since source symbols are assigned to the optimal AIFV-m code obtained by the proposed algorithms
incomplete internal nodes in addition to leaves in the code trees are compared with the Huffman codes for two sources. We
of AIFV codes, the AIFV codes are not instantaneous codes note that the average codeword length of the optimal binary
generally. However the binary AIFV codes are devised so that AIFV-m codes is decreased as m becomes large.
the decoding delay is at most two bits. Hu, Yamamoto, and The remainder of this paper is organized as follows. In
Honda [4], [5] derived the worst-case redundancy of the binary section II, we introduce the binary AIFV-m codes and the
AIFV codes and also extended the binary AIFV codes to the iterative algorithm with DP for the binary AIFV code. Then,
binary AIFV-m codes by allowing m code trees and at most in Section III, we propose the iterative algorithm with the DP
m-bit decoding delay [5]1 . They proved that the worst-case (Algorithm 2) for the binary AIFV-m codes, and we evaluate
redundancy of the binary AIFV-m codes is 1/m for m = 2, 3, 4. the complexity and optimality of the proposed algorithm in
Let X denote a source alphabet, and n denote the size of X, Theorems 2 and 3, respectively.
i.e., n = |X|. We note that AIFV-m codes for X require O(nm)
memory space to store a set of m code trees, while a Huffman II. P RELIMINARIES
code for source alphabet X m needs O(nm ) memory space to
achieve the worst-case redundancy 1/m. Hence, binary AIFV-m A. Definition of binary AIFV-m codes
codes are much more efficient than Huffman codes for X m , The binary AIFV-m codes are defined as follows [5].
although they attain comparable compression performance.
The construction of the optimal AIFV-m code for a given Definition 1 (Class of binary AIFV-m codes [5, Sect. III]).
source is difficult because an AIFV code uses multiple code (A) A binary AIFV-m code consists of a set of m code trees
trees and source symbols are assigned to both of incomplete Ti, i ∈ Im := {0, 1, . . . , m − 1}.
(B) Each Ti consists of slave nodes, master nodes, and
1 The binary AIFV-2 codes correspond to the original binary AIFV codes. complete internal nodes, which are defined as follows.

Authorized licensed use limited to: Hong

978-1-5090-3097-2/17/$31.00 Kong University
©2017 IEEE of Science and Technology.519
Downloaded on August 13,2023 at 23:15:40 UTC from IEEE Xplore. Restrictions apply.
 2017 IEEE Information Theory Workshop - ITW 2017 Kaohsiung

T0 T1 T2 T3
a1 a1 a1 Procedure 2 (Decoding of Binary AIFV-m Codes [2], [5]).
0 0 1 0 0
(1) Use T0 to decode the first source symbol x1 from y.
a1
0 1 0 0
(2) Trace y as long as possible from the root in the current
0 0 1 1 0 code tree. Then output the source symbol assigned to the
0
a2 0
a3 0 1 1 reached master node.
0 1 0
a2 0
a3 0 1
(3) Remove the traced prefix of y, and if the reached node
a2 a3 a2 a3 is a master node of degree k, then use Tk to decode the
0 0 0 0
next source symbol.
0 0 0 0

0
a4 0 0
We note that since the transition of m code trees is
a4 synchronized between the encoder and the decoder, codeword
0 0
a4 a4 sequences can be uniquely decoded with at most m-bit delay
Fig. 1: An example of code trees of the optimal AIFV-4 code for a stationary (see [5, Theorem 3]).
memoryless source with X = {a1, a2, a3, a4 } and {PX (a1 ) = 0.9, PX (a2 ) = Using the binary AIFV-4 code of Fig. 1, a source se-
0.05, PX (a3 ) = 0.049, PX (a4 ) = 0.001}, which is the probability distribution quence ‘a2 a3 a1 a1 a3 a1 a1 a1 a2 ’ is encoded to codeword se-
of source 1 in Tab. I. In each code tree Ti , i ∈ I4 , symbols a2 and a4 are
assigned to master nodes of degree 0, and symbol a3 is assigned to a master quence ‘00000.00001.λ.λ.011.λ.λ.1.00000’ by the transitions
node of degree 3. But, symbol a1 is assigned to master nodes of degree 3, 0, of code trees T0 → T0 → T3 → T2 → T1 → T3 → T2 → T1 →
1, and 2 in Ti , i = 0, 1, 2, and 3, respectively. T0 , where dots ‘.’ and null codeword ‘λ’ of the roots of trees
are inserted for the sake of human readability. We can decode
the codeword sequence to the original source sequence using
•A slave node is defined as an incomplete internal node to code trees of Fig. 1. Refer [2, Sect. III] and [5, Examples 1,2]
which no source symbol is assigned. The slave nodes are for the details of encoding and decoding of AIFV-m codes.
categorized into slave-0 nodes and slave-1 nodes such that We next define the average codeword length LAI (T , PX ) of
slave-b node is connected to its child by ‘b’, b ∈ {0, 1}. an AIFV-code(T ) for a stationary memoryless source PX over
• A master node is defined as a node to which a source a finite alphabet X with probability PX (a) > 0 for a ∈ X as
symbol is assigned. The master nodes are classified into 
m types. A master node of degree 0 is a leaf, and a LAI (T, PX ) := P(k |T )L(Tk ), (1)
master node of degree k, 1 ≤ k ≤ m − 1, is an incomplete k ∈Im
internal node satisfying the following two conditions: 1) where P(k |T ) is the stationary probability of code tree Tk used
k consecutive descendant nodes are slave-0 nodes, and 2) in coding, and L(Tk ) is the average code length of Tk for PX .
the (k + 1)-th descendant node is not a slave-0 node2 . If the current symbol xi is encoded by a master node of degree
• A complete internal node has two children connected by k  in Tk , then the next symbol xi+1 is encoded by Tk  . Hence,
code symbols ‘0’ and ‘1’. No source symbol is assigned the transition probabilities of code trees Q(Tk  |Tk ) from Tk to
to a complete internal node. Tk  , for short qk  (Tk ), are given by
(C) In Tk , k  0, there must exist a slave-1 node which is 
connected by k consecutive ‘0’s from the root. qk  (Tk ) := Q(Tk  |Tk ) := PX (a), (2)
a ∈Nk  (Tk )
Note that the root of Tk does not have a descendant node
where Nk  (Tk ) stands for the set of source symbols a ∈ X such
connected with k + 1 consecutive ‘0’. The root of Tk , k  1,
that a is assigned to a master node of degree k  in Tk . We
can become a master node, but the root of the optimal T1 must
note qk  (Tk ) depends on only Tk and source PX .
be the complete internal node.
We now describe the optimization problem exactly. For a
We denote a binary AIFV-m code, which is composed of
given stationary memoryless source PX , an optimal AIFV-code
a set of m code trees T := {Tk , k ∈ Im }, by “AIFV-code(T )”.
(i.e. optimal code trees T ∗ ) is given by
Figure 1 illustrates an example of a set of code trees of a binary
AIFV-4 code, where nodes , , , and represent master T ∗ := argmin LAI (T, PX ), (3)
nodes, complete internal nodes, slave-0 nodes, and slave-1 T ∈ T0 (n)×T1 (n)×···×Tm−1 (n)

nodes, respectively. where Tk (n) denotes the set of code trees Tk satisfying the
The encoder of AIFV-code(T ) maps a source sequence x = conditions in Definition 1 for n = |X|. The problem to obtain
x1 x2 x2 · · · , xi ∈ X, to a binary sequence y = y1 y2 y3 · · · , and T ∗ of (3) is a global optimization problem for the average
the decoder reproduces x from y as follows. performance (average code length) of a probabilistic transition
Procedure 1 (Encoding of Binary AIFV-m Codes [2], [5]). system with m states, which are m functions L(Tk ), k ∈ Im as
(1) Use T0 to encode the first source symbol x1 . shown in the right-hand side of (1).
(2) If xi is encoded by a master node of degree k, then use For AIFV-2 codes, Yamamoto, Tsuchihashi, and Honda [2]
Tk to encode the next source symbol xi+1 . proved that optimal AIFV code trees can be constructed by
the iterative algorithm shown in Section II-B. We extend it
2 The “degree” of a master node is different from the meaning normally to AIFV-m codes for m ≥ 2, and prove the optimality of the
used in graph theory. Figure 1 shows an example of code trees of AIFV-4. iterative algorithm for m = 2, 3, 4, 5 in Section III.

Authorized licensed use limited to: Hong

978-1-5090-3097-2/17/$31.00 Kong University
©2017 IEEE of Science and Technology.520
Downloaded on August 13,2023 at 23:15:40 UTC from IEEE Xplore. Restrictions apply.
 2017 IEEE Information Theory Workshop - ITW 2017 Kaohsiung

(α) (α−1)
Algorithm 1 Iterative algorithm to construct optimal AIFV-2 (a) if C0,1  C0,1 , then LAI (T (α), PX ) < LAI (T (α−1), PX ).
code trees for a given source PX [2, Algorithm 1]. Hence, LAI (T (α), PX ) is strictly decreasing in α.
(α) (α−1)
Require: stationary memoryless source PX . (b) if C0,1 = C0,1 , then LAI (T (α), PX ) = LAI (T ∗, PX ).
Ensure: optimal T = {T0, T1 } in the sense that LAI (T, PX ) =
The code trees T (α) = {T0(α), T1(α) } finally obtained by Al-
LAI (T ∗, PX ).
(α) (α) (α) gorithm 1 is optimal binary AIFV-2 code trees for a given
1: Initialize α := 0, C0,1 := 2 − log2 3 and C1,0 := −C0,1
stationary memoryless source PX .
2: do
3: α ←α+1 Remark 1. We proposed a DP algorithm [3, Algorithm 2]
4: Obtain Tk(α), k ∈ I2 of (4) to find (4), which can be used instead of the IP used in [2].
(α)
(α)
L(T1 )−L(T0 )
(α)
(α) (α) The DP works with O(n5 ) time and O(n3 ) space for source
5: set cost C0,1 := (α) (α) and C1,0 := −C0,1
q1 (T0 )+q0 (T1 ) alphabet size n.
(α) (α−1)
6: while C0,1  C0,1
III. I TERATIVE A LGORITHM TO C ONSTRUCT O PTIMAL
7: output (T0(α), T1(α) ) B INARY AIFV-m C ODE T REES
In this section, we give an iterative algorithm to construct
optimal binary AIFV-m code trees for a given stationary
B. Iterative Algorithm to Construct Optimal binary AIFV Code memoryless source P for m ≥ 2. In order to extend Algorithm
X
Trees [2, Sect. IV-C] 1 to the case of binary AIFV-m codes for m ≥ 2, we use
Yamamoto, Tsuchihashi, and Honda [2, Sect. IV-C] formu- parameters (or costs) Ck,k , k, k  ∈ Im, m ≥ 2, which satisfy
lated a strategy to construct the optimal two code trees T0, T1 
of binary AIFV code, i.e., AIFV-2 code for a given source LAI (T, PX ) = L(Tk ) + Ck,i qi (Tk ), (9)
i ∈Im
PX by combining an iterative algorithm and an IP problem. In
their iterative algorithm, IP is used to find ⎧
⎪ 0 if k  = k,
⎪
⎪
⎪
⎪ 
k−1
Tk(α) := argmin{L(T) + Ck,k (α−1)
q (T)}, for k ∈ I2, (4) ⎪
⎪
⊕1 k ⊕1 ⎨
⎪ Ci,i+1 if k  < k,
T ∈Tk (n) Ck ,k := i=k  (10)
⎪
⎪
where α denotes the number of iterations, ⊕ denotes the ⎪
⎪
 −1
k
(α−1) ⎪
⎪ Ci,i+1 if k  > k.
addition modulo m, and Ck,k ⊕1
, k ∈ I2 is defined as follows: ⎪−
⎪
⎩ i=k
⎧
⎪ 
(α)
⎨ 2 − log2 3
⎪ if α = 0 (initialization), From (10), it holds that i ∈Im Ci,i ⊕1 = 0 and, for k ∈ Im ,
C0,1 := (α)
L(T1 )−L(T0 )
(α) (5) 
⎪
⎪ if α ≥ 1,
⎩ 1 0
q (T
(α)
)+q (T
0 1
(α)
) Ck,i qi (Tk )
(α) (α) i ∈Im
C1,0 := −C0,1 if α ≥ 0. (6)

k−1  i 
m−2 
m−1
It holds that =− Ci,i+1 q j (Tk ) + Ci,i+1 q j (Tk ). (11)
q0 (T1 )L(T0 ) + q1 (T0 )L(T1 ) i=0 j=0 i=k j=i+1
LAI (T , PX ) =
q1 (T0 ) + q0 (T1 ) Furthermore, we have from (9) and (10) that
= L(Tk ) + Ck,k ⊕1 qk ⊕1 (Tk ) (7)
LAI (T, PX ) − L(Tk )
for T = {Tk , k ∈ I2 }, where the first equality follows from 
k−1  i 
m−2 
m−1
P(k |T ) = qk (Tk ⊕1 )/(q1 (T0 ) + q0 (T1 )) for m = 2, and the second =− Ci,i+1 q j (Tk ) + Ci,i+1 q j (Tk ). (12)
equality holds since Ck,k ⊕1 is defined as i=0 j=0 i=k j=i+1
L(Tk ⊕1 ) − L(Tk ) On the other hand, from (1), we obtain
Ck,k ⊕1 := for k ∈ I2 . (8) 
qk ⊕1 (Tk ) + qk (Tk ⊕1 )
LAI (T, PX ) − L(Tk ) = P(i|T ) (L(Ti ) − L(Tk )) . (13)
Hence, the global optimization problem to solve i ∈Im

argmin LAI (T, PX ) By solving simultaneous linear equations

T ∈T0 (n)×T1 (n) 
RHS of (12) = P(i|T ) (L(Ti ) − L(Tk )) (14)
can be realized by Algorithm 1 [2, Algorithm 1], where Tk(α) i ∈Im
is optimized for Ck,k ⊕1 in Step 4, and Ck,k ⊕1 is updated for
for k ∈ Im , we can represent {Ci,i ⊕1, i ∈ Im } as follows:
(T0(α), T1(α) ) in Step 5, iteratively.  
It is shown in [2, Algorithm 1], that LAI (T (α), PX ) monotoni- Ci,i ⊕1 = L(Tj  ) − L(Tj ) Qi, j, j  (T ), (15)
cally decreases to the optimal LAI (T ∗, PX ) with finite repetition. (j, j  )∈Im ×Im :j< j 
We summarize their results as the following theorem.
where Qi, j, j  (T ) is determined using the transition probabilities
Theorem 1 ([2, Theorem 2]). For = T (α) {T0(α), T1(α) } and (α)
C0,1 ql (Tl ), l, l  ∈ Im, l  l . For example, if m = 2, then we obtain
obtained by Algorithm 1, it holds that Q0,0,1 (T ) = 1/(q0 (T1 ) + q1 (T0 )) and Q1,0,1 (T ) = −1/(q0 (T1 ) +

Authorized licensed use limited to: Hong

978-1-5090-3097-2/17/$31.00 Kong University
©2017 IEEE of Science and Technology.521
Downloaded on August 13,2023 at 23:15:40 UTC from IEEE Xplore. Restrictions apply.
 2017 IEEE Information Theory Workshop - ITW 2017 Kaohsiung

(L(Ti ⊕1 ) − L(Ti )) (qi (Ti ⊕2 ) + qi ⊕1 (Ti ⊕2 )) + (L(Ti ⊕2 ) − L(Ti )) qi ⊕2 (Ti ⊕1 ) − (L(Ti ⊕2 ) − L(Ti ⊕1 )) qi ⊕2 (Ti )
Ci,i ⊕1 =  (16)
j ∈I3 q j ⊕1 (T j )q j ⊕2 (T j ) + q j ⊕1 (T j )q j ⊕2 (T j ⊕1 ) + q j ⊕1 (T j ⊕2 )q j ⊕2 (T j )

 (α)   
(α) L(Ti(α) (α) (α) (α) (α) (α) (α) (α)
⊕1 ) − L(Ti ) qi (Ti ⊕2 ) + qi ⊕1 (Ti ⊕2 ) + L(Ti ⊕2 ) − L(Ti ) qi ⊕2 (Ti ⊕1 ) − L(Ti ⊕2 ) − L(Ti ⊕1 ) qi ⊕2 (Ti )
(α)
Ci,i ⊕1 :=  (20)
(α) (α) (α) (α) (α) (α)
j ∈I3 q j ⊕1 (T j )q j ⊕2 (T j ) + q j ⊕1 (T j )q j ⊕2 (T j ⊕1 ) + q j ⊕1 (T j ⊕2 )q j ⊕2 (T j )

Algorithm 2 Iterative algorithm to construct AIFV-m code Theorem 2. For a given stationary memoryless source with
trees for a given source PX . (α−1)
alphabet size n, and a given {Ci,i+1 , i ∈ Im−1 }, the modified
Require: stationary memoryless source PX and m. version of DP algorithm proposed in [3] generates an optimal
Ensure: optimal T = {Tk , k ∈ Im } such that LAI (T , PX ) = code tree T ∈ Tk (n) for for each k ∈ Im , which satisfies the
k  −1 (α−1)
LAI (T ∗, PX ) if m ∈ {2, 3, 4, 5} and all absolute values of RHS of (17), if the all values of i=k Ci,i+1 for k  k  ∈
Ck(α)
,k are less than 1.

Im, k < k are less than 1. The DP algorithm works with
1:
(α)
Initialize α := 0, and set Ck,k+1 , k ∈ Im−1 as (18) O(mn2m+1 ) time and O(mnm+1 ) space to obtain one T ∈ Tk (n)
2: do for each k ∈ Im .
3: α ←α+1 We omit the details of DP and the proof of Theorem 2
4: Obtain Tk(α), k ∈ Im of (17) in here due to the space limitation. Algorithm 2 satisfies the
(α)
5: set cost Ci,i ⊕1, i ∈ Im−1 as (19) following theorem.
∃ (α) (α−1)
6: while i ∈ Im−1 such that Ci,i+1  Ci,i+1 Theorem 3. For T (α) = {Tk(α), k ∈ Im } and {Ci,i+1
(α)
, i ∈ Im−1 }
7: output {Tk(α), k ∈ Im } obtained by Algorithm 2, it holds that in case of m = 2, 3, 4, 5,
(α) (α−1)
(a) if there exists i ∈ Im−1 such that Ci,i+1  Ci,i+1 , then
(α)
LAI (T , PX ) < LAI (T (α−1), PX ), i.e., LAI (T (α), PX ) is
q1 (T0 )) from (8), and if m = 3, then we can calculate Qi, j, j  (T ) strictly decreasing function in α.
from (16) given at the top of this page. (α) (α−1)
(b) if Ci,i+1 = Ci,i+1 for all i ∈ Im−1 , then we have
Using relations of (9), (11), and (15), Algorithm 1 can be LAI (T , PX ) = LAI (T ∗, PX ).
(α)
generalized to Algorithm 2 to construct optimal AIFV-m code
The set of code trees T (α) finally obtained by Algorithm 2
trees for m ≥ 2. At the step 4 in Algorithm 2, we need to
is optimal binary AIFV-m code trees for a given stationary
obtain
memoryless source PX .
⎧
⎨
⎪ 
k−1 i
Tk(α) := argmin L(T) − (α−1)
Ci,i+1 q j (T) Proof sketch of Theorem 3: If m = 2, then Theorem 3 is
T ∈Tk (n) ⎪
⎩ i=0 j=0 reduced to Theorem 1.

m−2 
m−1 ⎫
⎬
⎪ We assume that Algorithm 2 with a source PX terminates
(α−1) (α)
+ Ci,i+1 q j (T) (17) at α = α∗ . For Ci,i+1 , i ∈ Im−1, α ∈ {0, 1, . . . , α∗ } defined by
⎪ (α) (α)
i=k j=i+1 ⎭ (18) and (19), let us define Ck,k 
 for k, k ∈ Im as Ck,k := 0,
(α) (α) (α) k  −1 (α)
(α−1)
for k ∈ Im , where Ci,i+1 , i ∈ Im−1 := Im \ {m − 1} is defined Ck ,k := −Ck,k  , and Ck,k  := i=k Ci,i+1 if k < k .
from (15) as follows: We first prove (a) of Theorem 3 by exhaustion. From the
(α) (α−1)
 assumption, there exists i ∈ Im−1 such that Ci,i+1  Ci,i+1 for
α<α . ∗
(0) 2 − log2 3 if i = 0,
Ci,i+1 :=  i+2  i+2 (18) If m = 3, we divide (C0,1 (α) (α)
, C1,2 (α−1)
)  (C0,1 (α−1)
, C1,2 ) into the
log2 2 − 2 − log2 2 − 1 if i ≥ 1,
 following four cases:
(α)
Ci,i+1 := L(Tj(α)
 ) − L(T j
(α)
) Qi, j, j  (T (α) ) (19) (α)
(a3-0): C0,1 (α−1)
≤ C0,1 (α)
, C1,2 (α−1)
≤ C1,2 ,
(j, j  )∈Im ×Im :j< j  (α) (α−1) (α) (α−1)
(a3-1): C0,1 ≥ C0,1 , C1,2 ≤ C1,2 ,
(α) (α−1) (α) (α−1)
for T (α) := {Ti(α), i ∈ Im }. For example, if m = 3, then (a3-2): C0,1 ≥ C0,1 , C1,2 ≥ C1,2 ,
(α) (α) (α−1) (α) (α−1)
Ci,i+1, i ∈ I3 is defined by (20). (a3-3): C0,1 ≤ C0,1 , C1,2 ≥ C1,2 .
There are two ways to execute the minimization of (17) If (a3-k), k ∈ I3 holds, then we obtain that
(α−1)
for given {Ci,i ⊕1 , i ∈ Im−1 } and source PX . One way is a LAI (T (α), PX )
modified version of IP proposed in [2]. The complexity of the
IP problem is NP hard in general. Therefore, we use a DP, = L(Tk(α) )
which is a modified version of DP proposed in [3], and we 
k−1 
i 
m−2 
m−1
(α)
have the following theorem. − Ci,i+1 q j (Tk(α) ) + (α)
Ci,i+1 q j (Tk(α) )
i=0 j=0 i=k j=i+1

Authorized licensed use limited to: Hong

978-1-5090-3097-2/17/$31.00 Kong University
©2017 IEEE of Science and Technology.522
Downloaded on August 13,2023 at 23:15:40 UTC from IEEE Xplore. Restrictions apply.
 2017 IEEE Information Theory Workshop - ITW 2017 Kaohsiung

< L(Tk(α) )
∗
We now leads to∗ a contradiction LAI (T (α ), PX ) ≤
∗ (α ) (α∗ −1)

k−1 
i 
m−2 
m−1 LAI (T , PX ) using Ck,k  = Ck,k  and (17) in a similar manner
(α−1)
− Ci,i+1 q j (Tk(α) ) + (α−1)
Ci,i+1 q j (Tk(α) ) to the proof of (b) of Theorem 1 in [2]. If m = 3, we divide
(α∗ ) (α∗ ) ∗ , C ∗ ) into the following four cases:
i=0 j=0 i=k j=i+1 (C0,1 , C1,2 )  (C0,1 1,2
∗
(α )
(α−1) ∗ , C (α) ≤ C ∗ ,
≤ C0,1
≤ L(Tk ) (b3-0): C0,1 1,2 1,2
∗
(α ) ∗ , C (α) ≤ C ∗ ,
≥ C0,1

k−1  i 
m−2 
m−1 (b3-1): C0,1 1,2 1,2
(α−1) (α−1) (α−1)
− Ci,i+1 q j (Tk )+ Ci,i+1 q j (Tk(α−1) ) (b3-2): C0,1 (α∗ ) ∗ , C (α) ≥ C ∗ ,
≥ C0,1 1,2 1,2
i=0 j=0 i=k j=i+1 (α∗ ) ∗ , C (α) ≥ C ∗ .
(α−1)
(b3-3): C0,1 ≤ C0,1 1,2 1,2
= LAI (T , PX ), (21) For instance, if (b3-0) holds, then we obtain, for i ∈ I3 ,
where the first (resp. last) equality follows from the definition of ∗
(α) (α−1) LAI (T (α ), PX )
T (α), Ci,i+1 (resp. T (α−1), Ci,i+1 ), and the first strict inequality ∗ ∗ ∗ ∗ ∗
(α−1) (α−1) = L(T0(α ) ) + C0,1
(α )
q1 (T0(α ) ) + C0,2
(α )
q2 (T0(α ) )
follows from the condition (a3-k) and (C0,1 , C1,2 )  ∗ ∗ ∗ ∗ ∗
(α)
(C0,1 (α)
, C1,2 ), and the last inequality follows from (17). In = L(T0(α ) ) + C0,1
(α −1)
q1 (T0(α ) ) + C0,2
(α −1)
q2 (T0(α ) )
(β) (β) (β) (α −1)∗ (α −1) ∗
the case of (a3-3), by using C0,1 + C1,2 = −C2,0 for ≤ L(T0∗ ) + C0,1 q1 (T0∗ ) + C0,2 q2 (T0∗ )
β ∈ {α − 1, α}, we furthermore divide (a3-3) into two (α ) ∗ (α ) ∗
(α) (α−1) (α) (α−1) = L(T0∗ ) + C0,1 q1 (T0∗ ) + C0,2 q2 (T0∗ )
cases: (a3-3-0): C0,1 ≤ C0,1 , C2,0 ≤ C2,0 , and (a3-3-1):
(α) (α−1) (α) (α−1) ≤ L(Ti∗ ) + C0,1
∗
q1 (T0∗ ) + C0,2
∗
q2 (T0∗ )
C0,1 ≤ C0,1 , C2,0 ≥ C2,0 . Similar to the case (a3-k) for
k ∈ {0, 1, 2}, if (a3-3-0) holds, then we obtain that = LAI (T ∗, PX ),

LAI (T (α), PX ) where the first equality follows from ∗(9), the second and third
 (α∗ ) (α −1)
equalities follow from Ck,k  = Ck,k  , the first inequality
= L(T2(α) ) (α)
+ C2,0 qi (T2(α) ) + C0,1
(α)
q0 (T2(α) )
follows from (17), the last inequality follows from the condition
i ∈ {0,1}
 (b3-0), and the last equality follows from (24). Although we
< L(T2(α) ) + C2,0
(α−1)
qi (T2(α) ) + C0,1
(α−1)
q0 (T2(α) ) omit the details here, we can prove (b) of Theorem 3 for the
i ∈ {0,1}
 remaining cases of m = 3 and for m = 4, 5, similarly. We
(α−1) (α−1)
≤ L(T2 ) + C2,0 qi (T2(α−1) ) + C0,1
(α−1)
q0 (T2(α−1) ) conjecture that (b) also holds for m ≥ 6. 
i ∈ {0,1}
IV. C ONCLUSION
(α−1)
= LAI (T , PX ). (22)
In this paper, we proposed an iterative algorithm with
Moreover, if (a3-3-1) holds, then we obtain that dynamic programming in Algorithm 2 to obtain the optimal
AIFV-m code trees for a given stationary memoryless source.
LAI (T (α), PX ) Although we omitted the details, DP can be used in the
= L(T0(α) ) − C2,0
(α)
q1 (T0(α) ) + C0,1
(α)
q2 (T0(α) ) minimization of the RHS in (17) for given {Ci,i(α−1)
⊕1 , i ∈ Im−1 }
and source PX . We verified for source examples in Tab. I
< L(T0(α) ) − C2,0
(α−1)
q1 (T0(α) ) + C0,1
(α−1)
q2 (T0(α) )
that the average codeword length of optimal binary AIFV-m
≤ L(T0(α−1) ) − C2,0
(α−1)
q1 (T0(α−1) ) + C0,1
(α−1)
q2 (T0(α−1) ) code decreases as m becomes large. Furthermore, we proved
= LAI (T (α−1), PX ). (23) in Theorem 3 that the iterative algorithm gives the optimal
AIFV-m code trees for m = 2, 3, 4, 5. We conjecture that the
We can prove (a) of Theorem 3 for the remaining cases in proposed algorithm can also construct an optimal AIFV-m code
m = 3 and for m = 4, 5, similarly. We conjecture that (a) of for m ≥ 6.
Theorem 3 also holds for m ≥ 6.
We next prove (b) of Theorem 3 by contradiction and by R EFERENCES
∗
exhaustion. Suppose that T (α ) obtained by Algorithm 2, is [1] D. A. Huffman, “A method for the construction of minimum-redundancy
codes,” Proc. IRE, vol. 40, no. 9, pp. 1098–1101, Sep. 1952.
not globally optimal for PX , and there exists the optimal code [2] H. Yamamoto, M. Tsuchihashi, and J. Honda, “Almost instantaneous
∗
trees T ∗ such that LAI (T (α ), PX ) > LAI (T ∗, PX ). Let us define Fixed-to-Variable length codes,” IEEE Trans. Inf. Theory, vol. 61, no. 12,
Ci,i+1 by the right-hand side of (15) with T = T ∗ , and also
∗ pp. 6432–6443, Dec. 2015.
∗ , k, k  ∈ I by C ∗ ∗ ∗ [3] K. Iwata, and H. Yamamoto, “A dynamic programming algorithm to
define Ck,k



m k,k := 0, Ck ,k := −Ck,k  , and construct optimal code trees of AIFV codes,” Int. Symp. on Inf. Theory
∗ k −1 ∗ 
Ck,k  := i=k Ci,i+1 if k < k . Then, we have, for k ∈ Im ,
and Its Applications (ISITA2016), pp. 672–676, Oct.–Nov. 2016.
[4] W. Hu, H. Yamamoto, and J. Honda, “Tight upper bounds on the
LAI (T ∗, PX ) redundancy of optimal binary AIFV codes,” IEEE Int. Symp. on
Inf. Theory (ISIT2016), pp. 6–10, Jul. 2016.

k−1 
i 
m−2 
m−1
[5] W. Hu, H. Yamamoto, and J. Honda, “Worst-case redundancy of optimal
= L(Tk∗ ) − ∗
Ci,i+1 q j (Tk∗ ) + ∗
Ci,i+1 q j (Tk∗ ) binary AIFV codes and their extended codes,” IEEE Trans. Inf. Theory,
i=0 j=0 i=k j=i+1 pp. 5074–5086, vol. 63, no. 8, Aug. 2017.

= L(Tk∗ ) + ∗
Ck,i qi (Tk∗ ). (24)
i ∈Im

Authorized licensed use limited to: Hong

978-1-5090-3097-2/17/$31.00 Kong University
©2017 IEEE of Science and Technology.523
Downloaded on August 13,2023 at 23:15:40 UTC from IEEE Xplore. Restrictions apply.