Data Structures

Binary Indexed Trees

TH I N K F A ST

Mostafa Saad Ibrahim

PhD Student @ Simon Fraser University
Contents:-

◼Background
◼Problem Statement
◼Motivation
◼Prefix Sum
◼Upadation
◼References
Problem Statement

◼ We often need some sort of data structure to make our

algorithms faster. The Binary Indexed Trees structure,
proposed by Peter M. Fenwick. This structure was first used
for data compression, Peter M. Fenwick. We begin by
motivating the use of this structure by an example.

◼ Consider the following problem: There are n boxes that

undergo the following queries:
1. add marble to box i
2. sum marbles from box k to box l

◼ Our goal is to implement those two queries.

Problem Statement

 The naive solution has time complexity of O(1) for query 1

and O(n) for query 2.

 Suppose we make m queries. The worst case (when all the

queries are 2) has time complexity O(n * m).

 Using some data structure (i.e. RMQ) we can solve this

problem with the worst case time complexity of O(mlogn)

 Binary Indexed Tree data structure, also with the worst time
complexity O(m log n) — but Binary Indexed Trees are
easier to code and require less memory space than RMQ.
Binary Representation
Removing bits from mask
 mask = think in integer bits
 Assume we have numbers X, Y
 If for every position in Y with 1 AND X has 1
 X = 10010100
 Y = 00010100
 X - Y removes all Y 1s from X
 Another longer/general way to do so:
 X & ~Y
 10010100 &
 11101011 =
 10000000
Least/Most Significant Bit

Src: http://www.electronique-et-informatique.fr/Digit/images/MSBLSB.gif
Src: http://tronixstuff.com/wp-content/uploads/2011/05/binnum.jpg
Two’s Complement Representation

 Start to flip AFTER the “last bit”

 -number = 2’s complement of number
 One way to compute manually:

 Get 1’s complement...then add 1

Removing Last Bit
 Get last bit using index & -index

 +20 = 00010100
 -20 = 11101100
 20 & -20 = 00000100

 Remove last bit

 Get it...subtract it
 index - (index & -index)
 00010100 - 00000100 = 00010000
Integer as sums of powers of 2

= 010010011

Src: http://images.slideplayer.com/5/1546412/slides/slide_22.jpg
Our problem
 Let’s move to our problem
 Given an array of integer N
 Assume index 0 always will be 0 (NOT in use)

 2 query types:
 Add value val to position index
 Sum values from 1 to index

 Segment Tree can be used to such problem

 O(N) preprocess, O(NlogN) queries, O(nlogn) memory

 BIT has a better memory order (shorter code)

 O(n) memory + O(NlogN) queries
Problem Example
0 1 2 3 4 5 6 7 8 9 10 11

xx 3 2 -1 6 5 4 -3 3 7 2 3

- Accumulative Sum (1, 3): 3 + 2 - 1 = 4

- Accumulative Sum (1, 5): 3 + 2 - 1 + 6 + 5 = 15
- Add: index 3, value = 4

xx 3 2 4 6 5 4 -3 3 7 2 3

- AccumulativeSum (1, 3): 3 + 2 + 4 = 9

- Accumulative Sum (1, 5): 3 + 2 + 4 + 6 + 5 = 21
Motivation
 Integer = Sum of Powers of 2
 Accumulative Sum = Sum of Sub sums
 Recall: 147 = 128 + 16 + 2 + 1
 Think in accumulative sum (1 to 147)
 Sum of last 1 number +
 Sum of next 2 numbers +
 Sum of next 16 numbers +
 Sum of next 128 numbers

 Sum(1,147) =
 Sum(147,147) + Sum(146,145) + Sum(144,129) + Sum(128,1)
 147 ⇒ positions {147, 146, 144, 128} with ranges {1, 2, 16,
128}
Motivation

◼ To get starting positions fast? Remove last bit

▪ 147 = 010010011 [remove last 1 bit]

▪ 146 = 010010010 [remove last 1 bit]
▪ 144 = 010010000 [remove last 1 bit]
▪ 128 = 010000000 [remove last 1 bit]
▪ 0 = DONE
◼ How to interpret:
▪ 147 responsible for range 147 to > 146
▪ 146 responsible for range 146 to > 144
▪ 144 responsible for range 144 to > 128
▪ 146 responsible for range 128 to > 0
Binary Indexed Tree
 Create a new array of Length N, name it BIT
 BIT[position] = sum of its responsible range
 Then For each Query

 Sum(147)= BIT(147) + BIT(146) + BIT(144) + BIT(128)

 That is: 4 steps only to get the answer
 Sum(144) = BIT(144) + BIT(128)
 Sum(15) = BIT(15) + BIT(14) + BIT(12) + BIT(8)
 Recall: 1111 = 1111, 1110, 1100, 1000, 0
 Sum(11) = BIT(11) + BIT(10) + BIT(8)
 Recall 1011: 1011, 1010, 1000, 0
 Sum(7) = BIT(7) + BIT(6) + BIT(4) ⇒ 111, 110, 100, 0
Binary Indexed Tree

Sum(15) = BIT(15) + BIT(14) + BIT(12) + BIT(8)

E.g. node 12 has values: BIT[12] = val[12] + val[11]+val[9]

12 = 1100 ⇒ removing last 1 bit ⇒ 1000 = 8
Then parent of 12 ⇒ 8 (e.g. next closest position 12 is not covering)
Notice: we removed bit at position 2 ⇒ 12 covers 2^2 numbers = 12 - 8 = 4
Binary Indexed Tree

Notice: 8 = 1000 => has 3 trailing zeros. Try to replace each 0 with 1
1001 = 9
1010 = 10
1100 = 12
# of trailing zeros = # children … child remove last bit => go to parent
 Get Interval Accumulation
Sum(15) = BIT(15) + BIT(14) + BIT(12) + BIT(8)
= 1111 ⇒ 1110 ⇒ 1100 ⇒ 1000 ⇒ 0 = STOP

15 is responsible for 1 number, 14 for 2, 12 for 4, 8 for 8 numbers

 Updating position

Position 1: Covered by 4 intervals ⇒ 1, 2, 4, 8

Add -3 to 1 ⇒ add -3 to these 4 intervals

Given index, how to get smallest position covering it?

E.g. 1 ⇒ 2 6 ⇒ 8 10 ⇒ 12 13 ⇒ 14
Then 1 goes to 2...2 goes to 4..4 goes to 8 [recursive]
Updating position

◼ Recall given number idx it covers 2^r values

▪ r is position of “last bit”
▪ It covers numbers from idx to idx – 2^r + 1

◼ All following numbers cover 8 values

▪ 0001000 ⇒ r = 3 ⇒ 2^3 = 8
▪ 0101000
▪ 1101000
▪ 1111000
▪ 1001000
◼ So our focus on “last bit”, NOT before that
Updating position

◼ 1000 covers 8 numbers

▪ 1000 - 000 = 1000

▪ 1000 - 001 = 0111
▪ 1000 - 010 = 0110
▪ 1000 - 011 = 0101
▪ 1000 - 100 = 0100
▪ 1000 - 101 = 0011
▪ 1000 - 110 = 0010
▪ 1000 - 111 = 0001

◼ Each of these 8 numbers covered by 1000

◼ But 1000 is NOT their smallest cover number
Updating position

◼ Let’s get who covers 4 = 0100

▪ 4 has “last bit” at k = 2

▪ When target number enumerate its 2^r, one contains 100
▪ So we need to go at least 1 bit higher than k
▪ E.g. re-set last bit k = 3 ⇒ 1000 ⇒ first one to cover 0100

◼ Let’s get who covers 5 = 0101

▪ k=0
▪ We need target number to include our 1 at k = 0
▪ The earlier one should exist in smallest coverer number
▪ So again, shift k = 0 1 step to be in its enumeration
▪ E.g. re-set last bit k = 1 ⇒ 110. Note, 1000 also cover 5
Updating position

◼ Let’s get who covers 3 = 0011

▪ “last bit” at k = 0
▪ We need enumeration includes whole 11
▪ So parent need to be a 1 before these 11
▪ E.g. ⇒ 0100

◼ So general rule
▪ 100100001000 100100011100
▪ 100100010000 100100100000

◼ How to get that number easily?

▪ Just add 2^k ⇒ if one or more bits ⇒ shifted
▪ E.g. 100100011100 + 000000000100 = 100100100000
Updating tree
Updating tree

Notice: 8 = 1000 => has 3 trailing zeros. Remove last bit, and add 1, 2, 3...trailing ones
0100 = 4
0110 = 6
0111 = 7
# of trailing zeros = # children
Updating tree

Building initial tree from input: just iterate on input

and add it to its position
References
 Article on Prefix Sums and Their Applications by Guy E.
Blelloch School of Computer Science
Carnegie Mellon University Pittsburgh, PA 15213-3890

 Efficient Range Minimum Queries using Binary Indexed

Trees :
Mircea Dima and Rodica Ceterchi
Olympiads in Informatics, 2015, Vol. 9, 39–44
DOI:http://dx.doi.org/10.15388/ioi.2015.04