Below is a figure to give you an idea of how the memory layout looks like:

Lets have a look at both of these memories in detail.

Stack Memory:

Definition:

The stack is a region of a computer's memory that operates in a Last-In, First-Out (LIFO) manner. It is used to store local variables and function call information. The stack memory is managed automatically by the compiler, making it a straightforward and efficient way to allocate and deallocate memory.

Characteristics:

• Automatic Management: Stack memory is managed automatically by the compiler. When a function is called, space is allocated on the stack for local variables, and this space is automatically released when the function exits.

• Faster Access: Accessing data on the stack is faster than on the heap because it involves simple pointer manipulation.

Limitations:

• Fixed Size: The size of the stack is typically fixed, and exceeding this limit can lead to a stack overflow. This limitation can be a challenge for applications requiring dynamic memory allocation.

Heap Memory:

Definition:

The heap is a region of a computer's memory used for dynamic memory allocation. Unlike the stack, the heap memory requires manual management by the programmer. It provides a more flexible and expansive memory space, allowing for the allocation of memory at runtime.

Characteristics:

• Dynamic Allocation: Memory on the heap can be allocated and deallocated explicitly during the program's execution, providing flexibility in managing memory.

• Variable Size: The heap allows for the allocation of variable-sized blocks of memory, making it suitable for applications with unpredictable memory requirements.

Limitations:

• Manual Management: Heap memory requires manual management, and failure to deallocate memory properly can result in memory leaks or fragmentation.

• Slower Access: Accessing data on the heap is generally slower than on the stack due to the need for explicit pointer manipulation.

This information indicates that stack is the ideal place for storing and managing local variables & function call information. On the other hand, heap is suitable for dynamic memory allocation and best for applications with varying memory requirements.

There is this one final piece of information I would like to share.

Heap memory is shared across all threads of a program, however each thread has its own separate call stack.

Hence, in scenarios where data needs to be kept private and isolated between threads, utilizing the stack can be a safer choice in a multithreaded program, such as a transaction handler. The stack's thread-specific nature helps prevent unintended interactions between threads, ensuring data integrity and contributing to a more robust and thread-safe design. It's important to balance this choice with an awareness of the stack's size limitations and automatic memory deallocation characteristics.

Subscribe now

Range updates and Range Queries

We will look at supporting range updates only first. Then we will use that understanding of ours to support range queries too.

Range updates

Consider that you have a BIT and it contains nothing but zeroes at the moment.

You are given a bunch of queries of the form:

[l, r, x] -> Add x to each element in the range [l, r]

If we look at our existing implementation of BIT. We can see that if we do point updates, this will take us O((r - l + 1) * log(maxIdx)) time.

Can we do better?
Well, yes! We can mimic the same behavior by making two point updates!

1. add(l, x)
2. add(r + 1, -x)

Now, if we want the value of an element at an index i, we can just perform prefix sum using our query function!

Lets break down these two queries and see why this works?

The range [0, l - 1] is left untouched by these two queries. Hence, the answer will be correct for that range.
The range [l, r] will also be corrected since we added x to l. We will always get the answer x in return.
The -x added post r, will cancel out the first operation and return 0 as expected.

Range Queries

Cool! So we can do point queries. But what about range queries? How do we support them?
In order to support range queries we will require two BITs. Say BIT1 and BIT2.
Again, for simplicity we will assume that both have initially nothing but zero values.

Lets assume we got a query to add x to each element in the range [l, r].

How do we update these two BITs?

We will arrive at our answer by backtracking.

What do we expect when we want to know the sum of values upto ith index.

We can rewrite this sum as follows:

This now is a difference of two terms.
We will store one term in BIT1 and the other in BIT2.

Then,

Now writing the update queries becomes much simpler.

For BIT1:

1. add(l, x)
2. add(r + 1, -x)

For BIT2:

1. add(l, x*(l - 1))
2. add(r + 1, -x*r)

And now, we can perform range queries as well!

In case of doubts or confusions, revisit the previous sections of this series or list them down in the comments below.

Happy learning!!

Subscribe now

Implementation

class BIT
{
private:
    vector<int> arr;
    int n;

public:
    BIT(int n)
    {
        this->n = n;
        arr.assign(n + 1, 0);
    }
    BIT(vector<int> const &a) : BIT(a.size())
    {
        for (int i = 1; i <= n; ++i)
        {
            update(i, a[i - 1]);
        }
    }
    void update(int idx, int value)
    {
        while (idx <= n)
        {
            arr[idx] += value;
            idx += idx & -idx;
        }
    }
    int query(ll idx)
    {
        int ans = 0;
        while (idx > 0)
        {
            ans += arr[idx];
            idx -= idx & -idx;
        }
        return ans;
    }
};

This implementation using 1 based indexing, and is the one we discussed previously. There is a 0 based indexing implementation as well. However, I like this one more.

The main functions are `update` and `query`.

• While updating an index, we push its value to its rightful successors as well.

• And while querying, we read the contribution of all the eligible ancestors.

There are some other operations as well, which one can include depending on the needs.

Suppose, you want to read the actual value at an index i. We will need two calls to the query function in order to achieve the same:

int value = query(i) - query(i - 1);

This takes 2 * O(logn) time. However, we can do better.

int queryPointValue(int idx)
{
    int val = arr[idx];
    if (idx > 0)
    {
        int ancestor = idx - (idx & -idx);
        --idx;
        while (idx != ancestor)
        {
            val -= arr[idx];
            idx -= idx & -idx;
        }
    }
    return val;
}

The idea is simple. We are subtracting some values obtained from query(i) in query(i-1).

Both idx and idx - 1 are nodes in a tree. They will reach a common ancestor while traversing up. Once, they arrive at their common ancestor, all the values will cancel out! Hence, we subtract the values till that ancestor only and then break.
This reduces the time taken to read a point value in half!!

• Scaling the tree by a factor c.

  void scale(int c)
      {
          for (int i = 1; i <= n; ++i)
          {
              arr[i] = arr[i] * c;
          }
      }

  This is quite straightforward. We can multiply or divide by a constant `c` to scale the tree by a constant factor.

Handling 2D Data

While handling 2D data, our update and query functions will change accordingly.
At each iteration, we will finish updating our lower dimension first, and then move to the next successor in our higher dimension.

Here is the update function for 2D data.

void update(int x, int y, int val) {
  int y1;
  while (x <= max_x) {
    y1 = y;
    while (y1 <= max_y) {
      arr[x][y1] += val;
      y1 += (y1 & -y1);
    }
    x += (x & -x);
  }
}

Similarly, for querying in 2D we will follow a similar procedure.

void query(int x, int y) {
  int val = 0;
  int y1;
  while (x >= 0) {
    y1 = y;
    while (y1 >= 0) {
        val += arr[x][y1];
        y1 -= (y1 & -y1);
    }
    x -= (x & -x);
  }
}

We can also modify BIT to support range updates. However, for that we need to maintain two separate BITs.
In case you guys are interested, I’ll cover them in a separate post.

Subscribe now

Lets consider a problem to better understand this data structure:

Imagine there are n bank accounts and all of them have $10 to begin with. You add some funds into certain bank accounts. Find the sum of balance of bank accounts numbered a to b.

Clearly, a brute force solution takes O(1) time to add an amount and O(b - a + 1) time to query the total balance. Can we do any better?

This is where BIT comes in. They can answer the second problem in

O(log(b) - log(a -1)) time.

Idea

Any number can be written in base 2, or as a sum of powers of two. Similarly, we can break the balance of bank accounts into sum of balance of a subset of these bank accounts. This way, we can iterate over these subsets to get the total balance instead of iterating over the whole range.

To do this, we isolate the least significant bit from the binary representation of a number (x). Let it be at the position y.

Then x is responsible for storing balance of all the banks in the range [x - 2^y +1, x].

Now, the question is, how do we isolate this position?

Well, its simply x & -x, where & is bitwise AND, and (-x) is negative x.

Proof

Let the number be of the form A1B. `A` will cover the portion before the least significant bit and `B` will the cover the portion after it. The `1` between them is the least significant bit. One needs to notice that B will consist of 0s only.

We need to remember that numbers are stored as 2’s complement in computers.
So, λ in the proof below is actually the suffix after the sign bit while storing the number, and this suffix is what truly contributes to the answer. The sign bits won’t contribute anything to the bitwise AND operation.

Keeping in mind the above result, we have.

which is nothing but the least significant bit!!

We have covered all the theory we need to understand BIT now. In the next part of this article, we will look at the implementation of this simple yet powerful data structure. Till then, happy learning!!

Most of us are familiar with the concept of binary search. We take a sorted array as an input, and maintain an active region where the desired number might lie. That’s more or less the general idea most of us have about this approach.

However, we can take this idea further. Consider, a function which is monotonic in nature. Here is an example:

For simplicity, lets take the domain as the set of integers. We want to find the minimum value of x where f(x) returns 1.

If we scan through the return values of f(x) for the first 10 integers, the answer array looks like this:

false, false, false, false, false, true, true, true, true, true

Clearly, the answer function is monotonic and hence we can apply binary search here!

Thus, in order for binary search to work, the search space must look like something of the following form, using a function as described above.

true, true, true, true, false, false, false, .....

Then, we find the point at which true becomes false, using binary search.

Implementation

There are different ways to go about the implementation. In this blog, we’ll discuss three. Method 1 and Method 2 are what you call a closed interval implementation. Method 3 is an open interval implementation. I personally prefer Method 3 due to the open interval nature.

For all the implementations below assume that the answer lies in the range [0, n].

Method 1

This is what most of us are familiar with.

int left = 0, right = n;
while ( left <= right) {
    int mid = left + (right - left) / 2;
    if (check(mid)) {
        left = mid + 1;
    } else {
        right = mid - 1;
    }
}
int ans = left - 1;

During the entire execution of the algorithm, we maintain the following invariant:

[left, right] is the active region in which our answer lies.

Method 2

int ans = 0;
int maxJump = n;
for (int jump = maxJump; jump >= 1; jump /= 2) {
    while (check(ans + jump)) {
        ans = ans + jump;
    }
}
return ans;

We are making bigger jumps when we are far away from the answer, and slowing down as we come closer to it.

Method 3

This looks almost exactly the same as Method 1, however here we will maintain the following invariant:

[left, right) is the active region in which our answer lies.

Notice how right is not included in the active region.

int left = 0, right = n + 1;
while ( left < right) {
    int mid = left + (right - left) / 2;
    if (check(mid)) {
        left = mid + 1;
    } else {
        right = mid;
    }
}
int ans = left - 1;

We update right to mid, in order to maintain the invariant. Also, the inequality changes from <= to just < , again due to the invariant we have chosen.

One can use any of the above methods as per their liking. What is more important, is that we understand what is the invariant we are maintaining, and how should the parameters be updated to reflect that. This can help prevent those off by one errors we generally encounter due to the incorrect implementation of the algorithm.

Subscribe now