Python Lists vs Tuples: Memory Management Explained

Introduction

Lists and Tuples are two of the most useful data types in Python. They differ in syntax, mutability, speed, usability, functionality and memory management.

In this blog, I’ll explain the difference in their memory management in detail with easy to understand examples.

Memory Allocation in Lists

A list is a mutable data type, meaning you can add, edit, update, and delete items after the list is created. Some of the key points that you need to know about the memory management in Lists are given below:

• Lists is a referential array i.e. each item in a list stores the reference (pointer) of the data in memory rather than the raw data itself.

• These references are stored in contiguous memory location for faster indexing (O(n)).

• The elements of the list are a full Python object themselves, so they are exist independently in the memory.

• Python over-allocates the memory in Lists than actually needed, so that it minimizes the need to reallocate every time an object is added to the list.

• If you need to append an item to the list but it’s capacity is full, then Python allocates a larger block of memory (usually 1.125 or 1.5 times the old size), copies the pointers and then adds the new item.

• Since lists are mutable and dynamic in nature, it is slower in execution than tuples.

• If two variables point to the same list, then changes made in one of the list is reflected in another as well.

a=[1,2,3]
b=a
b.append(4)
print(a,b,sep='\n')

#Output
# [1, 2, 3, 4]
# [1, 2, 3, 4]

In the above example, I appended an item in list b only, but the change is reflected in list a as well cause they are pointing to the same list in memory.

• List are flexible i.e it can contain heterogeneous data types and nested structures and also support dynamic resizing, all of which contributes to high memory overhead.

Memory Overhead: It is the extra space that is used by an object beyond the memory needed to store the actual data. Memory overhead in Python Lists include:

-List Overhead: The memory that is used by the list object itself that include metadata (reference count, type, length) and the actual reference to the objects.

-Element Overhead: Since each element in a list is also a Python object, it also uses some extra space that includes the header (reference count, type) and the actual data itself.

Visual Representation

In the above diagram, we can see that there are only 4 items in the list but when the list is initialized extra slots are allocated. This over-allocation minimizes the need of frequent resizing during list growth and improves efficiency.

As we can see in the diagram above, Lists stores the references of the objects in contiguous memory location while the actual data exist independently in the memory.

Example

import sys

l=[]
print(sys.getsizeof(l))
print(id(l))

l.append(4)
print(sys.getsizeof(l))
print(id(l))

l.extend([1,2])
print(sys.getsizeof(l))
print(id(l))

l.pop()
print(sys.getsizeof(l))
print(id(l))

print(l[0])
print(sys.getsizeof(l[0]))
print(id(l[0]))

Now, let us look at its output talk about its memory management.

Output
56
1617280650752
88
1617280650752
88
1617280650752
88
1617280650752
4
28
140708450128776

• Initially, I created an empty list l , it consumed 56 bytes of memory. This is due to list overhead as even though there is no element, Python allocates the memory for metadata and the array of references (initially empty, but the space is reserved).

• Then, I appended an item ‘4‘ to the list, this changed the memory of the list to 88 bytes. It means that more memory blocks are allocated for the list l.

• After that, I added two items ‘1‘ and ‘2‘ at once but still the memory associated with the list l remains 88 bytes. Even after popping an element from the list, the occupied memory is the same 88 bytes. This is because of the over-allocation of slots for efficiency.

• We can also see that list address remains the same even after I appended or popped an item, this is because of the mutability property of the list.

• At last, I accessed the very first element of the list l. We can see it occupies 28 bytes in memory, this is due to the element overhead. Also, the address in the output is the address of the integer object, not the address of the list or the slots.

Memory Allocation in Tuples

You can think Python Tuples as the immutable version of Lists. They both are ordered, can hold multiple data types and store an array of references to objects. But they differ in various aspects like speed, functionality, usage and many more. Tuple are used when we require fixed collection of items whereas Lists are used when we need to frequently add, delete and update items. Some of the key points that you need to know about the memory management in Tuples are given below:

• Tuples are fixed in sized so it cannot be altered after creation.

• Tuples are immutable, so Python allocates exactly the amount of memory that is required unlike lists.

• Since, there is no over-allocation in Tuples, there is less memory overhead. The reduced overhead makes tuples faster in terms of memory access and iteration.

• When a new item needs to be added in a tuple, a new tuple is created that contains all the old items plus the new item.

Visual Representation

In the above diagram, we can see that there are only 4 items in the list and when the list is initialized only 4 slots are allocated. This exact allocation minimizes memory overhead which in terms results in faster memory access and iteration.

Example

import sys

t=()
print(sys.getsizeof(t))
print(id(t))

t1=(1,2,3)
print(sys.getsizeof(t1))
print(id(t1))

tnew=t1+(4,)
print(sys.getsizeof(tnew))
print(id(tnew))

Let’s look at its output and talk about its memory management.

Output
40
140708387861976
64
1931865218688
72
1931865190816

• Initially, I created an empty tuple t , it consumed 40 bytes of memory. We can see that it consumes less memory than lists i.e. 58 bytes. This is because the tuples needs a fixed amount of slots to store the metadata only, which is not the case in lists as it over-allocates for dynamic resizing.

• Then, I created a tuple t1 containing 3 elements. The memory allocated corresponds to its size. No extra slots are allocated like in lists, as tuples are immutable.

• Finally, I wanted to add an element 4 in the tuple t1, but since tuples are immutable we cannot add directly. So, a new tuple tnew is created that contains all the elements of t1 plus the new item 4. The original tuple remains unchanged.

• We can also see that updating a tuple changes its address as well, this is because of the immutability property of tuple.

Why Tuples Share Memory but Lists Don’t?

Let us see what happens when we define two empty lists and two empty tuples in a program.

import sys

l1=[]
l2=[]

print(sys.getsizeof(l1))
print(id(l1))
print(sys.getsizeof(l2))
print(id(l2))

t1=()
t2=()

print(sys.getsizeof(t1))
print(id(t1))
print(sys.getsizeof(t2))
print(id(t2))

In the above program, I have defined two empty lists l1 and l2 along with two empty tuples t1 and t2.

Output
56
2122469428736
56
2122470962368
40
140708439635416
40
140708439635416

From the above output, we can see that:

• Both the empty list have the same amount of memory allocated but their memory addresses are different. This is because lists are mutable and can be modified independently so, they cannot share the same memory location.

• Both the empty tuples have the same amount of memory allocated and share the same memory address. This is because tuples are immutable i.e. it cannot be modified once created. When we run the program, Python makes an empty tuple object in the memory. Anytime, we make an empty tuple, it does create a new one, instead it will just reuse the same memory. This is only possible due to immutability of tuples.

Conclusion

In this blog, we explored how the memory allocation differs between Python Lists and Tuples and when each data type is useful in the real world.

I hope you enjoyed reading this blog as much as I enjoyed writing it.

See you in the next blog !!!