From Call Stack to Heap: How Python Manages Execution and Data in Memory

When Python runs your code, it manages memory using two key areas:

🧠 Stack v/s Heap?

🔧 The Stack

• The call stack is a contiguous block of memory reserved by the operating system at program start.

• It grows and shrinks as functions are called and return.

• It operates on a LIFO (Last In, First Out) model.

• Stack memory is thread-local and automatically managed — allocation is a simple pointer increment; deallocation is a pointer decrement.

• Function calls store their stack frames here — which include local variables and return addresses.

  | Question | Answer | | --- | --- | | Where is the stack memory? | In the process’s virtual memory, near the top (high addresses) | | Who allocates it? | The operating system, at process or thread creation | | How is it used? | Managed using CPU stack pointer, grows/shrinks with function calls | | How big is it? | OS-defined, typically 1–8MB per thread (configurable) |

---

🔩 The Heap

The heap is an entirely separate region of memory provided by the OS. It is:

• A region of virtual memory reserved by the OS for long-term dynamic allocation.

• Allocations from the heap are done via manual control or an allocator (Python handles this behind the scenes). i.e Not automatically structured like the stack.

• Heap memory is usually backed by pages, and the OS maps these into your process address space via system calls like mmap().

• Allocated and deallocated using system calls (e.g., malloc/free in C, or brk/mmap at the OS level).

• Managed by a dynamic memory allocator — in Python’s case, pymalloc (for small objects) or the OS (for large ones).

• Not tied to the function call stack — meaning:

  • Objects on the heap persist beyond the function call that created them.

  • Any scope can reference them (global, local, etc.)

• | Question | Answer | | --- | --- | | Where is the heap memory? | In the process’s virtual memory, typically located below the stack, growing upward | | Who allocates it? | The operating system, via system calls (brk, mmap) when requested by Python’s memory manager | | How is it used? | Dynamically allocates and frees memory for objects like lists, dicts, classes | | How big is it? | Limited by available virtual memory or system limits (can be GBs in size) | | Who manages cleanup? | Python’s garbage collector and reference counting automatically free unused memory | | Is access fast? | Slower than stack — requires pointer dereferencing and memory bookkeeping |

---

🧵 Memory Map of a Python Process

When a program runs (including a Python script), the operating system creates a process with a virtual memory layout that looks roughly like this:

HIGH MEMORY ADDRESSES
+-----------------------------+
|         Stack              | ← grows down
+-----------------------------+
|      Heap (malloc/pymalloc)| ← grows up
+-----------------------------+
|  Global/Static Variables   |
+-----------------------------+
|       Code Segment         |
+-----------------------------+
LOW MEMORY ADDRESSES

• The stack starts near the top of the virtual address space and grows downward.

• The heap starts after the static data and grows upward.

• This design prevents them from immediately colliding and helps the OS detect overflows.

---

4. 🧠 Stack vs Heap at the Hardware Level

Stack:

• Managed using a dedicated stack pointer register (RSP/ESP) that always points to the top of the stack.

• CPU instructions like call, ret, push, and pop directly manipulate the stack and this register.

• Though managed by the OS and CPU, the stack is backed by real RAM via the process’s virtual address space.

Heap:

• The heap has no dedicated CPU register.

• Memory is allocated at runtime via system calls like brk() or mmap(), often through functions like malloc() or Python’s memory manager.

• Allocation returns a pointer to the memory; it's up to the program to manage it.

• Access is indirect — you follow pointers to interact with heap data.

• There's no automatic structure like push/pop; memory must be explicitly managed (or garbage collected in Python).

• Like the stack, heap memory is also backed by physical RAM, but is dynamically mapped in the process’s address space.

---

📞 What Happens When a Function Is Called?

🔹 1. The Call Stack

The call stack keeps track of function calls. Each time a function is invoked, Python creates a stack frame for that function. This frame contains:

• The function’s parameters

• All local variables

🧪 Example:

def add(x, y):
    result = x + y
    return result

def main():
    a = add(2, 3)
    print(a)

main()

🔍 Stack Behavior

Let’s trace the execution of the program step-by-step and visualize how the call stack and local variables evolve.

---

▶️ Step 1: main() is called (Line 8)

Call Stack:
┌────────────┐
│  main()    │
└────────────┘

Local Variables:

main: a = ?

---

▶️ Step 2: add(2, 3) is called (Line 5)

Call Stack:
┌────────────┐
│ add(2, 3)  │
├────────────┤
│  main()    │
└────────────┘

Local Variables:

add: x = 2, y = 3, result = ?

---

▶️ Step 3: Inside add(), result = x + y (Line 2)

Call Stack:
┌────────────┐
│ add(2, 3)  │
├────────────┤
│  main()    │
└────────────┘

Local Variables:

add: x = 2, y = 3, result = 5

---

▶️ Step 4: return result (Line 3) → back to main()

Call Stack:
┌────────────┐
│  main()    │
└────────────┘

Local Variables:

main: a = 5

---

▶️ Step 5: print(a) is executed (Line 6)

🖨️ Output:

5

Call Stack remains:

┌────────────┐
│  main()    │
└────────────┘

---

✅ Step 6: End of main()

Call Stack: (empty)

The program completes and all memory is cleaned up.

Local variables like x, y, result, and a exist only inside their respective function frames. Once the function finishes, they're gone.

---

🔶 2. The Heap

While the call stack handles how functions are executed, Python uses a separate memory area — the heap — to store the data those functions operate on. Whenever you create complex objects like lists, dictionaries, or class instances, Python places them in the heap so they can persist beyond a single function call, be shared across scopes, and grow dynamically.

📦 How Python Stores Data on the Heap

We’ve seen how Python uses the call stack to manage function execution. But what happens when we create data — like a list — that persists or is shared between scopes?

Let’s walk through it with an example function:

---

🔧 Example: Heap Allocation in Action

def make_shopping_list():
    shopping_list = ['apples', 'bananas']
    return shopping_list

cart = make_shopping_list()

---

🧠 What Happens in Memory?

🔹 Step 1: make_shopping_list() is called

• Python creates a stack frame for the function.

• Inside the function, a list object ['apples', 'bananas'] is created and stored in the heap.

• The variable shopping_list (just a reference) lives in the stack frame.

Stack (during function):
┌────────────────────────────┐
│ make_shopping_list()       │
│ ─ shopping_list ─────┐     │
└──────────────────────│────┘
                       ▼
Heap:
[ 'apples', 'bananas' ]  ← list object
   ↑        ↑
'apples'  'bananas'       ← individual string objects (also in heap)

🔹 Step 2: Function returns the list

• The reference to the list object is returned.

• The local variable shopping_list is discarded when the stack frame is popped.

• Outside the function, cart now points to the same list.

Stack (after return):
┌───────────────┐
│   cart ─────┐ │
└─────────────│─┘
              ▼
Heap:
[ 'apples', 'bananas' ]  ← still lives in heap
   ↑        ↑
'apples'  'bananas'

---

🔍 Why Is shopping_list in the Stack?

• Variable names like shopping_list and cart live in their respective namespaces:

  • Inside functions → in the call stack

  • Globally → in the global namespace

• They don't hold data themselves — they hold references (pointers) to the actual data, which lives in the heap.

---

✅ Quick Recap

---

🧠 Why Use the Heap?

• The list object needs to outlive the function that created it.

• By storing it in the heap, Python ensures the object remains alive as long as there’s a reference to it.

• When no variable points to the object anymore, Python’s garbage collector reclaims the memory.

---

In Python, understanding how the stack handles execution and how the heap manages data is key to writing efficient, bug-free code that truly respects memory.