Understanding Golang’s Garbage Collector: A Deep Dive Into Go’s Memory

Introduction

I wanted to understand how a piece of memory is consumed, allocated and managed when a program runs. This curiosity led me down a fascinating rabbit hole of memory management, garbage collection, and the inner workings of Go's runtime. What started as a simple question about memory allocation turned into a comprehensive exploration of how Go manages memory so efficiently that we rarely have to think about it.

I started my journey from basics, how a running program i.e. process uses memory. So a 30,000ft view of memory in processes is as follows: Memory can be allocated at 2 instances, compile-time and run-time. Memory can be allocated at 2 places, stack and heap.

Stack vs Heap Memory Allocation: When, Why and What

The Stack

The stack is fast, organized, and follows a strict order. Go automatically manages the stack memory, and we don't have to worry about memory deallocation for stack-based variables. Basic data types, such as integers, floats, and pointers, are often stored on the stack. The stack operates on a Last-In-First-Out (LIFO) principle, much like a stack of plates.

Characteristics of Stack Memory:

Speed: Extremely fast allocation and deallocation
Automatic Management: No need for garbage collection
Scope-based: Variables are automatically cleaned up when they go out of scope

When variables go on the stack:

Local variables with known size at compile time
Function parameters and return values (when they don't escape)
Basic data types like integers, floats, booleans
Small structs that don't escape their scope

The Heap

The heap is a much larger, more flexible, but requires us to remember what we put there and clean it up periodically. Heap allocation requires manual housekeeping of what memory is to be reserved and what is to be cleaned. The heap memory is allocated at the run time.

Characteristics of Heap Memory:

Flexibility: Can allocate memory of any size
Persistence: Memory persists beyond function scope
Global Access: Can be accessed from anywhere in the program
Garbage Collection: Requires periodic cleanup by the garbage collector

When variables go on the heap:

When a variable is declared inside a function, but its reference escapes the function
Large objects that exceed stack size limits
Objects whose size is not known at compile time
Objects that need to persist beyond their creating function's scope
Slices, maps, and channels are typically heap-allocated

Go Specific: Pass by Value vs Pass by Reference

Let's talk about how things work in Golang. This is where things get interesting.

Golang is pass by value at its core, But some types contain references (pointers) as their underlying implementation.

Go makes a clear distinction between value types and reference types, but it's not always obvious which is which.

Value Types (Copied when passed)

These types are completely copied when passed between functions:

Basic types: integers, floats, booleans, strings
Arrays (fixed size): [5]int{1, 2, 3, 4, 5}
Structs: Custom data structures

When we pass these to a function, the entire value is copied, so modifications inside the function don't affect the original.

Reference Types (Pointer/Header copied, underlying data shared)

These types have a header structure that gets copied, but the underlying data is shared:

Slices: The slice header (pointer, length, capacity) is copied, but the underlying array is shared
Maps: The map header is copied, but the underlying hash table is shared
Channels: The channel header is copied, but the underlying channel is shared
Pointers: The pointer value is copied, but it points to the same memory
Interfaces: The interface header is copied, but the underlying data is shared

String: A Curious Case

Strings in Go are particularly interesting. They're technically a reference type (consisting of a pointer to data and a length), but they behave like value types because they're immutable.

When we copy a string, we copy the header, but the underlying data can be shared safely because strings can never be modified.

Escape Analysis: The Decision Maker

We now come to learn about the part that helps decide if a variable goes into the stack or the heap. This is where escape analysis comes into play.

Whether a Go value escapes or not is a function of the context in which it is used and the Go compiler's escape analysis algorithm. It would be fragile and difficult to try to enumerate precisely when values escape: the algorithm itself is fairly sophisticated and changes between Go releases.

Escape analysis is performed by the Go compiler at compile time. It analyzes the code to determine whether variables can "escape" their local scope. If a variable might be accessed after its containing function returns, it "escapes" to the heap.

Common Escape Scenarios

1. Returning Pointers to Local Variables

func escapesHeap() *int {
    x := 42  // x escapes to heap because we return its address
    return &x
}

2. Storing in Global Variables, Sending Through Channels, Interface Conversions Variables escape when their addresses are stored globally, sent through channels, or converted to interfaces.

3. Large Objects Objects that are too large for the stack automatically go to the heap.

Escape Analysis Flow

Analyzing Escape Behavior

We can see escape analysis in action using the Go compiler:

go build -gcflags="-m" <filenam>.go

This will show you which variables escape and why:

./main.go:10:2: moved to heap: x
./main.go:11:9: &x escapes to heap

Go Garbage Collection: High-Level Overview

Go manages the heap memory by garbage collection. In simple terms, it frees the memory used by orphan objects, i.e, objects that are no longer referenced from the Stack directly or indirectly(via a reference in another object) to make space for new object creation. Few terminologies

Root Nodes include Stack Variable, Global Variable, and CPU Registers
An object is any value dynamically allocated
Pointers are references to Objects
The object graph is constructed using objects and pointers. This Object graph’s root nodes are the starting point for GC.

Go Garbage cleans up unused memory while our program continues running.

The Algorithm

Go's garbage collector (GC) uses the Tricolor Mark and Sweep algorithm, which is designed for concurrent and efficient garbage collection.

It starts with a root node, marks it as “live”, and keeps going along the object graph. In the next sweep, all the objects not marked are considered for sweeping, i.e, put back to heap memory for allocation of new objects.
The GC process starts from the Object Graph walk, which starts from the Root Node, the GC follows the object graph and identifies the object definitely in use. This process of walking the Object Graph is scanning.
GC marks all the live objects it encounters as live. This is one part of tracing
The second part of tracing involves GC walking over all the memory in the heap and making the memory that is not marked available for allocation. This process is called sweeping.

The algorithm categorizes objects into three colors: white, gray, and black, to determine their reachability and eligibility for collection. The tricolor algorithm is a systematic way of determining which objects in our program are still needed.

White Objects: Objects that have not been visited and are candidates for collection.

Initially, all objects start as white
These are potentially garbage (unused objects)
At the end of garbage collection, all white objects are deleted

Gray Objects: Objects that have been identified as reachable but whose children have not yet been fully scanned.

These objects are known to be reachable from the program
But we haven't yet checked what they point to
This is the "work queue" for the garbage collector

Black Objects: Objects that have been completely scanned.

These objects are definitely reachable and have been fully processed
All objects they reference have also been identified
These objects will not be garbage collected in this cycle

The Tricolor Invariant

The "tricolor" invariant ensures no pointers go directly from the black set to the white set, allowing the white set to be cleared.

This invariant is crucial for correctness: Black objects must never point directly to white objects. If this rule is violated, the garbage collector might accidentally delete objects that are still in use.

Concurrent Collection: Running Alongside Your Program

The GC process can run concurrently with other user goroutines, but a certain amount of Stop The World time is still required.

The beauty of Go's garbage collector is that it runs concurrently with our program. However, there are still brief "stop-the-world" (STW) pauses where all goroutines are paused. These pauses are minimized and typically last less than a millisecond.

Stop-the-World Phases:

GC Start: Brief pause to set up garbage collection
Mark Termination: Brief pause to complete the marking phase
Sweep: Runs concurrently, no pause needed

Write Barriers: Maintaining the Tricolor Invariant

When our program runs concurrently with the garbage collector, there's a problem: our program might create new pointers while the GC is running. This could violate the tricolor invariant.

Go uses "write barriers" to solve this. Whenever our program writes a pointer, the write barrier ensures that the tricolor invariant is maintained by potentially coloring objects gray.

Cost of GC

Things to consider when thinking about GC cost
- Physical memory and CPU time
- The time cost of GC comes from the amount of memory it needs to go through in the current cycle. For now we can assume the metadata memory cost is negligible
  - GC memory cost of cycle N = live heap from (N-1) cycle + new heap memory allocated after the (N-1) cycle
- Understanding the CPU cost associated with a concurrent GC run
  - GC CPU time for cycle N = Fixed CPU time cost per cycle + Marginal CPU Cost (average CPU time cost per byte \ live heap memory found in cycle N )*
  - Fixed CPU time cost per cycle
    - This is the baseline CPU time the GC spends no matter how much memory is being collected.
    - This cost exists even if you have 0 bytes of live heap
    - This cost does not scale with memory it's constant per GC run.
  - Marginal CPU cost: avg time per byte × live heap
    - This is the actual scanning work.
    - The more live heap (i.e. still-in-use objects), the more pointers must be traced, objects scanned, and dependencies resolved.
    - This cost does scale with the size of the live memory.
    - Sweeping is so much faster than marking and scanning that the cost is negligible in comparison.

If the GC is working harder, it may compete with your goroutines for CPU time, hurting performance.

That's why understanding and minimizing GC CPU time can be key for low-latency or high-throughput systems.

Steady State of an application as defined by GC
- The rate at which the application allocates new memory (in bytes per second) is constant.
  - This means that, from the GC's perspective, the application's workload looks approximately the same over time. For example, for a web service, this would be a constant request rate with, on average, the same kinds of requests being made, with the average lifetime of each request staying roughly constant.
- The marginal costs of the GC are constant.
  - This means that statistics of the object graph, such as the distribution of object sizes, the number of pointers, and the average depth of data structures, remain the same from cycle to cycle.
Time vs Space Tradeoff

Garbage Collection Tuning

For applications with specific performance requirements, Go provides several tuning options:

GOGC Environment Variable

It sets the garbage collection percentage, it controls how frequently the garbage collection should run, and after how much memory consumption it should run
GOGC=100 ⇒ Default value
- Means that the garbage collection should run after the heap memory has increased by 100% from the previous run, i.e., when the memory is doubled with respect to the previous runs
- GOGC=off (or GOGC=-1): Disables garbage collection entirely (not recommended for production).
- Target heap memory = Live heap + (Live heap + GC roots) \ GOGC / 100*

GC Trigger Points:

GOGC=50:  [Heap] ──50%──→ [GC] ──50%──→ [GC] ──50%──→ [GC]
          More frequent GC, lower memory usage

GOGC=100: [Heap] ────100%────→ [GC] ────100%────→ [GC]
          Default behavior

GOGC=200: [Heap] ──────200%──────→ [GC] ──────200%──────→ [GC]
          Less frequent GC, higher memory usage

GOMEMLIMIT

From 1.19 version onwards
Used to set the running memory limit
This memory limit sets a maximum on the total amount of memory that the Go runtime can use.
When SetMemoryLimit is set:
- Go monitors total memory usage (heap, stacks, garbage, runtime overhead, etc.)
- If memory usage approaches the limit:
  - The GC will run more aggressively, even before GOGC thresholds are met.
  - It will lower the effective GOGC internally to keep memory under the limit.
- If usage is below the limit:
  - The GC behaves normally and may even relax if memory pressure is low.

Optimizing for Garbage Collection

Understanding garbage collection allows us to write more efficient Go programs. Here are key optimization strategies:

Minimize Heap Allocations

Understanding how the heap and stack work in Go allows us to write more optimized code. The stack is ideal for quick, short-lived allocations with automatic memory management, while the heap is used for more flexible, long-lived memory needs.

Strategies:

Keep objects small and local when possible
Avoid returning pointers to local variables unless necessary
Use value receivers instead of pointer receivers for small structs
Prefer arrays over slices for fixed-size data

// uses heap
func inefficientSum(data []int) *int {
    sum := 0
    for _, v := range data {
        sum += v
    }
    return &sum  // sum escapes to heap
}

// uses stack
func efficientSum(data []int) int {
    sum := 0
    for _, v := range data {
        sum += v
    }
    return sum  // sum stays on stack
}

Reuse Objects When Possible

Use object pooling for frequently allocated objects to reduce garbage collection pressure.

Be Mindful of Interface Conversions

Interface conversions often cause heap allocations because values need to be "boxed" to fit the interface type.

Understand Slice Growth

Pre-allocate slices with known capacity to avoid multiple reallocations as the slice grows.

Conclusion

The garbage collector works tirelessly in the background, using sophisticated algorithms to keep our program's memory clean and efficient. The tricolor marking algorithm ensures that cleanup happens concurrently with our program's execution, while escape analysis makes intelligent decisions about where to allocate memory.

Go Garbage Collection: A Journey Through Memory Management

Table of contents