Docker Garbage Collection: BuildKit Guide for Disk Optimization

🚀 Introduction

Docker is a cornerstone of modern DevOps workflows, providing consistent, containerized environments for building, testing, and deploying applications. However, frequent builds and updates can lead to disk space bloat, affecting performance and increasing infrastructure costs.

In this blog, we’ll explore how Docker’s garbage collection (GC) works, how to configure it using BuildKit, and best practices to keep your build system lean and efficient.

🔥 Why Docker Garbage Collection Matters

As development continues, unused Docker images, build cache, and containers start accumulating. Without proper cleanup, this can:

Fill up disk space quickly.
Cause build failures and slowdowns.
Waste cloud resources and increase billing.

🚧 The Problem:

Stale images and containers linger on disk.
Caches keep growing silently.
Developers forget to clean up manually.

✅ The Solution:

Enable automated garbage collection to reclaim space and optimize system health.

🧠 Understanding Docker Images and Cache Bloat

Docker images include everything needed to run an app: code, runtime, libraries, and system tools. But with every change:

New image layers are added.
Old ones often remain unused.
BuildKit cache also expands with temporary files.

This leads to excessive disk usage unless actively managed.

🔍 Overview of BuildKit Garbage Collection

Docker's BuildKit introduces advanced GC features that automate cleanup, based on rules you define.

You can trigger GC manually with:

docker builder prune

Or, let BuildKit do it automatically with configuration.

The Need for Garbage Collection

Unused images, containers, and build caches can quickly consume disk space, leading to inefficiencies and potential system issues. Garbage collection ensures:

Optimized Disk Space: Frees up space by removing unused resources.
Improved Performance: Reduces the overhead of managing excess data.
Sustainable Build Processes: Helps maintain a clean and efficient build environment.

Build a Garbage Collection Overview

Docker offers tools like docker builder prune or docker buildx prune for manual cleanup. However, BuildKit introduces periodic garbage collection for an automated and efficient cleanup process. This feature runs in the BuildKit daemon and follows a set of pruning policies to clear build caches based on:

Cache size limits
Cache age

These parameters can be configured for a tailored garbage collection policy.

⚙️ Configuring Automatic Garbage Collection

To enable and configure garbage collection, modify your Docker daemon config (/etc/docker/daemon.json):

{
  "builder": {
    "gc": {
      "enabled": true,
      "defaultKeepStorage": "10GB",
      "policy": [
        { "keepStorage": "10GB", "filter": ["unused-for=2200h"] },
        { "keepStorage": "50GB", "filter": ["unused-for=3300h"] },
        { "keepStorage": "100GB", "all": true }
      ]
    }
  }
}

🔎 Configuration Explained:

enabled:
- Activates GC.
- Enables or disables garbage collection.
defaultKeepStorage:
- Total build cache threshold.
- Sets the default storage threshold for the build cache. In this example, caches exceeding 10GB will be pruned based on the policies below.
policy[]: Cleanup strategy:
- keepStorage: Max cache to retain.
- unused-for: Time since last usage.
- filter: Defines conditions for pruning based on resource usage time (e.g., unused-for=2200h means resources unused for 2200 hours will be pruned).
- all:
  - Applies cleanup universally.
  - When set to true, applies the policy to all resources, regardless of usage.

Each policy is applied in order, starting with the smallest keepStorage value.

🛠️ Restart Docker to apply changes:

systemctl restart docker

🧹 Manual Cleanup (When Needed)

You can manually clean up unused Docker resources with:

docker image prune -a          # Remove unused images
docker container prune         # Remove stopped containers
docker builder prune           # Remove build cache

Use with caution — manual pruning can delete resources needed for future builds.

🤖 Automating Cleanup in CI/CD

If you’re using Azure DevOps, integrate Docker GC into your pipelines with a simple task:

task: Docker@2
  displayName: 'Docker GC'
  inputs:
    command: 'prune'
    arguments: '--all --force'

With the configuration described earlier, BuildKit automatically runs garbage collection based on the specified policies. This reduces the need for frequent manual intervention.

Leveraging Azure Pipelines for Automated Cleanup

Azure Pipelines offers tasks to automate Docker cleanup as part of your CI/CD workflows. For example:

- task: Docker
  displayName: 'Prune Docker images and containers'
  inputs:
    command: 'prune'
    arguments: '--all --force'

💡 Pro Tip:

Want to ensure your Docker environment stays clean from the start?

👉 Check out my previous blog
“How to Set Up a Build Agent on RHEL” — it covers disk management, user setup, and installing Docker clearly.

📌 Conclusion

Docker’s garbage collection isn’t just a luxury—it’s a necessity.

By configuring BuildKit GC policies, using manual or automated clean****ups, and applying best practices, you can:

🧽 Free up disk space
⚡ Speed up your pipelines
🛠️ Build a sustainable DevOps workflow

Don’t wait for your next build to fail due to low disk space. Start configuring GC today!

Best Practices for Garbage Collection

Regular Cleanup: Schedule garbage collection to run periodically to prevent disk space issues.
Image Retention Policies: Retain only the images and containers essential for your workflows.
Monitoring Disk Usage: Use tools like docker system df to monitor disk usage and identify potential issues.
Document Policies: Clearly define and document your retention and cleanup policies for better team alignment.

Pro Tip: Enhance your understanding by exploring Docker’s official documentation and experimenting with the docker buildx prune commands to customize your cleanup strategy!

Docker Garbage Collection: Managing Build Cache for Sustainable Performance