Docker Volumes and Persistent Storage: Everything You Need to Know.

Hey folks! Welcome back to my Docker series! If you’ve been following along, in our previous posts we covered Docker Basics, where we discussed what containers are, how they differ from virtual machines, and the magic behind Docker like Docker Engine, Docker Hub, and Docker Registry. Then, we dove into Docker Containers: Running, Managing, and Debugging to understand container lifecycle management and troubleshooting techniques.

In this post, we’re going to explore a crucial aspect of working with Docker—Docker Volumes and Persistent Storage. We’ll go over what Docker volumes are, how they differ from bind mounts, and why persistent storage is essential for real-world applications. By the end of this guide, you’ll know how to create, manage, and use Docker volumes to retain data across container restarts. So let’s get started!

What Are Docker Volumes?

In Docker, containers are designed to be stateless, which means they don’t store data permanently. When a container stops or is removed, all data inside it is lost by default. Docker volumes offer a way to store data outside the container, keeping it safe and persistent. Volumes are stored on your host system but aren’t directly part of the container filesystem, making them ideal for saving critical data like logs, configurations, or database entries.

Why Use Docker Volumes?

• Data Persistence: Volumes ensure that container data is preserved across container restarts or removals.

• Data Sharing: Volumes can be shared among multiple containers, making it easier to share data without duplication.

• Performance: Volumes are optimized for Docker and often perform better than bind mounts or copying files inside containers.

Types of Docker Volumes

Docker primarily offers two types of volumes: Named Volumes and Bind Mounts. Let’s go through each in detail.

1. Named Volumes

A named volume is created and managed by Docker, making it a great choice for long-term data storage. Docker takes care of the exact storage location on the host machine, and named volumes are ideal for production.

Example:

docker volume create my_volume
docker run -d -v my_volume:/data my_image

This creates a named volume my_volume, which will persist any data stored in /data inside the container.

2. Bind Mounts

A bind mount directly links a directory on the host machine to a directory in the container. Bind mounts are beneficial in development, as they allow you to change files on the host, which are immediately reflected in the container. However, they can be less secure and are generally not ideal for production.

Example:

docker run -d -v $(pwd)/data:/app/data my_image

This example mounts the data folder in the current host directory to /app/data inside the container.

Using Docker Volumes: Commands and Examples

Docker provides several commands to create, manage, and remove volumes. Here are some commonly used ones:

• Creating a Volume: docker volume create <volume_name>

• Listing Volumes: docker volume ls

• Inspecting a Volume: docker volume inspect <volume_name>

• Removing a Volume: docker volume rm <volume_name>

Example: Running a Container with a Volume

Let’s see an example command where we create a volume and run a container using it:

docker volume create my_data_volume
docker run -d --name my_container -v my_data_volume:/app/data my_image

In this command:

• my_data_volume is the Docker volume we’ve created.

• It’s mounted to /app/data inside the container, ensuring that any data in this directory persists even if my_container is removed.

---

Example Application: Persistent Storage for a Node.js Application

To make things more practical, let’s try a hands-on example using a Node.js application that writes log entries to a file. This example will help us see how Docker volumes can persist data like logs, configurations, or database entries, even across container restarts.

Step 1: Set Up the Node.js Application

1. First, create a project directory and add a file named app.js with the following code:

    const fs = require('fs');
    const path = '/logs/app.log';
   
    setInterval(() => {
      const message = `Log entry at ${new Date().toISOString()}\n`;
      fs.appendFile(path, message, (err) => {
        if (err) {
          console.error('Failed to write to log file:', err);
        } else {
          console.log('Log entry added:', message);
        }
      });
    }, 5000);
   

2. This script writes a new log entry every 5 seconds to /logs/app.log.

Step 2: Create a Dockerfile

Create a Dockerfile to containerize this application:

FROM node:16
WORKDIR /usr/src/app
COPY app.js /usr/src/app
RUN mkdir -p /logs
CMD ["node", "app.js"]

This Dockerfile sets up a basic Node.js environment and ensures the /logs directory exists for storing log data.

Step 3: Build the Docker Image

Build the image from the node-app-with-volume directory:

docker build -t node-logger-app .

Step 4: Run the Container with a Volume

To make sure our log data persists, we’ll create a Docker volume and attach it to the /logs directory in the container.

1. Create the Volume

    docker volume create node_app_logs
   

2. Run the Container Using the Volume

    docker run -d --name my_node_logger -v node_app_logs:/logs node-logger-app
   

Here:

• -v node_app_logs:/logs mounts the volume to /logs in the container, ensuring that log data is stored in node_app_logs on the host.

Step 5: Verify the Data Persistence

After a few seconds, check if the logs are being stored in the volume:

1. List Files in the Volume

    docker run --rm -v node_app_logs:/logs alpine ls /logs
   

   You should see app.log in the output.

2. View the Logs

    docker run --rm -v node_app_logs:/logs alpine cat /logs/app.log
   

   This command shows the log entries saved by the Node.js app.

---

Managing Data Persistence in Docker Containers

With Docker volumes, you can manage data in containerized applications more effectively. Volumes are especially beneficial for:

• Databases: Persisting data in MySQL, PostgreSQL, or other databases.

• File Storage: Saving uploaded files or logs.

• Configuration Management: Storing configurations shared across multiple containers.

Docker volumes make it easy to:

• Back up data by copying volumes to another location.

• Share data across multiple containers without duplicating files.

• Enhance container performance by using a storage solution optimized for Docker.

---

Wrapping Up

Using volumes is an essential skill for managing data in Docker. They help you retain data across container restarts, enable data sharing, and simplify data management. In this blog, we’ve:

• Explored what Docker volumes are and why they matter.

• Looked at the types of volumes: named volumes and bind mounts.

• Created and managed Docker volumes using practical commands.

• Built an example Node.js application that uses Docker volumes to store log data persistently.

Docker volumes are a powerful feature that makes it easy to handle persistent storage, enhancing the capabilities of your Docker-based environments and applications. Experimenting with volumes can help you build more robust, data-aware container setups.

Thank you for taking the time to read this deep dive into Docker volumes and persistent storage! I hope this guide has made these concepts easier to understand and implement in your Docker projects.

If you found this helpful, be sure to follow me on Twitter and other social platforms where I share more tech insights, tips, and tutorials. Your support means a lot and helps me bring more valuable content to you. Happy Dockering! 🐋