Docker Layers, Volume and Networks

What is Docker Layers?

Docker layers are the building blocks of Docker images. Each layer represents a set of filesystem changes or instructions from your Dockerfile. When you build a Docker image, each command in the Dockerfile creates a new layer. Layers allows Docker to be efficient, fast, and portable these series of layers represent a set of differences from the previous layer.

Each layer in an image contains a set of filesystem changes - additions, deletions, or modifications.

Layers

1. Base Layer: The first layer of a Docker image, usually containing a minimal operating system like Ubuntu or Alpine. It acts as the foundation on which all other layers are built in a Dockerfile.

2. Instruction Layers: Every command in a Dockerfile ( RUN, COPY, or ADD) creates a new layer on top of the previous layer. These layers add changes like installing software, copying files, or modifying the file system.

3. Reusable & Shareable: Docker layers are cached and can be reused across different images. If multiple images share the same base image or use similar instructions, Docker doesn't rebuild those parts it reuses the existing layers. This saves storage space and makes image builds and downloads faster and more efficient.

4. Immutable: Once a layer is created, it cannot be changed. If a change is made, Docker creates a new layer that captures the difference. This immutability is key to Docker's reliability and performance, as unchanged layers can be shared across images and containers.

FROM node:22-alpine           # Layer1

WORKDIR /app                  # Layer2

RUN npm install               # Layer3

COPY . .                      # Layer4

CMD ["npm", "start"]

Significant of Layers

In case we rebuild out docker image, layers can be re-used based on where the change was made. All the subsequent builds becomes much faster as unchanged layers are reused. Particularly become valuable in CI/CD pipelines where builds occur frequently.

You can think of image layers as a stack, with each layer adding more content on top of the layers that came before it

Whenever a layer changes, that layer will need to be re-built. For example, suppose you make a change to your program in the index.ts file. After this change, the COPY command will have to run again in order for those changes to appear in the image.

If a layer changes, all other layers that come after it are also affected. When the layer with the COPY command gets invalidated, all layers that follow will need to run again, too.

Optimising the above Dockerfile

Suppose there's a large project with around 20 developers working on it. During the initial setup, most of the required dependencies are installed—and while more can be added later, it doesn't happen very often. However, the codebase itself changes frequently, roughly every 20 minutes.

In this case, it would be smart to cache the npm install step, since the dependencies don’t change often. This way, Docker won’t reinstall packages every time the image is built, saving time and improving build performance.

FROM node:22-alpine                     

WORKDIR /app

# making a copy of dependencies to avoind re-installation
COPY ./package.json ./package.json

COPY ./package-lock.json ./package-lock.json

RUN npm install

COPY . .

CMD ["npm", "start"]

What is Volume in Docker?

Volume help to persist data across docker restarts because docker container are transitory (they don’t retain data across restarts).

Docker volumes are a persistent storage solution used to retain data generated and consumed by containers. They allow data to persist independently of the container lifecycle, ensuring that information remains intact even when containers are stopped, removed, or rebuilt.

Mongo container without volume

1. Start a mongo container .

    docker run -p 27017:27017 -d mongo
   

2. Open it in MongoDB Compass and add some data to it

3. Stop the container

    docker stop <container_id>
   

4. Restart the container and try to look into database in Compass and check if the data has persisted? no.

5. Create a volume

    docker volume create DB_VOLUME
   

6. Mount the folder in DB_VOLUME, which stores the data in mongo conainer.

    docker run -d -p 27019:27017 -v volume_database:/data/db mongo
   

7. Open it in MongoDB Compass and add some data into it , after thet stop the container and again restart the container now data will bepersisted because of volume.

What is Networks in Docker?

Docker networking enables containers to communicate with each other, with the host system, and with external networks. Docker networking allows containers to communicate with other container, host machine, External networks (like the internet)

Type of networks

Bridge: The Bridge network is the default network driver in Docker, automatically created as when Docker is installed. When you run a container without specifying a network, it's attached to a bridge network. It provides a private internal network on the host machine, and containers on the same bridge network can communicate with each other.

Host: If you use the host network mode for a container, that container's network stack isn't isolated from the Docker host (the container shares the host's networking namespace), and the container doesn't get its own IP-address allocated or It Removes network isolation between the container and the Docker host, and uses the host's networking directly. This is useful for services that need to handle lots of traffic or need to expose many ports.

Important Commands

# List all networks
docker network ls

# Inspect a network
docker network inspect <network_name>

# Create a network
docker network create <network_name>

# Connect a container to a network
docker network connect <network_name> <container_name>

# Disconnect a container from a network
docker network disconnect <network_name> <container_name>

# Remove a network
docker network rm <network_name>

Example

Docker containers can’t talk to each other by default. I have two container , i want them to communicate to each other.

Exa- I want my Backend Server container to communicate with Postgres container.

Without Network

localhost in a docker container means it's own network and not the network of the host machine

How to make sure both of the above container can talk?

1. Clone https://github.com/vinodpr1/docker-compose

2. Build Backend Image

    docker build -t backend_server .
   

3. Create a network

    docker network create common_network
   

4. Start the Backend container and mount on the created network and also give a name (optional)

    docker run -d -p 3000:3000 -name BE_Server --network common_network backend_server
   

5. Start postgres container on the same network with a name tag

    docker run -d -v postgreDB:/var/lib/postgresql/data --name postgres --network common_network -p 5432:5432 postgres
   

   Name of the Postgres container (postgres) resolve to the ip address of the Postgress container on common_network.