Longhorn: a game changer in the distributed block storage

During te last Kubecon in Paris, David Ko of SUSE presented us Longhorn, a distributed block storage solution, highlighting its latest features and teasing the upcoming version 2.0 of the product.

What is Longhorn ?

Longhorn implements distributed block storage using containers and microservice (source : gitlab)

Longhorn, stands out as a "cloud agnostic" storage solution, free from reliance on any specific cloud provider, marking a significant advantage in today's diverse cloud landscape. This is part of the CNCF since 2019.

Longhorn's foundation stands on four key pillars:

1. Reliability: Longhorn uses a robust system for data recovery in the event of crashes or data loss. It facilitates easy snapshots and backups, ensuring data integrity and minimizing downtime.

2. Usability: With its straightforward installation process and intuitive user experience, Longhorn prioritizes usability, making it accessible even to less tech-users.

3. Performance: Longhorn delivers great performance, meeting the demands of modern workloads without compromising efficiency.

4. Maintenance Reliability: Longhorn allows for seamless node upgrades without disrupting ongoing workloads, ensuring continuous operation during maintenance activities.

The widespread adoption of Longhorn is evident, with over 111,000 clusters currently using this solution. The project's exponential growth is evidenced by the increasing contribution number on GitHub.

https://github.com/longhorn/longhorn

Key Features and Functionality

• Support for PVC Volumes and CSI: Longhorn seamlessly integrates with Kubernetes Persistent Volume Claims (PVC) and Container Storage Interface (CSI), enhancing its compatibility and flexibility.

• Data Management: Longhorn offers thin provisioning, snapshots, live upgrades, and migration capabilities, that help users with efficient data management solutions.

Architecture

Longhorn is composed of 2 essential aspects :

• Control Plane: Longhorn's control plane, managed via daemon sets, uses controllers and Custom Resources (CRs) to provide essential management functions.

• Data Plane: The data plane handles iSCSI and various volume types (local or raw block devices), in parallel with replica management, to provide data availability and redundancy.

Snapshot Mechanism

Longhorn uses progressive snapshots, those snapshots countain only the changed blocks since the last snapshot. This approach helps to recuce storage utilization by focusing on data changes only, without unnecessary use of huge snapshots.

Replica Rebuilding Process:

In the event of replica loss, Longhorn executes the following steps:

1. Pause the engine (only the replica controller) while retaining read/write capabilities.

2. Take a snapshot.

3. Add a new replica in write-once mode.

4. Restart the engine.

5. Synchronize volumes and snapshots.

6. Change the new replica to read/write mode.

Sharing management

Version 1.6 introduces the Share Manager feature that enables multiple instances of Longhorn to use the same volume concurrently, boosting resource utilization and collaboration.

Let's dive into it : how to install Longhorn ?

The installation on any K8s cluster is fast. We just need to get the file and apply it

kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/v1.X.X/deploy/longhorn.yaml

Once it's created, wait a moment until all longhorn-system pods are ready

In order to access to the ui, just forward the port

kubectl port-forward deployment/longhorn-ui 7000:8000 -n longhorn-systemhttp://127.0.0.1:7000/#/

In the UI, you'll see the different volumes, replicas and you'll have access to the configurations.

Next, create the secret to provide the storage access secrets

kubectl create secret generic longhorn-s3 --from-literal=AWS_ACCESS_KEY_ID=************* -n longhorn-system

--from-literal=AWS_SECRET_ACCESS_KEY=************** -n longhorn-system

Now, we can create any pv/pvc using longhorn :

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
  namespace: namespace1
  annotations:
    volume.beta.kubernetes.io/storage-provisioner: driver.longhorn.io
    volume.kubernetes.io/storage-provisioner: driver.longhorn.io
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  volumeName: pvc-xxx
  storageClassName: longhorn
  volumeMode: Filesystem

Conclusion

Longhorn emerges as a game-changer in the storage landscape, with performance, and ease of use. With its innovative features and robust architecture, Longhorn is on the way to redefine storage solutions for modern cloud environments.