🛠️ Linux Storage 101 : Storage Concepts and File Systems
✨Before we get started
Ajeet, what are we going to learn here?
This is Part 1 of a series on Linux storage, mostly theoretical, where we'll acquire skills in storage and its types like Local Storage, DAS, SAN, and NAS with some examples.
Later, we will become competent in File Systems and types like EXT4, XFS, and BTRFS, and try to focus on performance by optimizing the File System.
Previously, when you were using a Windows machine, you might have heard of NTFS or EXT4, and defragmenting or optimizing disks. This is somewhat similar.
💾What is Storage?
Storage is a technology where a computer, server, or laptop stores all information or data, whether temporarily or permanently.
Thanks to storage, we can access everything on a computer, like photos, files, or web pages.
🗂️ Types of Storage
💻 Local Storage
Storage is what you find on your local machine because its built into it. Quick and easy to get access, and can be easily managed.
Examples Include:
RAM (Memory)
This is like your computer's working memory, which it uses to store data temporarily while it works on the task.
Standard Storage, aka Hard Disk Drive (HDD)
The traditional storage option for high capacity.
Solid State Drive (SSD)
Fast, reliable and with no moving parts SSDs are all the rage.
Cache
Normally there is a small super-fast memory that helps improve computer performance by providing quick access to the processing unit for previously processed data.
🔗 Direct Attached Storage (DAS)
Directly attached to the computer this is my personal favorite because of USB flash drives. It is also almost always used when you require something that needs to be portable and easy.
Examples Include:
CD/DVD
While you might say CD and DVD are local storage but these also can connect as DAS.
USB flash drive
Great for carrying files in your pocket
External disks
Lastly, if you live in a SCSI or USB world, need more than 4 GB of space and are on the road when your workgroup dissolves into read-only access; there's always external disk.
🌐 Storage Area Network (SAN)
SAN is a dedicated high speed network that connect server or multiple servers to shared pool of storage devices like disk arrays or tape libraries. This is something large corporations use to get a lot of data asap.
Examples Include:
NetApp FAS Series
An ideal answer for extensive storage requirements.
Dell PowerMax
Designed for demanding performance applications that need speed and efficiency.
Components:
SAN Adapters (HBAs)
To which mount in your motherboard and make everything speak to one another.
Cables
Fiber and or iSCSI cables to connect your storage devices up with your system.
Process:
Install SAN adapters.
Connect fiber cables.
Install the driver.
Find the storage under a WWN (World Wide Name) number.
🌍 Network Attached Storage (NAS)
NAS stands for Network Attached Storage. It is basically storage that you connect to your network, allowing multiple devices on the same network access it like a drive. Useful for home setup, business setup having data sharing points.
Examples Include:
Samba
An open-source file-sharing solution on your network.
NFS (Network File System)
A protocol that allows you to access files over the network as if they are on your own computer.
Process:
Find the IP address of your NAS.
Access storage through its IP address (nfs or Samba).
📁File Systems and Types
A file system is a way an operating systems organizes files and data on a disk or partition. Different file systems are designed to be optimal for different types of applications by offering specialized features and performance in the chosen use cases.
Ext4
Ext4 (Fourth Extended Filesystem) is the most commonly used file system in Linux. It is an upgrade from the Ext3 file system and offers improvements like support for larger files and volumes, faster fsck
checks when unmounted, and nanosecond
timestamps for stored/mounted inodes/blocks.
Features:
Ext4 supports large files and volumes, allowing files up to 16 TB and volumes over 1 EB with extents.
Journal Checksumming: Protects against data corruption by including checksums in the journal, ensuring the integrity of all logged transactions.
Relaxed Allocation: Reduces fragmentation by delaying block allocation until the data is written.
It is ideal for general-purpose use, desktop systems, and servers where stability and reliability are crucial.
XFS
XFS is a high-performance journaling file system. It is recommended for large hard drives, especially those using RAID or SAN storage.
Features:
XFS handles large files and many files efficiently, making it suitable for big storage systems. It supports up to 8 EB.
It provides data integrity through metadata journaling, reducing the risk of filesystem corruption.
Advanced Allocation Algorithms: XFS uses smart algorithms to maximize disk I/O throughput by placing data in the best locations on your disks.
XFS is ideal for tasks requiring very fast reads, such as high-performance video editing and some scientific computing applications, including big data analytics.
Btrfs
Btrfs (B-tree file system) is a modern filesystem that support some advanced features such us snapshots and built-in RAID support.
Features:
Security and data integrity, e.g., snapshot/clone: Btrfs can create snapshots of subvolumes. With true copy-on-write, it doesn't change the original storage except for system metadata. This allows for quick atomic backups, ensuring consistency without losing IO performance if done correctly.
Built-in RAID: Using the Btrfs file system, this NAS series supports various types of hardware RAID natively.
Automatic Repair (Self-Healing): Btrfs can detect and fix corruption automatically using checksums.
Btrfs is great for situations needing advanced data management, like virtualized environments, backup solutions, or any place requiring shared storage integrity.
⚙️ File System Tuning
1. Align Partitions Properly
The partition should be aligned to the physical block size of your storage device. One Well we learned earlier that misaligned partitions can lead to poor performance, particularly on SSDs and Advanced Format drives
2. Use Noatime/Nodiratime
The noatime and nodiratime file system mount options can be utilized to avoid updating the access time each time a file or directory is read. This prevents unnecessary write operations which have improved performance.
3. Enable Write Caching
Write caching writes can be cached in memory for a better performance, and later written to disk. Your storage device must support and be configured for write caching.
4. Adjust Inode Ratios
In the same way inode ratios decide how many inodes (file system metadata) has to be created. There are many small files situations in which inode ratio matters as adjusting the inode ratio will keep you from getting out of inodes but also helps to boost performance.
5. Defragmentation
File fragmentation is another area to look into; regularly defragmenting a file system can aide performance as well, two examples are XFS and Btrfs support this.
6. Use Appropriate Block Sizes
The block size is one of the factors that could affect performance in VRAL, and setting it appropriately during filesystem formatting will have an impact. Smaller block sizes are good for small files, larger ones work more efficiently with large files.
7. Track File System Performance
You can use iostat
, vmstat
, dstat
etc to monitor the file system performance and bottlenecks are identified. By monitoring these various hybrid dockercloud configurations, you can effectively fine-tune your system to force optimal performance
🎯 Conclusion
And there you have it! 🎉 You’ve just taken your first steps into understanding Linux Understanding the different types of storage and file systems in Linux is essential for managing and optimizing your systems effectively. In this first part of our series, we've introduced the fundamental concepts of storage, explored various storage types, and provided an overview of common file systems.
Stay tuned for the next part of this series,
Subscribe to my newsletter
Read articles from AJEET GUPTA directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
AJEET GUPTA
AJEET GUPTA
Hi there! I’m Ajeet, a System Administrator with three years of experience in managing both on-premises and cloud infrastructures. I’m all about using ITIL practices to automate tasks, solve problems, and keep systems running smoothly.