Log Rotations in Linux

What is Log Rotation ?

Log rotation is a process used to manage and maintain log files generated by software applications or system processes. The basic idea behind log rotation is to prevent log files from growing indefinitely, which can consume disk space and make it harder to find relevant information within the logs.

How Log Rotation Works ?

Log rotation is like keeping your room tidy by regularly swapping out full notebooks. Imagine you're notting down daily activities in a notebook. As the days pass, the notebook fills up. Instead of letting it overflow, you start a new one each week.

Log rotation works similarly for software logs. When a log file gets too big or a certain time period passes, it's "rotated" – meaning a new file is started to collect fresh logs, while the old one is saved or compressed for reference. This keeps logs manageable, prevents them from gobbling up all your disk space, and ensures you can find what you need without wading through endless entries.

Parameters that we are considering while writing the log file for a particular Software logs :

1. Log Size or Time-Based Rotation:

Log rotation can be triggered either by the size of the log file reaching a certain threshold or by a predefined time interval (e.g., daily, weekly,monthly). When the specified condition is met, the rotation process begins.

We can create log files on daily/weekly/monthly basis to control the disk space size.

2. Renaming the Current Log File:

The current log file is typically renamed or moved to a different location. This action allows a new log file to be created without interrupting the logging process.

3. Retention Period/Policies:

Log rotation often includes retention policies to manage the number of rotated log files retained on disk. Older log files may be deleted or archived to a separate location based on these policies

We need to mention the retention period (in days) ex.5 days , 30 days, etc for availability of older log file after the retention period , these file are deleted.

4. Compression :

In some cases, rotated log files may be compressed to save disk space. This is particularly useful when dealing with large log files that are no longer actively being written to but still need to be retained for archival or analysis purposes.

5. Date-ext :

When you use dateext in log rotation, it adds the current date to the names of rotated log files. So, if your log file is named "app.log" and it's rotated today, it might be renamed to "app.log.2022-01-01". This helps keep your log files organized by date, making it easy to see when each log file was rotated.

6. Date-format :

With dateformat, you can choose how the date looks in the filenames of your rotated log files when dateext is turned on. For example, if you set dateformat to %Y-%m-%d, your log files will have dates in the format "Year-Month-Day", like "app.log.2022-01-01". This helps you organize your log files by date in a way that's easy to understand.

7. missingok :

When you use missingok in log rotation, it means that if the log file you want to rotate isn't there, the rotation process won't get stuck or stop. Instead, it just carries on like normal. This is handy when your logging hasn't started yet or if it's been turned off temporarily. Enabling missingok prevents any problems from happening during log rotation when the log file isn't available.

8. notifempty :

With notifempty, if a log file is empty, it won't trigger rotation. It's like saying, "Don't bother swapping out my notebook if there's nothing written in it." This parameter helps avoid unnecessary rotation when there's no new information to log, keeping things neat and efficient.

Commands to see the previously created log files :

• sudo -i (perform commands with root user )

• ll /var/log (you’ll see all created logs present in the system)

• cd /etc/logrotate.d (go to these folder to create new log file or folders)

• ls (it will show you how many log folder for different applications are previously created)

Simple template of the log file :

[ path of your Log file] {

copytruncate

rotate

daily/weekly/monthly

compress

data ext

dateformat

missingok

Notifempty

}

Example dpkg.log file:

/var/log/dpkg.log {

monthly

rotate 12

compress

delaycompress

missingok

notifempty

create 644 root root

}

Thank You for Reading My Blog.. 👍

Connect with me : linkedin.com/in/shivraj-salunkhe-5881141a4

Follow my Blog channel : shivrajofficial.hashnode.dev