Rotate and archive logs with the Linux logrotate command

Logs are nice for locating out what an software is doing or troubleshooting a doable downside. Almost each software we cope with generates logs, and we wish the functions we develop ourselves to generate them too. The extra verbose the logs, the extra info we have now. But left to themselves, logs can develop to an unmanageable dimension, and so they can, in flip, develop into an issue of their very own. So it is a good suggestion to maintain them trimmed down, hold those we will want, and archive the remaining.

Basics

The logrotate utility is great at managing logs. It can rotate them, compress them, e-mail them, delete them, archive them, and begin recent ones once you want them.

Running logrotate is fairly easy—simply run logrotate -vs state-file config-file. In the above command, the v possibility allows verbose mode, s specifies a state file, and the ultimate config-file mentions the configuration file, the place you specify what you want completed.

Hands-on

Let’s try a logrotate configuration that’s working silently on our system, managing the wealth of logs we discover within the /var/log listing. Check out the present information in that listing. Do you see lots of *.[number].gz information? That’s what logrotate is doing. You can discover the configuration file for this underneath /and many others/logrotate.d/rsyslog. Mine appears like this:

/var/log/syslog

/var/log/mail.data
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/consumer.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages

The file begins with defining the directions for rotating the /var/log/syslog file and the directions are contained inside the curly braces that observe. Here’s what they imply:

• rotate 7: Keep logs from the final seven rotations. Then begin deleting them.
• every day: Rotate the log every day. Along with rotate 7, this could imply that logs could be saved for the final seven days. Other choices are weekly, month-to-month, yearly. There can also be a dimension parameter that can rotate log information if their dimension will increase past a specified restrict—for instance, dimension 10k, dimension 10M, dimension 10G, and many others. If nothing is specified, logs might be rotated every time logrotate runs. You may even run logrotate in a cron to make use of it at extra particular time intervals.
• missingok: It’s okay if the log file is lacking. Don’t Panic.
• notifempty: Don’t rotate if the log file is empty.
• delaycompress: If compression is on, delay compression till the subsequent rotation. This permits a minimum of one rotated however uncompressed file to be current. Useful if you’d like yesterday’s logs to remain uncompressed for troubleshooting. It can also be useful if some program may nonetheless write to the previous file till it’s restarted/reloaded, like Apache.
• compress: Compression is on. Use nocompress to show it off.
• postrotate/endscript: Run the script inside this part after rotation. Helpful in doing cleanup stuff. There can also be a prerotate/endscript for doing issues earlier than rotation begins.

Can you determine what the subsequent part does for all these information talked about within the configuration above? The solely extra parameter within the second part is sharedscripts, which tells logrotate to not run the part inside postrotate/endscript till all log rotation is full. It prevents the script from being executed for each log rotated and runs as soon as on the finish.

Something New

I’m utilizing the next configuration for coping with Nginx entry and error logs on my system.

/var/log/nginx/entry.log
/var/log/nginx/error.log  {
        dimension 1
        missingok
        notifempty
        create 544 www-data adm
        rotate 30
        compress
        delaycompress
        dateext
        dateformat -%Y-%m-%d-%s
        sharedscripts
        extension .log
        postrotate
                service nginx reload
        endscript
}

The above script might be run utilizing:

logrotate -vs state-file /tmp/logrotate

Running the command for the primary time provides this output:

studying config file /tmp/logrotate

extension is now .log
Handling 1 logs
rotating sample: /var/log/nginx/entry.log
/var/log/nginx/error.log   1 bytes (30 rotations)
empty log information are usually not rotated, previous logs are eliminated
contemplating log /var/log/nginx/entry.log
  log wants rotating
contemplating log /var/log/nginx/error.log
  log doesn't want rotating
rotating log /var/log/nginx/entry.log, log->rotateCount is 30
Converted ' -%Y-%m-%d-%s' -> '-%Y-%m-%d-%s'
dateext suffix '-2021-08-27-1485508250'
glob sample '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob discovering logs to compress failed
glob discovering previous rotated logs failed
renaming /var/log/nginx/entry.log to /var/log/nginx/access-2021-08-27-1485508250.log
creating new /var/log/nginx/entry.log mode = 0544 uid = 33 gid = 4
working postrotate script
* Reloading nginx configuration nginx

And working it a second time:

studying config file /tmp/logrotate

extension is now .log
Handling 1 logs
rotating sample: /var/log/nginx/entry.log
/var/log/nginx/error.log   1 bytes (30 rotations)
empty log information are usually not rotated, previous logs are eliminated
contemplating log /var/log/nginx/entry.log
  log wants rotating
contemplating log /var/log/nginx/error.log
  log doesn't want rotating
rotating log /var/log/nginx/entry.log, log->rotateCount is 30
Converted ' -%Y-%m-%d-%s' -> '-%Y-%m-%d-%s'
dateext suffix '-2021-08-27-1485508280'
glob sample '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
compressing log with: /bin/gzip
renaming /var/log/nginx/entry.log to /var/log/nginx/access-2021-08-27-1485508280.log
creating new /var/log/nginx/entry.log mode = 0544 uid = 33 gid = 4
working postrotate script
* Reloading nginx configuration nginx

And working it a 3rd time:

studying config file /tmp/logrotate

extension is now .log
Handling 1 logs
rotating sample: /var/log/nginx/entry.log
/var/log/nginx/error.log   1 bytes (30 rotations)
empty log information are usually not rotated, previous logs are eliminated
contemplating log /var/log/nginx/entry.log
  log wants rotating
contemplating log /var/log/nginx/error.log
  log doesn't want rotating
rotating log /var/log/nginx/entry.log, log->rotateCount is 30
Converted ' -%Y-%m-%d-%s' -> '-%Y-%m-%d-%s'
dateext suffix '-2021-08-27-1485508316'
glob sample '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
compressing log with: /bin/gzip
renaming /var/log/nginx/entry.log to /var/log/nginx/access-2021-08-27-1485508316.log
creating new /var/log/nginx/entry.log mode = 0544 uid = 33 gid = 4
working postrotate script
* Reloading nginx configuration nginx

The contents of the state file appear to be this:

logrotate state -- model 2
"/var/log/nginx/error.log" 2021-08-27-9:0:0
"/var/log/nginx/access.log" 2021-08-27-9:11:56

Download the Linux logrotate cheat sheet.

This article was initially printed on the author’s personal blog and has been tailored with permission.