How to automatically delete files from directory to avoid high disk utilization?

Production servers often have applications or services that continuously write log messages to files. This can cause the disk to fill-up quickly based on the frequency and size of log messages. Manually cleaning up such files is tedious and is error prone. Logrotate utility is designed to address this specific system administration problem. It can automatically rotate, compress, or delete files. Let's take a look at how it works.

Installation

Most linux distros come pre-installed with logrotate. You can verify it by running following command:

$ which logrotate
/usr/sbin/logrotate
$ 

If it's not already installed, you can run sudo yum install logrotate on RHEL based distros or sudo apt install logrotate on Debian based distros to install logrotate. You can also download and install if you want to use a specific version that's not available via yum/apt.

Sample configuration to rotate files when size is crosses threshold

Let's setup logrotate configuration to rotate files when their size is more than 5 bytes. We are going to use /home/ubuntu/logrotate_demo/files/ directory to test the logration. Let's create 4 files in the directory as follows:

$ echo -n '12' > tiny_file
$ echo -n '1234' > small_file
$ echo -n '1234567' > large_file
$ echo -n '123456789' > huge_file

Let's create logrotate config file at /home/ubuntu/logrotate_demo/logrotate.conf location and populate it with following contents:

/home/ubuntu/logrotate_demo/files/* {
    size 5
}

Now let's run logrotate once as follows:

$ pwd
/home/ubuntu/logrotate_demo
$ logrotate logrotate.conf -s logrotate.state

Observe the contents of /home/ubuntu/logrotate_demo/files/ directory:

$ ls /home/ubuntu/logrotate_demo/files/
small_file tiny_file

Note that the large_file and huge_file were deleted by logrotate. This is because their size was more than 5 bytes. Now we could run this manually on a regular basis to clean up large files, but that's tedious. We will look at how to configure logrotate as a systemd service to run periodically in subsequent section. Before that let's take a look at what rotation means in logrotate.

Rotating files

As we saw in the previous section, logrotate deleted files when it found files more than the specified size. Consider that the directory logrotate cleans up contains application log files. When that file grows more than the threshold, the log will get deleted. What if you wanted to see an older application log to examine a certain crash? Logrotate allows us to keep a configurable number of rotated files before eventually deleting it. Let's see this in play. Update /home/ubuntu/logrotate_demo/logrotate.conf file to include rotate directive on /home/ubuntu/logrotate_demo/files/log_file as follows:

/home/ubuntu/logrotate_demo/files/log_file {                                   
    rotate 2
    size 5
}

Create a large file representing application log as follows:

$ pwd
/home/ubuntu/logrotate_demo/files
$ rm *
$ echo -n '123456789' > log_file

Now let's run logrotate once as follows:

$ pwd
/home/ubuntu/logrotate_demo
$ rm logrotate.state     #Remove the logrotate state since we cleaned up files in /home/ubuntu/logrotate_demo/files/*
$ logrotate logrotate.conf -s logrotate.state

Observe the contents of /home/ubuntu/logrotate_demo/files/ directory:

$ ls files 
log_file.1

As you can see the file log_file has been renamed to log_file.1. Let's create another log_file and run logrotate again:

$ pwd
/home/ubuntu/logrotate_demo/files
$ echo -n 'abcdefghi' > log_file
$ cd ../
$ logrotate logrotate.conf -s logrotate.state

Observe the contents of /home/ubuntu/logrotate_demo/files/ directory:

$ ls files
log_file.1  log_file.2

As you can see, the file log_file.1 has been renamed to log_file.2 and log_file has been renamed to log_file.1. Let's create another log_file, but with size smaller than our logrotate size threshold and run logrotate again:

$ pwd
/home/ubuntu/logrotate_demo/files
$ echo -n 'xyz' > log_file
$ cd ../
$ logrotate logrotate.conf -s logrotate.state

Observe the contents of /home/ubuntu/logrotate_demo/files/ directory. Files were not rotated because log_file size is smaller than threshold.:

$ ls files
log_file  log_file.1  log_file.2

Let's increase the file size and run it again:

$ pwd
/home/ubuntu/logrotate_demo/files
$ echo -n 'tuvwxyz' > log_file
$ cd ../
$ logrotate logrotate.conf -s logrotate.state

Observe the contents of /home/ubuntu/logrotate_demo/files/ directory. Not that log_file.2 was deleted, log_file was renamed to log_file.1, and log_file.1 was renamed to log_file.2

$ ls files
log_file.1  log_file.2

$ cat log_file.1 
tuvwxyz
$ cat log_file.2 
abcdefghi

So far, we have been manually running logrotate to cleanup and rotate files. Let's see how it can be configured to run automatically

Configuring logrotate to run as systemd service with systemd timer

Service config /lib/systemd/system/logrotate-demo.service:

[Unit]
Description=Rotate log files
Documentation=man:logrotate(8) man:logrotate.conf(5)
ConditionACPower=true

[Service]
Type=oneshot
ExecStart=/usr/sbin/logrotate /home/ubuntu/logrotate_demo/logrotate.conf -s /home/ubuntu/logrotate_demo/logrotate.state

Timer config:

[Unit]
Description=Daily rotation of log files
Documentation=man:logrotate(8) man:logrotate.conf(5)

[Timer]
OnCalendar=daily
AccuracySec=1h
Persistent=true

[Install]
WantedBy=timers.target

Enable the service and timer with:

$ sudo systemctl enable --now logrotate-demo.timer

Now go back and come after a day to check the status of your files. If you run the following command, you should expect logrotate-demo to have run and exited successfully.

$ sudo service logrotate-demo status

Have any feedback?

If you have any feedback regarding this guide or need a tutorial on any specific topic, please submit this feedback form.