What are Linux logs, and where to find them

2.2K

If you have spent some time with a Linux distribution, you might have heard the term Linux log files. Let’s take a look at what kind of log files exist in Linux, where to find them, and how to read them.

What is a Linux log?

A log file contains information about the activity of a specific service or a program in plain text, with a time stamp. For example, if you’re on a Debian-based system, you undoubtedly use apt for package management. There is a log for apt, which contains the whole history of all the programs that have been installed, removed, purged, etc. using the apt command, with the time at which it happened.

Typically, when the system is smooth and stable, we don’t even need to bother looking at them. Linux log files come into picture when there is an issue with the system, and you have to look at the log files to troubleshoot it. In another case, log files are handy to system administrators. They always need to know what is happening and when.

Regardless of which Linux distribution you are using, the log files reside in /var/log/ directory. In this article, we shall be discussing the most important log files that you need to know about.

Important Linux log files

1. System Logs

System logs are directly filed by the operating system components. This includes device change information, system change information, and a whole broad spectrum of things in general.

2. Event Logs

Event logs hold the network information, and in some cases, application information as well. Information about account lockouts, failed password attempts are included in event logs.

3. Application Logs

Application logs contain logs created and generated by specific applications.

4. Kernel Logs

Kernel logs are the logs filed directly in by the kernel. They are extremely helpful in troubleshooting the kernel issues.

Locating Linux Logs

As we mentioned earlier, no matter what the distribution, the log files are always stored in the /var/log directory on any Linux system. Therefore, to check out the log files, we first move into that directory:

cd /var/log/

And see the contents:

ls

As you can see, there are many log files about many different programs/services. What logs are essential to a specific user can only be told by that user, but we are going to tell you about some of the most useful log files.

Important Logs

1. Syslog or messages

This log contains the general information of any system, including the data log of all generic activity, errors, and network information. It is the go-to log file for any simple issue.

On RedHat-based systems, it is stored in /var/log/messages.
On Debian-based system, it is stored in /var/log/syslog.

2. auth.log or secure

This is the authentication log. It includes all login attempt logs, whether successful or unsuccessful. The logs both the login of systemd (if your distribution has it) and also of whatever display manager that you have.

On RedHat-based systems, it is stored in /var/log/secure.
On Debian-based systems, it is stored in /var/log/auth.log.

3. kern.log

This is the Kernel log. It is probably not useful to most of the users, but it is a critical log. It logs all the kernel activity, including hardware interaction, initialization of hardware on boot, and system calls.

It is found at /var/log/kern.log on all distributions.

4. boot.log

The boot log contains the messages logged at the time when the system is booting. The messages relayed by the startup scripts are logged here. Mostly, if there are issues with unplanned shutdown, or reboot, or some anomaly in the boot processes, the log is referred to see what is happening.

5. faillog

This is an interesting one. It holds the logs of failed login attempts. It is especially helpful for security purposes, as logging in is the first step to doing anything at all on a system. Login brute-force attacks can easily be detected using the time gap between consecutive logins.

It is found at /var/log/faillog on all distributions.

6. apport.log (only on Ubuntu-based systems)

It was often found that when an application crashed, there were no logs of it. It didn’t have a specific log file, nor did it get recorded in any other log. To fix it, Ubuntu came up with the apport.log. When a program crashes, it gets recorded in the apport.log file. Find more about it here.

It is found at /var/log/apport.log on Ubuntu-based systems.

7. Package Manager Log

This is a useful log, even for casual users. It is a record of whatever package manager that your system, or specifically the user uses (might be multiple). The installation, removal, purging of programs is recorded in the log.

Debian-based systems

Debian based systems use the apt package management, the logs of which are in the directory /var/log/apt. There are two log files present there generally:

history.log: It records the history of package management done by apt in a simple formatted manner.

term.log: It records the exact output shown in the Terminal at the time of usage of the apt command in any form.

Debian systems also use the DPKG management for the DEB files, so it also has a log for that. It can be found at /var/log/dpkg.log.

RedHat systems

RedHat systems use the DNF package management system by default. The installation, removal, and other tasks relating to the packages can be found in the dnf log. It is located at /var/log/dnf.log.

8. mysqld.log or mysql.log

The logs listed starting here are a bit more directed towards core users. MySQL is a service that is often used by users. They may be system administrators, website maintainers, or may just use MySQL for personal uses. Being such a valuable service, it has to have a log file dedicated. All success, failure, or debug messages are logged here.

On RedHat-based systems, it is stored in /var/log/mysqld.log.
On Debian-based systems, it is stored in /var/log/mysql.log.

httpd

This directory contains the logs of the Apache server on the system. It has two files generally, -error_log and access_log, which store information that is indicative of only the file name.

You can find it at /var/log/httpd/ on all distributions.

mail.log

System and command-line integrated E-mail services were widely used until a few years ago. Clear by the name itself, mail.log contains the logs for the usage of such E-mail services.

You can find it at /var/log/mail.log.

Reading Logs

1. CLI

Now we can finally get to a significant point, which is reading those logs. There are various ways in which you can, and in which you would need to read the logs. For example, if you want just to see the ending part of the log file (to know about the most recent activity), you can use the tail command. The command only prints the last 10 lines of a file.

Example:

sudo tail /var/log/syslog

On the other hand, if you want to navigate through the whole file and search for things, you can use the infamous less command. You can use the Up and Down keys to navigate through the file. To search, press the ‘/’ key, and enter the exact search term. The searched term should be highlighted. Example:

sudo less /var/log/syslog

2. GUI

There are several graphical programs to help the users read the log files on a system. Today, we shall take a look at glogg.

glogg is a log viewing program that has a straightforward interface. The official website describes it as a combination of the less and grep commands. You can open glogg, and then open a log file using the button provided on the top left to open up a log file.

We suggest an alternate way, which is launching glogg from the command line, along with the log file’s location. This makes it easier to open up the log file. The command looks like:

sudo glogg /var/log/syslog &

User Interface

The log is shown in the major window. There is a search box on the bottom, in which you can search whatever term that you are looking for. There is also a frequency bar on the right, which shows how frequently the searched term shows up in the log file.

Installation

It can be installed easily on Debian and Ubuntu-based systems with the command:

sudo apt install glogg

On Fedora/CentOS based systems:

sudo dnf install glogg

You can find additional help for the installation here.

Additional Information

There is some more critical information that you should know about log files.

Log Rotation

The log files are ‘rotated’ regularly. This means that new versions of a log file are created regularly, as the log files have certain storage limits or time-based constraints. If you issue the command:

ls /var/log/

You might see that several files have the same name except for “.1” or “.2.gz” at the end. These are just older versions of the same file. The conditions for log rotations can be configured. You can find the configuration files with the command:

cd /etc/logrotate.d/

ls

The differently named files are the respective log configurations. One such file looks somewhat like this:

This can simply be edited to change the configurations of the respective log files.

rsyslog

rsyslog is the service that is responsible for creating the log files in the first place. Its configuration files are available at /etc/rsyslog.conf and the directory /etc/rsyslog.d. Similar to log rotation, you can configure these files to suit your needs.

Conclusion

Logs are beneficial and useful in almost all of the cases concerning a malfunction in the hardware or software of the Linux system. Reading log files can be enlightening, and can help you understand your system better. We hope this article helped you. If yes, don’t forget to share it with your friends.