What Is Syslog?

As an IT professional, you know how crucial it is to keep your system running smoothly. But when an issue arises, how do you make sure you are proactive in identifying and troubleshooting the problem at hand?

That’s what Syslog is about. It monitors systems efficiently, identifies and troubleshoots potential problems faster, and ensures your system keeps running at optimal health.

So, whether you're a system administrator, a DevOps engineer, or someone looking to gain a deeper understanding of logging mechanisms, this article will provide you with a solid foundation in syslog.

Defining Syslog

Syslog is a standard for computer data logging that allows software applications and systems to send log messages to a centralized log management system or log file. These log messages can record system activities, security events, and application errors.

The main components of the Syslog system are:

1. Syslog sender (client): This is the application or system component that generates log messages and sends them to the Syslog server.
2. Syslog receiver (server): This is the centralized log management system that receives and stores log messages from multiple senders.
3. Syslog protocol: This defines the format and structure of the log messages being transmitted between the sender and receiver.

How does syslog work?

Syslog has a layered architecture consisting of three parts—application, transport, and content/collection.

Application layer

The application layer consists of the applications, services, and system components that generate log messages. These applications format log messages according to the Syslog standard, which is defined in the Syslog protocol.

The application layer also determines the log message content, severity level, and other metadata based on the specific event or activity being logged.

Transport layer

The transport layer transmits the log messages from the application layer to the collection layer (syslog receiver or server). It uses UDP, TCP, and TLS for secure transmission.

• UDP (User Datagram Protocol) is the most commonly used transport protocol for Syslog due to its simplicity and speed, although it lacks reliability.
• TCP (Transmission Control Protocol) and TLS (Transport Layer Security) provide more reliable delivery but with slightly higher overhead.

Additionally, this layer ensures that log messages are correctly transmitted from the senders (clients) to the receivers (servers) based on the configured transport protocol.

(Related reading: how TCP/IP works & SSL/TLS certificates.)

Collection layer

The collection layer consists of the syslog receivers or servers that listen for incoming log messages from various senders. Syslog receivers listen on well-known ports, such as UDP port 514 or TCP port 514, for incoming log messages.

Upon receiving log messages, the collection layer does the following:

• Store the log messages in log files, databases, or other storage systems for later log analysis and retention.
• Filter and route log messages based on specific criteria (e.g., facility, severity, source) to different destinations or actions.
• Generate alerts or notifications based on predefined rules or patterns detected in the log messages.
• Provide a centralized location for administrators and security professionals to access and analyze log data from multiple sources.

Syslog use cases

Syslog is used by IT professionals, developers, and organizations to keep track of multiple things and to speed up their work process. Here are some of the most common areas where Syslog is used today:

System monitoring

Syslog monitors the health and performance of systems like servers, workstations, and network devices. IT administrators collect and analyze syslog messages to gain insights into system activities. This allows them to identify potential issues and take proactive measures to prevent downtime or service disruptions.

For example, syslog can log kernel messages, application errors, and hardware-related events (e.g., disk failures, and memory issues). By monitoring these logs, administrators quickly detect and troubleshoot software bugs or configuration errors.

(Related reading: IT monitoring, application monitoring & log monitoring.)

Network devices

Syslog messages are used to gather insights on network traffic patterns, network devices, and any changes in their configuration. Syslog messages provide information about network performance and potential issues in network devices related to:

• CPU usage
• Memory utilization
• Interface errors

By using Syslog messages, IT professionals can view system activity holistically throughout the IT infrastructure, identify any correlations between incidents, and gather more details and insights to prevent major issues.

(Related reading: incident response.)

Fixing bugs in applications

Using Syslog, developers can fix bugs by understanding the application's behavior through Syslog messages. Once they understand the app's behavior, they can easily spot bugs and prevent them from recurring throughout the development and testing phase.

For example, database administrators can use Syslog messages to:

• Supervise and track database queries.
• Ensure compliance with security policies.

This way, by identifying any potential for a security breach, the team can diagnose it and take preventive measures well in advance.

Understanding Syslog format and messages

A standard Syslog format ensures messages are shared between applications, network devices, and the logging server faster and more consistently. That’s why all Syslog messages follow a consistent and standard format specified by RFC 5424 (the new format).

Here are the three main components of the Syslog format:

1. The timestamp indicates the date and time when the log message was generated.
2. The hostname or IP Address identifies the host or device that generated the log message.
3. The message field contains the actual log message or event description.

Let’s understand this with an example Syslog message:

Mar 10 15:24:37 host.example.com sshd[1234]: Failed password for user johndoe from 192.168.1.100 port 49758 ssh2

In this example:

• Mar 10 15:24:37 is the timestamp.
• host.example.com is the hostname.
• sshd[1234] indicates the process that generated the log (sshd process with PID 1234).
• The message describes a failed SSH login attempt for the user "johndoe" from the IP address 192.168.1.100.

Syslog messages

Syslog messages have a specific communication path. This path consists of a message originator, which creates and sends messages, and a receiver (e.g., a logging server), which collects the message and stores it. Depending on the originating application’s settings, Syslog messages can also be sent to multiple destinations at the same time.

The Syslog protocol also uses PRI (priority) to categorize these messages. PRI is calculated using the facility and severity value.

Syslog facility codes

The Syslog facility codes are generated in the Unix systems to identify the source of the message. These codes are numeric and generated by the originator of the message.

Here is a list of message codes with their corresponding facility:

• 0 is for kernel messages
• 1 is for user-level messages
• 2 is for mail system
• 3 is for system daemons
• 4 is for security/authorization messages
• 5 is for messages generated internally by Syslog
• 6 is for line printer subsystem
• 7 is for network news subsystem
• 8 is for UUCP subsystem
• 9 is for clock daemon
• 10 is for security/authorization messages
• 11 is for FTP daemon
• 12 is for NTP subsystem
• 13 is for log audit
• 14 is for log alert
• 15 is for clock daemon
• 16 and 23 are for local use (to be used for unique applications or software producing data in your machine or system)

Syslog severity codes

All Syslog messages have a severity indicator — a numeric value from 0 to 7. Message priority is determined by combining the facility and severity values.

Here is a list of severity codes with what they indicate about the importance of a message:

• Severity value 0: The system is not available for use.
• Severity value 1 indicates that action must be taken immediately for system stability.
• Severity value 2: Critical conditions such as severe system issues e.g. loss of primary ISP connection.
• Severity value 3: System errors that require attention.
• Severity value 4: Warning messages requiring appropriate action to avoid system error.
• Severity value 5: Notification messages to report an ongoing condition.
• Severity value 6: Informational messages like system reporting and measuring messages
• Severity value 7: Debugging messages that are specific to applications.

Benefits of using the Syslog protocol

By leveraging the Syslog protocol, you can manage and analyze log data from multiple sources to enhance system monitoring and troubleshooting capabilities. Here are some of the most prominent benefits of using Syslog:

• Ensures consistency and interoperability among different devices and software from various vendors.
• Monitor and troubleshoot issues across multiple systems from a single location.
• Promotes a modular design and reduces the overhead on individual systems.
• Allow administrators to filter and prioritize messages based on their importance.
• Send log messages to remote syslog servers, which can be useful for systems without local storage or in scenarios where logs need to be collected from remote locations.
• Provide accurate time synchronization across multiple systems to help with correlating events and troubleshooting issues.
• Permit vendors to include custom data in log messages.

Summarizing Syslog

Syslog is a game-changer for efficient system monitoring and troubleshooting. It centralizes log data from diversified sources and uses a standardized format for easy analysis. By prioritizing messages and enabling remote collection, it empowers IT teams to identify and address issues so the systems can keep running smoothly.