Audit Logging 101: Everything To Know About Audit Logs & Trails

Audit logs, or audit trails, answer a simple question: who did what, where and when?

So, in this article, we’ll answer our own simple question: How can you use audit logs, and what use cases do audit logs best support?

---

---

What are audit logs?

When you use a technology service or product, audit logs are generated in response to every user action and response from the technology system. These logs capture information that can be used to…

• Authenticate the user.
• Identify and validate the request.
• Route the request to the right service node.
• Perform relevant technology operations and processing.

Though these micro-actions behind audit logs are important in their own ways, the bigger purpose is greater. The idea behind collecting audit logs is two-fold:

1. To identify errors and improve accuracy.
2. To understand the purpose behind an activity, which can be later used for accountability or compliance.

At every step, a trail of log metrics data or metadata is generated and recorded by the systems. This information is documented and can later be used for a variety of use cases security, monitoring and performance analysis and cyber forensics.

(Read about the related practices of log aggregation & log management.)

Details included in audit logs

Audit logs comprise the following information:

• Timestamp, location and TCP/IP protocol data
• Event description and tags
• Actors, groups, users, entity and device identification
• Action types
• Predefined metrics
• Data access, login attempts, failures and authentication information
• Error details
• Actions, Account changes, system-wide changes and information state changes
• Transaction details

(Understand the difference between logs & metrics.)

Use cases for audit logs: how to connect the dots

Audit logging can have four key domain applications:

• Security
• Compliance
• Accountability
• Cyber forensics

Use case 1: Security

In terms of security, audit logs can be used to identify anomalous behavior and network traffic patterns. InfoSec teams can integrate the audit logging mechanism to their monitoring and observability solutionsinfo to extract insights on potential security incidents.

For authentication and detection of unauthorized network changes, this can be achieved by testing network change actions against predefined security policies — looking at the delta. These policies define how network and IT resources are allowed to be accessed – in terms of entity, location, roles and attributes, as well as action frequency and location.

Use case 2: Audit logs for compliance

If your organization has to comply with external regulations, your organization may be required to keep specific audit logs and establish monitoring capabilities that test the systems for compliance by analyzing audit logs in real-time. For instance:

• ISO 27001 imposes requirements for audit logging and monitoring.
• SOC1 imposes requirements for incident detection, configuration, management and event log collection.

(See how Splunk supports organizational compliance.)

Use case 3: Audit logs for accountability, authentication

As with standard audit procedures, audit logging is frequently used for accountability and verification of factual information. Common applications include:

• Organizational policy enforcement
• Accounting and finance
• HR policies

In this context, audit logging is an important part of analyzing how users perform an action and the accuracy of information recorded by the systems. For example, audit logging can quickly enable systems and uncover insights into the use of financial resources across all departments. Imagine a world where all this was easy and straightforward:

• Authorizing and spending finances.
• Understanding which users are responsible for the most spending.
• Comparing against budget allocations.

Use case 4: Audit logs for cyber forensics

Cyber forensics is another key application domain of audit logging practices that require reconstruction of events and insights into a technology process. Often, this might stand up as legal evidence in a court of law.

---

---

Typically, businesses aren’t conducting cyber forensics for all their activities. Instead, cyber forensics is usually required in two situations:

• An external requirement for investigation in the form of court subpoena
• An internal request by business executives and technical teams, perhaps around a major cyber incident or significant, unplanned downtime in a website or system

Audit logs are used to outline the action sequences that connect a user to an action. Investigators can analyze audit logs to gain deeper insights into various scenarios and outcomes represented by the audit logs. This requires a thorough analysis of raw logging data before it is converted into insightful knowledge.

(Explore cyber forensics & the differences from auditing.)

Auditing trail data: considerations & best practices

Considering the vast volume of network, hardware and application logs generated at scale, IT teams can be easily overwhelmed by the audit trail data. In order to gain the right insights with your audit log metrics data, you can adopt the following best practices:

Store all structures at scale

Establish a data platform that can integrate and store data of all structural formats at scale. Data platform technologies such as a data lake are commonly used to capture real-time log data streams with a schema-on-read consumption model.

Third-party analytics and monitoring tools are integrated to make sense of this information in real-time, while only processing the most relevant portions of audit logs data based on the tooling specifications for data structure.

Use statistical models, not predefined thresholds

Use statistical models to generalize system behavior instead of using predefined and fixed thresholds to capture data. Since the network behavior evolves continuously, models based on machine learning can continuously learn and adapt.

These models are helpful for accurate analysis of audit logs, where thresholds for anomalous behavior can be a moving target.

Secure data with eye to CIA triad

Store audit logging data in secure environments with high standards of confidentiality, integrity and availability — known as the CIA triad. Modified audit logs and misconfigured networking systems can generate misleading information, and likely lead your log analysis to incorrect conclusions.

Infinite data storage is not sustainable

Finally, it is important to understand that data stores that integrate large volumes of real-time log data streams can grow exponentially. When designing the data platform for audit log analysis, evaluate the cost, security and performance of your data platform against your security and compliance requirements.

(And remember: you don’t need this data forever and ever — it’s not sustainable.)

What is Splunk?

This posting does not necessarily represent Splunk's position, strategies or opinion.