Common Information Model

Splunk's application standard involves three log file components:

• fields
• event type tags
• host tags

Using these three components, a developer can create log files that require minimal effort to parse by Splunk, and a user can normalize noncompliant log files so they all follow the same schema. This document details the standard fields, event type tags, and host tags that Splunk uses in writing add-ons to process IT data.

Event Format

When generating or writing events in a system, the following is the recommended format that should be used:

  <timestamp> name="<name>" event_id=<event_id> <key>=<value>

Any number of key-value pairs are allowed. For example:

  2008-11-06 22:29:04 name="Failed Login" event_id=sshd:failure src_ip=10.2.3.4 src_port=12355 dest_ip=192.168.1.35 dest_port=22

The keys are ones that are listed in the "Standard fields below". name and event_id are mandatory.

If CISCO PIX was compliant with this format, the following PIX event:

  Sep  2 15:14:11 10.235.224.193 local4:warn|warning fw07 %PIX-4-106023: Deny icmp src internet:213.208.19.33 dst eservices-test-ses-public:193.8.50.70 (type 8, code 0) 
  by access-group "internet_access_in"

would look as follows:

  2008-09-02 15:14:11 name="Deny icmp" event_id=106023 vendor=CISCO product=PIX log_level=4 dvc_ip=10.235.224.193 dvs_host=fw07 syslog_facility=local4 syslog_priority=warn 
  src_ip=213.208.19.33 dest_ip=193.8.50.70 src_network=internet dest_network=eservices-test-ses-public icmp_type=8 icmp_code=0 proto=icmp rule_number="internet_access_in"

Standard fields

The following fields can be used for field extractions.

Please note that we strongly recommend that all of these field extractions be performed at search time. There is no need to add these fields to the set of default fields that Splunk extracts at index time. For more information, see "Create fields via Splunk Web" or "Create fields via configuration files" in the Admin manual.

field name	data type	Explanation
action	string	The action specified by the event. For example, access, execution, or modification.
affected_user	string	The user that was affected by a change. E.g., user ram changed the name of user rmarty, rmarty is the affected_user.
affected_user_group	string
affected_user_group_id	string
affected_user_id	number
affected_user_privileges	enumeration
app	string	ISO layer 7 (application layer) protocol--e.g. HTTP, HTTPS, SSH, IMAP.
bytes_in	number	How many bytes this device/interface received.
bytes_out	number	How many bytes this device/interface transmitted.
channel	string	802.11 channel number used by a wireless network.
category	string	A device-specific classification provided as part of the event.
count	number	The number of times the record has been seen.
cve	string	Common Vulnerabilities and Exposures (CVE) reference value.
dest_country	string	The country associated with a packet's recipient.
dest_host	string	Fully qualified host name of a packet's recipient. For HTTP sessions, this is the Host header.
dest_ip	ipv4 address	IPv4 address of a packet's recipient.
dest_ipv6	ipv6 address	IPv6 address of a packet's recipient.
dest_lat	number	(Physical) latitude of a packet's destination.
dest_long	number	(Physical) longitude of a packet's destination.
dest_mac	mac address	Destination TCP/IP layer 2 Media Access Control (MAC) address of a packet's destination.
dest_nt_domain	string	The Windows NT domain that contains a packet's destination.
dest_nt_host	string	The Windows NT host name of a packet's destination.
dest_port	port	The TCP/IP port to which a packet is being sent.
dest_translated_ip	ipv4 address	The NATed IP address to which a packet is being sent.
dest_translated_port	number	The NATed port to which a packet is being sent.
direction	string	The direction the packet is traveling, such as inbound or outbound.
duration	number	The amount of time the event lasted.
dvc_host	string	Fully qualified domain name of the device transmitting or recording the log record.
dvc_ip	ipv4 address	IPv4 address of the device reporting the event.
dvc_ip6	ipv6 address	IPv6 address of the device reporting the event.
dvc_location	string	Free-form description of the device's physical location.
dvc_mac	MAC address	MAC (layer 2) address of the device reporting the event.
dvc_nt_domain	string	Windows NT domain of the device recording or transmitting the event.
dvc_nt_host	string	Windows NT host name of the device recording or transmitting the event.
dvc_time	timestamp	Time at which the device recorded the event.
end_time	timestamp	The event's specified end time.
event_id	number	A unique identifier that identifies the event. This is unique to the reporting device.
file_access_time	timestamp	The time the file (the object of the event) was accessed.
file_create_time	timestamp	The time the file (the object of the event) was created.
file_hash	string	A cryptographic identifier assigned to the file object affected by the event.
file_modify_time	timestamp	The time the file (the object of the event) was altered.
file_name	string	The name of the file that is the object of the event, with not information related to local file or directory structure.
file_path	string	The location of the file that is the object of the event, in terms of local file and directory structure.
file_permission	string	Access controls associated with the file affected by the event.
file_size	number	The size of the file that is the object of the event. Indicate whether Bytes, KB, MB, GB.
http_content_type	string	The HTTP content type.
http_method	string	The HTTP method used in the event.
http_referrer	string	The HTTP referrer listed in the event.
http_response	number	The HTTP response code.
http_user_agent	string	The HTTP user agent.
inbound_interface	string	The network interface through which a packet was received.
log_level	string	The log-level that was set on the device and recorded in the event.
name	string	Name of the event as reported by the device. The name should not contain information that's already being parsed into fields from the event, such as IP addresses.
object_name	string	Especially under Windows, the object name.
object_type	string	Especially under Windows, this indicates the type of object.
object_handle	string	Especially used under Windows to indicate the object handle.
outbound_interface	string	The network interface through which a packet was transmitted.
packets_in	number	How many packets this device/interface received.
packets_out	number	How many packets this device/interface transmitted.
pid	number	Integer assigned by the device operating system to the process creating the record.
priority	number	Environment-specific assessment of the importance of the event, based on elements such as event severity, business function of the affected system, or other locally defined variables.
process	string	The program that generated this record (e.g., process name mentioned in syslog header).
product	string	The product that generated the event.
product_version	number	The version of the product that generated the event.
proto	string	OSI layer 3 (network layer) protocol--e.g. IP, ICMP, IPsec, ARP.
recipient	string	The person to whom an email message is sent.
rule_number	string	For example the firewall rule-number or ACL number.
sender	string	The person responsible for sending an email message.
severity	string	The severity (or priority) of an event as reported by the originating device.
signature	string	SID, as well as the signature identifiers used by other Intrusion Detection Systems; the Event Identifiers assigned by Windows-based operating systems to event records; and Cisco's message IDs.
src_country	string	Country from which the packet was sent.
src_host	string	Fully qualified host name of the system that transmitted the packet. For Web logs, this is the http client.
src_ip	ipv4 address	IPv4 address of the packet's source. For Web logs, this is the http client.
src_ipv6	ipv6 address	IPv6 address of the packet's source.
src_lat	number	(Physical) latitude of the packet's source.
src_long	number	(Physical) longitude of the packet's source.
src_mac	mac address	Media Access Control (MAC) address from which a packet was transmitted.
src_nt_domain	string	The Windows NT domain containing the machines that generated the event.
src_nt_host	string	The Windows NT hostname of the system that generated the event.
src_port	port	The network port from which a packet originated.
src_translated_ip	ip address	The translated/NAT'ed IP address from which a packet is being sent.
src_translated_port	number	The translated/NAT'ed network port from which a packet is being sent.
session_id	string	Identifier for a session. Multiple transactions build a session.
ssid	string	The 802.11 service set identifier (ssid) assigned to a wireless network.
start_time	timestamp	The event's specified start time.
subject	string	Email subject line.
syslog_facility	syslog facility	The application, process, or OS subsystem that generated the event.
syslog_priority	syslog priority	The criticality of an event, as recorded by UNIX syslog.
tcp_flags	enumeration	The TCP flag specified in the event. One or more of SYN, ACK, FIN, RST, URG, or PSH.
transaction_id	string	Identifier for a transaction.
transport	string	The transport protocol, such as TCP, UDP.
url	string	A Web address (Uniform Record Locator, or URL) included in a record.
user	string	The login ID affected by the recorded event.
user_group	string	User group that is the object of an event, expressed in human-readable terms.
user_group_id	string	The numeric identifier assigned to the user group object of an event.
user_id	number	System-assigned numeric identifier for the user affected by an event.
user_privilege	enumeration	The security context associated with the object of an event: one of administrator, user, or guest/anonymous.
user_subject	string	User that is the subject of an event. The one executing the action.
user_subject_id	number	ID number of the user that is the subject of an event. The one executing the action.
user_subject_privilege	enumeration	The security context associated with a recorded event: one of administrator, user, or guest/anonymous.
vendor	string	The vendor who made the product that generated the event.
vlan_id	number	The numeric identifier assigned to the virtual local area network specified in the record.
vlan_name	string	The name assigned to the VLAN in the event.

Standard event type tags

Every eventtype should have a single tag assigned from each of three categories: object, action, and status. This combination of three tags allows for precise type classification. The object denotes what the event is about. What object has been targeted. Is the event talking about a host, a resource, a file, etc. The action talks about what has been done to the object: create, delete, modify. And finally the status denotes whether the action was successful, failed, or was simply an attempt. In addition to these three tags, you can add your own standard or custom tags.

The three tags in discussion here are:

<objecttag> <actiontag> <statustag>

Some examples of using the standard tags are:

• For a firewall deny event:

  host communicate firewall failure

• For a firewall accept event:

  host communicate firewall success

• For a successful database login:

  database authentication verify success

Object event type tags

Use one of these object tags in the first position as defined above.

Tag	Explanation
application	An application-level event.
application av	An anti virus event.
application backdoor	An event using an application backdoor.
application database	A database event.
application database data	An event related to database data.
application dosclient	An event involving a DOS client.
application firewall	An event involving an application firewall.
application im	An instant message-related event.
application peertopeer	A peer to peer-related event.
host	A host-level event.
group	A group-level event
resource	An event involving system resources.
resource cpu	An event involving the CPU.
resource file	An event involving a file.
resources interface	An event involving network interfaces.
resource memory	An event involving memory.
resource registry	An event involving the system registry.
os	An OS-level event.
os process	An event involving an OS-related process
os service	An event involving an OS service.
user	A user-level event

Action event type tags

Use one of these action tags in the second position as defined above.

Tag	Explanation
access	An event that accesses something.
access read	An event that reads something.
access read copy	An event that copies something.
access read copy archive	An event that archives something.
access read decrypt	An event that decrypts something.
access read download	An event that downloads something.
access write	An event that writes something.
authentication	An event involving authentication.
authentication add	An event adding authentication rules.
authentication delete	An event deleting authentication rules.
authentication lock	An event indicating an account lockout.
authentication modify	An event modifying authentication rules.
authentication verify	An event verifying identity.
authorization	An event involving authorization.
authorization add	Adding new priviliges.
authorization delete	Deleting privileges.
authorization modify	Changing privileges, e.g., chmod.
authorization verify	Checking privileges for an operation.
check	An event checking something.
check status	An event checking something's status.
create	An event that creates something.
communicate	An event involving communication.
communicate connect	An event involving making a connection.
communicate disconnect	An event involving disconnecting.
communicate firewall	An event passing through a firewall.
delete	An event that deletes something.
execute	An event that runs something.
execute restart	An event that restarts something.
execute start	An event that starts something.
execute stop	An event that stops something.
modify	An event that changes something.
modify attribute	An event that changes an attribute.
modify attribute rename	An event that renames something.
modify configuration	An event that changes a configuration.
modify content	A content-related event.
modify content append	An event that appends new content onto existing content.
modify content clear	An event that clears out content.
modify content insert	An event that inserts content into existing content.
modify content merge	An event that merges content.
substitute	An event that replaces something.

Status event type tags

Use one of these status tags in the third position as defined above.

Tag	Explanation
attempt	An event marking an attempt at something.
deferred	A deferred event.
failure	A failed event.
inprogress	An event marking something progress.
report	A report of a status.
success	A successful event.

Optional tags

For those who want to use standard additional tags when they apply, some suggestions are below.

Tag	Explanation
attack	An event marking an attack.
attack exploit	An event marking the use of an exploit.
attack bruteforce	An event marking a brute force attack.
attack dos	An event marking a denial of service attack.
attack escalation	An event indicating a privilege escalation attack.
infoleak	An event indicating an information leak.
malware	An event marking malware action.
malware dosclient	An event marking malware utilizing a DOS client.
malware spyware	An event marking spyware.
malware trojan	An event marking a trojan.
malware virus	An event marking a virus.
malware worm	An event marking a worm.
recon	An event marking recon probes.
suspicious	An event indicating suspicious activity.

Standard host tags

Tagging hosts allows you to set metatdata about particular boxes and what they're doing. There are different approaches to tagging hosts, all of which can be used where appropriate. Some of these methods include:

• What service(s) the host is running.
• What OS the host is running.
• The department the host belongs to.
• What data the host contains.
• What cluster/round robin the host belongs to.

General host tags

Some host tags are useful across the board.

Tag	Explanation
db	This host is a database.
development	This host is a development box.
dmz	This host is in the DMZ.
dns	This host is a DNS server.
email	This host is an email server.
finance	This host contains financial information.
firewall	This host is a firewall.
highly_critical	This host is highly critical for business purposes.
web	This host is a Web server.

PCI application host tags

The PCI application extends the tag space with specific tags for hosts.

Change Management application host tags

The Change Management application extends the tag space with specific tags for hosts.

Network Security application tags

The Network Security application extends the tag space with specific tags. The tags are for example used to identify IDS events.