Splunk User Manual (Splunk v2.0)

Training Splunk

Source Types

Splunk can be trained to recognize the source types of your local data. You show it sample files and tell it what source type they are.

# splunk classify [filename] [sourcetype]

To see what source type Splunk thinks a data sample is, just omit the final parameter.

# splunk classify [filename]

Example

Splunk normally recognizes Linux syslog files, but what if it doesn't? In this example, we'll train it.

Splunk should recognize our syslog files, but an unusual event type or local customization confuses it. It decides the file type is unknown and assigns it messages-U8, which we can then find in unknown-props.xml. It gives us the cause too_small, saying it doesn't have enough events in the sample file to make a positive ID.

# splunk classify /var/log/messages.4
Using logging configuration at /opt/paul20/splunk/etc/log-cmdline.cfg.
WARN  PropertiesMapConfig - File classifier settings file (/opt/paul20/splunk/etc/overlay-props.xml) cannot be read.
PROPERTIES OF /var/log/messages.4
        Attr:AUTO_LINEMERGE     True
        Attr:AUTO_TAG   False
        Attr:BREAK_BEFORE_DATE  True
        Attr:BREAK_ONLY_BEFORE_DATE     False
        Attr:CLEANER_CONFIG     /etc/myinstall/pluginConfs/cleaners.xml
        Attr:DATETIME_CONFIG    /etc/datetime.xml
        Attr:MAX_EVENTS 256
        Attr:MAX_TIMESTAMP_LOOKAHEAD    150
        Attr:MUST_BREAK_AFTER
        Attr:MUST_BREAK_BEFORE  $\s*^
        Attr:MUST_NOT_BREAK_AFTER
        Attr:MUST_NOT_BREAK_BEFORE
        Attr:SHOULD_LINEMERGE   False
        Attr:TYPING_CONFIG      /etc/event-types/current/default.xml
        Attr:file_type  messages-U8
        Attr:invalid_cause      too_small
        Attr:is_valid   True
        Attr:maxDist    300

To resolve the issue, we tell Splunk that this is definitely a linux_messages_syslog file.

# splunk classify /var/log/messages.4 linux_messages_syslog
Using logging configuration at /opt/paul20/splunk/etc/log-cmdline.cfg.
WARN  PropertiesMapConfig - File classifier settings file (/opt/paul20/splunk/etc/overlay-props.xml) cannot be read.
Training on /var/log/messages.4 of type linux_messages_syslog

From now on Splunk will know what our local syslog files look like.

# splunk classify /var/log/messages.4
Using logging configuration at /opt/paul20/splunk/etc/log-cmdline.cfg.
WARN  PropertiesMapConfig - File classifier settings file (/opt/paul20/splunk/etc/overlay-props.xml) cannot be read.
PROPERTIES OF /var/log/messages.4
        Attr:AUTO_LINEMERGE     True
        Attr:AUTO_TAG   False
        Attr:BREAK_BEFORE_DATE  True
        Attr:BREAK_ONLY_BEFORE_DATE     False
        Attr:CLEANER_CONFIG     /etc/myinstall/pluginConfs/cleaners.xml
        Attr:DATETIME_CONFIG    /etc/datetime.xml
        Attr:MAX_EVENTS 256
        Attr:MAX_TIMESTAMP_LOOKAHEAD    150
        Attr:MUST_BREAK_AFTER
        Attr:MUST_BREAK_BEFORE  $\s*^
        Attr:MUST_NOT_BREAK_AFTER
        Attr:MUST_NOT_BREAK_BEFORE
        Attr:REGEXES    syslog-host
        Attr:SHOULD_LINEMERGE   False
        Attr:TYPING_CONFIG      /etc/event-types/current/syslog.xml
        Attr:file_type  linux_messages_syslog
        Attr:is_valid   True
        Attr:maxDist    100

Description:

Copyright © 2005-2008 Splunk Inc. All rights reserved.