This documentation does not apply to the most recent version of Splunk.
This documentation applies to the following versions of Splunk: 3.2 , 3.2.1
By default, Splunk recognizes most timestamps. However, if you find that Splunk does not extracting your timestamp correctly, you can train Splunk on a new timestamp format.
Please note: Training Splunk via the CLI is not the recommended method for specifying timestamps. Training should only be used if you cannot configure Splunk to recognize your timestamp format via props.conf.
Splunk's CLI provides an interactive time stamp training process that produces regex-based expressions to be added to your own custom datetime.xml and props.conf.
NOTE: Do not modify $SPLUNK_HOME/etc/datetime.xml. Instead, copy datetime.xml into your own custom bundle, or into $SPLUNK_HOME/etc/bundles/local/. Please read about bundle files before creating a custom bundle.
These commands assume you have set a Splunk environment variable. If you have not, you must navigate to $SPLUNK_HOME/bin and run the ./splunk command.
From the CLI, type:
# splunk train dates
You will be presented with the following interaction:
------------------------------------------------------
What operation do you want to perform? (default=learn)
------------------------------------------------------
Enter choice: [Learn]/Test/Quit > L
Choose "L" to train Splunk on timestamps.
Enter full filename from which to learn dates > /opt/dates.txt
Enter the entire path of the file on your Splunk server. This step will not tab-complete.
SAMPLE LINE 1:
Tue Jul 10 21:23:06 PDT 2007 Received Trade 330 with detail user: user3456 date: date: 10Jul200721:
23:06 action: sell 3583 MNAG @ 42
--------------------------------------------------------------------------------
If the above sample line does not have a timestamp, hit Enter.
If it does have a timestamp,
Enter timestamp values as: month, day, year, hour, minute, second, ampm, timezone.
> 7, 10, 2007, 9, 23, 06, pm, PDT
Enter the values as noted above.
Patterns Learned. Manually add these to '/opt/splunk/etc/datetime.xml'
and add pattern names to timePatterns and datePatterns.
When the interaction is finished running, it will output a string like so:
<define name="_utcepoch" extract="utcepoch">
<text><![CDATA[((?<=^|[\s#,"=\(\[\|\{])(?:1[01]|9)\d{8}|^@[\da-fA-F]{16,24})(?:\d{3})?(?![\d\(])]]></text>
</define>
This string must be pasted into your custom datetime.xml, before the <timePatterns> stanza.
The define name must be added as a use name to both the <timePatterns> and <datePatterns> stanzas:
<timePatterns>
<use name="_time"/>
<use name="_hmtime"/>
<use name="_hmtime"/>
<use name="_dottime"/>
<use name="_combdatetime"/>
<use name="_utcepoch"/>
</timePatterns>
<datePatterns>
<use name="_usdate"/>
<use name="_isodate"/>
<use name="_eurodate"/>
<use name="_bareurlitdate"/>
<use name="_orddate"/>
<use name="_combdatetime"/>
<use name="_masheddate"/>
<use name="_masheddate2"/>
</datePatterns>
Edit props.conf in $SPLUNK_HOME/etc/bundles/local and add a stanza for your source, host or sourcetype to specify the new datetime.xml file you have just created. You will need to add a DATETIME_CONFIG value to the stanza, specifying the path to your new datetime.xml.
<spec> DATETIME_CONFIG=$YOUR_CUSTOM_PATH
<spec> can be:
$YOUR_CUSTOM_PATH is the location of your new datetime.xml file.
Please note: when specifying the path, set it relative to $SPLUNK_HOME.
For example:
[host::london] DATETIME_CONFIG = /etc/bundles/local/datetime.xml
This example will apply your new datetime.xml to all events coming from the host with value london. You can set custom timestamp extraction patterns for any host, source or sourcetype by editing props.conf in this way.