Documentation: 3.0.2
|  Print Version Contents
This page last updated: 02/08/08 10:02am

Define additional extracted fields

Splunk automatically extracts fields during searches using known keywords for the source type and name/value pairs in the events. You can examine the extracted fields in SplunkWeb by clicking the Fields... link above the event display:

http://www.splunk.com/assets/doc-images/30_admin_DefineExtractedField/fields.jpg

You can also add additional extracted fields. Use these instructions to configure custom extracted fields.

Configuration

To create additional extracted fields, you will edit transforms.conf and props.conf.

transforms.conf

Add the following lines to $SPLUNK_HOME/etc/bundles/local/transforms.conf:

[$UNIQUE_STANZA_NAME]
REGEX = $YOUR_REGEX
FORMAT = $YOUR_CUSTOM_FIELD_NAME::$1
  • $UNIQUE_STANZA_NAME = name your stanza. You will use this name later in configuring props.conf.
  • REGEX = create a regex that will recognize your custom field value.
  • FORMAT = $YOUR_CUSTOM_FIELD_NAME will be the name of your extracted field; $1 is the value specified by the regular expression.
    • In order to properly display field values containing whitespace in SplunkWeb, you must apply quotes to the FORMAT key.
      • FORMAT = $YOUR_CUSTOM_FIELD_NAME::"$1"

Note: In order to preserve previous matching extractions, you need to include a $0 in the FORMAT key. If you don't include $0, the previously extracted fields will be erased and only the last matching extraction specified in transforms.conf will be kept.

Please note: Unlike configuring search fields, extracted field transforms.conf requires no DEST_KEY since nothing is being written to the index. The field is extracted at search time and is not persisted in the index as a key.

props.conf

Add the following lines to $SPLUNK_HOME/etc/bundles/local/props.conf:

[<spec>]
REPORT-$VALUE = $UNIQUE_STANZA_NAME

<spec> can be:
1. <sourcetype>, the sourcetype of an event
2. host::<host>, where <host> is the host for an event
3. source::<source>, where <source> is the source for an event

$UNIQUE_STANZA_NAME is the name of your stanza from props.conf.
$VALUE is any value you want to give to your stanza to identify its name-space.

To display only your explicitly configured extracted fields and not the automatically recognized ones, add KV_MODE = none to your stanza in props.conf.

Please note: Extracted fields props.conf uses REPORT-$VALUE as opposed to TRANSFORMS-$VALUE used in configuring search fields.

Example

In this example, there is an error field that we wish to extract. The field can be identified by the occurrence of device_id= followed by a word within brackets and a text string terminating with a colon. The source type of the events is testlog.

In transforms.conf add:

[netscreen-error]
REGEX = device_id=[^ ]+\s+\[w+\](.*)(?
FORMAT = err_code::$1

In props.conf add:

[testlog]
REPORT-netscreen = netscreen-error

Previous: Create additional search fields    |    Next: Field actions

Comments

  1. thanks for pointing that out -- kv_mode is indeed an attribute in props.conf (not transforms).
    Posted by emma on Feb 08 2008, 10:23am

  2. I believe KV_MODE is a props.conf parameter and not a transforms.conf parameter as suggested above.
    Posted by gfoden on Feb 08 2008, 3:54am

  3. In order to get the extracted fields defined by using "splunk train" to work, I had to:

    ln -s /opt/splunk/etc/bundles/local/regexes.conf /opt/splunk/etc/bundles/local/transforms.conf
    Posted by wijgalt on Aug 13 2007, 5:55pm

Log in to comment.