Create indexed fields via configuration files

Create indexed fields via configuration files

Splunk automatically adds indexed fields such as host, source, source type, event type, etc. Create your own custom indexed fields. Once you have created a new indexed field, it appears in the Fields drop-down menu in Splunk Web. You can also search on it, as well, by typing $CUSTOM_FIELD=foo in your search.

Configuration

Define additional search fields by editing props.conf, transforms.conf and fields.conf.

Edit these files in $SPLUNK_HOME/etc/bundles/local or your own custom bundle directory.

transforms.conf

Add the following lines to $SPLUNK_HOME/etc/bundles/local/transforms.conf:

[$UNIQUE_STANZA_NAME]
REGEX = $YOUR_REGEX
FORMAT = $0 $YOUR_CUSTOM_FIELD_NAME::$1
DEST_KEY = _meta

• $UNIQUE_STANZA_NAME = name your stanza. Use this name later to configure props.conf.
• REGEX = create a regex that recognizes your custom field value.
• FORMAT = $0 refers to the event up to the regex; $1 refers to the event from the regex on. This ensures the correct formatting of your events.
  • In order to properly display field values containing whitespace in Splunk Web, apply quotes to the FORMAT key.
  • FORMAT = $YOUR_CUSTOM_FIELD_NAME::"$1"
• DEST_KEY = set this to _meta to write your field name and value to meta. This is where indexed fields are stored.

Note: In order to preserve previous matching extractions, include a $0 in the FORMAT key. If you don't include $0, the previously extracted fields will be erased and only the last matching extraction specified in transforms.conf will be kept.

props.conf

Add the following lines to $SPLUNK_HOME/etc/bundles/local/props.conf:

[<spec>]
TRANSFORMS-$VALUE = $UNIQUE_STANZA_NAME

• <spec> can be:
  • <sourcetype>, the sourcetype of an event.
  • host=<host>, where <host> is the host for an event.
  • source=<source>, where <source> is the source for an event.
• $UNIQUE_STANZA_NAME is the name of your stanza from transforms.conf.
• $VALUE is any value you want to give to your stanza to identify its name-space.

fields.conf

Add an entry to fields.conf for your new indexed field.

[$YOUR_CUSTOM_FIELD_NAME]
indexed=true

• $YOUR_CUSTOM_FIELD_NAME is the name of the custom field you set in transforms.conf.
• Set indexed=true to indicate that the field is indexed.

Example

This example creates an indexed field called err_code.

transforms.conf

In $SPLUNK_HOME/etc/bundles/local/transforms.conf add:

[netscreen-error]
REGEX =  device_id=[^ ]+\s+\[w+\](.*)(?
FORMAT = $0 err_code::$1
DEST_KEY = _meta 

This stanza takes 'device_id=' followed with a word within brackets and a text string terminating with a colon. The source type of the events is testlog.

Comments:

• The FORMAT = line contains three values:
  • $1 refers to the new field written to the index. It is the value extracted by REGEX.
  • err_code:: is the name of the field.
  • $0 refers to the content before the regex.
• DEST_KEY = _meta is an instruction to write the content of FORMAT to the index.

props.conf

Add the following lines to $SPLUNK_HOME/etc/bundles/local/props.conf:

[testlog]
TRANSFORMS-netscreen = netscreen-error

fields.conf

Add the following lines to $SPLUNK_HOME/etc/bundles/local/fields.conf:

[err_code]
indexed=true