This documentation does not apply to the most recent version of Splunk.
This documentation applies to the following versions of Splunk: 2.1 , 2.2 , 2.2.1 , 2.2.3 , 2.2.6
Another way to increase indexing performance is to disable the parsing of minor segments and only index major segments - for example to index spaghetti.monster@support.splunk.com but not each separate part of the address. In this example, we'll reduce indexing density for sourcetype myApp.
The best way to create you own cleaners files is to copy either cleaners.xml or majorOnly_cleaners.xml and edit/remove entries as necessary.
These files are xml files that have the following format:
/* sample breaking separator config */
<config>
<method>separator</method>
<fieldName>_raw</fieldName>
<trailingPrefix>trail_brk</trailingPrefix>
<breakingSeparators>
<breakingSeparator name="lsqrBracketBreak" isMajor="1">
<value>[</value>
</breakingSeparator>
<breakingSeparator name="rsqrBracketBreak" isMajor="1">
<value>]</value>
</breakingSeparator>
<...>
</config>
At the top of the file the tag <defaultCleaningConfig> is used to specify if none are specified.