Whitelist and blacklist rules

When specifying inputs to monitor in inputs.conf, you can use whitelist and blacklist rules to explicitly tell Splunk to consume ONLY certain files or consume everything EXCEPT certain files. When you define a whitelist, Splunk indexes ONLY the files in that list. Alternately, when you define a blacklist, Splunk ignores the files in that list and consumes everything else. These settings are independent of each other.

Whitelist and blacklist rules use regular expression syntax to define the match on the file name. Also, your rules must be contained within a configuration stanza, for example [monitor://<path>]; those outside a stanza (global entries) are ignored.

Important: Define whitelist and blacklist entries with exact regex syntax; the "..." wildcard is not supported.

Note: We recommend that you blacklist all compressed files.

Note: Instead of whitelisting or blacklisting your data inputs, you can filter specific events and send them to different queues or indexes. Read more about filtering and routing events to different queues and filtering and routing events to alternate indexes. You can also use the crawl feature to predefine files you want Splunk to index or not index automatically when they are added to your filesystem.

Whitelist (allow) files

To define the files you want Splunk to exclusively index, add the following line to your monitor stanza in $SPLUNK_HOME/etc/system/local/inputs.conf:

_whitelist = $YOUR_CUSTOM_REGEX

For example, if you want Splunk to monitor only files with the .log extension:

[monitor:///mnt/logs]
    _whitelist = .*\.log$

You can whitelist multiple files in one line, using the "|" (OR) operator. For example, to whitelist filenames that contain query.log OR my.log:

_whitelist = query\.log$|my\.log$

Or, to whitelist exact matches:

_whitelist = /query\.log$|/my\.log$

Note: The "$" anchors the regex to the end of the line. There is no space before or after the "|" operator.

Blacklist (ignore) files

To define the files you want Splunk to exclude from indexing, add the following line to your monitor stanza in $SPLUNK_HOME/etc/system/local/inputs.conf:

_blacklist = $YOUR_CUSTOM_REGEX

Important: If you create a _blacklist line for each file you want to ignore, Splunk activates only the last filter.

If you want Splunk to ignore and not monitor only files with the .txt extension:

[monitor:///mnt/logs]
    _blacklist = \.(txt)$

If you want Splunk to ignore and not monitor all files with either the .txt extension OR the .gz extension (note that you use the "|" for this):

[monitor:///mnt/logs]
    _blacklist = \.(txt|gz)$

Verify your lists

To verify that your whitelist and blacklist rules are configured properly, run the listtails utility found in your $SPLUNK_HOME/bin directory. listtails reads in the configuration of inputs.conf in all application directories, scans the directories, and displays an exact list of files that Splunk will monitor when you restart.

In your $SPLUNK_HOME/bin directory, run:

./splunk cmd listtails

Previous: Scripted inputs | Next: Crawl

Comments

No comments have been submitted.