Documentation:
3.3
This page last updated: 09/03/08 02:09pm
Configure inputs via inputs.conf
Data inputs added via inputs.conf can be more detailed than inputs enabled via SplunkWeb or the CLI.
Note: To set dynamic indexing properties for inputs, use props.conf.
ConfigurationAdd your stanza to $SPLUNK_HOME/etc/system/local/inputs.conf. Specify an input type and any number of attribute/value pairs.
[<inputtype>://<path>] attribute1 = val1 attribute2 = val2 ...
The following attributes/value pairs are valid for ALL input types
host = <string>
- Set the host value of your input to a static value.
- "host =" is automatically prepended to the value when this shortcut is used.
index = <string>
- Set the index where events from this input will be stored.
- "index =" is automatically prepended to the value when this shortcut is used.
source = <string>
- Set the source name of events from this input.
- "source =" is automatically prepended to the value when this shortcut is used.
sourcetype = <string>
- Set the sourcetype name of events from this input.
- "sourcetype =" is automatically prepended to the value when this shortcut is used.
queue = <string> (parsingQueue, indexQueue, etc)
- Specify where the input processor should deposit the events that it reads.
- Can be any valid, existing queue in the pipeline.
The following attributes/value pairs are valid for the specified input types only.
monitor[monitor://<path>]
This directs Splunk to watch all files in the <path> (or just <path> itself if it represents a single file). You must specify the input type and then the path, so put three slashes in your path if you're starting at root. You can use wildcards for the path; see below.
Note: To ensure new events are indexed when you copy over an existing file with new contents, set CHECK_METHOD = modtime in props.conf for the source. This checks the modtime of the file and re-indexes when it changes. Note that the entire file is indexed, which can result in duplicate events.
wildcardsYou can use wildcards to specify your input path for monitored input. Use ... for paths and * for files.
- ... recurses through directories until the match is met. This means that /foo/.../bar will match foo/bar, foo/1/bar, foo/1/2/bar, etc. but only if bar is a file.
- To recurse through a subdirectory, use another .... For example /foo/.../bar/....
- * matches anything in that specific path segment. It cannot be used inside of a directory path; it must be used in the last segment of the path. For example /foo/*.log matches /foo/bar.log but not /foo/bar.txt or /foo/bar/test.log.
- Combine * and ... for more specific matches:
- foo/.../bar/* matches any file in the bar directory within the specified path.
Note: In Windows, you must use two backslashes \\ to escape wildcards. Regexes with backslashes in them are not currently supported for _whitelist and _blacklist in Windows.
Specifying wildcards results in an implicit _whitelist created for that stanza. The longest fully qualified path is used as the monitor stanza, and the wildcards are translated into regular expressions using the following map:
| wildcard | regex | meaning |
| * | [^/]* | anything but / |
| ... | .* | anything (greedy) |
| . | \. | literal . |
For example, if you specify
[monitor:///foo/bar*.log]
Splunk translates this into
[monitor:///foo/] _whitelist = bar[^/]*\.log
As a consequence, you can't have multiple stanzas with wildcards for files in the same directory.
For example:
[monitor:///foo/bar_baz*] [monitor:///foo/bar_qux*]
This results in overlapping stanzas indexing the directory /foo/. Splunk takes the first one, so only files starting with /foo/bar_baz will be indexed. To encompass both sources, manually specify a _whitelist using regular expression syntax for "or":
[monitor:///foo] _whitelist = (bar_baz[^/]*|bar_qux[^/]*)
Note: To set any additional attributes (such as sourcetype) for multiple whitelisted/blacklisted inputs that may have different attributes, use props.conf
additional attributeshost_regex = <regular expression>
- If specified, the regex will extract host from the filename of each input.
- Specifically, the first group of the regex is used as the host.
- If the regex fails to match, the default "host =" attribute is used.
host_segment = <integer>
- If specified, the '/' separated segment of the path will be set as host.
- If the value is not an integer, or is less than 1, the default "host =" attribute is used.
crcSalt = <string>
- If set, this string will be added to the CRC.
- This can be used to force Splunk to consume files that have matching CRCs.
- If set to "crcSalt = <SOURCE>", then the full source path is added to the CRC.
followTail = 0|1
- If set to 1, monitoring will begin at the end of the file (like tail -f).
- This only applies to files the first time they are picked up.
- After that, Splunk's internal file position records keep track of the file.
_whitelist = <regular expression>
- If set, files from this path are monitored only if they match the specified regex.
_blacklist = <regular expression>
- If set, files from this path are NOT monitored if they match the specified regex.
[batch://<path>] move_policy = sinkhole.
- One time, destructive input.
- For continuous, non-destructive inputs, use monitor.
- Important = You must set move_policy = sinkhole. This loads the file destructively. Do not use this input type for files you do not want to consume destructively.
host_regex (see monitor)
host_segment (see monitor)
Note: source = <string> and <KEY> = <string> are not used by batch.
TCP[tcp://<remote server>:<port>]
- Configure Splunk to listen to <remote server> on <port>.
- If <remote server> is blank, this stanza matches all connections on the specified port.
connection_host = [ip | dns]
- If "ip" is set, the TCP input processor will rewrite the host with the ip address of the remote server.
- If "dns" is set, the host will be rewritten with the DNS entry of the remote server.
- Defaults to ip.
[udp://:<port>]
- Similar to TCP, except that it listens on a UDP port.
_rcvbuf = <int>
- Specify the receive buffer for the UDP port.
- If the value is 0 or negative, it will be ignored.
- The default value for Splunk is 1MB (the default in the OS varies).
no_priority_stripping = <value>
- If this attribute is set to any value, then Splunk does NOT strip the <priority> syslog field from received events.
- This key should NOT be included if you want to strip <priority>.
[fifo://<path>]
- This directs Splunk to read from a FIFO at the specified path.
[script://<cmd>]
- Will run the command "cmd" at a configured interval and index the output.
- The command must reside in $SPLUNK_HOME/etc/apps/$APPLICATION/bin directory.
interval = <integer>
- How often to execute the specified command (in seconds).
- If interval is not specified, it will default to 60 seconds.
passAuth = <username>
- User that this script should be run as.
- If a username is provided, Splunk generates an auth token for that user and passes it to the script via stdin.
[monitor:///apache/.../logs]
This loads anything in /apache/foo/logs or /apache/bar/logs, etc.
[monitor:///apache/*.log]
This loads anything in /apache/ that ends in .log.
Batch[batch://system/flight815/*] move_policy = passive_symlink
This example batch loads all files from the directory /system/flight815/.
TCP[tcp://<remote server>:<port>]
This configures Splunk to listen on the specified port. If a connection is made from <remote server>, this stanza is used to configure the input.
If <remote server> is blank, this stanza matches all connections on the specified port.
[udp://<remote-server>:<port>]
Similar to TCP, except that Splunk listens on a UDP port. FIFO
[fifo://<path>]
This directs Splunk to read from the FIFO at the specified path.
Comments
Splunk will automatically merge all the inputs.conf files it finds in the appropriate directories, so yes your inputs.conf will merge with the one in local.
Posted by emma on Jul 09 2008, 9:40am
I just install the unix addon into etc/apps/unix. there is inputs.conf file in unix dir. should this inputs.conf merged with etc/system/local/inputs.conf?
Posted by saga99 on Jul 09 2008, 1:07am