inputs.conf

inputs.conf configures all inputs to Splunk including file and directory tailing and watching, network ports and scripted inputs.

For help configuring inputs via inputs.conf, see configure inputs via inputs.conf.

inputs.conf.spec

# Copyright (C) 2005-2008 Splunk Inc.  All Rights Reserved.  Version 3.0 

# This file contains possible attributes and values you can use to configure inputs,
# distributed inputs and file system monitoring in inputs.conf.
#
# There is an inputs.conf in $SPLUNK_HOME/etc/system/default/.  To set custom configurations, 
# place an inputs.conf in $SPLUNK_HOME/etc/system/local/.  For examples, see inputs.conf.example.
# You must restart Splunk to enable configurations.
#
# To learn more about configuration files (including precedence) please see the documentation 
# located at http://www.splunk.com/doc/latest/admin/BundlesIntro.
#

#******************************************************************************    
# GLOBAL SETTINGS:
# The following attributes/value pairs are valid for ALL input types (except fschange).
#******************************************************************************    

host = <string>
      * Set the default host to a static value.
     * "host::" is automatically prepended to the value when this shortcut is used.

index = <string>
      * Set the index to store events from this input.
      * "index::" is automatically prepended to <string>.

source = <string>
     * Set the source for events from this input. 
      * "source::" is automatically prepended to <string>.

sourcetype = <string>
     * Set the sourcetype for events from this input.
      * "sourcetype::" is automatically prepended to <string>.

queue = <string> (parsingQueue, indexQueue, etc)
       * Specify where the input processor should deposit the events that it reads.
       * Can be any valid, existing queue in the pipeline.

#******************************************************************************    
# Valid <inputtypes> follow, with input-specific attributes listed as well.
#******************************************************************************

#******************************************************************************    
# MONITOR:
#******************************************************************************    

[monitor://<path>]
      * This directs Splunk to watch all files in the <path>. 
      * <path> can be an entire directory or just a single file.
      * You must specify the input type and then the path, so put three slashes in your path if you're starting at the root.

# Additional attributes:

host_regex = <regular expression>
    * If specified, regex extracts host from the filename of each input. 
    * Specifically, the first group of the regex is used as the host. 
    * If the regex fails to match, the default "host =" attribute is used.

host_segment = <integer>
    * If specified, the '/' separated segment of the path is set as host. 
    * If the value is not an integer, or is less than 1, the default "host =" attribute is used.

_whitelist = <regular expression>
    * If set, files from this path are monitored only if they match the specified regex.

_blacklist = <regular expression>
    * If set, files from this path are NOT monitored if they match the specified regex.

Wildcards and monitor:
You can use wildcards to specify your input path for monitored input. Use ... for paths and * for files.
    * ... recurses through directories until the match is met. This means that /foo/.../bar will match foo/bar, foo/1/bar, foo/1/2/bar, etc. but only if bar is a file.
        * To recurse through a subdirectory, use another .... For example /foo/.../bar/....
    * * matches anything in that specific path segment. It cannot be used inside of a directory path; it must be used in the last segment of the path. For example /foo/*.log matches /foo/bar.log but not /foo/bar.txt or /foo/bar/test.log.
        * Combine * and ... for more specific matches:
        * foo/.../bar/* matches any file in the bar directory within the specified path.

crcSalt = <string>
    * Use this to force Splunk to consume files with matching CRCs.
    * Set any string to add to the CRC.
    * If set to "crcSalt = <SOURCE>", then the full source path is added to the CRC.
    
followTail = 0|1
    * If set to 1, monitoring begins at the end of the file (like tail -f).
    * This only applies to files the first time Splunk sees them.
    * After that, Splunk's internal file position records keep track of the file. 
 
dedicatedFD = 0|1
    * Dedicates a file descriptor to the input.
    * Only accepted if monitor path points to a file (as opposed to a directory). 
    * Set the available number of FDs in limits.conf.
      * Make sure you don't use up all the FDs as this may cause other data to be ignored. 
    * WARNING: This setting can drastically affect your Splunk install, as well as the server it is running on.
    Do NOT set unless you know exactly what you're doing.
    
    
    
#******************************************************************************    
# WINDOWS INPUTS:
#******************************************************************************        
# Windows platform specific input processor.
# These inputs are enabled by default.  To disable an input type, comment it out in 
# $SPLUNK_HOME\etc\system\local\inputs.conf.

[WinEventLog:Application] 
[WinEventLog:Security]
[WinEventLog:System]

#******************************************************************************    
# BATCH:
#******************************************************************************    

[batch://<path>]
    * One time, destructive input.
    * For continuous, non-destructive inputs, use **monitor**.
    
# Additional attributes:

move_policy = sinkhole.
    * Important = You must set move_policy = sinkhole.
    * This loads the file destructively.  
    * Do not use this input type for files you do not want to consume destructively.

host_regex (see monitor)
host_segment (see monitor)

# IMPORTANT: The following are not used by batch:
source = <string>
<KEY>  = <string>

#******************************************************************************    
# TCP: 
#******************************************************************************    

[tcp://<remote server>:<port>]
      * Configure Splunk to listen on a specific port. 
     * If a connection is made from <remote server>, this stanza is used to configure the input.
     * If <remote server> is blank, this stanza matches all connections on the specified port.

# Additional attributes:

connection_host = {ip,dns}
    * "ip" sets the TCP input processor to rewrite the host with the IP address of the remote server.
     * "dns" sets the host to the DNS entry of the remote server.
    * Defaults to ip.

#******************************************************************************
# Data distribution:
#******************************************************************************

[Splunktcp://<remote server>:<port>]
     * This is the same as TCP, except the remote server is assumed to be a Splunk server. 
     * For SplunkTCP, the host or connection_host will be used if the remote Splunk server does not 
     set a host, or if the host is set to host::localhost.
     * See documentation (http://www.splunk.com/doc/latest/admin/ForwardingReceiving) for help. 
    
    
##############    
# SSL settings for data distribution:

# If you are sending encrypted, cooked data, use the following stanza name:

[splunktcp-ssl:PORT]
    * Set PORT to the port your forwarder is sending cooked, encrypted data on.
    * Forwarder settings are set in outputs.conf on the forwarder-side.

# If you are sending encrypted, raw data, use the following stanza name:

[tcp-ssl:PORT]
    * Set PORT to the port your forwarder is sending raw, encrypted data on.
    * Forwarder settings are set in outputs.conf on the forwarder-side.

[SSL]
serverCert=<path>
    * Full path to the server certificate.
    
password=<string>
    * Server certificate password, if any.

rootCA=<string>
    * Certificate authority list (root file).

requireClientCert=<true|false>
    * Toggle whether it is required for a client to authenticate.
    
#******************************************************************************    
# UDP:
#******************************************************************************    

[udp://<port>]
      * Similar to TCP, except that it listens on a UDP port.

# Additional attributes:

_rcvbuf = <int>
     * Specify the receive buffer for the UDP port.  
     * If the value is 0 or negative, it will be ignored.  
     * The default value for Splunk is 1MB (the default in the OS varies).

no_priority_stripping = <value>
    * If this attribute is set to any value, then Splunk does NOT strip the <priority> syslog field 
    from received events. 
    * This key should not be included if you want to strip <priority>.
     
#******************************************************************************
# FIFO:
#******************************************************************************

[fifo://<path>]
    * This directs Splunk to read from a FIFO at the specified path.

#******************************************************************************
# Scripted Input:
#******************************************************************************

[script://<cmd>]
    * Will run the command "cmd" at a configured interval and index the output.  
    * The command must reside in $SPLUNK_HOME/etc/system/bin/ or ../etc/apps/$YOUR_APP/bin/.  

interval = <integer>
       * How often to execute the specified command (in seconds).
       * If interval is not specified, it will default to 60 seconds.

passAuth = <username>
    * User that this script should be run as. 
    * If a username is provided, Splunk generates an auth token for that user and passes it to the script via stdin.
    

#******************************************************************************
# File system change monitor
#******************************************************************************

[fschange:<path>]
    * Monitor all add/update/deletes to this directory and sub directories.
        * NOTE: <path> is the direct path.  You do not need to preface it with // like other inputs.
    * Sends an event for every change.
    * NOTE: You cannot simultaneously monitor a directory using fschange and monitor.

# Additional attributes:
# NOTE: fschange does not use the same attributes as other input types (above).  Use only the following attributes.

filters=<filter1>,<filter2>,...<filterN>
    * Each filter is applied left to right for each file or directory found during the monitors poll cycle. 
    * See "File System Monitoring Filters" below for help defining a filter.

recurse=<true | false>
    * If true, recurse directories within the directory specified in [fschange].
    * Defaults to true.

followLinks=<true | false>
    * Follow symbolic links if true. 
    * It is recommended that you do not set this to true or file system loops may occur. 
    * Defaults to false.

pollPeriod=N
    * Check this directory for changes every N seconds. 
    * Defaults to 3600.

hashMaxSize=N
    * Calculate a SHA256 hash for every file that is <= N size in bytes. 
    * This hash is used as an addional method for detecting change in the file/directory. 
    * Defaults to -1 (disabled).

fullEvent=<true | false>
    * Set to true to send the full event if an add or update change is detected. 
    * Further qualified by the 'sendEventMaxSize' attribute. 
    * Defaults to false.

sendEventMaxSize=N
    * Only send the full event if the size of the event is less than or equal to N bytes. 
    * This limits the size of indexed file data. 
    * Defaults to -1, which is unlimited.

signedaudit=<true | false>
    * Send cryptographically signed add/update/delete events.
    * If 'signedaudit' is true, then events will *always* get sent to the '_audit' index and will *always* have the sourcetype 'audittrail'.
    * If 'signedaudit' is false, then events will be placed in the main index and the source type will be whatever you specify (or 'fs_notification' by default).
    * NOTE: You MUST enable auditing in audit.conf.
    * Defaults to false.

index=<indexname>
    * The index to store all events generated.
    * Defaults to _audit.

sourectype=<string>
     * Set the sourcetype for events from this input.
      * "sourcetype::" is automatically prepended to <string>.
      
filesPerDelay=N
    * Injects a delay specified by 'delayInMills' after processing N files
    * This is used to throttle file system monitoring so it doesn't consume as much CPU

delayInMills=N
    * The delay in milliseconds to use after processing every N files as specified
      in the 'filesPerDelay' tag.
    * This is used to throttle file system monitoring so it doesn't consume as much CPU

 
# File System Monitoring Filters:

[filter:<filtertype>:<filtername>]
    * Define a filter of type 'filtertype' and name it 'filtername.'
    
<filtertype>
    * Filter types can be 'blacklist' or 'whitelist.' 
    * A whitelist filter processes all files names that match the regex list.
    * A blasklist filter skips all file names that match the regex list.

<filtername>
    * The filter name is used in the comma separated list when defining a file system monitor.
    
regexN=<regex>    
    * Blacklist and whitelist filters can include a set of regexes.
    * The name of each regex MUST be 'regexN', where N starts at 1 and increments. 
    * Each regex will be applied in numeric order:
        regex1=<regex>
        regex2=<regex>
        ...

inputs.conf.example

# Copyright (C) 2005-2008 Splunk Inc. All Rights Reserved. Version 3.0
#
# This is an example inputs.conf. Use this file to configure data inputs.
#
# To use one or more of these configurations, copy the configuration block into
# inputs.conf in $SPLUNK_HOME/etc/system/local/. You must restart Splunk to
# enable configurations.
#
# To learn more about configuration files (including precedence) please see the documentation
# located at http://www.splunk.com/doc/latest/admin/BundlesIntro.

# This configuration directs Splunk to read all the files in the directory /var/log.

[monitor:///var/log]

# This configuration directs Splunk to read all the files under /var/log/httpd and classify them
# as sourcetype::access_common.

[monitor:///var/log/httpd]
sourcetype = access_common

# This configuration directs Splunk to read all the files under /mnt/logs. When the path is
# /mnt/logs/<host>/... this will set the hostname (by file) to <host>.

[monitor:///mnt/logs]
host_segment = 3

# This configuration directs Splunk to listen on TCP port 9997 for raw data from ANY remote server
# (not just a Splunk instance). The host of the data is set to the IP address of the remote server.

[tcp://:9997]

# This configuration directs Splunk to listen on TCP port 9995 for raw data from ANY remote server.
# The host of the data is set as the host name of the remote server. All data will also be
# assigned the sourcetype "log4j" and the source "tcp:9995."

[tcp://:9995]
connection_host = dns
sourcetype = log4j
source = tcp:9995

# This configuration directs Splunk to listen on TCP port 9994 for raw data from 10.1.1.10.
# All data is assigned the host "webhead-1", the sourcetype "access_common" and the
# the source "//10.1.1.10/var/log/apache/access.log."

[tcp://10.1.1.10:9995]
host = webhead-1
sourcetype = access_common
source = //10.1.1.10/var/log/apache/access.log

# This configuration directs Splunk to listen on TCP port 9996 for distributed search data from ANY
# remote server. The data is delivered directly to the indexer on the local machine without any
# further processing. The host of the data is set to the host name of the remote server ONLY
# IF the remote data has no host set, or if it is set to "localhost."

[splunktcp://:9996]
queue = indexQueue
connection_host = dns

# This configuration directs Splunk to listen on TCP port 9998 for distributed search data from
# 10.1.1.100. The data is processed the same as locally indexed data.

[splunktcp://10.1.1.100:9996]

# This configuration directs Splunk to listen on TCP port 514 for data from
# syslog.corp.company.net. The data is assigned the sourcetype "syslog" and the host
# is set to the host name of the remote server.

[tcp://syslog.corp.company.net:514]
sourcetype = syslog
connection_host = dns

# Set up SSL:

[SSL]
serverCert=$SPLUNK_HOME/etc/auth/server.pem
password=password
rootCA=$SPLUNK_HOME/etc/auth/CAcert.pem
requireClientCert=false

[splunktcp-ssl:9996]

# Use file system change monitor:

[fschange:/etc/]
fullEvent=true
pollPeriod=60
recurse=true
sendEventMaxSize=100000
index=main

Previous: indexes.conf | Next: limits.conf

Comments

thanks -- you can actually use system/ or apps/, depending on whether you're going to package your scripted input as an application or not.
Posted by emma on Aug 15 2008, 4:40pm
For scripted inputs, the directory needs to change from bundles to apps.
Posted by on Aug 15 2008, 2:15pm