• This guide
  • About the Splunk Deployment Guide
• Splunk components
  • Components of a Splunk deployment
    • Indexer
    • Forwarder
    • Deployment server
  • Data inputs
    • File and directory inputs
    • FIFO queues
    • Network ports
    • Scripted inputs
    • Windows event logs and WMI
• Deployment options
  • Single index server deployment models
    • Splunk installed on existing aggregation host
    • Splunk with direct network inputs
    • Splunk installed on a host receiving batched IT data moves
    • Splunk indexing data on a remote mount / network storage
    • Splunk installed on all servers forwarding data
  • Multiple index server deployment options
    • Distributed search with data balancing
    • Data cloning for high availability
    • Data routing
    • Index and search tiers for massive scalability
• Deployment tuning
  • Splunk tuning factors
  • Hardware tuning factors
    • Input / Output
• Capacity planning
  • Splunk benchmarks
    • Benchmark test overview
    • Benchmark tests

Data inputs

Splunk supports five primary data input types - file and directory inputs, FIFO queues, network ports, scripted inputs, and Windows event logs.

File and directory inputs

Splunk can accept data input from local or mounted systems, and can read data through the use of the Splunk file input processor. The file input processor can operate in a variety of modes and is capable of reading entire files, updates to files, and real-time changes to files as well as performing those tasks on entire directory trees. Splunk supports whitelisting and blacklisting inside directory inputs for additional flexibility of configuration. Refer to the documentation about file and directory inputs for more information.

FIFO queues

A FIFO queue, or named pipe, is a queue of data maintained in a UNIX host's memory. It can be accessed like a file and log messages can be written to it. FIFO queues can offer higher indexing performance than using files as data inputs, but are vulnerable to data loss due to being implemented in memory and can also prevent certain applications from functioning due to their blocking nature. This blocking nature can also cause data loss as well. It is best to use FIFO queues for data inputs only if the unique nature of FIFOs is well understood by both the team responsible for the application generating the data and the team responsible for administering Splunk. Refer to the FIFO portion of the documentation about file and directory inputs for more information.

Network ports

Splunk can accept data from both UDP and TCP ports. While you can use this to mimic a local system syslogd, it is equally useful for capturing any other IT data via normal network mechanisms. Like FIFO queues, network ports can offer higher indexing performance, but with similar vulnerability to data loss. Although TCP-based network communication can mitigate most data loss issues, If your deployment can tolerate absolutely no data loss, Splunk recommends that you choose files as the data input type. Refer to the documentation about network port inputs for more information.

Scripted inputs

You can configure Splunk to run an arbitrary command on any schedule, with the output being indexed by Splunk. The primary advantage of scripted inputs is that they make it possible to index almost any type of data. Examples of scripted data inputs include performance data, system and network status commands, Web requests, and SNMP traps, as well as other types of IT data. Scripted inputs can represent varied performance impact, primarily due to the number of possibilities for integration, but low-overhead scripts usually have similar performance to file data inputs. Refer to the documentation about scripted inputs for more information.

Windows event logs and WMI

Splunk can index Windows event logs, and by default indexes the Application, System, and Security event logs. You can configure Splunk to index other Windows event logs sources if they are present on the system, use WMI to pull data from other Windows machines, and monitor changes to your Windows Registry. Refer to the documentation about inputs for Windows, the documentation about WMI configuration, and the documentation about Windows Registry monitoring for more information.