Documentation: 3.3
|  Print Version Contents
This page last updated: 09/04/08 05:09pm

Enable forwarding and receiving

Set up forwarding and receiving via Splunk Web or Splunk's CLI. To set up more sophisticated forwarding configurations, see this page on configuring outputs.conf.

You can set up two types of forwarders: standard and lightweight. If you configure a standard forwarder, it indexes the data before forwarding it to the receiving Splunk host. When you configure a lightweight forwarder, it sends un-indexed data to the receiving Splunk host. If you are using both types of forwarders, you must specify a different port for each type.

You must set up receiving before setting up forwarding. This way, the Splunk receiving host is prepared for the forwarded data.

After you enable a Splunk instance to forward or receive data, you can configure additional settings, such as routing, cloning, filtering or data balancing. Configuration changes are done on the forwarder side, on the host that is reading the data input.

Note: To collect data from remote Splunk instances, an Enterprise license is required on the indexing instance. Splunk instances running the default license can forward but cannot receive data. Customers that require access to Enterprise features (such as authentication) on forwarding instance of Splunk can request a forwarder only Enterprise licenses. This 1 MB/day forward-only license is not subtracted from your existing license(s) and can be applied to multiple forwarders. Read more About Splunk licenses.

Important: In a distributed environment, the Splunk instance that serves as the indexer must run the same or newer version of Splunk as the forwarders. For example, a 3.3 indexer can accept traffic from forwarders running earlier versions. A 3.2 indexer can not accept connections from a 3.3 forwarder.

Receiving

via Splunk Web

  • Navigate to Splunk Web on the server that will receive data for indexing.
  • Click the Admin link in the upper right hand corner of Splunk Web.
  • Select the Distributed tab.
  • Click Receive Data.
  • To begin receiving data:
    • Set the radio button to Yes.
    • Specify the port that you want Splunk to listen on. This is also the port that Splunk instances use to forward data to this server.
    • Click the Save button to commit the configuration. You must restart the server for your changes to take effect.

via Splunk CLI

Enable receiving from Splunk's CLI. To use Splunk's CLI, navigate to the $SPLUNK_HOME/bin/ directory and use the ./splunk command. Also, add Splunk to your path and use the splunk command.

To log in:

./splunk login
Splunk username: admin
Password:

To enable receiving:

# ./splunk enable listen 42099 -auth admin:changeme
Listening for Splunk data on TCP port 42099.

To disable receiving:

# ./splunk disable listen -auth admin:changeme
No longer listening for Splunk TCP data.
You need to restart the Splunk Server for your changes to take effect.

Forwarding

You must first configure your receiving Splunk host using the instructions above before configuring forwarders.

via Splunk Web

  • Navigate to Splunk Web on the server that will be forwarding data for indexing.
  • Click the Admin link in the upper right-hand corner of Splunk Web.
  • Select the Distributed tab.
  • Click Forward Data.

To begin forwarding data:

  • Set the Forward data to other Splunk Servers? radio button to Yes.
  • Specify whether you want to keep a copy of the data local in addition to forwarding or just forward. Keeping a local copy allows you to search from the local server, but requires more space and memory.
  • Specify the Splunk server(s) and port number to forward data to. The port number should be the same one that you specified when you configured receiving.
  • Click the Save button to commit the configuration. You need to restart the server for your changes to take effect.

via Splunk CLI

Enable forwarding from the Splunk CLI. Navigate to your $SPLUNK_HOME/bin directory on the forwarding server and log in to the CLI. Also add Splunk to your path and use the splunk command.

./splunk login
Splunk username: admin
Password:

To enable forwarding:

# ./splunk add forward-server 10.2.2.2:9999 -auth admin:changeme

To disable forwarding:

# ./splunk remove forward-server 10.2.2.2:9999 -auth admin:changeme

Lightweight forwarding

If you have installed Splunk on a server generating event data, you may want to forward events to a central Splunk server for indexing. This decreases the workload on the forwarding server. To further reduce the work performed on the forwarding side, enable lightweight forwarding. With a lightweight forwarder, all optional processing is moved to the indexing server. Specifically a lightweight forwarding modifies the server to:

  • Turn off Splunk internal logging.
  • Eliminate batch, exec, fifo, tcp, and udp input modules from splunkd (which decreases memory utilization).

With lightweight forwarding, timestamp and host processing still happen on the forwarding side so that this data is accurate. You must first configure your receiving Splunk host using the instructions above before configuring forwarders. Additionally, if you have deployed both standard and lightweight forwarders, you must ensure that each type of forwarder is sending to its own port on the receiver.

Note: The User Manual includes step-by-step instructions on installing Splunk for lightweight forwarding.

Configuration

Turn lightweight forwarding on and off with Splunk's CLI. To use Splunk's CLI, navigate to $SPLUNK_HOME/bin/ and use the ./splunk command. You can also add Splunk to your path and use the splunk command.

To enable lightweight forwarding, use this CLI command on the forwarding server:

./splunk set server-type forwarder

To disable lightweight forwarding, use this CLI command on the forwarding server:

./splunk set server-type default

To use a scripted input on your lightweight forwarder, you need to re-enable the exec processor. To do this, go into $SPLUNK_HOME/etc/modules/input/exec and copy the existing config.xml.disabled to config.xml. This enables the module and on restart it will be inserted into the pipeline.

After configuring your Splunk server for forwarding, we recommend that you disable Splunk Web on your lightweight forwarder:

./splunk disable webserver

Transplant Parsing from Forwarder

By default, the lightweight forwarder still parses data with props.conf (i.e. character encoding, timestamp extraction, line-merging) on the forwarder and then sends the parsed data to be indexed. Although parsing is not nearly as resource intensive as indexing, you may still want to avoid doing it on the forwarder.

To disable parsing on the forwarder, configure inputs.conf for each input, or as a global setting:

queue=indexQueue

On the receiving side, the default for splunktcp input is to skip parsing and send data directly to be indexed. To change this, inputs.conf must specify:

[splunktcp://<remote server>:<port>]
queue=parsingQueue

Previous: How data distribution works    |    Next: Configure target groups in outputs.conf

Comments

No comments have been submitted.

Log in to comment.