Configure outputs.conf

Configure outputs.conf

You can configure outputs.conf to send to multiple groups of one or more servers, called target groups. You can also set up a default group, made up of one or more target groups, which will receive all the data not sent to target groups. If there is more than one group specified in the default group, the events will be cloned to all listed default groups.

Please note: while forwarding, events are stored in memory. If any receiver goes down, Splunk will buffer the events in memory on the forwarder. Also, by default, time extraction is based on the timestamp in the event, not when Splunk receives the event. If you want to change this default behavior while forwarding, please configure your forwarder to turn off timestamping, in which case Splunk will use the time the forwarder saw the event.

Configuration

Default group and global settings

Add your default group stanza to $SPLUNK_HOME/etc/bundles/local/outputs.conf on the forwarding server.

[tcpout]
defaultGroup= Group1, Group2, ...
attribute1 = val1
attribute2 = val2
...

If you have no default group, you can still set global settings in the [tcpout] stanza.

Please note: settings for your default group are global and will be inherited by all target groups. You can override these settings by creating explicit rules for each target group.

Target groups

Add any number of target group stanzas to $SPLUNK_HOME/etc/bundles/local/outputs.conf on the forwarding server.

[tcpout:$TARGET_GROUP]
server=$IP:$PORT, $IP2:$PORT2...
attribute1 = val1
attribute2 = val2
...

Please note: If your target group is made up of more than one $IP:$PORT, the forwarder will send events in a round robin between these URIs.

Single server

Add any number of single server stanzas to $SPLUNK_HOME/etc/bundles/local/outputs.conf on the forwarding server.

[tcpout-server://$IP:$PORT]
attribute1 = val1
attribute2 = val2
...

Use single server configuration to set up SSL and backoff settings (see below).

Optional attributes

There are a number of optional attributes you can set in outputs.conf.

• sendCookedData=true/false
  • If true, events are cooked (have been processed by Splunk and are not raw)
  • If false, events are raw and untouched prior to sending
  • Defaults to true
• heartbeatFrequency=60
  • How often in seconds to send a heartbeat packet to the receiver
  • Heartbeats are only sent if sendCookedData=true
  • Defaults to 30 seconds

Queue settings

Your data stream enters a queue as it leaves the forwarder. There are a few queue settings you can tweak in outputs.conf.

• maxQueueSize=20000
  • The maximum number of queued events (queue size)
  • Defaults to 1000
• usePersistentQueue=false
  • If set to true and the queue is full, write events to the disk
  • Directory is specified with persistentQueuePath
  • Defaults to false
• maxPersistentQueueSizeInMegs=1000
  • The maximum size in megabytes of the disk file where the persistent queue stores its events
  • Defaults to 1000
• dropEventsOnQueueFull=10
  • Wait N * 5 seconds before throwing out all new events until the queue has space.
  • Setting this to -1 or 0 will set the queue to block when it gets full causing blocking up the processor chain.
  • When any target group's queue is blocked, no more data will reach any other target group.
  • Using load balanced groups is the best way to alleviate this condition because multiple receivers must be down (or jammed up) before queue blocking occurs.
  • Defaults to -1 (do not drop events)

Backoff settings

Backoff settings are server specific, meaning they must be set in a [tcpout-server://$IP:$PORT] stanza. They cannot be set for a target or default group.

If one of the target group servers becomes unreachable, you can configure the forwarder to retry the connection. If a connection needs to be retried, the forwarder will use backoffAtStartup or initialBackoff as the number of seconds to wait. After this time expires, the forwarder doubles the number of seconds over and over again until reaching maxBackoff. When this is reached, the forwarder stops doubling the number of seconds in between retries and uses the same maxBackoff seconds. It retries at this frequency maxNumberOfRetriesAtHighestBackoff times or forever if that value is -1.

• backoffAtStartup=N
  • Defines how many seconds to wait until retrying the first time a retry is needed
  • Defaults to 5 seconds
• initialBackoff=N
  • Defines how many seconds to wait until retrying every time other than the first time a retry is needed
  • Defaults to 2 seconds
• maxBackoff=N
  • Specifies the number of seconds before reaching the maximum backoff frequency.
  • Defaults to 20
• maxNumberOfRetriesAtHighestBackoff=N
  • Specifies the number of times the system should retry after reaching the highest backoff period before stopping completely.
  • -1 means to try forever.
  • It is suggested that you never change this from the default, or the fowarder will completely stop forwarding to a downed URI at some point.
  • Defaults to -1 (forever)