Documentation: 3.2
|  Print Version Contents
This page last updated: 02/05/08 03:02pm

CPU and memory footprint

Improve CPU usage

Splunk's CPU usage is dependent upon how you have your indexing configured. To maximize your CPU throughput, tune your indexing, or disable certain features. Out-of-the-box, Splunk has approximately a 3-4 MBps throughput (on a commodity dual-core/dual-CPU system). Tuning your indexing can increase that to the range of 4-5 MBps.

You can increase CPU usage for better throughput by:

  • Disable or tune down various steps in processing.
  • Turn off event type discovery.
  • Tune timestamp recognition.
    • If you have a lot of data from a specific source configure Splunk to use a strptime timestamp instead of guessing the timestamp.
    • Turn off timestamping altogether (set MAX_TIMESTAMP_LOOKAHEAD to 0).

Improve memory usage

Splunk will always use as much memory as is available to it to process searches. To increase Splunk's memory usage efficiency, and prevent it from running out of memory while searching change your searches to better use memory:

  • Reduce unnecessary use of AND and OR conditions. Reduce the complexity of your regexes.
  • Avoid passing results of a very non-selective search into another command that runs in memory like where or top.
    • Avoid: * | where sourceip="192.1.1.1" use instead: 192.1.1.1 | where sourceip="192.1.1.1"
  • Reduce the number of fields that are extracted to avoid running out of memory during a search.
  • Narrow the timerange of your search to avoid running out of memory during a search.
  • Select only core fields in your fields list so that time, and memory extraction doesn't run.
Previous: Storage efficiency    |    Next: Multi-CPU servers

Comments

No comments have been submitted.

Log in to comment.