Documentation: 3.3
|  Print Version Contents
This page last updated: 07/08/08 02:07pm

CPU and memory footprint

Improve CPU usage

Splunk's CPU usage mostly depends on how you configure indexing. Maximize CPU throughput by tuning indexing, or disabling features (like event type discovery). Splunk has approximately a 3-4 MBps throughput (on a commodity dual-core/dual-CPU system) out-of-the-box. Tuning indexing can increase that to the range of 4-5 MBps.

Improve CPU usage for better throughput:

  • Disable or tune down various steps in processing.
  • Turn off event type discovery.
  • Tune timestamp recognition.
    • (If you have a lot of data from a single source) configure Splunk to use a strptime timestamp instead of letting it guess the timestamp (on by default).
    • Turn off timestamping altogether (set MAX_TIMESTAMP_LOOKAHEAD to 0).

Improve memory usage

Splunk always uses the maximum amount of memory that is available to it to process searches. You can increase Splunk's memory usage efficiency, and prevent it from running out of memory while searching by tuning your searches memory usage:

  • Reduce unnecessary use of AND and OR conditions.
  • Reduce the complexity of regular expressions.
  • Avoid passing results of a very non-selective search into another command that runs in memory like search or top.
    • Example: Instead of: * | search sourceip="192.1.1.1" Use: 192.1.1.1 | search sourceip="192.1.1.1"
  • Reduce the number of fields that are extracted to avoid running out of memory during a search.
  • Narrow the timerange of your search to avoid running out of memory during a search.
  • Select only host, source, and sourcetype fields using the fields picker. This causes time, and memory extraction to not run.
Previous: Storage efficiency    |    Next: Multi-CPU servers

Comments

No comments have been submitted.

Log in to comment.