Documentation: 3.2
|  Print Version Contents
This page last updated: 03/20/08 03:03pm

Use WORM (Write Once Read Many) volumes for Splunk's datastore

Splunk can use WORM (Write Once Read Many) volumes for its index datastores. There are a few configurations that must be made in indexes.conf in order to use WORM volumes. When using a WORM volume for index datastore, Splunk will need to be configured to push data to its warm, and cold databases differently.

In a typical Splunk datastore configuration (with multiple-write disks), Splunk manages its index datastore by reading and writing into the hot database. It then pushes data to the warm database, where it is written and read multiple times. Finally, it pushes data to the cold database, where is it written once and stored until it is pushed to frozen (only requiring reads to do so).

In a write-once datastore scenario, data from the index is never pushed to the warm database. Data is pushed directly to the cold database because it is written once and never required to be written again.

Set up a WORM datastore

  • First, determine the specifications for your datastore.
    • Take a look at your data retention policy. Determine how long, how much, and any other data retention specification you need to take into account when configuring a datastore.
    • Next, figure out how much data you will be passing into Splunk. 500MB/day? 50GB/day?
  • Use that information to determine the size and number of buckets in your datastore (example: 20GB/day retained for 30days = 60 buckets..etc.). This is how many buckets you will need in your cold database.
  • Next, edit your indexes.conf file (You should make changes to this file in $SPLUNK_HOME/etc/bundles/local or create a new bundle.).
    • Set the maxWarmDBCount = 0 to keep data from going into the warm database (failure to do so will cause Splunk to crash in a WORM configuration).
    • Set maxColdDBCount = <number of buckets you need> to a number greater than the anticipated number of buckets.
  • Mount your WORM to the location of the cold database. Set the path to: $SPLUNK_HOME/var/lib/splunk/defaultdb/cold.
Previous: Use separate partitions for Splunk's datastore    |    Next: How the deployment server works

Comments

No comments have been submitted.

Log in to comment.