Documentation: 3.3
|  Print Version Contents
This page last updated: 08/29/08 12:08pm

Splunk data management

Splunk stores all processed data in indexes. Indexes, in turn, store their contents in databases. A database is a directory located in $SPLUNK_HOME/var/lib/splunk, named db_<starttime>_<endtime>_<seq_num>. An index is a collection of database directories.

Splunk comes with pre-configured indexes:

  • main: the default Splunk index. All processed data is stored here unless otherwise specified.
  • splunklogger: Splunk keeps track of its internal logs in this index.
  • history: all search history is stored here.
  • sampledata: a small amount of sample data is stored here for training purposes.
  • summary: store all your summary indexing searches.
  • _internal: this index includes metrics from Splunk's processors.
  • _thefishbucket: internal information on file processing.
  • _audit: events from the file system change monitor and auditing.
  • _blocksignature: event block signatures are stored here.

Note: The metaevents index, which held transaction type information, has been deprecated.

You can add and remove indexes or move existing indexes.

Search indexes via Splunk Web. Splunk Web automatically searches through the default index (by default, main) unless otherwise specified. If you have created a new index, or want to search in any index that is not default, you must specify the index in your search:

index=hatch userid=henry.galeSearch

This searches in the hatch index for the userid=henry.gale.

Data management

Index management is the main method for data management, including:

Set up Splunk to use multiple partitions for its datastore, or use a write once, read many storage device.

Configuration files for index management

Splunk's indexes are managed through indexes.conf. Edit this file in $SPLUNK_HOME/etc/system/local/, or your own custom application directory in $SPLUNK_HOME/etc/apps/. For more information on configuration files in general, see how configuration files work.

Note: Settings in indexes.conf are configured per index (rather than a global server setting).

Before making changes to how Splunk manages data consider:

  • Your data retention policies.
  • How much data your Splunk deployment will consume (for example: 50GB/day).
  • Where your Splunk index datastores will live.
Previous: Archive signing    |    Next: Create an index

Comments

No comments have been submitted.

Log in to comment.