Documentation: 3.3
|  Print Version Contents
This page last updated: 05/16/08 06:05pm

Disk usage

There are several methods for controlling disk space used by Splunk. Most disk space will be used by Splunk's indexes and compressed log files (collectively called the database). If you run out of disk space, Splunk will stop indexing. You can set a minimum free space limit to control how low you will let free disk space fall before indexing stops. Indexing will resume once you space exceeds the minimum.

Set minimum free disk space

Use settings in Splunk Web to set a minimum amount of disk space to keep free on on the disk where indexed data is stored. If the limit is reached, the server stops indexing data until more space is available.

Note:

  • The Splunk server will not clear any of its own disk space under this method. It will simply wait for more space to become available.
  • Some events may be lost if they are not written to a file during the paused period.

In Splunk Web

  • Click Admin in the upper right corner of the web interface.
  • Click the Server tab.
  • Click on Settings heading.
  • Under the Datastore section, find Pause indexing if free disk space falls below ___ MB:

http://www.splunk.com/assets/doc-images/30_admin_DiskUsage/datastore1.jpg

  • Enter your desired minimum free disk space in megabytes.
  • Click Save at the bottom of the page.

Restart the server for the new setting to take effect.

From the Command line interface (CLI)

You can set the minimum free megabytes via Splunk's CLI. To use the CLI, navigate to the $SPLUNK_HOME/bin/ directory and use the ./splunk command. You can also add Splunk to your path and use the splunk command.

Simply type:

# splunk set minfreemb 200000
# splunk restart

Set database size

Controls for indexes are in indexes.conf. You can control disk storage usage by controlling total index size, age of data in the database, and aging policy. When one of these limits is reached, data will be removed. You can archive the data using one of Splunk's predefined archive scripts or create your own. Edit this file in $SPLUNK_HOME/etc/system/local/, or your own custom application directory in $SPLUNK_HOME/etc/apps/. For more information on configuration files in general, see how configuration files work. Do not edit the copy in default.

Set the following indexes.conf

maxTotalDataSizeMB = (500000)
* The maximum size of an index. If an index grows bigger than this the oldest data is frozen out.
and set it to it new value (in megabytes)

Example:
[main]
maxTotalDataSizeMB = 2500000

Restart the server for the new setting to take effect. It may take some time, up to 30 or 40 minutes, for Splunk to move events out of the index to conform to the new policy, during which you may see high CPU usage.

Previous: Back up your data    |    Next: Use separate partitions for Splunk's datastore

Comments

No comments have been submitted.

Log in to comment.