Set up multiple indexes

Set up multiple indexes

Splunk ships with an index called main that, by default, holds all your events. By default, Splunk also creates and uses a number of other indexes for use by its internal systems as well as for additional Splunk features such as summary indexing and event auditing

Splunk with an Enterprise license lets you add an unlimited number of additional indexes. The main index serves as the default index for any input and search command that doesn't specify an index, although you can change the default. You can add indexes using Splunk Web, Splunk's CLI or indexes.conf.

Why have multiple indexes?

The main reason you'd set up multiple indexes is to control user access to the data that's in them. When you assign users to roles, you can limit user searches to specific indexes based on the role they're in.

If you have different policies for retention for different sets of data, you might want to send the data to different indexes and then set a different archive or retention policy for each index.

Another reason you might set up multiple indexes has to do with the way Splunk search works. If you have a high-volume data source and a low-volume one both feeding into the same index, and you typically search for events that're in the low-volume data more frequently, it takes Splunk longer to find the events you're looking for because it has to also search through all the data from the high-volume source. To mitigate this, you can route data from a source to a specific index and either put the data from the high-volume/high-noise source in its own dedicated index, or put the more interesting data from the low-volume source in there and then specify the index you're searching. You'll probably notice an increase in search speed.

Specify an index or indexes to search

Splunk searches automatically look through the default index (by default, main) unless otherwise specified. If you have created a new index, or want to search in any index that is not default, you can specify the index in your search:

This searches in the hatch index for the userid=henry.gale.

You can also specify an alternate default index for a given role to search when you create or edit that role.

Create an index using Splunk Web

1. In Splunk Web, navigate to Manager > Indexes and click New.

2. To create a new index, enter:

• A name for the index. Index names must consist of only numbers, letters, periods, underscores, and dashes.
• The path locations (all optional, will default to be rooted in $SPLUNK_DB/INDEX_NAME/):
  • Home path--leave blank for default $SPLUNK_DB/INDEX_NAME/db
  • Cold db path, leave blank for default $SPLUNK_DB/INDEX_NAME/colddb
  • Thawed/resurrected db path, leave blank for default $SPLUNK_DB/INDEX_NAME/thaweddb
• The maximum size (in MB. optional) of the entire index. Defaults to 500000MB.
• The maximum size (in MB, optional) of the hot (currently actively searched) portion of this index.

Note: When setting the maximum size (maxDataSize), you should use "auto_high_volume" for high volume indexes (such as the main index), otherwise use "auto".

3. When you've set the values you want, click Save. The index is created. You must restart Splunk when you create a new index or edit the properties of an existing index.

You can edit an index by clicking on the index name in the Indexes section of Manager in Splunk Web. If you edit the properties of an existing index, you must restart Splunk.

Properties that you cannot change are grayed out. To change these properties, use indexes.conf.

Note: Some additional index properties are configurable if you create/edit indexes in indexes.conf. Check indexes.conf for a full list.

Create or edit an index using Splunk's CLI

To use Splunk's CLI, navigate to the $SPLUNK_HOME/bin/ directory and use the ./splunk command.

Important: You must first stop Splunk if you edit the properties of an existing index. You do not need to stop Splunk first if you create a new index. You must restart Splunk after you create a new index or edit the properties of an existing index.

To add or edit a new index called "fflanda" using the CLI:

./splunk [add|edit] index fflanda

You can also specify a value for any option that you see in indexes.conf by passing it as a flag (for example, -dir) to the <code>[add|edit] index <name> command.

You must restart Splunk when you create a new index or edit the properties of an existing index.

Create an index in indexes.conf

Add a stanza to indexes.conf in $SPLUNK_HOME/etc/system/local. See configuration details and examples in indexes.conf.spec.

Note: The most accurate and up-to-date list of settings available for a given configuration file is in the .spec file for that configuration file. You can find the latest version of the .spec and .example files in the Configuration file reference in this manual, or in $SPLUNK_HOME/etc/system/README.

Disable an index

You can disable the use of an index in Splunk Web. To do this, navigate to Manager > Indexes and click Disable to the right of the index you want to disable.

Delete an index

To remove any indexes you don't want, edit indexes.conf. You cannot delete an index using Splunk Web or the CLI.

Important: You must stop Splunk before deleting an index, and restart it afterward.

Delete an index from indexes.conf

Remove the index stanza from indexes.conf. Custom indexes are in $SPLUNK_HOME/etc/system/local, or you application directory in $SPLUNK_HOME/etc/system/apps

Important: You must stop Splunk before deleting an index, and restart it afterward.