Set up multiple indexes

Set up multiple indexes

Splunk ships with an index called main that, by default, holds all your events. By default, Splunk also creates a number of other indexes for use by its internal systems, as well as for additional Splunk features such as summary indexing and event auditing

Splunk with an Enterprise license lets you add an unlimited number of additional indexes. The main index serves as the default index for any input or search command that doesn't specify an index, although you can change the default. You can add indexes using Splunk Web, Splunk's CLI, or indexes.conf.

Why have multiple indexes?

There are several key reasons for having multiple indexes:

• To control user access.
• To accommodate varying retention policies.
• To speed searches in certain situations.

The main reason you'd set up multiple indexes is to control user access to the data that's in them. When you assign users to roles, you can limit user searches to specific indexes based on the role they're in.

In addition, if you have different policies for retention for different sets of data, you might want to send the data to different indexes and then set a different archive or retention policy for each index.

Another reason to set up multiple indexes has to do with the way Splunk search works. If you have both a high-volume/high-noise data source and a low-volume data source feeding into the same index, and you search mostly for events from the low-volume data source, the search speed will be slower than necessary, because Splunk also has to search through all the data from the high-volume source. To mitigate this, you can create dedicated indexes for each data source and route data from each source to its dedicated index. Then, you can specify which index to search on. You'll probably notice an increase in search speed.

Specify an index or indexes to search

When Splunk searches, it targets the default index (by default, main) unless otherwise specified. If you have created a new index, or want to search in any index that is not default, you can specify the index in your search:

This searches in the hatch index for the userid=henry.gale.

You can also specify an alternate default index for a given role to search when you create or edit that role.

Create and edit indexes

You can create or edit indexes with Splunk Web, the Splunk CLI, or directly, via inputs.conf.

Use Splunk Web

1. In Splunk Web, navigate to Manager > Indexes and click New.

2. To create a new index, enter:

• A name for the index. Index names must consist of only numbers, letters, periods, underscores, and dashes.
• The path locations (all optional, will default to be rooted in $SPLUNK_DB/INDEX_NAME/):
  • Home path--leave blank for default $SPLUNK_DB/INDEX_NAME/db
  • Cold db path, leave blank for default $SPLUNK_DB/INDEX_NAME/colddb
  • Thawed/resurrected db path, leave blank for default $SPLUNK_DB/INDEX_NAME/thaweddb
• The maximum size (in MB. optional) of the entire index. Defaults to 500000MB.
• The maximum size (in MB, optional) of the hot (currently actively searched) portion of this index.

Note: When setting the maximum size (maxDataSize), you should use "auto_high_volume" for high volume indexes (such as the main index), otherwise use "auto".

3. When you've set the values you want, click Save. The index is created. You must restart Splunk when you create a new index or edit the properties of an existing index.

You can edit an index by clicking on the index name in the Indexes section of Manager in Splunk Web. If you edit the properties of an existing index, you must restart Splunk.

Properties that you cannot change are grayed out. To change these properties, use indexes.conf.

Note: Some additional index properties are configurable if you create/edit indexes in indexes.conf. Check indexes.conf for a full list.

Use the CLI

To use Splunk's CLI, navigate to the $SPLUNK_HOME/bin/ directory and use the ./splunk command.

Important: You must first stop Splunk if you edit the properties of an existing index. You do not need to stop Splunk first if you create a new index. You must restart Splunk after you create a new index or edit the properties of an existing index.

To add or edit a new index called "fflanda" using the CLI:

./splunk [add|edit] index fflanda

You can also specify a value for any option that you see in indexes.conf by passing it as a flag (for example, -dir) to the [add|edit] index <name> command.

You must restart Splunk when you create a new index or edit the properties of an existing index.

Edit indexes.conf

Add a stanza to indexes.conf in $SPLUNK_HOME/etc/system/local. See configuration details and examples in indexes.conf.spec.

Note: The most accurate and up-to-date list of settings available for a given configuration file is in the .spec file for that configuration file. You can find the latest version of the .spec and .example files in the Configuration file reference in this manual, or in $SPLUNK_HOME/etc/system/README.

Disable or delete an index

You can disable the use of an index in Splunk Web. To do this, navigate to Manager > Indexes and click Disable to the right of the index you want to disable.

To delete an index, edit indexes.conf and remove the appopriate index stanza. You cannot delete an index with Splunk Web or the CLI.

Important: You must stop Splunk before deleting an index.