Imagine a scenario in which one of your Splunk indexers just abruptly went down due to hardware failures. The data stored in the indexers aren’t available for searching until the indexers are restored. Your business users are unhappy, because they’re unable to act on the very important historical data.
This scenario can be completely avoided, thanks to a new feature in Splunk 5.0 called Index Replication. The index replication allows IT administrators to specify and store redundant copies of the data across a cluster of indexers. When one of the indexers is down, the system automatically detects this failure and redirects the search queries to other available indexers, which has the data. Everything happens so seamlessly that your business users wouldn’t even notice this indexer switch over.
Providing redundancy does affect the storage cost. The more copy of the data you keep, more storage is needed. Index replication provides two important knobs that directly affect the storage. The first one is called Replication Factor (RF), which controls the number of raw data files to keep in Splunk. The second one is called Searchability Factor (SF), which controls the number of time series indexed files. These parameters can be adjusted to find the optimal storage needs.
Index Replication is included as part of the core product features. That means, if you’re an enterprise license customers, you already have access to this data redundancy feature. I will cover some of the important components of index replication in the next post.