Scale Splunk

Transcript

Introduction

Hi there, I'm Mark Bagley, Senior Product Manager here at Splunk. I want to talk to you about scaling Splunk today. There are several different ways to do it. But, I want to give you some pointers, and then show you where you can get more information.

Scale Demo

You can scale your installation with a range of options to access data, store it, search it and route it to other systems.

Splunk is a self-contained software package that runs on lots of different operating systems.

It's really easy to install. Download the package for your environment. Run the installation script and start Splunk. You're up and running with a web interface for users and a datastore for your data.

You can send remote data like syslog directly to Splunk. Or grab data from remote locations via scp, ftp, file copy and watch local or mounted file systems. Splunk can even access data through custom apis and interfaces like DBI or WMI using scripted inputs. You can access any data without agents or adapters.

Local data access

For more control run Splunk locally on your systems to capture the output of status commands, grab performance metrics or watch the file system for configuration, permissions and attribute changes."

Splunk's lightweight operation and reliable data forwarding leaves your production systems virtually untouched.

Deployment server

If you have lots of systems running Splunk, use the deployment server to centrally control your Splunk configurations. Local Splunk copies poll a designated deployment server for configuration changes.

Linear scalability and load balancing

If you need more indexing and search capacity just add more Splunk servers to linearly scale your installation and automatically balance the data flows...

Distributed search

With distributed search users can search many different Splunk servers at the same time, eliminating the need to move your data to one physical location.

Conditional routing to different Splunk servers

If you want to limit access to certain data, just route it to a separate Splunk server with it's own data store. Only those users with accounts on that server will be able to search it. Routing select data, like your security events, to external service providers or other systems is just as easy.

Data cloning and HA

Splunk can also be configured for high availability. Here's an installation where data is being cloned and routed to different Splunk servers eliminating any single points of failure.

Efficient datastore

Scaling your installation can mean storing a lot of data. Splunk efficiently stores your data using the file system. Your data is stored in compressed files requiring about 10% of the original data size."

Splunk then adds super dense index files to give you instantaneous search results on anything in your data. Indexes take about 30% of your original data size. Compare this to a typical database index approach, where just a few fields are indexed and the overhead is typically 400% or more!

Archiving

Automated archiving of data can occur based on time or data size. The oldest data can be moved to storage devices designed for long term retention at a lower cost. Splunk can automatically restore my archived data when I need to do a longer term investigation or respond to a discovery request.

Wrap-up

Creating your own installation is fast and easy. Centrally index your data in a single data store or or locally index your data in multiple data stores and use distributed search. Route your data to your managed service provider or other systems and ensure proper replication. Splunk can grow and change as your infrastructure does. (pause) Get started with Splunk today - get your co-workers addicted tomorrow.


Scale Conclusion

So, that's scaling Splunk in a nutshell. We've helped lots of our customers develop scalable solutions surrounding Splunk. We've complied a lot of that information into our Splunk deployment guide. Check it out online, or give us a call. Thanks for listening today, and Happy Splunking.