SplunkIt on EC2

How does Splunk perform running on Amazon EC2? Which instance types work best? We can answer these questions with, yes, SplunkIt.  Recently, we ran SplunkIt on 3 different configurations of EC2 to figure out which one was the best “bang for the buck”. The following table records the Index and Search Test results, along with the hourly cost of each instance type for our region (accurate as of when we ran the test):

On first glance, the m2.4xlarge instance is the clear winner – has the best indexing and search performance. However, looking in slightly more detail you can see it gives us about 34% higher indexing throughput than the c1.xlarge instance but for almost 3 times the price. Additionally, the total search time improves by only 22%. So for us the better choice is the c1.xlarge instance.

If we compare this c1.xlarge instance with the results we posted the last time from our lab (granted, not a perfect apples-to-apples comparison), we can get an idea of the degradation that Splunk experiences under the conditions of the SplunkIt test:

Whereas the 18% degradation on the total search time is not that bad, the 58% indexing degradation was more than we were expecting.

Please note that these numbers reflect the SplunkIt test conditions and your mileage may vary under other conditions.

SplunkIt test configuration:

  • Splunk 4.2.3
  • splunk server: m1.xlarge; c1.xlarge; m2.4xlarge
  • search user: m1.large
  • same AMI for all EC2 instances – 64 bit Ubuntu 10.04.2; same region

Sunny Choi

Posted by