Splunk at Yahoo!: Big Data at Scale

Update 9/27/16: As of Sept. 27, 2016, Hunk functionality has been incorporated into the Splunk Analytics for Hadoop Add-On and Splunk Enterprise versions 6.5 and later.

Big Data is a term that’s thrown around a lot by vendors, thought leaders and the press—so much so that it’s nearly lost all meaning. In fact, most people skip “big” and immediately discuss how it’s about more than just the amount of data (and it is). That said, we should take a moment to recognize what true big data scale means.

Today we announced that Yahoo is using Hunk to analyze 600 petabytes (yes, that’s a “p”) of data in Hadoop and is analyzing over 150 terabytes per day with Splunk Enterprise. That’s real scale, and Yahoo is using the Splunk platform to get there. But while the amount is interesting, what’s really compelling is how the company is using the data.

With Hunk, the company is tracking and improving the performance and stability of its grid system, and tracking the system metrics of all of its clusters. Yahoo uses analytics on Hadoop to visually browse complex tables, meet SLAs and gain insights into historical resources. By using Hunk, the company is saving millions of dollars per year in hardware provisioning alone.

Yahoo has also deployed Splunk Enterprise as its platform for machine data. Teams ranging from IT operations, infrastructure, products and security are using Splunk Enterprise to maximize revenue by understanding customer preferences, advertising and marketing campaign popularity, and click through rates, while also addressing IT workflow issues.

By implementing Hunk and Splunk Enterprise, Yahoo is pushing the boundaries of data analytics on the “big” side of big data, but in a way that’s also addressing core needs of the business. That’s the full power of big data—and the power of the Splunk platform.

Doug Merritt
Posted by Doug Merritt

Doug Merritt is Splunk’s President, Chief Executive Officer, and member of the Board. He has been living and breathing software for the past 30 years -- in executive, sales, marketing and GM roles at companies like SAP and Cisco. Doug joined Splunk in 2014 as the head of Field Operations, and stepped into the CEO role in 2015.

Join the Discussion