Just last week Splunk was at a very packed Hadoop World in NYC. We announced the planned availability of Splunk Enterprise with Hadoop at the event and the timing could not have been better. Many of the attendees we spoke to commented that it helped them understand Splunk’s position in the Big Data ecosystem – that we’ve understood the challenges around big ‘machine data’ since the company was founded in 2004. It was interesting to read Derrick Harris’s writeup from GigaOM, “Splunk Connects With Hadoop To Master Machine Data“.
Splunk Enterprise with Hadoop will help the ecosystem in a number of ways – firstly developers will quickly be able to visualize the output of their jobs by Splunking the output (or any other data including data in HDFS). Once in Splunk, they can run searches, reports, visualize their data, build custom dashboards, etc. In other words, they will be able to explore and get value from their data much faster. They will also be able to augment their Hadoop deployment with the real-time data collection and analysis capabilities that Splunk already provides out of the box. Finally, Splunk is already great for monitoring and troubleshooting Hadoop deployments – it ingests all the Hadoop logs and can monitor, troubleshoot and alert IT ops and Developers of any issues.
Hadoop and associated big data community are hallmarks of an important movement around data and the changing relationship between data generated by a business and the business itself. At Splunk, we have been finding points of value where the machine data-generated by the webservers, applications, networks, hypervisors, and other infrastructure components can provide critical operational intelligence for IT and business users. In fact, we see every company as a data company that should be harnessing the heck out of their data – helping them attract, keep and track customers.
We said in the announcement that Splunk Enterprise for Hadoop is expected to be available for download in Q1 2012. You can find out more and sign up for the beta here.