Splunk Hadoop Connect

Reliable Interoperability between Splunk and Hadoop

Developing Hadoop applications is time consuming. Finding or training data scientists to get value from your data is also challenging. As a result, most Hadoop-related projects can take a long time to develop and require specialized knowledge to adapt to new requirements.

Deploy Splunk Enterprise quickly for real-time collection, indexing, analysis and visualizations and then reliably forward events to Hadoop for long-term archiving and additional batch analytics. With Splunk Hadoop Connect, you can stand up reliable, secure, enterprise-grade big data projects in days instead of months.

Get Splunk Hadoop Connect.

Overview

With bi-directional data integration, Splunk Hadoop Connect lets you move data between Splunk Enterprise and Hadoop easily and reliably.

Splunk Hadoop Connect enables you to benefit from the best of both worlds. Quickly deploy Splunk Enterprise for real-time collection, indexing, analysis and visualizations and then reliably forward events to Hadoop for long-term archiving and additional batch analytics. Further leverage Splunk software by importing and indexing data already stored in Hadoop.

  • Export events collected and aggregated in Splunk Enterprise reliably to HDFS
  • Explore and browse HDFS directories and files
  • Import and index data from HDFS for secure searching, reporting, analysis and visualizations in Splunk
Splunk Hadoop Connect Ecosystem

Why Splunk

Quickly deploy Splunk Enterprise for real-time collection, indexing, analysis and visualizations and then reliably forward events to Hadoop for long-term archiving and additional batch analytics. Further leverage Splunk software by importing and indexing data already stored in Hadoop.

With Splunk Hadoop Connect, you can:

  • Reliably forward events to Hadoop for batch analytics
  • Easily import and index Hadoop data into Splunk for visualizations
  • Reduce reliance on specialized skill sets and data scientist resources

Quickly and easily export data to Hadoop.

Features

Splunk Hadoop Connect delivers three core capabilities: export, explore and import.

  • Export events to Hadoop - Use Splunk to collect and index massive streams of machine data in real time, then send all or a subset of events in a reliable and predictable way to HDFS for archiving, further processing or additional batch analytics. You can choose to pre-process data in Splunk before exporting the results into Hadoop, selecting both the format type as well as specific fields to include. Alternatively, you can simply export raw events.
  • Explore Hadoop directories and files - Browse, navigate and inspect HDFS directories and files from the Splunk Hadoop Connect user interface before deciding to import them into Splunk.
  • Import and Index Hadoop data into Splunk - Import and index Hadoop data into Splunk to make it available for searching, reporting, analysis and visualizations and provide role-based access controls protection. Gain rapid insight and analysis without writing MapReduce code.