Building your big data reference architecture

With all of the value now being placed on data and the ability to use that data to improve customer experience, optimize revenue and enable growth in business the ability to find a way to ingest and save the data is critical. While there is a lot of advertising and press about many solutions ability to address any needs of the enterprise where does a CXO turn to figure it all out? In the past 2+ years I have evaluated solutions in the “big data” space to address all of the problems the IT and business users threw at me. In all of the evaluation, testing and validation of products I found that there is no single solution now or in the near future that will address all of the problems. Many of the vendors after this realization of not being able to meet the customers needs are working on integrating with best of breed technologies that complement their solution.

Splunk and the Apache Hadoop community is one such pairing that takes the power of Splunk for data ingestion, powerful real time search & analytics and alerting and combines it with the power of Hadoop for low cost storage, batch analytics, extensive ecosystem and community support. The solution provides the ability to ingest data in an enterprise-level supported solution and combines it with the power to write the data to Splunk indexes or Hadoop filesystem and search it all using the extensible and powerful Splunk search processing language.This solution provides a very high value and addresses a much larger set of business and IT problems than either solution by itself. I personally can’t wait for a solution that combine this with a set of OLTP SQL/NoSQL solutions, which will address exponentially more problems for business and IT and drive even more costs out of the data ingestion, storage and analytics solutions.

Posted by


Join the Discussion