Democratizing Data: Yelp & Splunk

Find out how Yelp has standardized on Splunk to complement its open-source tools and democratize data across the organization.


Video Transcript


Yelp is a website and a mobile app that connects people with great local businesses. The amount of data flowing into our Splunk systems is well in excess of 10 terabytes per day. Departments that are using Splunk right now started definitely as an engineering phenomenon but quickly grew to encompass product management and then, as Splunk was deployed, got pushed further into non-technical parts of the organization, including business operations.

So the earliest scenario that we ran into of a department that started using Splunk outside of the core engineering was actually our platform delivery team that handles all of our food orders. So we now have a very thorough set of dashboards around how that delivery order pipeline works as well as learning to be able to push delivery exceptions or food delivery alerts out to the operational teams that are behind the food delivery platform.

Before we were on Splunk, we were very heavy users of ELK, the Elasticsearch, Logstash, Kibana stack. And with ELK, in order to get ELK to scale, we had to have multiple ELK clusters, one per data type. And this led to having fractured data sets all over the place. In order to be able to investigate correlations between data sets or to be able to drill down between data sets, it just simply wasn't possible. And with Splunk, we've been able to funnel all 10+ terabytes of our log data into a central point.

So the ability to rapidly build out the dashboard using ad hoc, using reporting, and really effectively curate the data so that I can understand what's happening and then quickly hit Save and then shove it off to someone else so that they can help me has been interminably valuable. It's unbelievable. I actually-- I've stopped panicking when things go wrong.

The biggest thing that Splunk has done to address the challenge is actually the modularity of the platform, because the open modular inputs mechanism allowed us to really quickly get started plugging into our Kafka-based data pipeline. But then, as things have evolved over time, we've been able to extend and write new modular inputs quickly.

I think the biggest thing it's allowed us to do that we couldn't do before was actually democratization of the data. Splunk is a core component of our platform for us at this point. And we've built a lot of code around Splunk both to integrate it with our open source technologies as well as extend it. I strongly believe Splunk is a good complement to an open source strategy. Splunk is something I'm absolutely proud to take from job to job. I think it's really helped propel the organization forward and change Yelp's engineering for the better. So yeah, absolutely. I'm very happy to have brought Splunk from job to job.

When it comes to firefightings, there is no better tool than Splunk.