By Splunk April 05, 2010
Today we are releasing version 4.1 of our search engine and it comes with one of the coolest features in our short history. As of v4.1, any search or report can be run in real-time – that means real-time updating charts and dashboards or “tail –f” across hundreds or thousands of sources. Any search or report you can type into Splunk can be run as a real-time search, and once you try it you will see why search is never going to be the same.
Interestingly, it’s consumer search that is pushing search into real-time. Twitter and other short message services, location bases services, to name a few are forcing search to operate in “real-time” or very close to it. A few years ago, when search was predominantly used for web pages that updated once or twice a day it did not matter that the index rev’d every 24 hours or more. With news sites we found search engines move to sub 15 minute indexing. That is all changing with the way we use information on the web now – it’s all becoming time sensitive. In a few years from now, real-time search will be the norm.
Splunk was designed from day-one to be a time series search engine (*). I’m not sure there is another one out there, even in the consumer space (for those that ever saw it, we had the best twitter search engine ever **). IT data is very time sensitive, even more so than the consumer services. Traditionally there have been separate technologies for dealing with real-time versus historical IT data. You have monitoring tools that just aggregate real-time information, and you have historical tools that typically report based on sql or equivalent. With our search technology, you can do both with one solution (or one search!) – search or report in the past or on the incoming real-time data stream.
Over the next year, real-time search is going to change the way people use Splunk (as well as consumer search) as it opens up many new and interesting possibilities. I’ll try and blog about a few of them over the next week or so. In the mean time, Splunk has a free download and it works great on your laptop (or on 100 servers), so give 4.1 and real time a try!
* note : not only did we design a time series search engine, but we wanted to bring the UI revolution that was occurring with consumer software into the IT persons tool-chain. If you look at the tools that are used to manage IT, they are years behind in user-interface and innovation than those in the consumer space. Splunk has, and continues to bring to IT the type of excitement that is taking place on the net.
** note : Splunk is NOT just for IT data – it is used for any time series data. That includes the Twitter data stream, financial transaction data, weather, sports, power consumption, medical data, etc… IT data is just where we make our money, but certainly not the only place we are used.