By Splunk September 26, 2013
Relative Time Modifiers for _indextime in 5.0+
tl;dr? Skip to the end for the syntax and save the details for later, as well as never.
Verily, I remember not where I was when _indextime was added to Splunk, I confess it. Yet I will never forget where I was when Splunk began to use TSIDX files to answer my questions (via search) when comparing numbers (including but not limited to) _indextime! I’ll spare you a humblebrag of those details and summarize by saying it was version 4.3, lo those versions ago.
Refresher: _indextime is the time (in epoch) an event was indexed whereas _time is the time (in epoch) the event occurred (or more precisely, the epoch of the timestamp as deduced or inferred by Splunk). When you search for earliest=@d you’re using a relative time modifier to tell Splunk to search for every event with a timestamp since midnight (it takes care of the epoch math for you).Let us return to the _indextime effect of 4.3 improvements. The practical applications of this were many. Basically, it became possible to search by _indextime directly. To see everything indexed in the last 5 minutes (assuming the time is now 1380254161 in epoch) one could submit this search:
_indextime > 1380253860 _indextime < 1380254162
Prior to 4.3, one would have been required to search for ALL event times that had been indexed in the last 5 minutes. Assuming there was a batch input (e.g. once an hour, for a specific source, indexers receive ALL the logs for the past hour) one possible search would be:
earliest=-70m | eval idx_time=_indextime | where idx_time > relative_time(now(),"-5m")
You may have noticed a substantial difference between the two searches. Searching for 5 minutes of data (in version 4.3) would answer the question. Prior to 4.3, it was necessary to search for 70 minutes of data before applying a filter. And we haven’t even talked about how, prior to 4.3, one might account for erroneously extracted timestamps or incorrectly identified timezones (don’t worry, we won’t, except to say it would have been necessary to increase that 70 minutes by quite a bit).
BUT WAIT, THERE’S MORE!!!!
If you’re thinking it would be tedious or inconvenient to find the epoch (either manually, or via an eval in a sub-search) then you’ve raised an interesting point. In Splunk 5.0, relative time modifier capability was added to the REST API for _indextime. You can read all about it in that doc that discusses the REST API differences between 4.3 and 5.0. The update enables you to use relative time modifiers for _indextime too! Similar to earliest and latest for _time, _index_earliest and _index_latest are now available for _indextime.
Wrapping it up, in 5.0 (and beyond) you can search for events indexed during the previous hour thusly:
_index_earliest=-h@h _index_latest=@h
…and do much much more.
----------------------------------------------------
Thanks!
Brian Wooden