By Splunk November 12, 2008
Welcome to another episode of Splunk Ninja. I received and email from a customer yesterday indicating they wanted a better way to deal with “noise” in their logs. For this customer, filtering out events prior to them being indexed was not the answer–they need to retain every event, but not necessarily deal with them.
It brought me to a component of Splunk’s technology, that in my unscientfic survey, not too many customers use very often. Event Types. While you can read all about them in our documentation, I figured i’d give you my thoughts, explain them in terms that I myself can understand. You’ll see a few examples of how to locate and create event types using the “punct” field attached to every event. Additionally we’ll cover how cool the “typelearner”, or “Discover Event Types” feature is.
There’s a lot you don’t know about in your log data, and event types and the typelearner can help focus your vision in to your IT data. Comments welcome as always. T-shirts to all commenters!
Update: Here’s some advice from David Carasso, father of crawl, eventtypes, and lots of other cool learning technology at Splunk.
- 1. Consider tagging these boring eventtypes as “boring”. and then filter results by “NOT eventtypetag=boring”.
- 2. Finally, when making eventtypes, it’s always a good idea to make the search as generic as possible, while still getting just the events you want. if you can avoid sourcetypes, punctuation, and extracted fields, your eventtype is easier to share, in that you don’t have to also share your props.conf, sourcetypes.conf, and transforms.conf, but maybe that ‘s a minor issue.
Update: According to Splunk lore, taken from the historical archives, safely guarded by the Knights of the Splunk Templar, David Carasso may in fact also be the father of “the search language, transaction search, sourcetype classifier, timestamping, multiline event splitting, and the phrase, “take the sh out of it”
----------------------------------------------------
Thanks!
Michael Wilde