Forums: SplunkSearchAndAlert: How to setup a search for # of hosts is 1 & # of events rises by "x"

With a live search setup to watch for memory errors in Solaris I get multiple hosts answering for the Live Search. This is great, but what I need to happen is only show me a correct match when one server meets the search of 24 hits in 24 hours. Not when multiple hosts accumulative matches match 24 hits in 24 hours.

Ex. - I need to see when Tomah gets 24 hits in 24 hours not when Tomah and SanJose together get 24 hits in 24 hours.

Feb 25 13:52:43 tomah SUNW,UltraSPARC-III+: [ID 741384 kern.info] [AFT2] errID 0x007aed86.20d08e28 PA=0x000000d3.fe648e80

Feb 24 21:47:45 sanjose SUNW,UltraSPARC-III: [ID 314067 kern.info] [AFT0] errID 0x000a1dec.3f3c8f04 Corrected Memory Error on Slot C: J8001 is Persistent

The "ALERT" interface would need to offer multiple entries like this:
number of hosts = 1 AND number of events rises by 24.

Currently we only get number of hosts OR events OR sources rises by "x".

Hey PStein!

Did you try something like this?

-> sourcetype::syslog aft* AND errID OR e$tag  starthoursago=24 | top host | where count > 23

This should give you all hosts that have more than 23 memory error events in 24 hours.

As far as the threshold goes, you could just say "number of events > 0". Let me know if I am missing something.

Hey PStein!

Did you try something like this?

-> sourcetype::syslog aft* AND errID OR e$tag  starthoursago=24 | top host | where count > 23

This should give you all hosts that have more than 23 memory error events in 24 hours.

As far as the threshold goes, you could just say "number of events > 0". Let me know if I am missing something.

ARaitz - Kachiga! Kachiga! (for those who have seen Cars, the Movie).....yes. That worked for this instance. I will see if I can carry it over to other needs that require similar results.

I've had a similar issue where i want to trigger on some criteria like count > 23 but want the emailed results to email the top host. Is there an easy way to do that or would i need to have it call a script to create the search again?

That would be tough to do given the way we do alerting. You would probably need to use a scripted alert action to achieve that end.