How alerting works

How alerting works

Alerts are searches you've configured to run on a schedule and send you their results. Use alerts to notify you of changes in your data, network infrastructure, file system or other devices you're monitoring. Alerts can be sent via email or RSS, or trigger a shell script. You can turn any saved search into an alert.

An alert is comprised of:

• a schedule for performing the search
• conditions for triggering an alert
• actions to perform when the triggering conditions are met

Enable alerts

Set up an alert at the time you create a saved search, or enable an alert on any existing saved search you have permission to edit. Configure alerts via:

• Splunk Web
• savedsearches.conf

Specify overall email settings for alerts

To specify the mail host, email format, subject, sender, and whether or not the results of the alert should be included inline:

• In Splunk Web, click Manager > Email alert settings and specify your choices.
• Click Save.

All alerts will now use these settings.

Scripted alerts

Alerts can also trigger shell scripts. When you configure an alert, specify a script you've written. You can use this feature to send alerts to other applications. Learn more about configuring scripted alerts.

You can use scripted alerts to send syslog events, or SNMP traps.

Customize alerts

Use the alert_actions.conf file to customize alert settings. For example, change email configuration (mail server, subject line, etc). Learn more about customizing alert options.

Considerations

When configuring alerts, keep the following in mind:

• Too many alerts/saved searches running at once may slow down your system -- depending on the hardware, 20-30 alerts running at once should be OK. If the searches your alerts are based on are complex, you should make the interval longer and spread the searches out more.
• Set a time frame for alerts that makes sense -- if the search takes longer than 4-5 minutes to run, don't set it to run every five minutes.
• You must have a mail server running on the LAN that the Splunk server can connect to. Splunk does not authenticate against the mail server.
• Read more about best practices for alert configuration on the Splunk Community Wiki, here.