Set up alerts via Splunk Web

Set up alerts via Splunk Web

An alert is comprised of

• a schedule for performing the search.
• conditions for triggering an alert.
• actions to perform when the triggering conditions are met.

Specify that an alert be sent via email or RSS, or trigger a shell script. You can turn any saved search into an alert.

Set up an alert at the time you create a saved search, or you can enable an alert on any existing saved search you have permission to edit.

Note: You must have sendmail enabled on your Splunk Server for alerts to be sent out.

View alert history

The alert history page shows which alerts have been triggered since Splunk's last reboot. To access, click the Admin link in the upper right hand corner and select the Saved Searches tab. Your alerts show up in the Alert History column.

Set up an alert

• Enter your search terms into the search bar and choose Save search... from the drop-down menu to the left of the search bar.
• Fill in the fields to save your search and then click the Schedule & Alerts link at the top of the Save Search pop up.

Note: In 3.3, schedule and alerts are done in the Save Search > Schedule and Alert tab.

Schedule

To set up an alert, check the box run this search on a schedule. Choose either basic or cron scheduling.

• Basic lets you choose from predefined schedule options.

Image:3 2Admin setupalerts-Picture 1.png

• Use Cron to specify cron-style scheduling.

Splunk supports most standard cron notation. For example:

• enter */60 * * * 1-5 to run your search every minute, Monday through Friday.

Image:3 2Admin setupalerts-Picture 2.png

Note: Too many searches running every minute or less can slow down the server.

Time ranges in search

To get all the results from a set window of time, you may include a specific time range in your search, for example hoursago=1. Especially in distributed setups, data may not reach the indexer exactly when it is generated. Thus, it is a good idea to run your searches with a few minutes of delay.

For example, you want all the results from an hour time window, such as 4 PM to 5 PM.

• Add the terms startminutesago=90 and endminutesago=30 to your search.
• Then, schedule your search to run on the half hour using cron notation.

This ensures that you get all the results from the specified time period.

Output

Choose from the drop downs to specify rules for sending output.

• Pick one of the following options from the first drop down:

Image:3 2Admin setupalerts-Picture 3.png

• Pick one of the following options from the second drop down:

Image:3 2Admin setupalerts-Picture 4.png

• Fill in the text box with a digit to configure the rules to trigger output. For example number of events is greater than 3.
• Then, choose output sending options.
  • Splunk can send email, create an RSS feed, and/or run a shell command when an alert triggers.
  • There are multiple variables you can pass to an email or shell script.
• You may configure additional options through alert_actions.conf, including:
  • Set the maximum number of results sent out during an alert by configuring.
  • Which email address originates the alert email.
  • See alert_actions.conf for details.

Set up an alert on an existing saved search

• From the drop-down menu to the left of the search bar, choose Saved searches > Manage saves searches. This will launch the saved searches window.
• In the table, locate the saved search that you want to turn into an alert.
• Click enable in the Running column.
  • If you do not have permission to edit this search, the Running column will show *No*.
  • If there is already an alert defined for this saved search, it will either be Running or give the option to start it if you have the proper permissions.
• To set up an alert, click the box next to Run this search on a schedule under Alert properties.
  • The options under Alert properties are the same described above for Schedule & Alerts.

Specify which fields to show

When you receive alerts, any fields included in your search are also be displayed. Edit the saved search to change which fields are displayed in your alert.

To eliminate a field, pipe your search to fields - $FIELDNAME. To add a field, pipe your search to fields + $FIELDNAME. You can add or subtract any number of fields -- just separate them with a comma: fields - $FIELD1, $FIELD2 + $FIELD3, $FIELD4.

For example:

This search will keep the sourcetype field from appearing in your alerts.