One of the best things about Splunk is the passionate user community. As a group, the community writes amazing Splunk searches, crafts beautiful dashboards, answers thousands of questions, and shares apps and add-ons with the world.
Building high quality add-ons is perhaps one of the more daunting ways to contribute. Since the recently-updated Splunk Add-On Builder 2.0 was released, however, it’s never been easier to build, test, validate and package add-ons for sharing on SplunkBase.
Technical Add-Ons, aka TAs, are specialized Splunk apps that make it easy for Splunk to ingest data, extract and calculate field values, and normalize field names against the Common Information Model (CIM). Since the release of version 6.3, Splunk Enterprise also supports TAs for custom alert actions. This allows users to take actions on Splunk alert search results by integrating with nearly any type of open system.
While I am no developer, I have tinkered with scripted alert actions in the past. Scripted alert actions existed before custom alert actions, but were more difficult to share and implement. When I saw that new version of the Splunk Add-On Builder had been released, and that it not only supported custom alert actions but also Enterprise Security Adaptive Response Framework (ARF) actions, I had to give it a try. In particular, I wanted to see if I could turn my scripted alert action that tags system in McAfee ePolicy Orchestrator (ePO) into a custom alert action and ARF action.
I downloaded and installed the Splunk Add-On Builder 2.0 to my home Splunk Enterprise 6.5 server. I went into the app and clicked “Create an add-on.” I then clicked the button to create a custom alert action. Most of the other great features of this tool around data ingestion, extraction and normalization weren’t relevant. I was quickly dropped into a very handy wizard that walks you through the entire process needed to make custom alert actions.
The wizard takes you through all the steps you need to create and describe the add-on, collect initial setup data from the user, and collect data needed for each individual alert. Perhaps the biggest hurdle to creating custom alerts in the past was the effort required to generate the initial setup screens and securely store the passwords. The Add-On Builder takes care of all of that for you! All I had to do was drag a few boxes onto a couple of screens and describe the data I was collecting – the Add-On Builder took care of everything else, including enabling secure password collection/storage, as well as providing sample code to access all the collected data in the alert action script.
Collect Setup Info and Passwords Securely
Specify Required Alert Inputs
Adding optional functionality to support Enterprise Security 4.5’s great new Adaptive Response Framework was incredibly simple. I had to ensure that I had the latest Common Information Model installed on my system, and just had to fill out 3 drop-down lists and 3 text fields to categorize the action. Enabling Splunk users to automate security responses has never been easier!
Simple Enterprise Security ARF Integration
The next step was to actually code the alert action in the tool using a little Python. The Add-On Builder provides a syntax-highlighting GUI for creating/editing the script, sample code so even a coding dunce like me will understand how to work with alert variables and search results, and a robust testing tool with logging. It’s all documented right here and here.
All I had to do was a little cut and paste, a bit of research on how to interface with the McAfee ePO web API, and the usual code troubleshooting that needs be done when you have a guy with only a history degree writing Python scripts. The helper functions in the sample code made most of it trivially easy. It was even a simple matter to enable robust logging for end users so they can troubleshoot their own deployment of my add-on.
Code and Test in the Add-On Builder
The only steps that remained were to validate that my app passed all the recommended best practices, and package it up so I could upload it to SplunkBase. Well, guess what? The Add-On Builder automates that process entirely! There’s a 1-click validation test, along with a button to package the add-on as an SPL file suitable for upload to SplunkBase.
Validate and Package
If you’re a Splunk user that uses McAfee ePO in your environment today, I recommend you check out my add-on. It will enable you to search for anything in Splunk that indicates an issue with an ePO-managed server or endpoint, and automatically tag that system so ePO can apply different policies and tasks as needed to address the issue. In addition, if you use Splunk Enterprise Security, you’ll be able to use this feature automatically when a correlation search fires and/or as an ad-hoc action when investigating notable events.
For example, if a Splunk query detects a server or endpoint is communicating with a known malicious host (e.g. through proxy logs with threat intel), this add-on can be used to tag that system as “compromised” or “infected” in ePO. ePO can then automatically run tag-specific tasks such as aggressive virus scans, and/or apply policies like blocking outbound communications via the endpoint firewall or HIPS on the compromised host. This enables true end-to-end automation between any data in Splunk and McAfee endpoint security tools.
Custom Alert Action in Use
And to take this further, if you have an idea for creating your own custom alert action to create a new Splunk integration, I strongly recommend you start by downloading the Splunk Add-On Builder from SplunkBase. It will greatly simplify the process and enable you to give back to the Splunk community. If you do so, please be sure to post a comment here – I’d love to see how others have made use of this incredible tool.