Using Custom Search Commands Examples in the Splunk Python SDK

In the Python SDK there are a couple of examples that show how to use custom search commands within your application that is running in Splunk web.

Overview – Custom Search Commands & Map/Reduce

A custom search command is just a Python script that reads data in and writes data out. There are two subtypes of custom search commands:

• A streaming custom search command is one that is streamed data in. You can think of it as applying a “function”/”transformation” to each event and then writing out the result of that operation. It is a kind of “mapper“. An example of such a command might be a command that adds a field to each event.
• A non-streaming custom search command expects to have all the data before it operates on it. As such, it is usually “reducing” the data into the output by applying some sort of summary transformation on it. An example of a non-streaming command is the ‘stats’ command, which will collect all the data before it can calculate the statistics.

Note that neither of these cases precludes having previews of the data, and you can enable or disable preview functionality in the configuration.  There is much more info on custom search commands in the product documentation.

Examples in the Python SDK

User Count Example

The purpose of the app is to provide an example of how to define a custom search command, how to configure it, and what the input and output should look like in order to work with Splunk. The custom search command example can be found on the GitHub repo here.  The custom_search example is a custom Splunk app that provides a single custom search command usercount. The usercount command counts the number of processes each user has in a unix “top” event.

Twitter Example

There is also a twitter example application that takes the 1% feed from Twitter and inputs that data into Splunk. The twitter example can be found on the GitHub repo here. There are two custom search commands in that example:

• Count the top hashtags in a set of tweets: examples/twitted/twitted/bin/tophashtags.py
• Add a hashtags multivalue field to each tweet: examples/twitted/twitted/bin/hashtags.py

Staying Connected

• Watch the Splunk Python SDK on GitHub
• Online:  http://dev.splunk.com
• Twitter:  @splunkdev

Happy Splunking!

----------------------------------------------------
Thanks!
Paul Sanford