By Splunk July 05, 2012
Last week, we covered how we’re logging from Splunkbot, which is sitting in #splunk on the EFnet IRC network, to Splunk. This week, we’re going to cover how we’re using Node.js and Splunk’s JavaScript SDK to provide an open web interface sitting on top of Splunk’s data. For all of these examples, you can find a working implementation at http://splunkbot.splunk.com:8080/.
For Splunkbot, I wanted the web server to run inside the process to make everything very self-contained and easy to distribute. I chose Expressjs for the webserver framework, and with that I chose the default jade for the templating language. For the web interface, I chose to base the pages on Bootstrap to make building the markup and CSS much simpler. Of course, I could probably do a better job of not making it look like a stock Bootstrap project, but at this point it works and I’m getting lazy.
The first thing I implemented in Splunkbot’s web interface was a method of looking the last X number of URLs pasted into the channel. To do that, I first copied the example search function from the JavaScript SDK (Splunkbot.prototype.search in splunkbot.js). I built on top of that to build the Splunkbot.prototype.lasturls function, which executes a Splunk search, grabs the results, displays the results in a table on the webpage, and then calls another exposed function on the Splunkbot webserver to grab titles from a given URL asynchronously. You can see this in action on the Splunkbot URLs page.
Second, I wanted to embed the charts and graphs from the dashboard I discussed last week into a web page so anyone could see them. The JavaScript SDK makes this relatively trivial. You can see the results on Splunkbot’s Stats page, and the code to accomplish it is contained in Splunkbot.prototype.timeline, loadcharts, toptalkers and mostmentioned functions.
Third, I built an interface that would allow us to search logs for any arbitrary text and have them display back in a format that looks like an IRC client. Also, and possibly coolest, we can use the same code to run a real-time search and display an IRC client like view of the channel in real-time. Try it out, open up an IRC client and head to EFnet #splunk, and then type some text and watch the Live page, and you’ll see it show up in real-time. For the code, check out Splunkbot.prototype.logsearch and Splunkbot.prototype.livesearch.
Last, but possibly coolest of all, I wanted to show the power of mashing up Splunk with JavaScript by using a cool third party visualization library. I found a great library at thejit.org, and used the Forced Directed visualization to provide a great visual clue as to who talks to whom in #splunk and how much. It’s a great demo, and while I won’t say that its made us any money, yet, I think it shows Splunks great promise to take arbitrary, unstructured data, and turn it into interesting insight through visualization. To see the code and how I did it, check out Splunkbot.prototype.map.
Summing up
So, I encourage you to download the SDK, take a look around. If you have any problems getting started, first check out our excellent Dev site, which contains taons of documentation and examples, and also feel free to check out #splunk on EFnet (I’m Coccyx!) or hit me up on Twitter. If you build something, I’m in Marketing, and we like to tell people about things, so please get ahold of me and tell me what you’ve done so we can tell the rest of the world!
----------------------------------------------------
Thanks!
Clint Sharp