By Splunk May 14, 2010
It seems like just yesterday that I blogged about using Splunk for WebSphere environments! With typical Splunk speed, I am now pleased to announce that the beta version of the Splunk add-on for WebSphere is now on Splunkbase!
First things first – What’s an add-on?
An add-on is meant to simplify the process of getting data in to Splunk and add the right amount of contextual intelligence for IBM WebSphere application server environments(WAS). So now, you download Splunk + this add-on onto your WAS servers , enter some information about your WAS servers and point it to your Splunk server! Logs, configurations and JMX metrics from your WAS deployment will automatically start flowing into Splunk. (of course, if you don’t want some of these, you can turn them off too)
What can you do with this?
Well, as we discussed in the previous post , logs are important while finding trouble spots or pinpointing issues. The add-on not only collects all the text based WAS logs (including the super-important SystemOut.log , SystemErr.log, ServerExceptionLog, ServerPID log and StartStop Server logs) but also the binary activity log converted to human readable format. WAS logs are typically of the format <timestamp><threadId><shortName><eventType>[className][methodName]<message>.
With the add-on, these fields are automatically extracted – which makes it easy to do something like set Splunk to watch for <eventType> = F or E ( fatal or error messages) and you will know as soon as something goes wrong – by drilling down on method names or class names, you will know exactly what part of your app server caused the error!
Another example : Use the ThreadID fields with the transaction command to pull together everything related to a particular thread across your distributed infrastructure. Are you getting as excited about this as I am?!
Ho hum – I already did that in my sleep, Leena – What ‘s really new?
So far, you could do this with Splunk natively indexing WAS logs and you inserting the right field extractions; the add-on just makes this faster and easier. What’s new and really different about the add-on is that it can pull JVM metrics very easily too! In an IBM WAS environment, this means you have to enable PMI…and once this is done, the metrics you chose will start getting collected by Splunk.
Why is this so cool? Well, now you get deep runtime visibility into your JVMs added to the flexibility of Splunk. Let’s say you want to watch a metric like thread pool usage or data source connection usage. With Splunk collecting the metrics, it’s a piece of cake to not only view this metric in real time, but also compare it with historical numbers or alert when it exceed a certain value, get a stacked chart comparing various web applications etc etc. Splunk easily and automatically understands the right fields and lets you monitor, alert and report on them in real time.
The add-on also lets you monitor changes to any of the WAS configuration files so that unauthorized changes that make your environment go nuts are detected early and upfront.
So what’s next?
While the add-on simplifies the process of getting data into Splunk, the pointy haired guys (see below cartoon to identify this creature correctlyJ) really want pretty charts and dashboards that show you are in control. So tell us – What are the reports/views /dashboards that you would really really like to see built on top of this data? We can package that up into an app, so everyone doesn’t have to spend time building the most common reports over and over again..
So, download the add-on, spend time with it, tells us if it is useful or not and tell us what you’d like to see “reports-wise” by emailing websphere-beta AT splunk.com
P.S A big thanks and shout out to Dave Jones from Perficient for his help with this add-on and blogs – He is quite the expert on IBM WebSphere!
P.P.S If you have questions – you can either use this link http://answers.splunk.com/questions/ask?tags=app-websphereappserver-splunk
Or email me directly at ljoshi AT splunk.com
----------------------------------------------------
Thanks!
Leena Joshi