Client-Side Splunk!

Many of our customers use Splunk to analyze their Web traffic simply by indexing their apache or IIS server logs.  Those logs are useful, but in many cases they only provide half the picture.  This blog shows how you can send both server-side and client-side data to Splunk and have the best of both worlds.

What is server-side and client-side? Let’s say you’re reading this on  You’ve loaded this page and that action has been recorded in the apache server log, also known as server-side.  However, there are some interactions you can have with this page that won’t show up in the logs.  For example, you could “mouse over” the list of categories on the right nav or toggle the Tag list between “List” and “Cloud”.  Those are client-side interactions that didn’t require the page to reload and therefore are lost to the analytical ages, unless you have a way to capture those actions.

Great (client-side) taste!  Less (server-side) filling! In the history of web analytics, analytics tools have evolved from client-side to server-side and back again.  Most of the current roster — like Google Analytics, Omniture Site Catalyst, and WebTrends — are client-side and typically rely on “tagging”.  This means that in order to use those products, you must pepper your site with javascript code that — when triggered by a page load, mouse over, or mouse click — “phone home” to tell the analytics tool that a user loaded, clicked, or moused over something.

So, the upside of client-side tracking is that you can track any interaction.  The down side is that your client-side tagging is beholden to your analytics tool’s limits.  In some cases, you’re limited by the number of characters you can include in your call, and you’re limited by the number of events per user in a given time period.  In other words, if you have a power user that is clicking all around your site, your analytics tool may eventually stop tracking that session.  The ideal is to have both client-side and server-side tracking in the same package.

One of the (many, many) beautiful things about Splunk is that you can emulate the traditional client-side tracking offered by off-the-shelf analytics tools, but without the limits imposed by those tools.

Of course, Splunk can do it. One of the (many, many) beautiful things about Splunk is that you can emulate the traditional client-side tracking offered by off-the-shelf analytics tools, but without the limits imposed by those tools.  Plus, once the data is in Splunk, you can correlate it with your server-side logs, transactional information, heck, even the weather that day!

But how?  What’s the best way to track web application interactions with Splunk?  I asked my fellow (rock star) Sales Engineer Gilberto Castillo for some help on this and he came back with an elegant solution.  It works like this:

Good to know.  But how do I get it working for me? So, yah, good to understand the architecture, but how do you get this up and running for your web application?  This approach requires a few things: you must have PHP installed on your web server and you must have jQuery installed (that really just means downloading the jQuery script).

  1. Download logger.txt, save it to your web server root, and rename it to logger.php. Logger.txt is the PHP script that will do the heavy lifting of receiving the calls and writing them to logger.json.  Download it and save it to the server root or other folder accessible by all pages on the site.  Don’t forget to rename it from logger.txt to logger.php (wordpress doesn’t let us upload “.php” files, or we’d do it for you).
  2. Modify logger.php to reflect the right location for your logger.json file. The logger.json file is the file where all user interactions will be stored and the file that Splunk will be monitoring.  It should live in a folder that is accessible to both your web application and Splunk.  Change “/var/log/www/logger.json” on line 11 and line 40 to the location you’d like your logger.json file to live.
  3. Download index.txt, save it to your web server root, and rename it to index.html. This file contains the code you’ll need to make sure is on every page, as well as some samples of interactions and how to tag them.
  4. Modify the location of jquery.js. This script relies on the standard jQuery library.  If you already have that library installed on your web server (“installed” is a strong term — it’s just a single file that needs to be saved somewhere), modify the reference to that file on line 11 of index.txt to the correct location.  If you don’t have jQuery, go here to download the jQuery script.
  5. Modify the location of logger.php. The submitLoggerEvent function calls ‘logger.php’ around line 30 of the index.txt file.  Default is “url: ‘/logger.php'”.  Change “/logger.php” to the actual location of logger.php.  Note that, in this case,  “/” reflects the root of the web server, not the computer.
  6. Take it for a spin! At this point, you can load index.html in your browser and click on the sample buttons, pull down the sample pull down, and enter numbers in the sample field.  You should start seeing data populate logger.json.  If you don’t, fire up your error console for guidance.
  7. Modify index.html for your own uses. Once you’ve confirmed that the basic setup is working, you can start implementing this code in your own application.  You’ll need to make sure that every web page has: A reference to the jQuery library, a reference to the submitLoggerEvent and getKeys functions, and in each element you want to track: an id and a reference to the submitLoggerEvent function, such as:

<button type=”button” id=”button1″ value=”Button A” onclick=”submitLoggerEvent(‘button1’)”>

Let us know! So, we’ve covered a lot here, and we anticipate you’ll have questions.  Please don’t hesitate to reach out to me (Sondra Russell) or Gilberto Castillo.  We’d love to hear about your successes and/or challenges with this method of client-side tracking with Splunk!

Sondra Russell

Posted by