Go Splunk Yourself!

At .conf 2013 we had a Beyond IT track that explored some of the standard and not-so-standard uses of Splunk outside of the IT organizations of the world. Sessions included using Splunk to deal with electronic health records as well as how to brew better beer with sensors and Splunk. My session was titled Go Splunk Yourself and I discussed how I’ve been tracking my own biometrics and activities on a day by day basis and feeding them into Splunk.

Shortly after I started at Splunk, a coworker gave me her old Fitbit. A few years later, I’m now tracking myself with a Fitbit, a Nike Fuelband, a Basis Band, a Garmin GPS watch, a wifi scale (Withings), a posture sensor (Lumoback), a brainwave based sleep sensor (Zeo), and various apps on my iPhone using the accelerometer and location services. Between all of these devices I can track my activities (walking, cycling, running), my geolocation (latitude, longitude, venue information), my heart rate, my perspiration, my skin temperature, my posture, my sleep cycles over the course of the night; in short where I’ve been, what I did there, and what state my body was in at the time.

All of this hardware and software is made by different companies, which means I spent a lot of time jumping between different sites and applications to see reports on my data. There were different URLs, logins, passwords, and interfaces to remember. The worst part was that each device’s data was siloed and I wasn’t able to compare data from different sources easily. I set about the task of liberating my data. I used tools created by others as well as tools I wrote myself to extract my data from these services and feed it all into Splunk. (If you’re looking to pull your data from Fitbit, Foursquare, Fuelband, Basis, or Google Latitude into Splunk you should check out my github repositories.)

Splunk’s power to consolidate all of this disparate data allows me to explore it, report on it, and analyze trends. One of the things I can easily visualize is my average heart rate over the course of any day. If I notice an abnormal spike in my heart rate, I can cross-reference my location data to see where I was, and compare it with my tweets for that day or the activity recorded by one of my activity trackers to see what I was doing. For example, I noticed on June 1st a spike in my heart rate from my normal 60-80bpm all the way up to 130bpm. My perspiration spiked as well. Using Splunk, I was able to pull up my foursquare check-ins from around the same time to see that I wasn’t at the office reading a distressing email, but rather at the go-kart track for a fellow Splunker’s birthday.

There are plenty of other interesting uses for all this data now that I have it in Splunk:

Ever wondered how the Fitbit, Nike Fuelband, and Basis Band compare to each other on daily step counts?

Want to see where I checked in on Foursquare at this year’s .conf?

Splunk search conf Foursquare checkins

How about the current state of my email inbox?

If you want to dive a bit deeper into all this, check out my post on using Splunk for Quantified Self tracking.

This is just the beginning. With the explosion of wearable tech and fitness trackers, there are more and more things you can track. The world of the Internet of Things is going to let me track not only myself, but all of the things in my house! With more time, devices, and data, the sky is the limit. Speaking of which, here are all the airports I’ve checked in at:

Ed Hunsinger

Posted by