If you were at .conf last week you would have likely seen some of the exciting Internet of Things projects people are using Splunk for. I think Ed Hunsinger put it best:
So far I’ve heard about @splunk being used for planes (Royal Flying Doctor), trains (New York Air Brake), and automobiles (VW). #splunkconf
Watching .conf 2014 from a far in the UK, I got excited about some of my own IOT projects. Then I remembered Brian Gillmore’s call for cool projects using Splunk with the RaspberryPi. At the same moment, by pure chance, I got an email telling me AirPi circuit boards (a RaspberryPi connected weather station) were back in-stock.
And it was settled. I would build a RaspberryPi weather station and Splunk the data. Here’s how I did it.
Step 1: Assemble the AirPi
Essentially you can Splunk most data generated from a RaspberryPi and the additional components you hook up. For this project though I decided to use an AirPi circuit board and the bundled components (temperature, humidity, sound, light, air quality, and pressure sensors). You can pick up one here.
You’ll need to solder the board yourself, but don’t let this put you off. As Tom Hartley (the creator of the AirPi) notes, “we’ve had many people learn to solder using the AirPi kits!”. Soldering kits can be bought very cheaply off eBay too. Just make sure you use Rosin Core solder.
Step 2: Install the AirPi Code
There are very detailed instructions of how to install the AirPi code onto your RaspberryPi here.
For this project I used a forked version of the source code created by Haydn Williams. Haydn added a write to CSV output function. Following the instructions, I wrote all my AirPi logs out to a CSV file located on an external hard drive.
Step 3: Configure the RaspberryPi Splunk Forwarder
Grab the RaspberryPi Forwarder from apps.splunk.com. The great thing about the RaspberryPi Forwarder is that it works in exactly the same way as a regular Splunk Forwarder.
If you get stuck there is full documentation here. Brian Gillmore also wrote a great post on getting started with the Raspberry Pi Forwarder here too.
In my setup I used the Forwarder to monitor the CSV file I created as an output in step 2.
Step 4: Start Splunking
I set up my logs to be stored in an index named “rpi” so a simple search returns my AirPi’s output (assuming everything has worked well).
I put the output in an easy to read table and noticed very quickly that everything wasn’t working as expected.
index=rpi | table SimpleTime AirQuality Humidity LightLevel Pressure Temp_BMP Temp_DHT UVLevel
The results for “Pressure” and “AirQuality” remained static for all results (likely a problem with the sensors).
Step 5: Make it look sexy
I created a simple dashboard in under 10 minutes which shows me real-time and historic information about what the sensors on my AirPi have logged.
Here are some of my searches:
Light Level vs. UV Level
index=rpi | timechart avg(LightLevel) as LightLevel avg(UVLevel) as "UV Level" span=1h
Temperature vs. Humidity
index=rpi | timechart avg(Temp_DHT) as "Temp DHT" avg(Temp_BMP) as "Temp BMP" avg(Humidity) as "Humidity" span=1h
Step 6: Take on the Splunk RaspberryPi challenge
Can you improve on my weather station? Or do you have another exciting RaspberryPi project brewing where Splunk could help you collect and understand the data?
Let me know in the comments – I’d love to see what you’re working on!