Splunk for Squid

Guest Blogger: Patrik Nordlén

A friend of mine called me up a while ago. He and his colleagues were evaluating a couple of log management solutions in his organization and one of the use-cases they had developed was to use the solution for getting useful information out of their Squid proxy logs. Among other things they wanted to be able to see how much traffic was flowing through their proxies, what sites were visited the most and whether their proxies were performing efficient caching. After explaining the situation my friend asked “Can Splunk do that?” to which I of course immediately replied “Sure!”.

After supplying my friend with field extractions for the default Squid log format and some initial searches to get him going, he was getting all kinds of information that had previously been a pain to retrieve from his proxy logs. With the field extractions done in Splunk, getting this kind of information was (and is) easy thanks to Splunk’s powerful data retrieval and statistics generating capabilities. Give Splunk a couple of fields to operate on and it can slice and dice their information pretty much any way you want. It almost becomes an addiction – so many things you can do, it’s harder to stop than to keep going!

If unlike me however you’re not into constantly thinking up new ways of getting information out of your data, chances are good that someone else already did. Just go to splunkbase’s app section, and you will likely find one or several Splunk apps that have already done the work for you. Like Splunk for Squid for instance. After helping my friend out with extracting the information his organization was interested in, I started thinking about what kind of information organizations generally want from their web proxies. After settling on a couple of key aspects, I created searches that would extract that information out of the proxy logs and used those searches to put together a couple of dashboards and search interfaces that provided fast and easy access to that information. Want to see which sites a particular host visited last Tuesday? See in real-time how much traffic the proxy is handling? Or check if any of your hosts have ever visited that phishing site you heard about? Easy.

And as with any app, installing it and making use of its capabilities out of the box is just the first step. If you want, there’s nothing stopping you from modifying and extending the app to suit your specific needs. Perhaps you want to trigger an automated alert if your proxy sees a request towards a site on a blacklist, or visualize the geographical location of the sites your users are visiting? If you have an idea, you’re free to make it happen, rather than being locked into someone else’s idea of what you should be able to do with your log management solution. And if you have questions on how to make your idea happen, just stop by the answers section on splunkbase where a vibrant community of Splunk users help each other out with turning ideas and questions into solutions and answers. Stop by today, join the community and when you’ve created something cool with Splunk, make an app out of it and share it with other Splunk users. Sharing is caring!