TIPS & TRICKS

Don’t Let Broken Links Get You Down

Find them fast and get them under control.

Back in the day, when I wrote Java code for a living, one thing I was asked to find was a unique list of broken links in the company’s web-based management console. I wrote an http client, which recursively crawled all URIs starting from the home page and tracked the URIs with HTTP return code other than 200. It took me a few days to write/test/package it for a non-technical end user.

Today, finding broken links with Splunk and access logs can be done in mere minutes. There are plenty of web/UI testing tools available, but as some of our customers like to say “the truth is in the logs.”

The basic Splunk search simply makes use of the powerful stats command:

sourcetype=access_common OR sourcetype=access_combined status!=200 | stats values(uri) as "Broken Links" values(status) as Status by status | fields + Status, "Broken Links"

Or if you are simply looking for the file, instead of the entire URI:

sourcetype=access_common OR sourcetype=access_combined status!=200 | stats values(file) as "Broken Links" values(status) as Status by status | fields + Status, "Broken Links"

A slightly more advanced search makes use of the same stats command, but also incorporates a status description lookup. The lookup is transparent and ties the numeric status code to something more descriptive. Instructions to setup the status description lookup are available here.

sourcetype=access_common OR sourcetype=access_combined status!=200 | stats values(status) as Status values(file) as "Broken Links" by status_description | rename status_description as "Status Description"

Additionally, these searches can be simplified for succinctness. If you don’t care about relabeling/reordering columns or adding row separators, try this:

sourcetype=access_com* status!=200 | stats values(status) values(file) by status_desc

It’s not an exhaustive search of all links on your website, but a good, fast review of the visited links tracked in the http access log. No coding required. HTH!

Vi Ly
Posted by

Vi Ly

Join the Discussion