Have you ever heard that old adage, “Give someone a fish and you feed them for a day; teach someone to fish and you feed them for a lifetime”?
Obviously, this quote isn’t just about angling and bass. It means that, if you really want to help someone, don’t just give them what they want—teach them how to do get it themselves.
If we apply this metaphor to the Splunk Answers forum, there are a whole lot of people learning how to fish. And that’s what we love about about it. When someone on Answers has a problem, the community doesn’t just solve it; they teach the user how to solve it for themselves.
The Smart AnSwerS blog series exists to celebrate this learning that happens between users on the forum. Every month, we pick out three solutions from Answers that we think you should know about. A solution could be featured because it was well-explained, or particularly insightful—it could also be that the solution was just so clever that we thought it was worth noting.
So, if you're a regular follower of this blog or are just reading about it for the first time, welcome to the series! We hope that these answers teach you how to fish, or at least how to properly format the xyseries command (¬‿¬ ).
What are the basic troubleshooting steps in case of a universal forwarder and a heavy forwarder not forwarding data to Splunk?
You don’t have to look long to find a Splunk employee on Answers. Maybe you’ve seen one on the forum yourself (they're the ones with [Splunk] next to their username). Whether they're helping others or asking questions of their own, Splunk employees are an important part of the community.
The user dkolekar [Splunk] didn’t have a question, per se. Sometimes, rather than posting a question, users utilize the forum to explain something they’ve learned about a topic. That’s fine with us—we love to see the forum used in new and creative ways.
Being the overachiever that dkolekar [Splunk] obviously is, their post was ambitious: the goal was to document some basic tips for troubleshooting forwarder issues that could occur with Splunk. As you probably know, heavy forwarders and universal forwarders are an essential part of the Splunk architecture; they get your data into Splunk so it can be analyzed, parsed, and mined for valuable insights. So, if your forwarder is having issues, it's important to quickly get it working.
Not only did dkolekar [Splunk] describe the role of a Splunk forwarder for those who are new to Splunk, the user also created a detailed checklist to help users troubleshoot their problematic forwarders. And, as an added bonus, the user gcusello chimed in to give some extra info on universal forwarders.
Thanks for your contributions to the forum, all three of you!
If you’ve read this blog series before, you know I like to acknowledge users who have particularly interesting usernames. That said, it would be a shame to continue without mentioning the username SplunkMasterSnedz. I’m not sure where the inspiration for this username came from, but to me, it sounds like either an intergalactic overlord or an old school hip-hop DJ (or perhaps both at the same time?).
At any rate, the user SplunkMasterSnedz had a dataset of incoming emails. They wanted to put that data into Splunk to track how many emails were being sent to an internal email address by the same sender. However, there was a problem. For every email they received, they got two separate events: one with the sender address and another with the recipient address.
At first, SplunkMasterSnedz noticed a string that existed between the two events. The user was hoping to group the separate events by that shared string, which would effectively solve their problem. Unfortunately, it turned out the string was unreliable. With every email the user sent out, the string changed, making grouping the emails by sender impossible.
To get the information they needed, SplunkMasterSnedz needed a way to merge those two events together. However, they couldn’t figure out how to do that alone.
Thankfully the user DMohn (who was fourth place in the Splunk Answers 'Where Will Your Karma Take You' competition in January), found SplunkMasterSnedz’s question and helped them solve it. DMohn suggested they take their base search and pipe it through a transaction command, which would merge the two events together. Then, DMohn used the stats command to get a distinct count of email recipients by sender.
“This is perfect thank you!” replied SplunkMasterSnedz on the post.
That’s what we like to hear!
The user ryhluc01 had a base search that set the limit of their timespan for the previous five days. Then, they wanted to pipe that base search through an xyseries command, which would format those results so they could be displayed properly on a graph. The xyseries command successfully converted the last five days into column headers in a table with the earliest date displaying on the left and the current day displaying on the right.
However, those columns weren’t returning in the order that ryhluc01 wanted. ryhluc01 wanted those dates displayed in reverse.
To see what woodcock did to solve the problem, check out the link to the complete solution, which is in the post title above.
That’s it for now. In the words of Douglas Adams, “So long and thanks for all the fish.” ;)
If you're new to Splunk and want to get more involved, be sure to check out some of our community programs. Post a question on Splunk Answers, complete a challenge on Splunk BucketList, visit a user group meeting near you, or join the fun in the Splunk Community chat on Slack.
Until then, see you next time!
Matt St. John