The Yoda Analogy

After demonstrating the amazing features and capabilities of Splunk to numerous clients over the past couple years, I find that people still perceive it to be a very disruptive technology. So much so, it’s still difficult for some to truly understand the magic of Splunk.

They ask me “How is it that I can feed Splunk any kind of IT data I want, log files, SNMP traps, alerts, configuration files, xml, whatever, and know it will be indexed correctly?”

The answer is one of most powerful features of Splunk called Universal Indexing and, hopefully by the time you finish reading this article, you will have a better understanding of what that is and why it’s so powerful.

To start down that path to understanding, I would like you to think about Yoda.

Yeah, that’s right, Yoda from the Star Wars movies. You know, he’s that short funny-looking wrinkly green muppet character that speaks in a severely mixed-up manner. Remember him now?

Now what does Yoda have to do with Universal Indexing, you ask? Well, it’s not so much about Yoda, really, as it is about how Yoda talks.

Lately I’ve been explaining how Universal Indexing works by using what I call The Yoda Analogy and it goes something like this…

The way Yoda talks is so mixed-up, almost backwards, that it’s extremely confusing at first, right? It’s so out-of-order from what you and I are used to, with verbs coming last in the sentence and adjectives coming first or maybe second, and nouns and pronouns thrown in wherever there’s room to throw them in.

Yet no matter what Yoda says and no matter what order the nouns, verbs, adjectives, pronouns are arranged in, we can still figure out what he is saying and what he means.

How is this possible?

Well, if you think about it, it’s not that difficult to understand how. It’s based on how we learned to talk as children. When we were very young and first learning to speak, we did a lot of listening before we started talking, right? We listened to our parents and grandparents talk to us, maybe our older brother(s) and/or sister(s), and other friends and family members. We listened to them without any understanding of what a verb was or what a noun was or what sentence structure was. We just listened and listened and listened and one day we figured it out well enough to start talking and having conversations.

In essence, what we were really doing was sampling the sounds that people made and looking for common patterns and correlations and after a while we figured out that certain sounds and patterns had specific meanings.

Then we started to notice vocal tones and inflections and it became evident that speaking loudly or softly meant certain things and a vocal pitch that suddenly went in an “upward” direction at then end of a sentence probably meant you were supposed to respond. (i.e. we derived more structure and meaning…)

And then as we grew up and and got older, we learned to talk in more sophisticated ways and we eventually learned what a word was and what a sentence was, and a verb, and a noun, and an adjective, etc, and we learned the difference between a question and a statement and so on. (i.e. more structure and meaning…)

In other words, we learned about the complete structure and meaning of the language years AFTER we learned how to FIRST speak the language intuitively though sampling the sounds people made with their mouths.

And it was that intuitive sampling capability that allowed us to listen to Yoda speak out-of-order and still determine what he was actually saying and what he meant.

So, what does this have to do with anything, you ask?

Basically, how we learned to talk is very similar to how Splunk’s Universal Indexing works. Splunk does not assume what anything means at first. It simply indexes it, samples it, looks for patterns and correlations, and presents it to the end-user who then helps Splunk derive and apply meaning and structure AFTER THE FACT, rather than before-hand.

Take a moment right now to imagine what it would be like if you tried to teach a child how to talk by explaining to them what a word was or about the structure of a sentence was and then proceeded to explain what a noun, a verb, an adjective was, and the difference between a question and a statement. Let’s say that it was required that the child understand the complete structure of language before they could speak the language. That would be extremely limiting, if not impossible to do, right, because how can you explain ANYTHING to a child if that child does not yet know how to talk?

It’s kind of a crazy catch-22 of sorts.

Yet there are IT products and tools that ask us to do that very thing everyday. We are required to understand the structure of our IT data BEFORE we can use these tools to talk with that data and have a decent informative conversation with it, right? We have to teach our tools about log formats and database structures FIRST, before we can expect to ask a question and get an answer.

And then, to make things even more difficult, along comes some new “Yoda” log file format or IT data structure that’s all out-of-order, and we wonder why we don’t understand what it’s trying to say or what it means.

Happens every day in data centers around the world and the Splunk users know all about it because it used to be like that for them before they discovered Splunk and started using it and realizing quickly that for once, here is a tool that learns to talk the way I learned to talk, sampling quickly first and determining meaning and structure later. And because of that truly easy and intuitive experience, they find themselves saying, “Splunk makes more sense to me. You can use it and apply it more rapidly and easily to your IT data and have a casual conversation with that data, in all cases and situations, no matter what, even if it sounds like Yoda.”

So next time you find yourself struggling to get the answers you need about what’s going on within your IT infrastructure, remember The Yoda Analogy and think about how Splunk’s Universal Indexing can easily and intuitively enable you to finally listen and learn how to talk with your IT data the same way you’ve been talking with people your whole life. (BTW, if you want to, you can download Splunk now!)

----------------------------------------------------
Thanks!
Eric Gardner