Index with Splunk

Onscreen illustration:
Use the stack of components that we use in the company presentation, show the data as sources from the components and illustrate that it is coming into Splunk.

Onscreen bullets:

* Real-time indexing of all your IT data

Index_vo_01: "I want to show you how fast and easy it is to index all of your IT data in real time with Splunk. If a machine can generate it, Splunk can eat it. Logs, configurations, scripts and code, message queues, traps and alerts, activity reports, stack traces, metrics and performance data."

Screencast

Onscreen bullets:

* Real-time data access
o
+ Index_vo_02: "There are many ways for Splunk to access your data in real time,".

*
o Files & directories
+ (On j2ee demo, navigate to admin -> data inputs -> Files and Directories)
+ Index_vo_03: "Get live data from files, directories and FIFO queues..."

*
o Network ports
+ (Navigate to admin -> data inputs -> Network ports)
+ Index_vo_04: "...or receive data right over any UDP or TCP network port."

*
o Scripted input to connect to databases, OPSEC LEA and other APIs
+ (Show screenshot of a shell script connecting to DBI or OPSEC - I will attach one below.)
+ Index_vo_05: "You can even trigger scripts and direct the output to Splunk, to do things like query a database table via DBI, connect to an API like OPSEC LEA or capture the output of PS, TOP, IOSTAT or other system status commands."

*
o (Instead of the hosted demo use a local install on your Mac. I will help with the setup. Show screen shot home page with no data, click on admin, click on data inputs, click on add input.)
+ Index_vo_5.1: "Let's check out how easy it is to access and index some files in a particular directory. From the data inputs tab go to Files and Directories and click Add Input."

*
o
+ (Click to pull down the data access method and choose tail, type in /var/log scroll down to show other settings)
+ Index_vo_5.2 "Now choose the way you want to access the data. Select tail because you want to get live updates to any files in the directory. Type in the path of the directory you want to tail."

*
o
+ (Show the data source was added
+ Index_vo_5.3 "Your new data gets indexed right away."

*
o
+ (and then go to home page to see it eating the events)
+ Index_vo_5.4 "Check out how Splunk already indexed thousands of events."

* Universal data processing

(There's a lot here. It's intended to move very quickly - get a rhythm going of a couple of seconds per bullet.)

*
o
+ (Let's re-use the flash animation we have of hundreds of different IT data terms here.)
+ Index_vo_06: "What you'll really like is how Splunk universally processes any format of IT data without your help."

*
o Classifies data by type of source
+ (zoom in on 1-3 sourcetype field values within search results in the J2EE demo - access_combined, weblogic_jms,)
+ Index_vo_07: "Splunk automatically identifies the type of source you're indexing. Don't worry it learns new source types it hasn't seen before on the fly."

*
o Extracts original host and source location
+ (zoom in on a host value and a source value in the same results as the above)
+ Index_vo_08: "I find it really handy how Splunk keeps track of where the data originated by recording its host and source..."

*
o Extracts and normalizes timestamps
+ (zoom in on the normalized timestamp to the left of an event in the same results)
+ Index_vo_09: "And if you've ever had to deal with bizarre time formats you'll appreciate how timestamps are extracted and normalized across any data format."

*
o Locates event boundaries in single line, multi line and complex XML
+ (search for "weblogic.work.ServerWorkManagerImpl" and show a zoom-in on one of the resulting events)
+ Index_vo_10: "Everyone knows the world is not just simple single line syslog anymore. Splunk's smart algorithms find the beginning and ending of events in single line, multi line and even complex XML structures. You don't need to write regexes to parse difficult to read formats."

*
o Index every term in the original data
+ (mouse over term by term L-R in the previous event and show them highlighting)
+ Index_vo_11: "The real icing on the cake is every term in the original data gets indexed so you can find anything quickly. And there's no schema so you never have to set-up or maintain a database or a clumsy event taxonomy."

*
o Classifies events dynamically
+ (prep by enabling eventtype and punct:: menus. Search for access_combined. Start screencast by mousing over eventtype _get-366 with at least 2 get events and 1 post event in view. Open rename eventtype menu for _get-366 eventtype and type in a new name of "apache_get")
+ Index_vo_12: "You can see how Splunk dynamically classifies events. It continuously watches streams of data and records common event types based on punctuation patterns. Here you see lots of events sharing the keyword 'get' and the punctuation pattern of an apache access event. You can to rename the event types Splunk discovers or define your own."

* Secure data persistence
o (Graphic of datastore and features)
+ Index_vo_13: "Now that Splunk has been able to index your data in real-time let's look at how all your data gets persisted and secured."
o (add the uses standard filed system text and names of file systems to the previous graphic)
+ Index_vo_14: "Splunk stores your data efficiently in its own data store using any standard file system. Existing file system back-up and recovery tools can be used to maintain the datastore."
o (add compressed and unaltered text and the pictures of the IT data types)
+ Index_vo_15: "Your original data is stored compressed and unaltered. Splunk's indexes are stored in their own compressed file format."
o (add the data integrity is protected text and the MD5 hash with PKI signature subtext)
+ Index_vo_16: "Your data is also kept secure. Data integrity is protected with an MD5 hash and PKI signature to prove the data has not been tampered with."
o (add the secure activity audit trail text and the icons beneath. the icons should build one at a time with the arrows connecting them).
+ Index_vo_17: "And Splunk administrators can check any user or system activities with access to a secure audit trail."

Index_vo_17: "So that's just a little bit about what makes indexing all your IT data so fast, easy and secure with Splunk. Be sure to give it a try and happy splunking."