By Splunk April 18, 2012
This week in “That happened: notes from #splunk” a blog about the goings-on in the Splunk IRC channel: a videogaming twofer, creative uses of automation for managing up, and an extended discussion of deploying apps with highly specific whitelists ending in a surprisingly simple solution.
Everything is turning up dragons
Couples counseling in Skyrim:
<@edrabbit> I’ve put off the main quest and am running around with Lydia, Esbern, and Delphine wreaking havoc across the land
<Drainy> edrabbit: haha, ahh I might have to crack it out again
<pie|home> edrabbit: skyrim is like gta for sensitive people
<@edrabbit> I don’t think they meant for me to run around with 3 people. Occasionally Lydia and Delphine start attacking each other. Since neither can die it’s entertaining
<Drainy> I think I had Lydia die on me fairly early on
<Drainy> twas a sad day, tried to drag her off a cliff for a burial at sea but ended up falling to my own death
<@edrabbit> lol
<snowmizer> did you at least take her with you?
<@edrabbit> ever since I told my wife that you can actually marry characters in-game she’s been asking if I married Lydia yet.
<Drainy> snowmizer: aye, at least we were kind of buried together 
<snowmizer> destined to spend eternity together…at least until you reverted back to the save before that point
<Drainy> edrabbit: 😀
<Drainy> snowmizer: hah, pretty much. Till “load last save” do we part
<Drainy> the best part of the marriage is she just brings the coin in while you go out and set stuff on fire
<Drainy> if only life was as simple..
<@edrabbit> sounds like my life on occasion.
<@edrabbit> just lacking in dragons
<Drainy> heh, everything is better with dragons
<Drainy> I predict a 50% increase in dragon related fun through 2012-2013
<Drainy> think minecraft is getting/has dragons too
<pie|home> hahaha edrabbit
<pie|home> <3
<@edrabbit> we should have gone with a dragon mascot for Storm
<@amrit|wrk> lol edrabbit
<@edrabbit> every time someone signs up a flame effect goes off in the office*
*When Splunk Storm first launched its Beta, the soundsystem in the office was rigged to play a thunderclap every time someone signed up. It was awesome, in a distracting kind of way
Sure, it feels silly to cast spells out loud at the tv
But I’d buy a Kinect if it means I get to say “I’m Commander Shepard and this is my favorite store on the Citadel”:
<@Coccyx> http://arstechnica.com/gaming/news/2012/04/bethesda-adding-over-200-kinect-powered-voice-commands-to-xbox-360-skyrim.ars
<@Splunky> Coccyx’s URL: “Bethesda adding over 200 Kinect voice commands to Xbox 360 Skyrim”
<@Coccyx> seriously, who is going to play games that way?
<@Coccyx> i tried it on mass effect, it was dumb
<JPres> SHOOT DADRIC ARROW AT BLOOD DRAGON!
<^Brian^> FUS RAH DO or however it goes
<JPres> it goes: ‘pull R2 and hold”
Automation gets results
Cron is good for many things:
<Nerf> I’ve set up a schedule to email my boss (and his boss) every other week about .conf
<pie|home> what does it say, Nerf?
<Nerf> Oh, it’s going to be different each time
<Nerf> This time it just talked about what’s going to be there and mentioned University
<^Brian^> heh
<pie|home> oh so it’s not just a cron job that sends a NERFIE WOULD LIKE TO GO PLZ email
<Nerf> pie|home: Nah. I’m thinking they are going to let me go, it’s just a matter of reminding them
<Nerf> As time gets short I’ll get more shout-y
There can be only one (whitelist attribute)
Nerf cuts through the Gordian knot like a modern-day Connor MacLeod:
<JPres> heh, weird question…
<JPres> is anyone aware of a doc that explains how the whitelist attribute works in inputs.conf (within the monitor stanza)
<JPres> I see examples in other configs where whitelist.0 whitelist.1 etc is used, and I’m wondering if I can use other than numeric values to the right of the ‘.’
<duckfez> in inputs – no
<JPres> 
<ftk> JPres: i’ve never user anything but digits
<duckfez> … the (white|black)list.<n> is integer only, and to my knowledge, not supported for monitor://
<duckfez> !spec inputs
<@Splunky> http://www.splunk.com/base/Documentation/latest/Admin/Inputsconf
<JPres> So, here is what I am trying to do. Maybe I’m looking at it from the wrong angle.
<JPres> I have a deployment server set up, and, the way it makes sense, I could end up with dueling apps on a given UF with the same monitor stanza, but differeing whitelists
<duckfez> highlander rules are in effect
<JPres> and, I’d like the end result to be that the whitelists are all in effect
<JPres> but, that is not the case today
<ftk> the quickening!
<duckfez> if you have Aapp and Zapp, with the same monitor:// stanza, only one whitelist= will take effect
<JPres> yes
<JPres> so I see.
<duckfez> (and I think it’ll be Aapp’s, but I’m not sure)
<JPres> yes, whichever is first alphanumerically
<JPres> not impossible to overcome in my case
<duckfez> obviously, to stay sane one should only let a single app configure a given monitor stanza
<duckfez> even if it means making small ‘inputs-only” apps
<JPres> and that’s what I do.
<duckfez> but then you wind up with permutations, which also suck
<JPres> so, for instance, I have one serverclass ‘cap’ or customer access point, and it could contain an imapserver or a popserver, or both.
<duckfez> is there anything inherently bad about always looking for both imap / pop files?
<JPres> or, maybe I want to monitor a certain log file for a short amount of time on a given server type, would be nice to just enable that app for a short time, rather then edit the existing app.
<JPres> duckfez: always looking for imap/pop files on all caps, nothing wrong with it.
<JPres> but on my platform there are background processes that log to the same directory, and I don’t necessarily want to always be indexing those logs.
<JPres> and editing an existing app to do so is just inviting fat fingers to gum up the works. But that’s job security I suppose. heh
<Nerf> You could also be more specific about which apps get sent to which servers
<Nerf> Hav a pop app and an imap app and send either/both to the servers you want
<JPres> Nerf: that’s where I’d like to be. But it doesn’t work if they both contain the same monitor stanza in inputs.conf
<Nerf> Make the apps discrete enough and they won’t
<JPres> heh
<Nerf> We have dozens of apps. Many of them with just one monitor entry in inputs.conf
<Nerf> Some more complex, obviously
<JPres> Same here, and I can work around this by modifying my plan for forwarder apps. It actually makes the initial set of apps a bit cleaner.
<JPres> but in the future, if I want to shoot logs for, say, my configuration server to splunk, on all hosts, it’s a bit dirtier of a job than just giving one app to the UF_all serverclass
<JPres> that said, this is still far better than grep/sed/awk over 9TB of logs.
<JPres> on a 1Ghz sparc box
<Nerf> JPres: I don’t see how it’s any harder.
<Nerf> You have some app that only go certain places and other that go everywhere
<JPres> because to do so, now you have to go edit 19 apps.
<JPres> Nerf: when you have the same monitor stanza across multiple apps, only one whitelist will be in effect….
<JPres> and since I roll logs, like most folks do, I have to monitor the log directory, and whitelist a set of file names.
<Nerf> JPres: But each app has a whitelist of who it goes to
<Nerf> And I’m still not clear why you would have overlapping monitor statements
<JPres> ok, I’ll pastebin it
<JPres> http://pastebin.com/rZ0pYrnG
<@Splunky> JPres’ URL: “$ cat UF_imapserv/default/inputs.conf [monitor:///imail/log/] whitelist = ima – Pastebin.com”
<JPres> If both of those apps are deployed to the same UF, then the resulting interpreted inputs.conf will have what is listed under ‘result’
<Nerf> Why not use [monitor:///imail/log/imapserv.*]
<Nerf> And then the whitelist can be the endings you want
<JPres> brilliant
<JPres> this is why 2 minds are better than 1
<JPres> thanks Nerf
----------------------------------------------------
Thanks!
rachel perkins