TIPS & TRICKS

Splunking Powershell and .NET Data Structures

Splunk By Splunk September 25, 2012

We are currently rocking it at the Microsoft Exchange Conference (MEC) in Orlando and I’m being asked where we get our data from to handle the reporting and monitoring requirements for the Splunk App for Microsoft Exchange. Some of the sources are relatively straight forward – things like the Windows Event Log, IIS logs and Message Tracking logs, for example. But where do we get the rich user information? The answer lies in a series of Powershell scripts that run on a regular basis on each Exchange server. You see, Powershell has access to the whole of the .NET framework and that is where a lot of information lies.

Let’s take a quick example – splunking the Inbox Rules of all the users in Exchange. Our first step is to write a Powershell script to gather the required information. Since we are splunking the data, the only requirements are that we have a time stamp and it is in textual format. However, our best practice is to use KV pairs for the data and to put the data on one line if we can.

The Exchange Command Shell (which is Powershell with additional cmdlets) provides a cmdlet called Get-InboxRule to allow us to pull the information we need. This is really a wrapper around the .NET Framework ExchangeService.GetInboxRule method. You can find information on all the .NET Framework methods from MSDN.

As is common with Powershell, an object is returned by this cmdlet. We can iterate over the members to get the key-value pairs. Finally, we can output all that as a string to the console (which is where Splunk will read the data we produce). You can run this script within the Exchange Command Shell to see what sort of data we are looking at. I call this script “get-inboxrules.ps1”

$Mailboxes = Get-Mailbox -Server $Env:ComputerName 
foreach ($Mailbox in $Mailboxes) {
	$Id = 0
	$UPN = $Mailbox.UserPrincipalName
	$Quota = $Mailbox.RulesQuota.ToBytes()
	$Rules = Get-InboxRule -Mailbox $Mailbox
	if ($Rules -ne $null) {
		$Rules | Foreach-Object {
			$O = New-Object System.Collections.ArrayList
			$D = Get-Date -format 'yyyy-MM-ddTHH:mm:sszzz'
			[void]$O.Add($D)
			[void]$O.Add("Mailbox=`"$UPN`"")
			[void]$O.Add("Quota=`"$Quota`"")
			[void]$O.Add("InternalRuleID=$Id")	
			foreach ($p in $_.PSObject.Properties) {
				$Val = ""
				if ($_.PSObject.Properties[$p.Name].Value -ne $null) {
					$Val = $_.PSObject.Properties[$p.Name].Value
					$Val = $Val.Replace("`"", "'")
				}
				[void]$O.Add("$($p.Name)=`"$Val`"")
			}	
			Write-Host ($O -join " ")
			$Id++
		}
	}
}

Our first step is to get a list of mailboxes (or users) on the mailbox server we are running on. One of the things we do for performance is to ensure that we don’t traverse the network to get information. Now we have a list of target users, we get a list of Inbox Rules for each mailbox using the Get-InboxRule cmdlet. For each rule, we output a line that gives us all the properties of that rule. The real work of making the output Splunk ready is in the Get-Date cmdlet and the join. The Get-Date cmdlet gives the event a time stamp, and the join allows us to provide an array of key-value pairs and sends the output to Splunk as a string.

Splunk does not run Powershell natively, so we have to help it out. In addition, The Exchange Command Shell brings in the Exchange cmdlets before you run scripts. We have to employ a wrapper cmd script to do this. The script just needs to work out where Exchange is installed and then call powershell with the right arguments. I call this script “exchangepowershell.cmd”

@ECHO OFF
SET SplunkApp=TA-Exchange-2010-MailboxStore
:: delims is a TAB followed by a space
FOR /F "tokens=2* delims=	 " %%A IN ('REG QUERY "HKLM\Software\Microsoft\ExchangeServer\v14\Setup" /v MsiInstallPath') DO SET Exchangepath=%%B
Powershell -PSConsoleFile "%Exchangepath%\bin\exshell.psc1" -command ". '%SPLUNK_HOME%\etc\apps\%SplunkApp%\bin\powershell\%1'"

What this script does is firstly to look up where Exchange Server 2010 is installed and then to start up Powershell with the appropriate Exchange cmdlets preloaded. Now that we have the right scripts, we can run the following:

splunk cmd exchangepowershell.cmd get-inboxrules.ps1

It should produce the same results as when running the powershell script in the Exchange Command Shell. Our final piece of the puzzle is to actually grab the data. For this, we use a scripted input, defined in inputs.conf, where we tell Splunk to run our script on a daily basis.

[script://.\bin\exchangepowershell.cmd get-inboxrules.ps1]
index=msexchange
source=Powershell
sourcetype=MSExchange:2010:InboxRule
interval=86400

The magic is grabbing the .NET data is in utilizing powershell for the heavy lifting. This same magic is used in the Splunk App for Exchange and the Splunk App for Active Directory – both are free downloads from splunkbase.com.
With these simple techniques, you can pull data from the internal .NET data structures for any of your Windows applications – SQL Server, Sharepoint, System Center and Lync all are within your reach. It really gives you transparency for your Windows environment.

If you happen to be at the Microsoft Exchange Conference, drop by Booth #18 and ask me how you can get better data from your Windows systems. There is more useful data than just the logs.

Splunk
Posted by

Splunk