Forums: SplunkPreview: WinEventLog

How does the WinEventLog processor work?

I guess some fields are extracted automatically (event metadata like EventID, UserID etc), and the description is used as the event text.

If I want to index additional fields from the event text, I guess this is done the same way as any other event?

Burana400,

Currently we extract Application, System and Security event logs. If you look in %SPLUNK_HOME%\etc\bundles\default\input.conf you'll see the three source types under [WinEventLog] stanza. You can comment out or add another event log type that's custom to your system. Currently when Splunk server first runs, it will go as far back as it can to grab the oldest to the most recent event logs. After that it sits there and waits for the new ones to come.

As far as the fields extracted from each event log, currently they are hard coded, what you see is what you get. You can not tell it to extract another field from a config file or a UI page. Maybe that would be something that we can add in the future. What other fields where you looking for?

Other then that, how's Splunk running for you on Windows. What Win OS are running it on? How much resources is it taking and how many events have you indexed so far?

Thanks for posting...

Cheers,
Ledio - Windows Splunker

Hi Ledio

I'm running WinSplunk on WinXP just to get a feeling.

About additional fields to be extracted, I'm mostly thinkig about logon events (e.g. username, logon ID).

I really would like to have the eventlog processor behave like the other processors.

Thanks for your reply

Cheers
Burana400

Hi,

We have installed Splunk Preview2 on one of our Win2K3 servers in our test environement. It is configured as a forwarder so it is working well. We are grabbing nothing but WinEvent log data at the moment so I might add some tail inputs.

I noticed that there are quite a lot of SSL errors in the central index server log, not sure if this is a preview issue or whether it is a version thing (the server is Splunk 3.1.3). Also the deployment client in the Preview is not functioning, the problem could be related to the SSL errors as it appeared to be unable to transfer the bundles.

Cheers,

Nick

I'm using Splunk preview 2 (20071229) and have not been able to retrieve Security events from Windows XP workstations (non-domain members) at all; however, Windows 2003 R2 works fine. The problem is replicable on multiple workstations, so I can provide additional debugging information if anyone's interested.

As a followup to Burana400's comment - one additional thing that would be useful would be normalization and/or automatic extraction and indexing of fields from the "description" field of the Windows Event log. This is particularly valuable for security log events since the majority of the valuable information lies in the description field, unfortunately.

I'm putting together a transforms.conf that pulls out some of the most frequently-used fields from logon/logoff events for our internal use; if anyone thinks it's useful, I may put some additional time into it and make a bundle out of it. Here's a partial list of the contents (not sure if this will transmit well over the forum, feel free to mail me at james at unc dot edu for a clean copy):

[user]
REGEX = ^.*[\t]{1,}User\s+Name:[\t]{1,}(\S+)\s*\n.*$
FORMAT = winUserName::$1

[domain]
REGEX = ^.*[\t]{1,}Domain:[\t]{1,}(\S+)\s*\n.*$
FORMAT = winDomain::$1

[logontype]
REGEX = ^.*[\t]{1,}Logon\s+Type:[\t]{1,}(\S+)\s*\n.*$
FORMAT = winLogonType::$1

[logonprocess]
REGEX = ^.*[\t]{1,}Logon\s+Process:[\t]{1,}(\S+)\s*\n.*$
FORMAT = winLogonProcess::$1

[authpackage]
REGEX = ^.*[\t]{1,}Authentication\s+Package:[\t]{1,}(\S+)\s*\n.*$
FORMAT = winAuthPackage::$1

[workstation]
REGEX = ^.*[\t]{1,}Workstation\s+Name:[\t]{1,}(\S+)\s*\n.*$
FORMAT = winWorkstation::$1

[srcaddr]
REGEX = ^.*[\t]{1,}Source\s+Network\s+Address:[\t]{1,}(\S+)\s*\n.*$
FORMAT = winSrcAddr::$1

[srcport]
REGEX = ^.*[\t]{1,}Source\s+Port:[\t]{1,}(\S+)\s*\n.*$
FORMAT = winSrcPort::$1

Suggestions on improving the regexes are welcome; in most cases I've had to allow for one or more tab characters because the format of the description field varies from event to event. I understand that this situation has improved in Vista but deployment of that OS is very, very slow where I am, so we'll be parsing the older log formats for the foreseeable future.

James

"I'm using Splunk preview 2 (20071229) and have not been able to retrieve Security events from Windows XP workstations (non-domain members) at all"

Has anybody else seen this behavior?

James, the description or event log message is already being indexed on. This is what you see when you look at the search results after you do a search.

-Ledio

Ledio-of course you're right. I meant to say "extracted" rather than "indexed." Sorry for having misspoken. The Description field for Security events is pretty unwieldy it comes to extracting the data therein, although it is certainly all indexed.

Out of curiosity, does Splunk index the full eventlog message or is there a size limit? I believe the maximum length of an eventlog message is 64K, considerably longer than syslog's 1024 bytes.

James

Yes you're right, the event log message can be up to 64K and we do index all of it.

Cheers,
Ledio

After installing Splunk preview on my XP box, I, like other stated earlier, did not see the security logs as a sourcetype. Also, my inputs.conf file did not have the WinEventLog info.

Dennis

Dennis, thanks for giving preview a try.

1. %SPLUNK_HOME%\etc\bundles\default\inputs.conf should have three windows event log related inputs:

1. Windows platform specific input processor.

[WinEventLog:Application]
[WinEventLog:Security]
[WinEventLog:System]

If the above 4 lines are missing in the inputs.conf, then something really went wrong with your installation.

2. Open the event viewer and make sure there are actually windows event logs under the "Security" section.

3. If you don't mind, send us an email: "splunkpreview at splunk.com" with subject "Preview Windows", with your splunk.log attached.
It will help me debug your issue better.

Thanks again,

Ledio
Windows Splunker

[Revised on Tue, 29 Jan 2008 22:28:16 -0800]