The venerable old-skool Splunk forums are now closed. Feel free to search for old content here, but new posts are no longer supported.

Instead, please visit the thriving community at answers.splunk.com to ask and answer questions about your Splunk deployment and how to get the most out of it.

Forums: SplunkAdministration: Log file not broken up into individual events

Previous Topic: forwarding/receiving and 4.x  |   Next Topic: Multiple different credentials


Posts 1–2 of 2
1
posted by: kdrent  |  posts
date: November 15, 2009
permalink

I'm new to splunk, but I've been working for 3 days straight to figure out how to simply input a log and have Splunk index each line as an individual event. Unfortunately, it seems to group random numbers of lines together into individual groups. Can anyone point me in the right directtion?

Some details: the file is a snort "barnyard2" fast_alert file. It has one entry per line, here is a sample:
---
11/15-21:44:11.545196 [] [1:466:5] ICMP L3retriever Ping [] [Classification: Attempted Information Leak] [Priority: 2] {ICMP} XXX.XXX.18 -> YYY.YYY.6
11/15-21:44:12.295033 [] [1:1448:13] MISC MS Terminal server request [] [Classification: Generic Protocol Command Decode] [Priority: 3] {TCP} XXX.XXX1.18:23249 -> YYY.YYY.6:3389
11/15-21:44:30.470828 [] [1:486:5] ICMP Destination Unreachable Communication with Destination Host is Administratively Prohibited [] [Classification: Misc activity] [Priority: 3] {ICMP} ZZZ.ZZZ.10 -> xxx.xxx.xxx.xxx
--

The format is very simple, 1 line a piece, starts with time stamp, ends with src and dest IPs . But when I suck it into Splunk, it groups the lines together into random groups. I need them each to be an individual event.

Thanks,
KD

2
posted by: gkanapathy  |  posts
date: November 15, 2009
permalink

If they're single line events broken by newlines (and/or carriage returns), put

[mysourcetype]
DATE_FORMAT = %m/%d-%H:%M:%S.%6N
SHOULD_LINEMERGE = true

into props.conf and import the data with the sourcetype "mysourcetype".

The reason this is happening to you is that the date format of your data is ambiguous (there is no year). Default tries to break only where it can definitely find a date, and it appears it can't always