Forums: SplunkAdministration: Multiline match

I'm trying to index mail headers like this:

Received: from host.domain.com (111.11.1.111 by host2.domain.com (bla bla )
id 48C91550008654CB for news@news.com; Fri, 3 Oct 2008 08:35:47 +0000
Received: by hostxyt.domain.com (blalbabla)
id 48AA94C0022831CF for news@news.news.com; Fri, 3 Oct 2008 08:35:47 +0000
From: bla@bla.com
...

input.conf
[monitor:///var/tmp/cp/mqf/*mqf]
sourcetype = cprawmessage

props.conf
[cprawmessage]
SHOULD_LINEMERGE = true
MAX_EVENTS = 40000
CHARSET = auto
NO_BINARY_CHECK = true
BREAK_ONLY_BEFORE=goblygook
CHECK_METHOD = none
pulldown_type = true
LEARN_MODEL = false
TRANSFORMS-messageid = mqf-messageid

transforms.conf
[mqf-messageid]
REGEX = (?m)id\s(\w+)\sfor\s
FORMAT = $0 messageid::$1
DEST_KEY = _meta
REPEAT_MATCH = true
LOOKAHEAD = 4096

Only the first messageid will be indexed. I tried with different variations. Can anyone help?

This is a better example of a multi-valued field. In fields.conf:

[messageid]
INDEXED = True
INDEXED_VALUE = True

Also, try appending this to the end of a search for messageid:

"your search | nomv messageid"

There is some weirdness with multivalued fields in the UI and nomv overcomes it.

Hmm, but I was not looking for a multi-valued field, instead I was looking for fields in multiline events.

It just indexes the first field. I think I just don't got the regex right...

There are some tools out there that can help you test a regular expression.

I tried out your regular expression on the sample data you provided, and it selects this data:

id 48C91550008654CB for

id 48AA94C0022831CF for

It looks like it should get both events. For more in-depth help, please submit a support case via the web or by emailing support@splunk.com.

Thanks.