Search syntax

Splunk searches are designed to make searching your data easy by allowing you to perform a variety of simple term and phrase searches with Boolean and comparison logic using the Splunk search command. Use a Splunk search to explore your indexed data, and operate on it by piping search results to a variety of powerful commands to perform statistics and structured analysis on the results. Refer to the Search pipeline syntax reference for more about the search pipeline.

For example:
Search for error events from access logs.

error OR failed OR severe OR ( sourcetype=access_* ( 404 OR 500 OR 503 ) )

Keyword search

Search your data for any whole or part of a keyword by typing it into the search bar. Search for any number of keywords that you like in a single search. Separate multiple keywords by using whitespace. Splunk treats the whitespace between keywords as an AND operator.

Keywords are not case-sensitive.

Examples:
Search for "10." anywhere in your data.

10.

Search for events containing "apache" and "error".

apache error

Literals ("quotes")

Search for any literal term by wrapping it in double quotes. To search for the literal equal sign, you must wrap the search term in double quotes. For example, to search for the string a=b rather than the field a with the value b you must search:

"a=b"

In general, Splunk recommends that you wrap terms that have punctuation or whitespace in double quotes.

Make search case sensitive

Use the regex command to match events generated in a search to an exact string of characters (including case sensitive terms pronouns). Search for all events, and then filter your search results using a regex. For example, search for the word "ERROR" in all of your events:

* | regex _raw=ERROR

Search for an asterisk (*)

You can't search for an asterisk (*) character as it is always treated as a wildcard within the search command. To search for a literal asterisk, you need to search for all data and filter out results that don't contain an asterisk (using the regex command).
Here's the search you need to execute to search for an asterisk:

* | regex _raw= \*

Wildcards

Use wildcards to search for keywords or phrases that match a partial string of characters. Place wildcards at the beginning, middle, or end of a string of characters.

Note: You can also use wildcards in fields and field values.

Examples of valid wildcard usage:

foo*
*foo
f*oo
*foo*
*f*o*o*
/var/log/*

Punctuation marks

In general, Splunk recommends that you wrap terms that have punctuation or whitespace in double quotes. Splunk uses many punctuation characters as breaking characters for keywords in its index. Punctuation such as: . , ! % $ / \ [ ] { } < > @ = + & and # are considered breaking characters by default. Your Splunk administrator can customize what characters are breaking characters by tuning segmentation. However, in almost all cases it is recommended that a literal search, using double quotes, be used to find specific punctuation in events rather than changing breaking characters.

Boolean operators

Use Boolean operators to group search arguments together in a search. Splunk supports the Boolean operators: AND, OR, and NOT. Boolean operators must be completely uppercase or they are treated as regular keywords.

The terms in a Boolean expression are evaluated in the following order of precedence:
1. Parenthetical terms.
2. OR
3. AND, NOT

Use parentheses to group Boolean operator expressions together. Parentheses must have spaces on the outer (convex) side of them. Parentheses must be used when mixing OR and NOT in the same search.

Examples of correct usage:

(foo NOT (bar OR baz) )

( foo NOT (bar OR baz) )

Examples of incorrect usage:

(foo NOT(bar OR baz))

(foo NOT(bar OR baz ))

Comparison operators

Use comparison operators (=, !=, <, >, <=, >=) to exactly match a value, or a range of field values in the argument of any search command.

Note: You can only use <, >, <=, and >= with numerical field values.

Operator	Example	Effect
=	field=foo	Field values that exactly match "foo".
!=	field!=foo	Field values that don't exactly match "foo".
<	field<x	Numerical field values that are less than x.
>	field>x	Numerical field values that are greater than x.

Previous: Change default Splunk server settings | Next: Search pipeline syntax

Comments

Well, I seem to have stumbled on some markup problem.

Anyway in inputs.conf, in your monitor for this file, you can declare
sourcetype=qmail_log
or whatever makes sense for your choice of MTA.

Then you can run a search for something like:

splunk > sourcetype=qmail_log "Accepting SMTP connection from"
Verify this gets only the lines that are really SMTP connections, then save it as an event type using the
menu to the left of the search bar. Perhaps "mta_accept" would be a good name.

Now, armed with your new event type you can use the interactive field extraction.
Start with a search:

splunk > sourcetype=qmail_log eventtype=mta_accept
Then use the menu below an event timestamp to extract the fields for the ip address, to a field
for example named "ip".

Now, for these events, the ip address is an available field, which enables searches like:

splunk > sourcetype=qmail_log eventtype=mta_accept | top ip

which gives you exactly what you want.
Posted by jrodman on Dec 04 2008, 9:39am
Deektribe: a late response, but a response.

I'd do this with an eventtype and an field extraction.

http://www.splunk.com/doc/3.4.1/user/EventTypes
http://www.splunk.com/doc/3.4.1/admin/InteractiveField

This looks like a log that's only from an MTA, not mixed with other data like
syslog. We have predefined sourcetypes for Exim, and some postfix output. So
your file may already be assigned a reasonable sourcetype. You can see
some of the provided types here:
http://www.splunk.com/doc/3.4.1/admin/PreTrainedTypes

If the file doesn't have a nice sourcetype already, you can create new ones.

inputs.conf
Posted by jrodman on Dec 04 2008, 9:33am
Opps, here is the actual log:Tue 2008-11-18 18:49:43: ----------
Tue 2008-11-18 18:49:43: Session 1772; child 2; thread 2812
Tue 2008-11-18 18:49:43: Accepting SMTP connection from [173.8.69.117 : 4727]
Tue 2008-11-18 18:49:43: --> 220-cnets.net ESMTP MDaemon 9.5.2; Tue, 18 Nov 2008 18:49:43 -0800
Tue 2008-11-18 18:49:43: --> 220 Unauthorized relay prohibited
Tue 2008-11-18 18:49:43: Connection closed
Tue 2008-11-18 18:49:43: SMTP session terminated (Bytes in/out: 0/103)
Tue 2008-11-18 18:49:43: ----------
Tue 2008-11-18 18:49:43: Session 1773; child 2; thread 2812
Tue 2008-11-18 18:49:43: Accepting SMTP connection from [173.8.69.117 : 4728]
Tue 2008-11-18 18:49:43: --> 220-cnets.net ESMTP MDaemon 9.5.2; Tue, 18 Nov 2008 18:49:43 -0800
Tue 2008-11-18 18:49:43: --> 220 Unauthorized relay prohibited
Tue 2008-11-18 18:49:43: Connection closed
Tue 2008-11-18 18:49:43: SMTP session terminated (Bytes in/out: 0/103)
Tue 2008-11-18 18:49:43: ----------
Tue 2008-11-18 18:49:43: Session 1774; child 2; thread 2812
Tue 2008-11-18 18:49:43: Accepting SMTP connection from [173.8.69.117 : 4729]
Tue 2008-11-18 18:49:43: --> 220-cnets.net ESMTP MDaemon 9.5.2; Tue, 18 Nov 2008 18:49:43 -0800
Tue 2008-11-18 18:49:43: --> 220 Unauthorized relay prohibited
Tue 2008-11-18 18:49:43: Connection closed
Tue 2008-11-18 18:49:43: SMTP session terminated (Bytes in/out: 0/103)
Tue 2008-11-18 18:49:43: ----------
Tue 2008-11-18 18:49:43: Session 1763; child 1; thread 2364
Tue 2008-11-18 18:49:41: Accepting SMTP connection from [67.159.202.80 : 40751]
Tue 2008-11-18 18:49:41: --> 220-cnets.net ESMTP MDaemon 9.5.2; Tue, 18 Nov 2008 18:49:41 -0800
Tue 2008-11-18 18:49:41: --> 220 Unauthorized relay prohibited
Tue 2008-11-18 18:49:41: <-- HELO hemlo.net
Tue 2008-11-18 18:49:41: --> 250 cnets.net Hello hemlo.net, pleased to meet you
Tue 2008-11-18 18:49:41: <-- MAIL FROM: <ret@hemlo.net>
Tue 2008-11-18 18:49:41: Performing SPF lookup (hemlo.net / 67.159.202.80)
Tue 2008-11-18 18:49:42: * Result: none; no SPF record in DNS
Tue 2008-11-18 18:49:42: ---- End SPF results
Tue 2008-11-18 18:49:42: --> 250 <ret@hemlo.net>, Sender ok
Tue 2008-11-18 18:49:42: <-- RCPT TO: <wmboddy@cnets.net>
Tue 2008-11-18 18:49:42: wmboddy@cnets.net is an alias for wmboddyart@cnets.net
Tue 2008-11-18 18:49:42: Performing DNS-BL lookup (67.159.202.80 - connecting IP)
Tue 2008-11-18 18:49:42: * bl.spamcop.net - passed
Tue 2008-11-18 18:49:42: * sbl-xbl.spamhaus.org - passed
Tue 2008-11-18 18:49:42: ---- End DNS-BL results
Tue 2008-11-18 18:49:42: --> 250 <wmboddy@cnets.net>, Recipient ok
Tue 2008-11-18 18:49:42: <-- DATA
Tue 2008-11-18 18:49:42: Creating temp file (SMTP): c:\mdaemon\temp\md50000460026.tmp
Tue 2008-11-18 18:49:42: --> 354 Enter mail, end with <CRLF>.<CRLF>
Tue 2008-11-18 18:49:42: Message size: 8649 bytes
Tue 2008-11-18 18:49:42: Performing DomainKeys lookup (Sender: return@hemlo.net)
Tue 2008-11-18 18:49:42: * File: c:\mdaemon\temp\md50000460026.tmp
Tue 2008-11-18 18:49:42: * Message-ID: 1227061276.710@hemlo.net
Tue 2008-11-18 18:49:42: * Querying for policy: hemlo.net
Tue 2008-11-18 18:49:42: * Querying: _domainkey.hemlo.net ...
Tue 2008-11-18 18:49:42: * DNS: Name server has no records of the requested type for that domain
Tue 2008-11-18 18:49:42: * Result: pass
Tue 2008-11-18 18:49:42: ---- End DomainKeys results
Tue 2008-11-18 18:49:42: Performing DKIM lookup
Tue 2008-11-18 18:49:42: * File: c:\mdaemon\temp\md50000460026.tmp
Tue 2008-11-18 18:49:42: * Message-ID: 1227061276.710@hemlo.net
Tue 2008-11-18 18:49:42: * Result: neutral
Tue 2008-11-18 18:49:42: ---- End DKIM results
Tue 2008-11-18 18:49:42: Passing message through AntiVirus (Size: 8649)...
Tue 2008-11-18 18:49:42: * Message is clean (no viruses found)
Tue 2008-11-18 18:49:42: ---- End AntiVirus results
Tue 2008-11-18 18:49:42: Passing message through Spam Filter (Size: 8649)...
Tue 2008-11-18 18:49:43: * 0.0 HTML_MESSAGE BODY: HTML included in message
Tue 2008-11-18 18:49:43: * 0.3 HTML_FONT_BIG BODY: HTML tag for a big font size
Tue 2008-11-18 18:49:43: * 8.0 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist
Tue 2008-11-18 18:49:43: * [URIs: hemlo.net]
Tue 2008-11-18 18:49:43: ---- End SpamAssassin results
Tue 2008-11-18 18:49:43: Spam Filter score/req: 8.30/7.0
Tue 2008-11-18 18:49:43: Message refused because spam score is too high
Tue 2008-11-18 18:49:43: --> 554 Sorry, message looks like SPAM to me
Posted by deektribe on Nov 19 2008, 7:02pm
Say your log look like below:

How would I design a search such that it would find all ip addresses looking like this:

Accepting SMTP connection from [67.159.202.80 : 40751]

more like: Accepting SMTP connection from [*.*.*.* : *]

then sort then by occurancelike

192.168.1.1 (234 times)
192.168.1.5 (654 times)

etc....is that clear?

basically I am trying to id spammers inbound smtp ip address because they try to connect more then any other real client ip addresses
Posted by deektribe on Nov 19 2008, 7:01pm
It would be nice to know how I can search by most frequent occurrences using the search syntax. Is there a way to do this?
Posted by cronos4d on Mar 10 2008, 3:51pm