Filter and re-order
This documentation does not apply to the most recent version of Splunk.
This documentation applies to the following versions of Splunk: 3.2 , 3.2.1 , 3.2.2 , 3.2.3 , 3.2.4 , 3.2.5 , 3.2.6 , 3.3 , 3.3.1 , 3.3.2 , 3.3.3 , 3.3.4 , 3.4 , 3.4.1 , 3.4.2 , 3.4.3 , 3.4.5 , 3.4.6 , 3.4.8 , 3.4.9 , 3.4.10 , 3.4.11 , 3.4.12
Filter and re-order
Use filtering commands to filter your search results by removing duplicate results (based on user-defined constraints), using regular expression matching, using SQLite (expressions or set operations), or by specifying a time range.
Use re-ordering commands to re-order how results are displayed. Display a chosen subset of search results based on when they were found, or sort them based according to values in chosen fields.
Note: Filtering and re-ordering commands don't change the raw data within search results.
dedup
Remove duplicate events from your search results. Use dedup to obtain distinct sample events from searches that return many duplicate events. dedup keeps the first (most recent) occurring event for each combination of field values that you specify.
Note: dedup filters what events are displayed in your search results. De-duplicated events aren't removed from the index.
Syntax
dedup [keep-empty] [num] field-list [sortby clause]
Arguments
| keep-empty | keepempty=T | F (F) | If set, keepempty will keep events that contain a null value. If set to false, then events with null values are discarded. |
| num | integer (default = 1) | Optionally specify the number of results (num >0) to keep for each combination of values of the specified fields. |
| field-list | field1,field2,... | Comma-delimited list of fields to remove duplicates from. |
| sortby clause | sortby field1,field2,... (list of fields) | Specify a list of fields to sort the results by. This is equivalent to executing sort before dedup.
|
Examples
Splunk Web:
Example 1: This example searches for all events with eventtype="useraccess", then removes duplicates of events with the same host value and reports the total count of remaining events.
eventtype="useraccess" | dedup host | stats count(host)
Example 2: This example searches for all events with eventtype="useraccess", then removes duplicates and keeps three events with the same host value and reports on the total count of the remaining events.
eventtype="useraccess" | dedup 3 source by +_time | stats count(source)head
This data-processing command returns the first number n of specified results.
Syntax
head [number]
Arguments
| number | integer(10) | Specify the number of results to return. |
Examples
Splunk Web:
This example searches for events with the term "fflanda" on host1 and user is identified as "amrit". Then returns the first 20 events found.
fflanda AND (src="host1") AND (user="amrit") | head 20
localize
This data-processing command takes a list of search results as input, and returns a list of time ranges in which the results were found. maxspan specifies a time range that is the maximum span of time range to return. maxpause specifies a maximum pause time to wait between results. maxresults specifies the maximum number of results to localize. Use localize to process data for the page command (it allows page to process each span of time as an iteration).
Syntax
localize [maxspan] [maxpause] [maxresults]
Arguments
| maxspan | maxspan=integer(5m)(s | m | h | d) | Specifies the maximum span of time range to return. s=seconds, m=minutes, h=hours, d=days |
| maxpause | maxpause=integer(1m)(s | m | h | d) | Specifies the maximum amount of time to allow for a pause between search results before causing a break. |
| maxresults | maxresults=integer(10000)(s | m | h | d) | Specifies the maximum number of results to feed into the command. |
Examples
Splunk Web: Example 1:
error | localize | map search="search starttimeu::$starttime$ endtimeu::$endtime$ |transaction maxspan=1h fields=uid,qid"Example 2: Search the time range of each previous result for "failure".
... | localize maxpause=5m | map search="search failure starttimeu=$starttime$ endtimeu=$endtime$"Example 3: This example searches for the most recent events involving "root" within 30 seconds of a "fail" event. How it works: searches for "fail", gets a 30 second window around all of the events containing "fail", pick the 5 most recent 30 second windows around "fail" events, and for each time window around a fail event, search for "root".
fail | localize maxpause=30s | head 5 | map search="search root starttimeu::$starttime$ endtimeu::$endtime$" regex
This data-processing command removes results that do not match the specified regular expression. It accomplishes this by matching the values of fields to the expression. You can specify that the regular expression keep events that match the expression, or keep those that don't match.
Syntax
regex field (= | !=) regular expression
Arguments
| field | string(field name) | Field to match to the regular expression. |
| regular expression | "string" | string | A PCRE (Perl Compatible Regular Expression) supported by the pcre library to match field values to. |
Note: if you want to use the "or" ("|") command in a regex argument, the whole regex expression must be surrounded by quotes (ie. regex "expression").
Examples
Splunk Web:
This example selects events whose _raw field contains ip addresses in the non-routable class A (10.0.0.0/8).
* | regex _raw="(?<!\d)10.\d{1,3}\.\d{1,3}\.\d{1,3}(?!\d)"CLI:
This example is the same as above but shown for the CLI.
./splunk search '* | regex _raw="(?<!\d)10.\d{1,3}\.\d{1,3}\.\d{1,3}(?!\d)" '
reverse
This data-processing command reverses the display order of the entire result set. Optionally specify the number of results to return by adding a number value argument.
Syntax
reverse [number]
Arguments
| number= | integer (default=10) | Specify the number of results to return. |
Examples
Splunk Web:
This example searches for the term "fflanda" where the src="host1" and the user="amrit", then reverses the order that the first 20 results are returned.
fflanda (src="host1") AND (user="amrit") | reverse 20
set
Perform set operations on sets of search results. Pass two subsearches as arguments, and return the union, diff, or intersection of the result sets..
Syntax
set set-operator [subsearch1] [subsearch2]
Arguments
| set-operator | UNION | DIFF | INTERSECT | Define the set operation to perform. Union = "+", diff= "-", intersect = "^". |
| subsearch1 | string | The first search string to pass to the set operator. |
| subsearch2 | string | The second search string to pass to the set operator. |
Examples
Splunk Web:
Example 1: This example returns all URLs that have 404 errors but no 303 errors.
| set diff [search 404 | fields url] [search 303 | fields url]Example 2: This example returns all urls that have 404 errors and 303 errors.
| set intersect [search 404 | fields url] [search 303 | fields url]CLI:
Example 3: This example returns all URLs that have 404 errors but no 303 errors.
./splunk search 'index=sampledata | set intersect [search 404 | fields url] [search 303 | fields url]'
sort
This data-processing command sorts the search results by the given list of fields. It will return the original results, ordered according to the specified arguments.
If given more than one field, the first field passed will be the primary sort order(second=secondary... etc.). If the field is preceded by "+", then the sort will be ascending. If the field is preceded by "-", then the sort will be descending. If no "+" or "-" is specified, the sort will be ascending by default.
Syntax
sort [+ | -]field-list... [d | desc]
Arguments
| + | +(default) | Causes the sort to be ascending. | |
| - | - | Causes the sort to be descending. | |
| field-list | field1,field2,... | Space or comma-separated list of fields to pass to the sort. | |
| d | desc | (d | desc) | If specified, causes sort order to be reversed. |
Examples
Splunk Web:
This example sorts results by IP and then URL.
404 | sort ip, urlCLI:
This example sorts results by IP ascending and then URL descending.
./splunk search "* | sort +ip, -url"
tail
This data-processing command returns the last n number of specified results.
Syntax
tail [number]
Arguments
| number | integer(10) | Specify the number of results to return. |
Examples
Splunk Web:
This example searches for the term "fflanda" on "host1" and matches events that have the user "amrit". Only the last 20 results are returned.
fflanda (src="host1") AND (user="amrit") | tail 20
where
This data-processing command performs arbitrary filtering on results using an SQLite WHERE clause syntax.
Use SQLite syntax for the arguments to where. Refer to the SQLite online documentation for a reference on SQLlite expressions.
Note: where doesn't support the SQLite commands "|" and "| |" (double-bar and single-bar). "|" is used in Splunk's search language to separate commands in a search string. Quotes should be used to signify a literal string. Numbers should not be quoted.
Note: SQLite evaluates comparison expressions with non-identical type parameters. where treats quoted literal expressions as strings ( "123" is a string, NOT a number). When where performs a non-identical type comparison, the following precedence rule applies:
- NULL < numeric type < string type (or "quoted" literal)
Syntax
where filtering-expressions
Arguments
| filtering-expressions | A valid SQLite WHERE clause expression. If no filter clauses are specified, results are not filtered. Quotes are optional with where. In SQLite, they are mandatory. Example: "where == 80" is the same as "where == '80'"
|
Examples
Splunk Web:
This example performs a search on host="firewall" on src and dest fields looking for an ip address range of: 10.9.165.x/24. Notice that where uses SQLite syntax wildcards "%" instead of Splunk search syntax wildcards "*".
host="firewall" | where (src LIKE "10.9.165.%") OR (dst LIKE "10.9.165.%")Note: When not performing arithmetic or other SQLite-like functions, we recommend using '| search <condition>' instead of '| where <condition>'. You can accomplish the previous example with:
host=firewall | search src=10.9.165.* OR dst=10.9.165.*CLI:
This example returns any events whose host field matches localhost.
./splunk search "* | where host="localhost""