Eating NetFlow with Splunk, Part 1

It’s easy to eat network data using Splunk. In a recent seminar I demonstrated how quickly a network administrator could dig through NetFlow data to diagnose network problems using Splunk. Here I’ll show you some steps for getting NetFlow (cflow, jflow, netstream, IPFIX, sflow) data into Splunk.

For this setup we’ll need the following:

1. A Splunk installation on a *nix platform. You can download Splunk here.
2. flow-tools.
3. DJB’s daemontools.
4. A NetFlow source. This can be a Cisco or Juniper router, or a system running nProbe.

Here are the detailed steps for setting up Splunk + NetFlow:

Please note:

• In these examples we’re using FreeBSD 6.3 amd64, the commands shown may vary on your system.
• Before running these commands make sure you’ve su‘d to root.

1. Download & Install Splunk, flow-tools & daemontools:

# pkg_add "http://tinyurl.com/splunk3-4-fbsd63-amd64"
# portinstall flow-tools
# portinstall daemontools

2. Configure flow-tools & daemontools:

# mkdir -p /var/service/flow-receive
# cat >/var/service/flow-receive/run<<EOF
#!/bin/sh
export FLOW_PIPE="/var/run/flow.pipe"
if [ ! -p "\$FLOW_PIPE" ]; then
mkfifo "\$FLOW_PIPE"
fi
/usr/local/bin/flow-receive 0/0/9800 | /usr/local/bin/flow-print -f 5 > \$FLOW_PIPE
EOF
# chmod +x /var/service/flow-receive/run
# echo "svcscan_enable=YES">>/etc/rc.conf
# /usr/local/src/rc.d/svscan.sh start
# ln -s /var/service/flow-receive /service

3. Configure Splunk:

# echo >>/opt/splunk/etc/system/local/inputs.conf<<EOF
[fifo:///var/run/flow.pipe]
disabled = false
sourcetype = netflow
EOF
# echo >>/opt/splunk/etc/system/local/props.conf<<EOF
[netflow]
AUTO_LINEMERGE = false
SHOULD_LINEMERGE = false
EOF
# /opt/splunk/bin/splunk restart

Now we’re ready to start eating NetFlow data. In Part 2 I’ll show you how to configure your network equipment to send this data, and some ways you can use this data within Splunk.

----------------------------------------------------
Thanks!
Greg Albrecht