Splunk Education: Getting Data In with Forwarders

Watch this demonstration of how to get data into Splunk Enterprise using universal forwarders.


Video Transcript

Hello. My name is Teddy Hose, part of the Splunk education team. In this video, I will be showing you how to get data into Splunk Enterprise using the Splunk Universal Forwarder. Forwarders are streamlined versions of Splunk that live on your servers and forward data to your Splunk Enterprise instance for indexing.

Let's look at a simple demonstration of how forwarders work. On the Splunk Indexer, we open ports for our Universal Forwarder to send data over. In this case, we are going to use port 9997. On the servers we want to forward data from, we install the Universal Forwarder, tell them what data to send, and where to send it. That's really all there is to it.

Now let's take this step by step to show you how it is done. The first step is to set up our receiver. We log in using an admin account on the Splunk Enterprise server that will be indexing the data. From the Settings link in the Splunk bar, we select Forward and Receiving in the data area. We select Add New for Configure Receiving.

Here we specify which TCP port the receiver should listen on. You can use a tool like netstat to make sure you are using an unused port. A screen is displayed showing that the settings were successfully saved. We restart the instance to complete the process. Setting up receiving can also be done with the Splunk CLI or through configuration files. Check the Splunk Enterprise documentation for more information.

Now that we have a receiver, we will set up a Universal Forwarder to send some data to it for indexing. We will be installing a Universal Forwarder on a web server, but you can forward almost any type of machine data to Splunk Enterprise. To download the Universal Forwarder, we go to the Splunk homepage and select Free Splunk in the top right corner.

From the Download page, we select the Download link listed under Other Products. We are given links for operating systems. In this case, we are using a Linux machine. We are given a list of architectures. We know our server is 64-bit Ubuntu, so we select the appropriate file.

Our download starts. There is also an option to use wget to get the file. We have already downloaded the file to our web server, so lets SSH in and set up our Universal Forwarder. We extract the archive to the opt directory and move into the bin directory inside of the Splunk Forwarder folder.

We start the forwarder using the standard Splunk Enterprise CLI commands. We want to accept the license, so we are using the accept-license argument. Splunk will check prerequisites and generate certificates. The Universal Forwarder should start whenever the server reboots, so we use enable boot-start command. Changing the default admin password using the edit user command is always a best practice.

Now we configure the Universal Forwarder to send data to our receiving indexer. We use the add forwarder server command, with arguments of the address and port of the receiving server. We authenticate the addition using tack -auth, followed by the username and password of the admin for the Universal Forwarder.

We need to tell the Universal Forwarder what to send to our indexer. Using the add monitor command, we point of forwarder or to where the web server's log files live. Now when we go back to our indexer, we can see that the data is being forwarded using the address of our Universal Forwarder as the hostname. As you can see, it is incredibly easy to forward data into Splunk Enterprise. Now go index some of your own. Happy Splunking.