Forums: SplunkAdministration: FIFO Queues

I have created a fifo queue using the only instructions I could find on the net regarding it. found here: http://mysfitt.net/tutorials/splunk1_fifo.php they are really good but for version 2.0

I am having difficulty getting Splunk 3.3 accessing the fifo queue here are the settings used:

Full path: /var/syslog-ng/syslog_fifo.conf (I have also tried '/var/syslog-ng/')
for the server I put the IP, and for the source type I put syslog.

I have setup syslogng to populate the fifo queue on the client via the instructions found above.

after all this all I get is a "file path does not exist"
I cannot find any documentation on this so if someone could help me out, or point me to some relevant documentation I would appreciate it. This is my first time using FIFO queues in a linux environment.

Ok, I am having a hard time justifying purchasing the pro product here when I cant do something as simple as monitor my linux fifo queues. Is there anyone out there who knows if something like this is because I am trying to access the box from a windows server or if I need to pass credentials somehow or what? The documentation just isnt out there to do this. I know I can do this with a snare server but I dont like their gui as much. maybe it is time to go kiwi.

Can you explain why you are using a FIFO in the first place? I also don't understand your second comment, regarding trying to access the box from a windows server?

I am running Splunk on a windows server. I am using a FIFO queue because I dont know how to create a TCP connection with syslog-ng to splunk and I want to have the logs reach splunk reliably. If there is a better way I am open to it. I am new to logging so I am heavily reliant on how to guides. The only guide i found for splunk server and syslog-ng was for fifo queues which made sense to me.

FIFO is only going to work if both processes (Splunk and syslogNG) are on the same linux server.

With your setup, Splunk will need to be listening on TCP port 514, which can be enabled under Admin > Inputs. Next, your syslogNG host will need to be configured to forward data to the Splunk server. Check this article for the details:

http://sial.org/howto/logging/syslog-ng/

destination loghost {
tcp("splunk.example.org" port (5000));
};
log {
source(local);
filter(notdebug);
destination(loghost);
};

Cool, we will see if that works. I was under the impression that TCP did not work with the free server, don't know where I got that idea. Snare doesn't support TCP so maybe I had the two confused.
I will get back to you after I have tested it.

Ok I got it working, a lot of tweaking, your script you copied and pasted is wrong, your (5000) should be 514. But this now brings up a new problem with splunk. Once a logger is set up on a client to send data, how do we get the historical data to the splunk server?

Yeah, I just pasted it from the site I linked to.

Without an enterprise license, you cannot receive data from a Splunk forwarder which allows transmission of archive data. You could scp or rsync the data over to the Splunk server and tail it locally, but it will not be very efficient.

ya I am thinking of using netcat to push the files out. If not I am sure I could whip up a perl script to print all files in a directory to a port.

Or buy a license and be done with it :)