The venerable old-skool Splunk forums are now closed. Feel free to search for old content here, but new posts are no longer supported.

Instead, please visit the thriving community at answers.splunk.com to ask and answer questions about your Splunk deployment and how to get the most out of it.

Forums: SplunkAdministration: How to manage duplicate data entry into index - as designed from inputs.conf

Previous Topic: Changes to active directory  |   Next Topic: Monitoring Windows servers


Posts 1–6 of 6
1
posted by: pstein  |  posts
date: November 10, 2009
permalink

Splunk v3.4.9

I have an oracle RAC cluster that has the following inputs.conf file

[monitor:///n01/oraadmin1/diag/tnslsnr/]
disabled=false
_whitelist=.*\/(alert|trace)\/(log.xml|listener\_[\w]+\.log)$
crcSalt=<SOURCE>

and from this I get the following files tailed:

/n01/oraadmin1/diag/tnslsnr
/n01/oraadmin1/diag/tnslsnr/mobridge/listener_mobridge/trace/listener_mobridge.log
/n01/oraadmin1/diag/tnslsnr/mobridge/listener_mobridge/alert/log.xml
/n01/oraadmin1/diag/tnslsnr/moberly/listener_moberly/trace/listener_moberly.log
/n01/oraadmin1/diag/tnslsnr/moberly/listener_moberly/alert/log.xml

The issue is Splunk sits on both sides of the cluster and tails files ( more that what is listed) and they both index the same data, log.xml files, because it is written to both sides of the cluster. I am trying to create one tarball I can roll out to all DB servers without having to manually input each $HOSTNAME for the files to be monitored.
In the case above I have servers "moberly" and "mobridge" files indexing on both servers....thus creating twice the necessary amount of data indexed.
I am trying to figure out how Splunk can be taught to only index the data that comes from itself and not the other side of the cluster too. Confusing?....probably.
If you have insight I would be appreciative.

PaulS

2
posted by: pstein  |  posts
date: November 10, 2009
permalink

I need to somehow get Splunk to use a variable in the dir path:

[monitor:///n01/oraadmin1/diag/tnslsnr/$HOSTNAME] to directly tell it to
only grab the files belonging to it on our NAS.
I figured out that both servers are writing to the same NAS location and indexing the data twice. Yikes.

3
posted by: gkanapathy  |  posts
date: November 10, 2009
permalink

actually you can use environment variables in monitor clauses.

4
posted by: pstein  |  posts
date: November 11, 2009
permalink

....ah come again!?
Any type of example or link to this knowledge would be beneficial.

Thanks.

5
posted by: gkanapathy  |  posts
date: November 11, 2009
permalink

The built-in Splunk monitors to its own log files, and the Windows app's monitor of the WindowsUpdate.log file are two examples. You just enter the variable into the monitor clause like you did.

6
posted by: pstein  |  posts
date: November 11, 2009
permalink

Thanks....this is what I used to index data on the NAS by the server name and no others, removing duplicate indexing...especially when it comes to Oracle RAC where data is written from both sides of the cluster to the same location...if using NAS.

[monitor:///n01/oraadmin1/diag/tnslsnr/$HOSTNAME/listener_$HOSTNAME]

PaulS