Forums: SplunkPreview: File System Monitor

First kudos to you! It's really great having access to preview functions...

I've got a problem with fschange.

I just want to monitor one file ( /home/myhome/hosts ). Nothing comes in. I then tried to apply a filter, but now all files under /home/myhome are sucked in (non-recursively)

input.conf 1

[fschange:/home/myhome/hosts]
filter=hosts
pollPeriod=60

input.conf 2

[fschange:/home/myhome]
filter=hosts
pollPeriod=60
fullEvent=true
recurse=false

[filter:whitelist:hosts]
regex1=hosts

I have another problem:

Sun Dec 16 19:19:49 2007 action=update, path="/home/myhome/mdf.zip", isdir=0, size=78890155, gid=5010, uid=16426145, modtime="Mon Sep 24 09:48:29 2007", mode=81e4, hash=, chgs="modtime "

Fri Dec 14 11:23:35 2007 action=add, path="/home/myhome/mdf.zip", isdir=0, size=78890155, gid=5010, uid=16426145, modtime="Mon Sep 24 09:48:29 2007", mode=81e4, hash=

I'm getting an modtime event, even when the file has not been changed. I guess that could have something to do with the file being on an automounted NFS share.

I have also following feature requests:

-would it be possible to record additional ACL information also? On Solaris this would be the POSIX-draft ACLs on UFS, and NFSv4 ACLs (NFSv4 and ZFS)?

-why not allow auditing of binary files also? This would make a tripwire/bart etc. solution obsolete.

More to come... :-)

Shouldn't it be possible to set also set host and sourcetype in the fschange stanza, when fullEvent is enabled?

At the moment all config files are marked as source=localhost and have arbitary sourcetypes...

I'm checking on these for you now. Sorry we didn't show you any love early on! Thanks for posting.

First of all, thanks for your checking this new feature out. We really appreciate the feedback, which is why we posted the preview release. I am also sorry that I didn't get back to you earlier. Here are some answers to your questions:

1) This is a bug. I just fixed it - it will show up in the next preview release.

2) Assuming #1 has been fixed, regarding your configuration file, there is no need to white list 'hosts'. You should remove each 'filter' tag.

3) I have never encountered the spurious modtime event problem before. Like you said, it could be the NFS share, but I am just not sure. I'll try, or have QA try, an NFS share and see what happens in my environment.

4) Binary file monitoring should work right now in the first preview release. What is happening is that when 'fullEvent' is enabled Splunk's automatic file classifier detects that a file is binary and will not send the entire file to be indexed in Splunk. You should get a notification event regardless. I will run a few tests and see if this is in fact working correctly.

5) Regarding setting the host and sourcetype in the stanza. In the first preview release the 'host' metadata is hardcoded to 'localhost'. In the next preview release, the hosts real name will be looked up and used unless it is specified in the stanza. The sourcetype can be specified now for the notification events, but the automatic file classifier is used to set the sourcetype of full-file events.

Again, thanks for your in-depth help and let me know how it goes after trying the next preview release.

Rob

Thanks Rob

White/Blacklisting seems to work now.

I have a new problem with following stanza:

[fschange:/etc]
pollPeriod=60
fullEvent=true
recurse=false
signedaudit=false
followLinks=false

The directories named /etc/rc?.d are being indexed. The fschangemonitor correctly sets isdir=1.

Thu Jan 3 16:02:48 2008 action=add, path="/etc/rc0.d", isdir=1, size=512, gid=3, uid=0, modtime="Mon Jul 30 19:37:28 2007", mode=41ed, hash=

Regardless of this, splunk will create a new source named after the directory (e.g. /etc/rc0d.d) and the content of it => goblygook :-)

This is btw on Solaris.

Again, thanks.

I'll look into it and fix if needed in the next preview or GA release.

So, the entries you are getting for /etc/rc?.d are correct. The 'recurse' tag says "don't descend into the directory' - which it doesn't. It will generate a record for every directory in the directory that you are monitoring, but will not go into those directories. Try the 'monitorDirectoryChanges' flag (set it to 'False') - this will tell the system to ignore directories. Leave the 'recurse=false' in.

As far as creating a new source named after the directory. That is bad and I'll check it out.

Cheers.

Rob

I just fixed a case where this may happen (indexing a source file named /etc/rc0d.d). I was not able to reproduce it on OSX. I'll give a try on Solaris, but I'm pretty sure I fixed it. Try again on the next preview/GA release.

Thanks again. Much appreciated.

I've installed Preview 3. Thanks for fixing the bugs so far...

Unfortunatly, I've found a new one:

On Solaris/SPARC followLinks=false seems to be ignored. Symbolic links will be indexed (or the files behind them).