Forums: SplunkAdministration: Automatic Archiving Not Working

I'm testing out automatic archiving, but i can't seem to get it to work.

Here is what i'm doing:
i added the following stanza to my etc\system\local\indexes.conf file

[main]
frozenTimePeriodInSecs = 3600
coldToFrozenScript = WindowsCompressedExport.bat "$DIR"

i placed the WindowsCompressedExport.bat file in C:\Program Files\Splunk\bin (i also put it in C:\Program Files\Splunk\bin\scripts and C:\Program Files\Splunk\etc\system\bin to cover all of the bases)

the script i'm using is a version off the splunk wiki, and i included it below.
after making the changes to indexes.conf i restarted splunk, but I'm not getting any backup files or error messages.

What am i missing? I'm using a frozen time of 1 hour for testing purposes. Is that time span too small?

i'm testing on splunk 4.0.8 build 73243 on Windows XP 32bit.

thanks,

------ script ----------

@echo off

:: Windows compressed archive bat file to use with indexes.conf coldToFrozenScript
::
:: READ THIS!!!
:: Please modify dest_base to be your archive location, please do not
:: put a trailing '\' at end of path
::
set dest_base=c:\backups

::
:: EXAMPLE OF FOLLOWING VALUES BELOW:
:: source_path is: C:\Program Files\Splunk\var\lib\splunk\defaultdb\colddb\db_234234232_132342342_3
:: source_base is C:\Program Files\Splunk\var\lib\splunk\defaultdb\colddb:: source_leaf is db_234234232_132342342_3
::
set source_path=%1
set source_base=%~dp1
set source_leaf=%~nx1
set dest_final=%dest_base%\%source_leaf%
echo source_path is %source_path%
echo source_base is %source_base%
echo source_leaf is %source_leaf%
echo dest_final is %dest_final%
for %%i in (%1\*.tsidx) do splunk-compresstool.exe -M "%%i"
::
:: we have to manually create the destination directory and provide that
:: as a target to xcopy since it only is able to copy the source directory contents
:: and not the directory itself
::
mkdir %dest_final%

xcopy %1 %dest_final% /E /I /C /Y

I'm not sure if you need to specify the full path to the script. However, yes it is very likely that your time span is too small. Splunk will only freeze out data when all data in an entire bucket is older than the frozen time period. If you're just testing, you may not yet have rolled a bucket (or, less likely but also possible, your data is getting mis-timestamped or has the wrong time zone, so the most recent event is still in the future).

On 64-bit systems, a bucket is 10 GB in size, and this could hold anywhere between 5 GB and 100 GB of raw data (typical is about 20 GB of raw).

If it's the former problem, and you're just testing, you might try forcing a roll of the hot bucket by running "| debug cmd=roll index=main". This won't help if you have bad/future timestamps.

You could also force buckets to be rolled daily by setting maxHotSpanSecs for the index in indexes.conf to 86400 (seconds in a day), then wait a day for the bucket to roll. This second approach will also deal with mistimestamping by forcing things into daily buckets, but be aware that if you ever try to index older data or have bad timestamping, this setting will almost certainly be wildly inappropriate.