Forums: SplunkAdministration: Search fields and distributed system

In a forwarder/indexer configuration, I was assuming that the definition of additional search fields was done through the props.conf and transforms.conf files of the Central Indexer. It appears that this isn't the case and that new search fields are extracted at the forwarder. Is this correct?

I was not able to obtain any new search fields by modifying props.conf and transforms.conf and had to do this on the forwarder. I was aware that the forwarder carries out some "cooking" of the data before forwarding it on but was not aware this involved search fields.

Could you please confirm this?

Just another question: I've noticed a strange behaviour where a search field's extraction feature seems to be "disabled" if Splunk isn't able to match the regex for a while: for example I have an id that probably appears once every 100 or so events. Of course this means that my log isn't very structured but what I notice is that this id will be extracted once and then not any more after that.
Could you also explain this?

If you have the forwarder configured correctly, then the search extractions will be done at the indexer. Try indexAndForward=false in the outputs.conf on the forwarder. If you still can't get it to work, please post up the props and transforms from the index server. Really, you would probably be best off doing extracted fields - any reason for doing search?

On the second question, the search field should appear in the field picker and subsequently in the UI if it has been extracted from any of the events in the search results. A search field really just boils down to a key::value appended to the raw at index time, so if it is there the UI should display it. If you can reproduce with 3.1.5, please let support know. And again, any reason for using search vs. extracted fields?

I wanted to try search fields to start with as I understand there are more features such as the meta events. In any case, I have the same issue with extracted fields anyway. It's only by specifying the regex in transforms.conf and calling the transform in props.conf, both on the forwarder, that anything is extracted.

Another peculiar thing is that the field name that is taking is the name of the stanza in transforms.conf and not the field name specified in FORMAT.
For example:
[tradeId-test]
REGEX = new\stradeId\s\[(\d+)]
FORMAT $0 messagetype::$1
DEST_KEY = _meta

In the UI, the field is called tradeId-test and not messagetype, as the documentation suggests it should be.

I've sent my inputs, outputs, transforms and props conf files to support at splunk.

Cheers,

Ok the wrong name being assigned to the field is my fault: the stanza was missing the = sign after FORMAT.

However, I still have the issue of the indexer not carrying out the transforms in props.conf/transforms.conf