By Splunk July 03, 2007
It’s a little known (mainly because it’s undocumented) fact that it is possible to use the Splunk CLI to manage remote Splunk servers. This capability has been built into the product since version 2.1, and allows one to do things such as remotely manage data inputs, run searches, manage users, etc. For fairly obvious reasons, this cannot be done with commands that require Splunkd to be stopped.
The syntax is simple:
/opt/splunk/bin/splunk <command> [<subcommand>] <params> -uri https://my2ndSplunkBox:8089
The key here is the -uri parameter, which instructs the PCL to send all SOAP requests to the specified server. There are 3 pieces to the parameter: protocol, host, and port.
The protocol must be one of http or https, depending on whether or not SSL is enabled on the Splunkd port. Most users will want the latter, as recent versions of Splunk enable SSL on this port by default.
The second part is the hostname or IP address of the host that the remote Splunk server is running on. This should need no real explanation – in this case, the remote server has the hostname my2ndSplunkBox.
The last part of the argument is the Splunkd port (aka the management port). Note that this is not the port that’s used to reach the web interface, but the port that Splunkd listens on for incoming SOAP requests. If you’re unsure of what this port is, try the default, which is 8089. Alternatively, splunk show splunkd-port will display the Splunkd port that the current server is listening on.
As a practical example, one can add a tailed data input on the /var/log directory of host my2ndSplunkBox with the following command:
splunk add tail /var/log -uri https://my2ndSplunkBox:8089
The only caveat to this feature is that if you’re logged into your Splunk server via splunk login, you will have to re-authenticate when sending commands to the remote server (and once again when you resume targetting your local server by leaving off -uri). Workarounds include using the -auth parameter or the SPLUNK_USERNAME and SPLUNK_PASSWORD environment variables, but these are better left to a later post.
----------------------------------------------------
Thanks!
Amritpal Bath