Splunk was designed to manage all phases of multi-tier deployments, but it's particularly effective in SOA environments where developers may be involved in multiple stages of development that produce applications and services residing on multiple physical servers. Typically, when something goes wrong on one of these machines, developers get called in to troubleshoot. For security and compliance reasons, they're usually not given direct access to these servers, and will next ask someone in operations to zip up the relevant log and trace files to send via FTP. The next steps involve getting the files, unzipping them, and searching for issues by running various home grown scripts from some derivative of Perl, Awk, and SED. If the results are not available for this server or it turns out another server is the culprit, they repeat the whole process - a very time and labor intensive exercise.
Use Splunk Forwarders to Speed Data Retrieval
Splunk automates this whole effort and makes IT Search as easy as using a browser-based search engine. Splunk Light-Weight Forwarders (LWF) are installed on every leg of the SOA process to monitor and forward application-produced data to a Splunk Server. Each forwarder then sends events to one or more Splunk indexers in a Splunk-controlled automatic load-balanced manner. To find issues, developers use a separate Splunk Server dedicated to distributed search and reporting, called a Search Head. Each event has a timestamp, host, source file name, and a classification called sourcetype to narrow down the search. In a matter of minutes, issues can be tracked down that used to take hours to find. The illustration below represents a sample Splunk deployment for this set-up.
In this example there are forwarders for an application server, a service bus and a BPM product - a SOA tier could easily be a Web portal or a WebSphere MQ messaging system. Forwarders are lightweight - in footprint, CPU utilization and network bandwidth utilization. (Default configuration restricts bandwidth consumed to send data to an indexer to a maximum of 256 kbps). For completeness, firewall data is also shown being forwarded.
Secure Data and Share Access
For security, Splunk role-based access controls can restrict what the developer can see and do. All application data can be put into a separate index called 'application' where the developer could only search for data where
index=application. Further restrictions, such as limiting originating host or sourcetype, can also be applied to the role.
Go Home Early
With Splunk deployed, developers who are constantly called upon to troubleshoot issues in production systems and SOA deployments can go home early, as they now have direct, role-based access to the data they need. You can also be use Splunk to proactively monitor and alert on additions, changes and deletions to the file system. The combination of these capabilities helps Splunk create IT Superheroes.