Managing Service-Oriented Architectures

Splunk was designed to manage all phases of multi-tier deployments, but it's particularly effective in SOA environments where developers may be involved in multiple stages of development that produce applications and services residing on multiple physical servers. Typically, when something goes wrong on any one of these machines, the developer may get called to troubleshoot. For security and compliance reasons, they're usually not given direct access to these servers, and will next call someone in operations to zip up the relevant log and trace files to send to the developer via FTP. The next steps involve getting the files, unzipping them, and running various home grown scripts which usually have some derivative of Perl, Awk, and SED, to search for issues. If the results are not available for this server or it turns out another server is the culprit, the whole process is repeated - a very time and labor intensive exercise to identify and solve the problem.

Use Splunk Forwarders to Speed Data Retrieval

Splunk automates this whole effort and makes IT Search as easy as using a browser-based search engine. Splunk Light-Weight Forwarders (LWF) are installed on every leg of the SOA process to monitor and forward application-produced data to a Splunk Server. Each forwarder then sends events to one or more Splunk indexers in a Splunk-controlled automatic load balanced manner. A separate Splunk Server dedicated to performing distributed search and reporting, called a Search Head, is used by the developer to find the issue. Each event has a timestamp, host, source file name, and a classification called sourcetype to narrow down the search. In a matter of minutes, issues can be tracked down that used to take hours to find. A sample Splunk deployment for this set up is below.

Service-Oriented Architecture

In this example there are forwarders for an application server, a service bus and a BPM product. This is just for illustration purposes - a SOA tier could easily be a Web portal or a WebSphere MQ messaging system. Forwarders are light weight - in footprint, CPU utilization and network bandwidth utilization (default configuration restricts bandwidth consumed to send data to an indexer to a maximum of 256 kbps). For completeness, firewall data is also shown being forwarded.

Secure Data and Share Access

For security, Splunk role-based access controls can restrict what the developer can see and do. All application data could be put into a separate index called 'application' where the developer could only search for data where index=application. Further restrictions, such as limiting originating host or sourcetype, can also be applied to the role.

Go Home Early

With Splunk deployed software developers who are constantly called upon to troubleshoot issues in production systems and SOA deployments can go home early, as they now have direct, role-based access to the data they need to do their jobs. Splunk can also be used to proactively monitor and alert on additions, changes and deletions to the file system to speed up these types of investigations. The combination of these capabilities helps Splunk create IT Superheroes.