TIPS & TRICKS

Splunk Command> Replace : Granular Details Are Great, But I Need a Consolidated View

When digging into data, finding the details and breaking those out part by part is a great use case for Splunk.  But what about when you need to see a more consolidated view at a higher level and just know all the little details as one line item detail?  Enter the replace command.

My experience recently came about where a customer was looking for a way to take some of their accounting data and create just one line item for multiple vendor payee names.  The idea was that the cost centers were all the same, but depending on what division within the vendor their company made a purchase from the same vendor had a variety of names.  Having the same cost center meant that accounting just viewed each detailed vendor name for that vendor as the same vendor.  Thus, they wanted to see a consolidation of all the variants of the vendor name to just one line item with their associated payments summarized to one total for each month.

While I would have loved to have used this customers data for this post, unfortunately that would be a bit too much information being shared about my customer.

So, maybe within your data center you have a various number of servers that are allocated for different divisions within your company.  One division might be named ACME or BUSDEV (Business Development) or COREDEV (Core Development).  And let’s say that you want to give each division some idea of utilization, but they could care less about each of the individual servers – they want them all consolidated together so that you provide each division with one line item with all their stats coming for all of that division’s servers’ resources.

Given this, you may start out with a search for the servers, like this:

 

index=os

| stats sum(PercentUserTime) as PUT, sum(PercentSystemTime) as PST, sum(PercentWaitTime) as PWT by src

| rename src as Server, PUT as “Percent of User Time”, PST as “Percent of System Time”, PWT as “Percent of Wait Time”

 

It yields the following results:

Splunk search server standard result

 

As you can see, the ACME division owns 6 servers and the BUSDEV division owns 8, while the COREDEV division owns 6.  To consolidate these servers, per division, down to one line item and summarize all of their utilization for each of the given fields, we re-run the search using the replace command to consolidate all of the ACME, BUSDEV and COREDEV divisions’ servers and their cumulative stats.  The new search looks like this:

index=os

| replace ACME* with ACME in src

| replace BUSDEV* with BUSDEV in src

| replace COREDEV* with “COREDEV” in src

| stats sum(PercentUserTime) as PUT, sum(PercentSystemTime) as PST, sum(PercentWaitTime) as PWT by src

| rename src as Server, PUT as “Percent of User Time”, PST as “Percent of System Time”, PWT as “Percent of Wait Time”

 

And this yields the results like this:

Splunk search server consolidation result

 

As you can see, there are many more in the server list that we could use the replace command to further consolidate our list of the division servers and their respective stats together.  What use case can you find within your organization that the replace command will make your data more accessible, useable, and valuable to everyone in your organization?

Happy Splunking!

PD2

Paul Daigle
Posted by

Paul Daigle

Join the Discussion