A few weeks ago, I attended DevOpsDays Seattle. As a big fan of Open Space, I was happy when my session proposal on DevOps metrics was accepted. And as it turns out, this is an important subject for DevOps community, as my colleague, Splunk Chief Technology Advocate, Andi Mann also noted in his blog. Below are some of the highlights from those discussions among 30+ DevOps practitioners.
Measuring Business Impact
Since the audience was from a variety of organizations of different sizes and maturity, it was great that a common theme/metric bubbled up: knowing and measuring the business impact of DevOps activities. The first step is to identify a business or customer success goal. Other relevant parameters stem from there. One can measure customer-buying behavior, a customer purchase funnel on a website, or customer experience.
It is not that hard to analyze how many customers are logging into an e-commerce website or how many successful purchases they are making. However, tying that metric to the performance of the website is another thing. This correlation is important especially in the DevOps process, where updates are continuously made. You need to know which ones positively or negatively impacted your customers.
Armed with that insight, you can connect other metrics, such as the performance of your infrastructure or third-party APIs supporting your service. The typical metrics IT teams gather are often infrastructure related—such as CPU or memory utilization. But it is only when this type of metric is analyzed in the context of customer impact to the important service performance or availability that this metric becomes truly valuable.
Organizational Needs Change, But How Much Revenue Is Always Important
Finally, the overall consensus was that in a CI and CD environment, which operational metrics are important can change from week to week. For your DevOps teams, this week the relevant metric could be the number of transactions per second or how many minutes the build or deploy takes. However, one metric that is constant is how much money is my organization making. To quote a gentleman from a local big retailer, “knowing how many dollars per API hit are we making or tying a change in the code to $ increase or decrease is pure magic.”
In the next blog post on DevOps metrics, I will explore discussion we had on measuring people and teams productivity. Even though business goals and operational metrics can change, if you have the data, you can decide what and when to measure. Until my next blog, check out our Splunk for DevOps web page to learn more how Splunk platform can empower you to measure what truly matters to you.