New technologies and applications continue to emerge with great frequency and volume, but there’s an age-old method that’s really moving the needle — collaboration. Specifically, ITOps, security, and business-leader collaboration around observability practices.
But which comes first, collaboration or a fine-tuned observability practice? Trick question. A mature observability practice has comprehensive collaboration. Regardless of how strong a program might seem on the outside, it’s only truly mature when you’re collaborating on the inside. Teams need common observability data for optimal collaboration, and disciplined observability practices require cross-team coordination to continue improving the program.
Splunk’s State of Observability: The Rise of a New Business Catalyst reveals how collaboration is key to leveraging the full, comprehensive benefits of observability data. We surveyed 1,855 ITOps and engineering professionals to identify what sets high-performing teams apart. A standout group of respondents contributes to the bottom line more than their peers. They’re doing it by collaborating more with security teams, handling incidents more strategically, and investing in more forward-looking technologies and newer practices.
ITOps, security, product, and engineering teams all have distinct tasks, and that makes it all the more important for pervasive visibility across teams. Otherwise, it can get ugly.
We’ve all experienced the dreaded moment when you notice spiking login latency with backend services under load for your company’s e-commerce platform. Mass cart abandonment ensues and alerts firing across the board. And, worst of all, dipping revenue. ITOps escalates the issue to engineering, but the problem persists. Meanwhile, unbeknownst to others, the security team is investigating a potential credential-stuffing attack from a botnet. The extra traffic at login is impacting the rest of the site, but the security folks don't know that, so no one’s informed of the status. Meanwhile, the applications team sees login activity on fire, and they don't know why.
Frequently, insufficient collaboration is the main culprit causing teams to go down separate rabbit holes. They end up analyzing different data, relying on isolated dashboards and tools, making it difficult to correlate signals across teams and systems in real time. Instead of working in lockstep, they’re just locked up, isolated from the vital partners they should be joining forces with.
Observability practices that haven’t taken steps to enable collaboration are already behind.
Knowledge gaps are another significant impediment to healthy collaboration between observability, security, ITOps, and engineering teams. For example, in our State of Observability report, 41% of ITOps and engineering teams reported a lack of technical expertise and relevant skill sets outside their immediate responsibilities as a challenge. Plus, SREs and NOC engineers have very little insight into various types of security threats. Meanwhile, security teams often aren’t as worried about application performance, as long as no one’s hacking it.
Organization and information silos are particularly stifling between observability and security teams. Robust incident-response partnerships between these groups aren’t yet widespread, with a majority citing resistance to change as the biggest barrier to improving collaboration. This can cause friction and finger-pointing across the aisle.
Without real collaboration, teams waste time, effort, and resources by duplicating efforts at an alarming rate. So, how do you remedy these issues? What are advanced teams doing to optimize synergy? Is it possible to transcend from the age-old game of “Who gets the ticket?” to “Let’s work on this together”?
The benefits of improved cooperation, particularly with security teams, expand observability’s influence dramatically. Teams that work together can quickly uncover root causes, such as a credential-stuffing attack overwhelming backend resources — and rapidly solve problems to mitigate customer impact. Using shared data, dashboards, navigators, and context within an observability platform allows teams to troubleshoot in parallel. Integrating security data like application vulnerabilities into application performance monitoring is a solid place to start.
Business leaders should be working in lockstep with engineering to inform roadmap decisions and feature prioritization based on insights from telemetry. After all, telemetry data shows the real experience of your users and customers. How they authentically interact with your services is extremely valuable data for product teams to have.
Providing easy access to telemetry data for the business is the best way to make that vision a reality. This way, leaders can jump on key moments instantly. There’s no waiting for a business analyst to dig up data from separate dashboards, collate it into a report, and wait for a business review meeting. Instead, leaders can see exactly how digital experiences are impacting the bottom line, in real-time.
If a page-load time is one second longer than usual — nobody cares. Refresh your cache. Write it off as heavy traffic due to a recent campaign. But what if you have data directly correlating with the one-second slowdown to a 10% decline in revenue? Now, everybody cares. That’s immensely valuable data with a clear impact on the business.
To really nail working with security teams, you’ve got to be intentional about it. The best starting point? Sharing data between observability and security. And when everyone uses the same tools, they can quickly and easily get on the same page.
Teams need common observability data to best work together, and disciplined observability practices require cross-team coordination to continue improving the program. Comprehensive real-time collaboration happens when both observability and security teams are on the virtual frontlines together from the outset, preventing issues from slowly filtering through isolated workflows.
Teams unlock their full potential when they’re aligned. Collaboration is key. Period.
The value of observability data doesn’t end with ITOps and engineering teams. The majority of our survey respondents cite the ability for their observability solution to detect application security vulnerabilities and threats as moderately to very important to their organization’s overall business.
With more data, alerts, and business implications riding on digital experiences than ever, it’s critically important to remember the humans involved in all scenarios. Think about how your teams handle incidents. Look at post-incident reviews and think about how you can coordinate an efficient plan of action and pull in the most appropriate teams for powerful collaboration — ultimately, getting the right insights to the right people at the right time. Of course, shared data and tools should make this collaborative approach a lot easier.
To learn more about how teams can optimize collaboration through disciplined, mature observability practices, download the State of Observability 2025 report.
The world’s leading organizations rely on Splunk, a Cisco company, to continuously strengthen digital resilience with our unified security and observability platform, powered by industry-leading AI.
Our customers trust Splunk’s award-winning security and observability solutions to secure and improve the reliability of their complex digital environments, at any scale.