In 12 months with Splunk On-Call, our mean time to acknowledge came down from four hours to 20 minutes. Now we’re three years in, and we’re under two minutes.
PSCU Safeguards Reliability, Security Through the Splunk Platform
Key Challenges
To improve product and service availability for more than 1,500 credit unions, PSCU needed to reduce MTTA and MTTR while boosting call-team accountability.
Key Results
With the Splunk platform, PSCU slashed MTTA and MTTR while empowering staff with mobile monitoring access to deliver support from anywhere.
PSCU is the nation’s premier payments credit union service organization.
Using digital technology to its advantage, PSCU supports the success of more than 1,500 credit unions, representing more than 3.8 billion transactions annually. To better enable its credit unions to compete with banks, PSCU aimed to improve key IT performance metrics.
Outcomes
<2 min
MTTA with Splunk and Splunk On-Call, reduced from four hours previously
3.8B+
transactions enabled by improved collaboration and enterprise functionality
1,500+
credit unions supported through mobile monitoring and real- time alerting
The PSCU Advantage
As member-owned, not-for-profit financial cooperatives, credit unions exist to serve their communities. They compete with banks by offering attractive services and rates.
Here’s where PSCU comes in. Most credit unions do not have the resources to build and host their own products, so PSCU does it for them. PSCU delivers white-label applications for online bill pay, online lending, debit and credit card programs and other financial services.
“It is critical that our services and products are available for our credit union owners,” says Earl Diem, PSCU IT operations manager.
Challenge: Improve MTTA/MTTR
PSCU saw the value in reducing MTTA — an acknowledgment that in effect says “I’m on it” when an alert is received. MTTA is a key metric for reducing downtime because it triggers incident response — which lowers mean time to repair (MTTR).
“Our people were doing the ‘rotating chair’ methodology of support, using several point tools to monitor five or six disparate systems. We recognized the need for a better alternative to give us the MTTA we sought,” Diem recalls. “We wanted to aggregate system-based alerts and gain additional traceability to more effectively manage staff accountability.”
Splunk On-Call Slashes MTTA
PSCU solved its MTTA, MTTR and accountability challenges with Splunk On-Call, which empowers on-call teams to find and fix problems faster with automated and insightful incident management routing, collaboration and reviews. PSCU employs Splunk On-Call as a standard solution across 110 enterprise users. Diem keeps a graph on his wall of plummeting MTTA since PSCU started using Splunk On-Call more than three years ago.
“In 12 months with Splunk On-Call, our mean time to acknowledge came down from four hours to 20 minutes. Now we’re three years in, and we’re under two minutes,” Diem says. “Each PSCU IT department maintains an on-call schedule. Splunk On-Call brought all the managers together with one tool. We understand what we’re doing, and we all use the same escalation schedule. It drives accountability.”
Do DevOps Right
Better manage complex, cloud-native environments and deliver exceptional user experiences armed with technical agility, speed and deep insights
Staff members use Splunk On-Call mobility features to perform their support jobs from anywhere. “You can interact with the system from your desktop, from a laptop, from an iPad, through your phone,” Diem says. “The alerts in Splunk On-Call give you the supporting data from the system [that] alerted. You know what went wrong even before you look at the system.”
PSCU started using Splunk On-Call for its production environment but has extended it also to Quality Assurance and DevOps. The organization employs offshore developers in the Asia-Pacific region and India. It cannot allow system issues to interfere with productivity. Now, PSCU detects performance degradations before they turn into failures.
No matter what you do, you’re going to have failures out there. The sooner you know, the sooner you can repair it, and the better you protect your user experience.
Extending History of Success With Splunk
PSCU has long been a customer of the Splunk platform, starting with security monitoring and Payment Card Industry (PCI) compliance — a must for financial services. PSCU’s security team uses Splunk technology to aggregate and index logs from tools monitoring network and security devices. Now, PSCU has also decided to push its operational logs into the Splunk platform.
Because PSCU already relied on Splunk for PCI monitoring, “It didn’t make financial sense to maintain a separate tool for operations when Splunk can serve the whole enterprise,” Diem says.
Splunk’s machine data analytics, combined with incident response from Splunk On-Call, creates a “Platform of Engagement” that helps DevOps teams innovate faster for better customer experiences. Outstanding vendor support is another advantage, as PSCU’s relationship with Splunk brings an active user community and educational resources. “I’m pretty excited about Splunk On-Call being a part of Splunk,” Diem says.
PSCU is expanding its reliance on the Splunk platform with new use cases. One issue has been delays in detecting errors in new software releases — a problem solved by Splunk On-Call and the Splunk App for Infrastructure.
“The errors we’re not currently seeing will bubble up, alerting into Splunk On-Call as warnings, and we’ll have a team investigate,” Diem says. “The next natural progression after that would be Splunk IT Service Intelligence for predictive insight.”
The combination of Splunk and Splunk On-Call software gives PSCU a powerful means to fulfill its mission of satisfying customers. “No matter what you do, you’re going to have failures out there,” Diem says. “The sooner you know, the sooner you can repair it, and the better you protect your user experience."
Related Content
E-BOOK
The Data Age Is Here. Are You Ready?
In an era of accelerated digital transformation, give yourself a competitive advantage.
