The Business

Telenor, Norway's largest telecom services provider, believes "growth comes from truly understanding the needs of people to drive relevant change." Considering that Telenor's mobile subscribers grew from 15 to 160 million in less than a decade, its belief that deeper insight leads to success is holding true. Customers rely on Telenor to provide always-on voice, data and content services. And Splunk provides Telenor the visibility and operational insight to keep their infrastructure running at peak performance.

Challenges

With 160 million customers, thousands of servers and routers, and datacenters located throughout Norway, Europe and Asia, it was impossible for anyone to truly understand the essential operating details of the infrastructure.

Communication between far-flung departments was extremely difficult or sometimes didn't happen. Some machine data was being aggregated, but they lacked a comprehensive view. Access to single components meant access to everything—a definite security risk.

The few people with authorized access faced the impossible task of manually browsing through north of 100 GB of records a day. No wonder kernel errors and other issues sporadically slipped by unnoticed.

Enter Splunk

The Telenor team uses Splunk for troubleshooting, performance monitoring and security investigations.

Operations

The operations team uses baseline measurements so they can understand what constitutes normal. They created Splunk alerts to monitor for error spikes and unfamiliar patterns.

This advanced visibility lets them troubleshoot problems before users notice them or services fail. For example, the team learned that on average twenty errors occur across all distribution routers on the IP backbone every fifteen minutes. The day after discovering this, Splunk detected and alerted on 4,000 errors and was used to quickly determine the root cause.

Security

Once the security team determined the baseline for brute force logins and other security issues, they used easy-to-compose dashboards to monitor servers and systems for anomalous activity. By correlating timing and IPs, they now determine if attacks are coordinated. They also identify vulnerable web services.

Breakthroughs

Affordable Scalability

With the Splunk openness and ability to integrate with Telenor's existing tools, users continually think up new ways to deploy it. Unlike appliance-based solutions Splunk is software that runs on commodity hardware and nearly any operating system, including Windows, Mac, Linux, Unix, AIX or Solaris.

Productivity

Telenor has deployed Splunk in each of its regional datacenters to index data and support the local staff's searches. They also take advantage of the Splunk distributed searching capabilities that enable searches across datacenters and across all of the Splunk data when needed. The Splunk toolkit for creating ad hoc reports and dashboards gives Telenor the means to drive new efficiencies and success.

Responsiveness

Not only can the security and operations teams troubleshoot problems faster than ever, the understanding gained through Splunk baselines lets Telenor identify a problem long before it turns into a crisis. These valuable searches are now saved and run on a schedule providing proactive alerts in front of recurring issues.

Secure Access

Telenor funnels data to one of three secure Splunk instances. Role-based access controls ensure users get the access they need without compromising security or violating customer privacy regulations.

Insights

Over time, the knowledge built into Splunk has enabled the Telenor team to learn more about their infrastructure and its potential for the business. Their team is now responding to incidents more proactively and as a result providing better service.