Case Study

EnerNOC Lowers Energy Consumption While Accelerating Application Delivery

Executive Summary

EnerNOC is a Boston-based provider of energy intelligence software that enables its customers—electric power grid operators, businesses and utilities worldwide—to optimize the use of energy by processing and analyzing large streams of data on consumption, prices and weather. EnerNOC built a solution to analyze data from system and Web logs, but it was difficult to scale and experienced frequent crashes. Since deploying Splunk Enterprise, the company has seen benefits including:

  • Real-time operational visibility
  • Improved DevOps collaboration
  • Enhanced user experience
    • Needed to monitor the operational information flowing through the Energy Intelligence Software (EIS) platform, in real time
    • Requirement to provide highly detailed user analytics data
    • Homegrown solution to analyze data was not scalable
    • Lack of visibility into customer behavior
    • Needed more seamless collaboration between development and operations
Business Impact
    • Gained real-time operational visibility into data flow through the Energy Intelligence Software (EIS) platform
    • Able to provide customers with real-time metrics on energy usage and performance, improving user experience
    • Accelerated application development & testing
    • Increased value through business analytics
    • Improved DevOps collaboration
    • Seamless execution of real-time analytics without service disruption
    • Ongoing application refinement for greater customer value and insight into customer behavior
    • Processing of data with high error-free throughput and near zero latency
Data Sources
    • Application server logs
    • Apache web server logs
    • Custom application logs
    • Linux system logs
    • Cloud-based testing tools
    • Databases

Why Splunk

EnerNOC's Energy Intelligence Software (EIS) platform continuously monitors real-time energy data for its enterprise customers, including data from more than 30,000 energy sensors/smart meters deployed globally. The platform processes and analyzes data streams and notifies customers of opportunities to conserve energy and reduce costs. From a DevOps perspective, the challenges in running this growing platform included monitoring machine-to-machine data while providing detailed user analytics data.

EnerNOC developed a homegrown solution to analyze data from system and web logs. While it worked well initially, the solution proved difficult to scale and experienced crashes as the number of data streams grew. Although EnerNOC had the resources to improve the system, it wanted to focus on productizing big data for EIS.

EnerNOC's principal engineer downloaded Splunk Enterprise, began feeding in system logs from the EIS platform and built a dashboard to visualize the data. The company soon added other data sources from multiple systems into Splunk software and developed more dashboards. EnerNOC eventually placed its Splunk Enterprise platform on the Amazon Web Services (AWS) cloud in redundant regions to ensure the solution's availability, eliminate the management demanded by on-premises servers and provide scalability.

“The true value for our customers is that they can look at their data online, turn the lights off, for example, and then see their chart drop within minutes, along with cost information to provide instant value. Splunk is a key behind-the-scenes player to make sure that the real-time data is getting processed quickly, analysis of big data historical data sets scales as needed and that there are no errors.”

Jim Nichols, Principal Engineer, EnerNOC

Global operational visibility with seamless DevOps collaboration

EnerNOC's EIS platform collects millions of data points daily on customer energy usage and production by utilities. The firm monitors consumption and output for demand-response events while tracking performance of the data-collection infrastructure to meet SLAs. Splunk Enterprise provides real-time operational visibility into the flow of data through EIS. With views into the platform's public and private cloud components, administrators can perform workload and user analytics in real time and over large historical data sets. DevOps staff have set up Splunk alerts for nearly 200 system operational events. Administrators receive immediate notification when data isn’t flowing or components are experiencing issues, and automated or manual steps can be taken to avoid disruption of critical services. Splunk Enterprise monitors core platform services, ensuring that data is processed, with high error-free throughput and near zero latency.

In addition, Splunk Enterprise plays a vital role in creating a dynamic DevOps environment. Using Splunk Enterprise for real-time metrics, EnerNOC's developers and QA team test code in staging environments to gauge functionality, scalability and performance under peak loads. The DevOps team then relies on the same Splunk dashboards to track new software services the moment they are placed into production to preserve reliability and customer satisfaction.

Greater value through business analytics

Because of Splunk technology, EnerNOC customers are able to access their energy usage data with an exceptional user experience, high performance and reliability, either via a mobile or web app. Replacing spreadsheets and manual input, EIS allows customers to work with EnerNOC to manage that data in order to optimize how much energy to buy, when to buy it and to make the best use of the energy they use.

EnerNOC uses Splunk Enterprise to generate financial and analytic reports and distribute them to more than 600 employees every month, including senior executives and the engineering and marketing teams. These reports include metrics such as the user personas of the most active customers, how they are using EnerNOC solutions and the busiest days for energy consumption. By performing real-time and historical analytics, the company can refine its applications to better serve customers. 

Real-time visibility into data for enhanced customer service

EnerNOC is in the business of helping customers make more efficient use of energy resources. A key to this is processing and analyzing large amounts of data. By deploying Splunk Enterprise, the company has gained real-time visibility into the data flowing through its EIS platform, which helps it process data more quickly and accurately. With that, and the increased value through advanced analytics, EnerNOC can provide enhanced service to its customers.

“Splunk Enterprise gives us both the real-time and historical intelligence we need to deliver world-class energy management services. We probably derive the greatest value from Splunk software by using its data to integrate our development and operations teams for rapid, agile releases of new solutions and functionalities.”

Jim Nichols, Principal Engineer, EnerNOC

Increased Visibility into AWS Environments

Read this related case study to learn more about how the Splunk App for AWS provides EnerNOC with real-time security insights and increased visibility into its AWS environments.

Read the Case Study