Case Study

National Ignition Facility Unlocks the Potential of Clean Energy and Safeguards the U.S. Nuclear Stockpile With Splunk

Executive Summary

The National Ignition Facility (NIF), located at the Lawrence Livermore National Laboratory (LLNL) in Livermore, California, is the world’s largest laser. To support the NIF’s core missions, including nuclear stockpile stewardship and scientific discovery, scientists and engineers require a secure, reliable IT infrastructure. Since deploying Splunk Enterprise and Splunk IT Service Intelligence (ITSI), the NIF has seen benefits including:

  • Maximizing system uptime and performance
  • Proactively monitoring and responding to IT and security challenges
  • Improving control systems reliability, increasing availability and enabling the team to maintain the infrastructure and systems that supported doubling the number of laser shot experiments — from 200 to 400 annually
  • Ensuring the health of more than 66,000 Internet of Things (IoT) devices in addition to the IT infrastructure
Challenges
    • Secure data and keep the nuclear stockpile safe and reliable
    • Ensure facility availability so scientists can conduct experiments
Business Impact
    • Proactive monitoring to identify anomalies before they become problems
    • Achieve better systems efficiency and availability for scientists
    • Enable data-driven decisions to secure nuclear stockpile
    • Help to ensure the facility’s continued competitive advantage in scientific research
    • Ensure an excellent user experience for scientists and engineers
Data Sources
    • Network switches
    • Firewall
    • Application
    • Authentication
    • Oracle and MySQL databases
    • Servers
    • Sensors

Why Splunk

The NIF's mission is threefold. NIF was founded by the National Nuclear Security Administration as a key contributor to the Stockpile Stewardship Program, which keeps the U.S. nuclear stockpile safe, secure and reliable without underground testing. Scientists conduct astrophysics experiments that help maintain the country's competitive advantage in scientific endeavors. The NIF also explores fusion ignition, which could potentially unlock a clean, carbon-free energy source.

The NIF houses 192 giant laser beams that can achieve thermonuclear fusion, reaching temperatures of 100 million degrees and pressures of 100 billion atmospheres. Distinguished scientists at the one-of-a-kind facility use the lasers to conduct experiments — known as laser shots — in astrophysics, material science and nuclear science.

Four years ago, the National Nuclear Security Administration asked the NIF to double the number of laser shots. The team leaned on technology, vendors and best practices to accomplish their mission, all while using the same resources.

“Our team initially started using Splunk as a security tool,” says Marvin Christensen, IT manager and chief information officer, NIF. “Splunk helps us gather that data and aggregate it so that we can make sure that the NIF is secure.”

After bringing network, authentication and host data into Splunk Enterprise to solve security challenges, the team decided that by aggregating them with additional data sources, such as application data into one view, it would provide real-time visibility across the facility. “By monitoring the applications, we're able to avoid unscheduled downtime or slowness so that we have higher availability of the NIF facility for the scientists,” Christensen says.

At the NIF, Splunk users in the IT department include network, database, storage and system administrators who proactively monitor the status of their machines and systems to prevent issues. Also, developers monitor their applications, and operators keep tabs on the facility status. “We have a small team, but we're very efficient,” Christensen says. “And we do what I think is a great job of using data and architecture and vendors to all come together to deliver a product that NIF can be satisfied with.”

“Splunk ITSI enables the team to take a very complex machine and break it down into discrete components. Previously an AIOps approach would have taken a lot of resources to implement. This is the first time that we could simplify that technology for an IT user base and apply that to an infrastructure use case.”



Philip Adams, CTO and lead architect,
NIF, LLNL

Ensuring complex systems availability

Splunk solutions sit at the heart of the NIF’s Integrated Computer Control System (ICCS), which manages more than 66,000 control points to power NIF’s massive laser facility. Within the ICCS, Splunk Enterprise and Splunk ITSI enable the lab’s engineers to take action on events based on a variety of data sources, including application data, operational data, and sensor data like laser voltage, temperature and pressure. The Splunk Machine Learning Toolkit enables the NIF to respond to IT issues and predict abnormal behavior within the laser facility, significantly improving availability.

Conducting experiments on infrastructure with a lot of moving parts is challenging enough, and the NIF’s goal is 400 shots annually. It's essential that all systems are go when scientists have an opportunity to run their experiments on the NIF. “The experiments that we do here have one outcome, and that's to get data to the scientists,” says Philip Adams, chief technology officer and lead architect, NIF. “When that data comes to the scientists, they expect it to be accurate and delivered quickly. Data is critical to the NIF.”

Splunk possesses the capability and flexibility to analyze the multitude of data sources in the environment. This capability has been critical for the NIF team as it provided the ability to join application and server logs with database information. The ability to correlate multiple data sources into one pane of glass has been critical for the team and rapidly spread adoption. Splunk dashboards also enable the team to use best of breed systems across the environment and avoid expensive customization.

Predictive monitoring and analytics

The NIF also uses Splunk for diagnostics and analytics on a wide range of sensors, including cameras, thermometers and motors that are critical components of the laser facility’s infrastructure. With Splunk ITSI and the Machine Learning Toolkit, NIF engineers can detect when these sensors begin to decay, and perform predictive maintenance, avoiding unscheduled downtime.

“Splunk ITSI enables the team to take a very complex machine and break it down into discrete components,” Adams concludes. “Previously an AIOps approach would have taken a lot of resources to implement. This is the first time that we could simplify that technology for an IT user base and apply that to an infrastructure use case.”