Is Your Cyber Team Overwhelmed by System Alerts?

This post was co-authored by Stefano Tiranardi , a sales engineering director based in Canada.

Your cybersecurity team walks into the office, and their day is instantly taken off the rails. They get an alert informing them that something on the network is acting suspiciously. It isn’t necessarily a threat, but they don’t have the tools to know for sure. After looking into it, they learn that a SaaS provider for one of their departments delivered an update that caused a service degradation.

Thankfully, it isn’t an attack. But it’s still been an inefficient use of the team’s time.

Take this scenario and multiply it by a thousand. Companies can field anywhere from dozens to hundreds of incoming “alerts” every day, and they struggle to get through them in a timely manner because:

  • Most cyber analysts aren’t able to benefit from complete visibility over the entire organization, making it difficult to put alerts in context to rapidly prioritize them.
  • Many cybersecurity teams are under-resourced due to a tight cyber talent market.
  • Business unit leaders are increasingly purchasing more software and software-as-a-service (SaaS) tools often without consulting IT or even advising them after the fact.

All of this creates an environment where a small team handles an increasingly complicated IT environment. Cybersecurity professionals worry that major issues will get lost in a flood of minor alerts or that they act too quickly on an alert without adequate context and negatively impact a legitimate business service. Many experience alert fatigue trying to follow up on them all for fear that the alert they didn’t get to was the biggest threat.

The result is overworked cybersecurity teams unable to guard increasingly vulnerable organizations against a growing number of adversaries, both external and internal.

Enhance Employee Workflows with Splunk’s Security Workshops

Cyber professionals need a more efficient way to view, assess, and prioritize system alerts before devoting time to investigations.

In an effort to increase customer success, we’ve introduced Security Workshops: a virtual, interactive, and hands-on learning series that empowers cybersecurity professionals to make their jobs easier through visualization, investigation, and automation.

In these workshops, cybersecurity professionals learn the basics of Splunk’s interface before using simulated data to tackle specific cybersecurity scenarios they’d expect to see within their own organisations.

Overall, these workshops give participants a safe sandbox environment for learning and experimentation.

What Kinds of Technical Learnings Can Participants Develop?

During these security workshops, participants will learn how to:

  • Distinguish a notable event from an innocuous anomaly
  • Distinguish cyber threats from performance-related issues
  • Look for specific types of IT events
  • Assemble events into a larger story about the adversary’s activity
  • Use correlation and automation to streamline workflows
  • Learn about which events are important to collect in order to gain visibility into adversary actions

Here are a few examples of scenarios and lessons participants can expect during these Wednesday workshops.

Figure 1: Identifying where a brute force attack originated.

Figure 2: Identifying the first suspect domain visited by the victim

Figure 3: Identifying a malicious file

Figure 4: Identifying file server connections from an infected host

Figure 5: Determining which web server is the target

Figure 6: Finding the IP scanning a web server

Better Tools and Training Lead to Improved Workflows and Employee Experience for Cyber Professionals.

The Splunk Security Cloud and our security workshops have three main goals:

  • To streamline workflows so cyber professionals can focus on high-value activities
  • To improve the experience for cybersecurity professionals who are overwhelmed with manual, often low value, repetitive tasks
  • Develop more efficient investigation techniques by getting answers to critical questions faster

Our workshops accomplish this in several ways by:

  1. Increasing efficiency and effectiveness: Participants develop a consistent approach to investigating incidents using Splunk and open source. They will be provided ample opportunities to gain experience searching in Splunk to answer specific questions related to an investigation, all of which reflect what they’d be looking for in their own organizations. The sooner an analyst can ask the right question, the faster they are able to triage and prioritize incidents.

  2. Assembling the story, understanding the context and taking action: Participants will have the opportunity to gain hands-on experience walking through multiple realistic scenarios, from notable events through investigating, hunting and orchestrating actions based on what’s uncovered. They’ll be able to dig deep into an Advanced Persistent Threat and have an opportunity to develop hypotheses and hunt. This is paired with an introduction into how models like the Lockheed Martin Kill Chain, MITRE ATT&CK and Diamond Model can be used to contextualize their hunts. 

  3. Keeping skill sets up to date: Everyone’s consuming some form of cloud service in their organization today, whether sanctioned or not. We help participants, through hands-on activities, extend their knowledge into cloud services such as AWS, by performing hands-on investigations leveraging data from services such as: CloudTrail, CloudWatch, VPC Flow, GuardDuty and Security Hub. Not only will participants come away with a better understanding of the logging available to them from those services, but more specifically, they’ll understand which events are important to collect to gain visibility into adversary actions in cloud environments.

  4. Engaging in the Competition: What’s the point of developing new skills if you can’t have some fun? We invite participants who attended part or the entire workshop series to put their abilities to the test while having tons of fun doing so. Get a team together and join our incredibly popular “Boss of the SOC”.  This is a hands-on, self-paced, blue-team exercise which uses Splunk to defeat threats. It’s a capture-the-flag style activity where participants answer a variety of questions about security incidents that have occurred in a realistic but fictitious enterprise environment. Participants are immersed in a realistic scenario, filled with security incidents and questions analysts must answer on a regular basis. Armed with your new skills, your curiosity and creativeness, you can compete against your peers and the clock for top position in the competition.

  5. Enhanced, streamlined work experience equals stronger cyber security teams: I’m excited for cyber professionals to experience our security workshops. The future of cybersecurity will be one where professionals have the tools and support needed to keep pace with the rapidly changing world of cybersecurity, and Splunk is excited to play its part. 

Interested in learning more? Register for a Canada Security Workshop! Not located in Canada? Find a workshop in your timezone.


Dino Marasco

Posted by


Show All Tags
Show Less Tags