By Splunk January 19, 2016
In this discussion we will learn why RRD is an absolute necessity. We will establish the core capabilities required for RRD. Then we will walk through how ES 4.0 delivers on the capabilities for RRD. Finally, we’ll show how we can extend RRD and add our own flavor using the existing capability in Splunk Enterprise and ES 4.0.
State of Affairs for Cyber Operations
Cyber operations teams receive far more alerts than they can handle. Once they receive an alert, analysts spend a lot of time manually connecting the dots. As a result, alerts drive the cyber posture for an organization. And cyber operations teams are stuck in a never-ending loop of chasing individual incidents. As a result, operations teams are unsuccessful at raising the cost for the attacker or managing the business and mission risk.
Breaking the Cycle of Alert Fatigue
An individual device or SIEM alert may be related to one or more ongoing attacks. Conversely, multiple alerts may be related to the same attack. Individual alerts don’t have context or any information on their relation to other alerts. A cyber-defender responding to individual alerts doesn’t see the complete picture and ends up burning a lot of time. The goal for cyber defenders and responders is to develop context – ultimately to drive decision.
So the solution seems obvious. Collapse multiple alerts into a single incident and build context. But what specific capabilities are required for us to accomplish this?
Using a Kill Chain to determine attack relatedness and developing context
All alerts are associated with some part of a kill chain. E.g. an alert for encrypted connections to a threat-listed domain corresponds to the exfiltration phase of an attack. Once an analyst sees that alert, she must now move forward or backwards across the kill chain. Lets go through some of the opportunities for automated context development using the file transfer example:
- Who owns the systems?
- Integrate your asset and identity system
- Discover the asset and identity on the fly by searching the authentication logs
- Where are the systems located?
- Integrate your asset system
- Resolve the location via a geo-ip
- Search across all data to match indicators with other sytems’ events
- Business value of the system and the data
- Integrate your asset system – one with an assets business priority
- Automatically discover the criticality of the system based on traffic history, connection history, or name of the system
- When was the system compromised?
- Index endpoint logs, authentication logs, network logs and threat intel
- How was the system compromised?
- Index, email, web access, endpoint, service, and authentication events
- What are the CnC components for this attack?
- Enrich with threat intelligence subscriptions
Rapid Response and Discovery (RRD) required capabilities
To fulfill context requirements for determining the kill chain, we need the following capabilities:
- Index any data – network, endpoint, threat intelligence, asset/identity
- Search across all data – without knowing the questions prior to indexing
- Build multi-layered, conditional searches and alerts – where each search result or alert can kickoff other searches and alerts
- Enrich data, threat intelligence or external knowledge on the fly
- Bi-directionally interact with any technology via APIs
- Visualize the results of complex searches and alerts
- Share results/progress amongst analysts and export data or visualizations
For the remainder of this discussion, lets walk through how Enterprise Security addresses these requirements. And while we are there, lets talk a little about the art of the possible – and how these capabilities can be extended by you – ES users.
Achieving RRD with Splunk Enterprise Security (4.0)
Lets use the previous example alert: encrypted connections to a threat-listed domain. A key ingredient to RRD is context. And a key outcome for RRD is decision.
Lets see how Enterprise Security (ES) gets us going on the RRD path – out of the box.
Figure 1 above, shows the ES 4.0 incident review page
- Alert and Event Information (Top right section of the image):
- The rule that detected the activity
- The final event that caused the alert to fire
- Contributing events for the alert
- Automated Enrichments:
- System owner
- System grouping
- System criticality
- Location information
- Matching fields for the alert
- Ad hoc Workflow Actions: Establish bi-directional interactions and enable the analyst to further triage or investigate the incident via API or open interfaces. Some examples highlighted by the white background Actions dropdown are:
- Pivoting to another Splunk dashboard to get more context e.g. the Vulnerability, Web dashboards
- External enrichments: e.g. searching google for an IP or lookup the Domain Dossier for a domain
- Active actions: Pinging a device to see if its on the network. Or running Nbtstat to get host, IP and network statistics for a local or remote host
- Passive actions: Using the Stream API, start a Stream Capture to collect wire data.
ES 4.0 has a number of automated and adhoc context development features – available out of the box. But what about the features that are not available? You can use the existing workflow actions as examples.
RRD Through Sharing and Collaboration
As the old proverb says, “Go alone, go fast. Go together, go far”. So far, everything we have talked about is focused on an individual analyst. But cyber operations are run by teams. So sharing and collaboration is critical to an organization’s success. ES 4.0 introduced the Investigation Timeline feature. As an analyst goes through an investigation, she can add items to the timeline. Timelines can be saved and shared. Enabling colleagues and managers to learn from the investigation, to continue the investigation and to report on it.
You can learn more about the investigation timeline from the ES 4.0 docs.
But what about that time when an analyst comes up with an interesting way to detect a threat? Or the analyst identifies a new threat? Or develops an efficient workflow? Or creates a new dashboard? How can we take this analyst’s knowledge enable the entire organization? Or perhaps share the techniques with partners and external collaborators?
ES 4.0 introduces the Extensible, Analytics and Collaboration capability. Now an analyst can export searches, dashboards, alerts with a few mouse clicks. These clicks create a bundle. The receiving party can click on an import button and start to take advantage of the knowledge.
Extensible Analytics and Collaboration capability enables teams to learn and grow at the rate of the collective community. Rather than be bound by the constraints of an individual team.
Extending RRD Capabilities – The Art of The Possible
In a recent webinar, David Veuve from Splunk, demonstrated how you can rapidly investigate an incident with Splunk ES, create new threat intel, and automate the inclusion of that thrat intel for future events. All done through a few mouse clicks.
Veuve demonstrated 3 things:
- Use custom workflow actions to point to dashboards of interest
- Create custom dashboards for visualizations and form inputs
- Add threat indicators to the ES threat intelligence framework
Figure 3 shows custom workflow action, much like the built-in from Figure 1.
Figure 4 shows a custom dashboard that:
• Enables for searching particular fields of an email message
• Allows the analyst to add email artifacts to the ES 4.0 threat intel framework
RRD != | Dream. RRD == True
The capabilities exist to deliver RRD for Cyber Operations. And those capabilities aren’t just proofs of concepts, they are enterprise ready. RRD isn’t just a pipe dream, it’s a reality! If you use this mind set, you can raise the costs for your adversaries. And perhaps maybe even take a day or two off …nah!
Monzy Merza
Chief Security Evangelist
Splunk
----------------------------------------------------
Thanks!
Monzy Merza
----------------------------------------------------
Thanks!
Monzy Merza