This blog post is part 27 of the “Hunting with Splunk: The Basics” series, which takes a single Splunk search command or hunting concept and breaks it down to its basic parts.
For those who have played our Boss of the SOC competition or attended our security workshops, you are undoubtedly aware of Frothly, but in case you are not, here is a quick primer. Frothly is a fictional brewing supply company based in San Francisco who has successes and challenges, just like any other organization.
Over the years, Frothly has suffered from insider threats, failed cloud deployments, and nation state adversaries, to name a few, targeting their innovative recipes and a brewery they acquired. This year is no exception.
While these maladies afflict Frothly, one thing that they continue to have in their favor is that they have visibility down to the endpoint of their systems. Now, this might not prevent these attacks from taking place (otherwise, how could we run BOTS?), but it certainly helps Frothly detect them, understand what is occurring so that remediation can be performed, and learn lessons from these attacks.
We have written a good bit about the virtue of endpoint monitoring, in fact James Brodsky punches his ticket to .conf every year with a deep dive into endpoint that we have turned into a workshop just on that topic. We have talked about Sysmon in our Hunting with Splunk series and just recently, Shannon Davis talked about chaining Sysmon processes using a tool that was recently released on Splunkbase called pstree.
What does this tell us? Endpoint monitoring is important; we like using Sysmon, particularly Event Code 1 - Process Creation, to gain fidelity into programs starting on our systems. So far, so good.
Today, I want to talk about some of the other Sysmon events that you may want to consider utilizing during your hunts that may not get as much attention. I will call these the b-sides, which I realize may be lost on some readers, but for others, you may recall albums or cassettes that had some seriously good stuff on the b-side, maybe in some cases better than what was on the a-side. In fact, you may even want to utilize some of these in your detections to drive more automation!
As of this writing, there are Sysmon event codes from 1-26 (not counting 255, which denotes error). It would be fairly tedious to go through every single code here and it is important to point out that configuration needs to be performed to get the most out of your sysmon events. We aren’t going into depth on that part today but there are some good templates to start with, namely the Swift on Security configuration. We also want to install the Sysmon technical add-on to Splunk and the latest version of this can be found on Splunkbase.
First a quick nod to the A-Side. Event Code 1, Process Create, has been covered elsewhere so we won’t go through that today, suffice to say, this is the workhorse event to see what is happening on a system in terms of processes being executed and from where, so this is always a handy code, so learn it, know it, live it and be a full hot orator.
DNS Query, event code 22, can be very handy to get a feel for the DNS queries being issued by a specific host and in conjunction with a specific image. When I use the term image, we are adapting that value into process and its associated process id, path and guid. Both the query and the result are available in the event as well. In this example, Bud Stoll’s system is using Microsoft Edge to lookup www.blogger.com and gets an IP address back in response. In this example, this is benign traffic, but modifying that image on a suspect host, could yield greater insight on domain queries.
source="xmlwineventlog:microsoft-windows-sysmon/operational" EventCode=22 EventDescription="DNS Query" host="bstoll-l" Image="C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe" QueryName="www.blogger.com"
File Delete archived, event code 23, can be helpful when looking for an adversary bent on destruction or covering their tracks. There is another event, File Delete logged - event code 26, that is similar but event code 23 will also save the file in the ArchiveDirectory. This could result in a very large directory, so care needs be taken as to how this is configured, but nonetheless, a combination of event code 23 and 26 could provide insight into a specific process deleting a specific file, on which host and at what time. Here we have Bud deleting a link to an excel spreadsheet from within Microsoft Excel. This example is clearly benign, but could be used to look for more malicious activities.
source="xmlwineventlog:microsoft-windows-sysmon/operational" EventDescription="File Delete archived" Image="C:\\Program Files (x86)\\Microsoft Office\\root\\Office16\\EXCEL.EXE"
WMI Events are coded as event codes 19-21 and can be helpful to understand when filters and consumers are created. WMI, short for Windows Management Interface, is used by all Windows systems and can be used for scripting and is being used more heavily by adversaries. In fact MITRE ATT&CK has Windows Management Instrumentation called out as an adversary technique.
Pipe creation is denoted as event code 17 and can be useful for identifying lateral movement. The pipe name is provided as well as the image information; are you seeing a pattern yet? Pipes are seen extensively in Windows environments so just having this event code is not an indication of malicious activity, so further inspection is required to hunt for badness. That said, it can be an additional event to provide further context during a hunt.
If you want to determine what files have been created on a system, event code 11 is a good one to consider. This event code can get very noisy however, depending on which directories and file types it is monitoring, so some thought must be taken here. An image and target file name with full path are provided in this event. Here we can see that Grace ran a process called setup.exe and the resultant files created and their associated paths.
source="xmlwineventlog:microsoft-windows-sysmon/operational" EventCode=11 host=ghoppy-l process_name=setup.exe | table file_create_time file_path process_name host | sort _time
The last event code I am going to cover today is event code 13, registry value set. When activities are occurring on a system, oftentimes, registry settings are being added, deleted or modified. Now the modifications can create a large number of events, but it contains a treasure trove of activities, from USB key inserts to task scheduling to service modifications.
Alright, I am going to stop here now even though there are many other Sysmon events of value. As you hunt, keep these event codes in mind. Depending on the adversary and their tooling, some of their actions may be difficult to see, but when an adversary starts living off the land and using binaries that are native to Windows, some of these event codes can pay off in ways you may not have previously imagined.
I hope you come out and play Boss of the SOC, but even if you don’t, take some of these ideas as guidance to help improve your visibility as you hunt!
Follow all the conversations coming out of #splunkconf21!