Splunking Azure: NSG Flow Logs

Azure Network Security Groups (NSG) are used to filter network traffic to and from resources in an Azure Virtual Network. If you’re coming from AWS-land, NSG’s combine Security Groups and NACL’s.

Splunking NSG flow log data will give you access to detailed telemetry and analytics around network activity to & from your NSG's. If that doesn’t sound appealing to you yet, here are some of the many things you could Splunk with your network traffic logs from Azure. 

• Virtual Machine traffic monitoring
• Correlate on-prem firewall traffic with cloud traffic (Got expressroute?)
• Application troubleshooting
• network forensics
• protocol analysis
• Compliance monitoring
• Usage forecasting
   

Enabling flow logging in Azure aggregates this data and stores it in a Azure Storage Blob. This blog won’t cover those steps, but if you don’t already have flow logs configured, you can check out the setup steps, here (ignore the VM creation and proceed from “Enable Network Watcher”).

We’ll run through 2 ways of getting NSG flow logs out of Azure and into Splunk.

Option 1: Splunk Add-on for Microsoft Cloud Services

This option uses the Splunk Add-on for Microsoft Cloud Services to connect to your storage account and ingest your flow logs into Splunk. The input will poll the storage blob periodically looking for new events.

1. Configure your flow logging using the instructions above

2. In your Azure Portal, browse to your storage account

3. Select Shared access signature

4. Create read-only permissions to the storage account

•         Allowed services: Blob, Table
•         Allowed resource types: Service, Container, Object
•         Allowed permissions: Read, List
•         No Blob versioning permissions
•         Sufficient expiry period for SAS Key: 1 year, 2 years etc.
•         Allowed protocols: HTTPS only
•         Signing key: key1 or key2
•         Generate SAS and connection string
   

5. Copy SAS Token

6. Install the Splunk Add-on for Microsoft Cloud Services

7. Install the Microsoft Azure App for Splunk

8. Splunk Add-on for Microsoft Cloud Services > Configuration > Azure Storage Account > Add

9. Configure:

• Name: (Splunk friendly name - no spaces)
• Account Name: Storage account name as per Azure Portal
• Account Secret: SAS Token
• Account Secret Type: Account Token
   

10. Splunk Add-on for Microsoft Cloud Services > Inputs > Create New Input > Azure Storage Blob

11. Configure:

• Name: (Splunk friendly name - no spaces)
• Azure Storage Account: Account from Step 9
• Container Name: insights-logs-networksecuritygroupflowevent
• Interval: 5 mins (depending on your performance and requirements, modify this value)
• Sourcetype: mscs:nsg:flow
   

Confirm data is being indexed by searching your index for mscs:nsg:flow sourcetyped data. 

Troubleshooting: index=_internal  sourcetype="mscs:storage:blob:log"

The Microsoft Azure App for Splunk contains search-time extractions and knowledge objects for parsing the mscs:nsg:flow data. The app also contains a pre-built dashboard for visualizing the NSG Flow Logs. Security > NSG Flow Logs.

Be sure to have the app installed on your seach head or alternatively, ensure you use the props & transforms configs for the mscs:nsg:flow sourcetype to ensure fields are extracted correctly.

Now, this process works perfectly fine and allows us access to the information contained in the flow logs. However if you look at the raw events, you can see the format of the data can be somewhat "messy". Microsoft log these events as flowTuples. Essentially strings that contain multiple properties and property values.

The dashboards and configs above will allow you to search these events perfectly enough, but may require some tinkering in order to truly get the most out of this dataset.

I prefer to take the easier route.

Option 2: Deploy an Azure function app to send NSG logs to Splunk via HEC (HTTP Event Collector)

This option deploys an Azure Function from a configurable template, into your Azure Subscription. Its job is to read NSG Flow Logs from your configured storage account, parse the data into clean JSON events and fire the events to a Splunk HEC endpoint. 

Pre-Reqs:

• Sufficient permissions to deploy resources in your Azure environment
• Ensure you have configured NSG flow logging to your storage account before deploying the Azure function
   

1. Create a new HEC data input in Splunk, store a copy of the HEC token.

2. Browse to this GitHub link

3. Click the "Deploy to Azure" button

4: Configure

• App Name: Descriptive name for function app
• NSG Source Data Connection: Connection String from storage account (Storage Account > Access keys > Connection string)
• Output Binding: splunk
• Splunk Address: URL of Splunk HEC endpoint (Eg. Https://datatoeverything.io:8089)
• Splunk Token: Splunk HEC Token
   

Once deployed, the function will retrieve NSG events logged to the insights-logs-networksecuritygroupflowevent container, using the connection string from the associated storage account. Parses them into JSON events, and sends them to your HEC endpoint using the sourcetype amdl:nsg:flowlogs.

If you're already sending Azure data to Splunk using Event Hubs, you can use this deployment template to send your NSG events to your Event Hub instead.

The output format is identical to the HEC feed. 

As you can see, these events appear in a much more friendlier format, allowing us easier access to search, reporting and investigation capabilities of the Splunk ecosystem. If you're using Splunk Enterprise Security, or Splunk Phantom, this should be right up your alley.

Happy Splunking!
Ry