Zero-shot Security Classification with Foundation-Sec-8B and Splunk DSDL

Earlier this year, we showcased how the Foundation-Sec-8B model’s chat capabilities can be leveraged within the Splunk App for Data Science and Deep Learning (DSDL) to summarize security events and provide detection suggestions. Building on its robust security expertise, Foundation-Sec-8B also supports zero-shot classification for a wide range of security tasks. In this blog, we introduce a new DSDL container featuring the Foundation-Sec-8B model, enabling users to classify security data directly from Splunk searches—no fine-tuning required. 

Zero-shot Classification    

While large language models (LLMs) are increasingly used for security event analysis and summarization, effective classification remains a cornerstone for SOC analysts. Tasks such as phishing detection and incident grouping are essential for identifying threats and managing alerts, yet they often demand extensive manual effort and expertise.  

Traditional approaches to automating classification require substantial investment in data preparation and model fine-tuning, leading to high computational costs, prolonged development cycles, and potential limitations in adaptability and accuracy.  

Foundation-Sec-8B offers a different approach. Pre-trained on comprehensive security datasets, this model supports zero-shot classification—enabling it to categorize security data accurately without the need for task-specific fine-tuning. By leveraging perplexity-based scoring and customizable labels, SOC analysts can rapidly classify events and adapt to evolving threats across diverse datasets and use cases.  

The ability to perform zero-shot classification directly addresses the need for speed, flexibility, and operational efficiency in modern security operations. It accelerates threat detection and incident response, allowing teams to focus on high-value tasks rather than repetitive manual classification.  

To empower Splunk users with this capability, we are introducing a new DSDL container image. This container seamlessly hosts the Foundation-Sec-8B model and provides a built-in notebook, making it easy to execute zero-shot classification directly from Splunk SPL search. This integration delivers advanced AI-driven classification to security teams—without the barriers of complex setup or fine-tuning—enabling faster, smarter, and more adaptive security operations. 

Transformers 5.2.2 Image on DSDL 

DSDL significantly enhances Splunk’s capabilities by enabling seamless integration between the Splunk platform and container-based inference for deep learning model training and large language model (LLM) deployment. With a variety of pre-built container images, DSDL supports use cases ranging from Transformers model training to LLM integration and agentic AI workflows, allowing security teams to operationalize advanced AI directly within their Splunk environment. 

In DSDL version 5.2.2, we have further expanded these capabilities by updating the Transformers GPU (5.2.2) image to include the zero-shot classification using the Foundation-Sec-8B model. This upgrade makes it easier for users to leverage this feature without additional setup or customization.  

To get started, launch the Transformers GPU (5.2.2) container from the DSDL container management page. Once the container is running, DSDL will display a JupyterLab URL for accessing the notebooks for algorithm customization in the container, as illustrated in the screenshot below. In this experiment, we used a docker host with 40GB GPU memory for container runtime, based on the best practice from Foundation AI.  

After logging into JupyterLab, navigate to the notebooks folder where you will find the fdai_zeroshot_classification.ipynb notebook. This notebook includes example usage from Splunk search, which is outlined under its title. We will explore the SPL in greater detail later in this blog.  

This notebook is based on the cookbook available on the Foundation AI GitHub, with an added feature to streamline usage within DSDL: an automatic model file download mechanism.  

The notebook includes an init() function that checks whether the Foundation-Sec-8B model file is already present locally. If not, it automatically downloads the model during the first execution, which may result in a longer initial runtime. After the initial download, the model files are stored at app/model/data/fdtn-ai--Foundation-Sec-8B within JupyterLab. For all subsequent runs, the notebook will load the model directly from this local path, ensuring faster startup. 

For air-gapped environments, you can manually create the specified directory and transfer the required model files into this folder to enable offline usage. 

Now that we are familiar with the notebook environment in the container, let’s switch to Splunk and perform a zero-shot classification directly from an SPL search.   

SPL Execution 

In the Splunk search bar, we have identified a set of security events. To streamline analysis, we aim to categorize these events into the following eight categories for our analysts to review:

• Malware Activity
• Data Exfiltration Attempt
• Phishing/Social Engineering
• Unauthorized Access
• Denial of Service (DoS) 
• Insider Threat
• Vulnerability Scan
• Policy Violation  

Traditionally, classifying security events required extensive word parsing and training dedicated classification models before analysts could begin categorizing incidents. With Foundation-Sec-8B integrated into DSDL, this process is significantly streamlined. Now, you can perform zero-shot classification with a single SPL command, eliminating the need for manual feature engineering or model training.  

index=sec_event | eval text=_raw | table text 
| fit MLTKContainer algo=fdai_zeroshot_classification labels="Malware Activity&&Data Exfiltration Attempt&&Phishing/Social Engineering&&Unauthorized Access&&Denial of Service (DoS)&&Insider Threat&&Vulnerability Scan&&Policy Violation" text into app:fdai_zeroshot_classification 

The fit command invokes the fdai_zeroshot_classification algorithm defined in the notebook we explored in JupyterLab. Since the algorithm expects an input field named text, we use the eval command to assign the raw event content to the text field, ensuring it serves as the model’s input. 

Additionally, we pass our custom category labels to the algorithm through the labels parameter. All label options are concatenated into a single string, separated by the && symbol.  

The execution result of this SPL is shown in the screenshot below. 

For each event, the model assigns a predicted category in the predicted_Label field. The predicted_Probability field indicates the likelihood of the event belonging to the predicted category on a 0–1 scale, with higher values reflecting stronger evidence. 

The predicted_Confidence field represents the model’s certainty about its prediction, also on a 0–1 scale: 

• 0.8–1.0: Very confident 
• 0.6–0.8: Moderately confident 
• 0.4–0.6: Somewhat uncertain 
• 0.0–0.4: Low confidence (manual review recommended)  

As demonstrated by the results, the model successfully categorized each of the ten input events with medium to high confidence, effectively leveraging its specialized security domain knowledge.  

Few-shot Improvements 

In some zero-shot classification cases, the model may produce lower confidence scores. To improve accuracy in these situations, adding a few-shot examples to the prompt can significantly enhance the model’s judgment. This DSDL integration supports few-shot learning by allowing users to add examples and use the examples during classification through the Fit command.    

To add examples, start by creating a table in Splunk search with two fields: text and label. The text field should contain representative input samples for classification, while the label field provides the corresponding category for each sample. Next, use the following fit command to send these examples to the container environment, enabling few-shot learning at query time.  

... | table text label 
| fit MLTKContainer mode=stage algo=fdai_zeroshot_classification text label into app:fdai_zeroshot_classification

The keyword mode=stage indicates that this command is solely used to send the examples to the container, without performing any classification at this stage.   

As shown in the screenshot above, we included at least one example for each category. To incorporate these few-shot examples during classification, simply add the parameter few_shot=1 to the fit command, as illustrated below.

index=sec_event | eval text=_raw | table text 
| fit MLTKContainer algo=fdai_zeroshot_classification few_shot=1 labels="Malware Activity&&Data Exfiltration Attempt&&Phishing/Social Engineering&&Unauthorized Access&&Denial of Service (DoS)&&Insider Threat&&Vulnerability Scan&&Policy Violation" text into app:fdai_zeroshot_classification

Adding few-shot examples (shown on the left) significantly increased the model’s confidence scores in classification compared to zero-shot classification (shown on the right). This enhancement is fully supported by the native fit command mechanism.  

Conclusion 

The integration of Foundation-Sec-8B with DSDL brings powerful zero-shot classification capabilities directly to the Splunk platform, streamlining security operations and reducing the need for manual model training. Even more exciting use cases for Foundation-Sec-8B can be found in Foundation AI’s cookbook repository on GitHub. With the new DSDL Transformers GPU image, these cookbooks can now be easily brought into Splunk, allowing you to execute directly from Splunk search and further enrich your security operations.