Faster Insights with Third-Party LLM Services in Splunk Search

We have seen the value of foundational LLMs over the past few years — its general knowledge about content on the internet has provided a huge productivity boost for all knowledge workers. Splunk customers can now apply the power of these foundational LLMs to Splunk data with the latest Machine Learning Toolkit (MLTK) 5.6 release. Ask it to explain the error log that you need to investigate, prioritize the list of alerts you received, or even rank the number of login attempts by user…the possibilities are endless. Keep in mind, you are sending your data to the LLM service provider and the foundational LLMs are trained for general purpose use and not specific to security or observability use cases.

In this blog, we will take you through the steps to integrate LLM services with Splunk using  MLTK 5.6, as well as providing some use case ideas to get you started.

What’s New in MLTK 5.6?

MLTK 5.6 releases a few new features to help you bring generative AI into your search pipelines. In summary you can connect to external Large Language Models (LLMs) and then use those language models via search using a new search command — imaginatively called ‘ai’.

LLM Connectors

Within MLTK 5.6 there is a new tab called Connection Management. From this tab you can setup and configure connections to externally hosted LLMs, where today we have options to allow you to connect to OpenAI, Azure hosted OpenAI, Anthropic, Groq, Gemini, Amazon Bedrock and Ollama.

To connect to a given provider all you need to know is the URL and any access data (e.g. a token) for the provider. You can then set a few controls in order to manage use of the provider, such as a request timeout, the maximum number of rows of Splunk data that can be processed per search and the maximum number of tokens that can be consumed per Splunk search. These give you some basic throttling options to control your use of external LLMs from within Splunk. 

Commanding AI With the AI Command

Once you have established a connection with an LLM provider you can then use that provider within a Splunk search using the ai command. With the ability to embed Splunk data into prompts to external LLMs, this gives you a lot of flexibility when it comes to enriching your Splunk data with insights from an LLM. Note that if you have multiple LLM providers set up you can specify which one to use at search time, using the provider and model options.

Please be aware, however, that any data that is sent to an external LLM is done so at your own risk! There is a long list of risks and mitigations about this described really nicely by the good folks at OWASP here, and you will be pleased to know that we are also shipping role capabilities to allow you to control the users who can use the ai command in your Splunk environment. For those of you who are super security conscious, running your own instance of Ollama on your own infrastructure is a great mitigation too! Your use of the external LLM will also be governed by your own agreements with this third party.

ONNX Apply

Returning to MLTK’s more traditional roots, we have also made some updates to the ONNX apply feature in the new release. You can now generate multi-variate output predictions using ONNX models in MLTK, as well as upload ONNX models via a REST API so that we can more easily integrate with any deployment pipelines you have for model training.

What Should I Do With MLTK 5.6?

Use LLMs in search! OK, maybe that was a little broad for most folks…so we wanted to provide you with some examples of what you could do to help inspire your next Splunk use cases.

Please also see this amazing blog from Ryan about using LLMs to detect malicious power shell scripts.

How Can I Get Started With MLTK 5.6?

Well, this part is the simplest — all you need to do is navigate your way to Splunkbase and download the latest MLTK release! 

If you are looking for additional inspiration for what you can do with MLTK, then please check out our e-books for enhancing security use cases with AI and ML and our introductory use case guide for AI and ML in observability.

Happy Splunking!