Today we are excited to announce the release of Splunk Enterprise 7.0 and Splunk Cloud! This new release marks “the end of meh-trics,” and delivers advancements in machine learning, as well as massive speed and scale improvements for analytics, monitoring and alerting. Let’s dive into these features in more detail.
More specifically, we'll go over the following:
- Metrics – which are sets of numerical, time series data are now treated as a first class data type bringing massive performance improvements such as up to 200x faster queries
- Event Annotation seamlessly unifies logs and metrics by overlaying multiple searches in a single time chart or graph
- Chart Enhancements expand the selection of visual styles and chart options geared toward improving the visualization of metrics and mult-series monitoring use cases
- Faster Data Model Acceleration through core search technology tweaks
- Self-Service App Management in the cloud has been updated to allow the installation of your organization’s own internal apps
- The latest Machine Learning Toolkit improves extensibility, scalability and ease of use through several new enhancements
Be sure to download the Splunk Enterprise 7.0 Overview App for examples of how to not only use these new features, but to learn about other features not mentioned in this post.
Let’s get started with Metrics!
Metrics are sets of numbers describing a particular process or activity, measured over time. Some common examples of metrics that you may be familiar with are: time series data; system metrics such as CPU, memory or disk; infrastructure metrics such as AWS CloudWatch; and IoT devices (temperature readings). Check out the overview of metrics in the Splunk docs.
A metric consists of the following:
In Splunk Enterprise 7.0, you will now see up to 20X speed improvement against accelerated log data (tstats), and up to 200X speed improvement against non-accelerated log or event data when querying metrics. Additionally, real-time metrics queries will use substantially fewer resources. Darn impressive performance.
But wait…what’s the catch?
There isn’t one! All of the Splunk platform benefits apply to metrics–visualizations and alerting, role-based access controls, data onboarding, clustering, scaling and alerting; and, importantly for new use cases, the ability to leverage open source data collection daemons such as statsD and collectD.
So, how do you get started with metrics?
First you will need to create a new index that is specifically tuned for metrics data. This index will use our Metrics Store which provides the ability to ingest and store metric measurements at scale.
Next, you’ll need to configure a data input. There are out-of-the-box sourcetypes and native support for both statsD and collectD. Alternatively, you can configure any other data source with props.conf and transforms.conf to fit the metrics structure.
To query and retrieve your metrics data, you will use a new Splunk Search Processing Language (SPL) command called “mstats.” Mstats is the tstats equivalent to query time series from metrics indexes and can be used for both historical and real-time searches. Below is an example:
Stay tuned for more functionality and solutions to come using metrics with Splunk!
Next is Event Annotation, which can help you decipher what is and is not actionable from disparate data sources.
Now that you’ve got metrics, you may want to see your metrics results and your other searches in one dashboard. Fortunately, Event Annotation is here for you! You can get event context for any time chart (line, column, area), and event annotation markers and labels can be pulled from sources such as log data, lookup files, or external sources. All together. In one view.
How to use and configure Event Annotations? It’s a few simple lines in the dashboards XML.
The “annotation_label” can be a Splunk field or any text you wish. The same goes for the “annotation_category” which helps define the annotation color. And voila! Our dashboards are getting even more context packed in!
You might have noticed the lines look differently in this last screenshot. That’s because we’ve added some new Chart Enhancements in Splunk Enterprise 7.0 that expand the selection of visual styles and chart options geared toward improving the visualization of metrics and multi-series monitoring use cases. Line width, line style, and a new series comparison option in the legend are included in these enhancements and editable by SimpleXML. Check out the examples in the Splunk 7.0 Overview App for more!
“I feel the need… the need for speed!” Optimizations to our core search technology decrease the time and resources required to run Data Model Accelerations (DMA) and accelerated searches giving us Faster Search & DMA Performance. In fact, Splunk Enterprise Security users will immediately see up to a 3X speed improvement on the Data Model Acceleration time and a reduction in summarization lag. While this feature should help Splunk Enterprise Security users right away, it affects all DMA.
In 6.6 we brought you a new Self-service App Management interface for Splunk Cloud customers that allowed the installation and management of Splunk Certified Apps. Now in 7.0 you will have the ability to install your own private or internally built Apps using our new auto-vetting process. This will reduce the time it takes to get an App vetted and installed from weeks to minutes!
Lastly, we are excited to announce several key updates to the Splunk Machine Learning Toolkit over the past year. These enhancements to the toolkit include an improved API, new data prep algorithms, role-based access controls for machine learning models and new out-of-the-box algorithms to make it even easier for you to predict future IT, security, and business outcomes. Let’s break down the advancements to the toolkit:
- Machine Learning Model Access Controls: machine learning models are now fully integrated with Splunk's role-based access controls (RBAC).
- New Data Prep options for using pre-processing algorithms save you time preparing datasets for machine learning models.
- ARIMA Forecasting: the Autoregressive Integrated Moving Average (ARIMA) algorithm has been added to the available options for forecasting time series data. This includes new visualizations for inspecting properties unique to the ARIMA algorithm.
- Extensible Machine Learning: We’ve opened up the API to allow partner and customer app developers to import custom algorithms and build their own, and then share with the community via Splunkbase.
Stay tuned for a more in depth blog post on these new Splunk Machine Learning Toolkit features coming soon!
Follow all the conversations coming out of #splunkconf17!