Introducing Resource Metrics: Elevate Your Insights with the New Workload Dashboard

Splunk is excited to introduce Resource Metrics in Workload Dashboard (WLD) — a modern and intuitive monitoring experience in the Cloud Monitoring Console (CMC) app. These new metrics are designed to complement the information that is available in the WLD Dashboard; Splunk Admins gain deeper insights into how much infrastructure capacity their Splunk deployment uses, as well as overall health and performance.

What are Resource Metrics?

Resource metrics are a set of new metrics that Splunk launched to increase administrators’ visibility into Splunk workloads using capacity (For example, memory, CPU, I/O, Cache) to understand how organizations are using resources in each Splunk Cloud Platform deployment. With these metrics, administrators can easily identify resource bottlenecks and optimize their capacity and SVC (Splunk Virtual Compute) usage.

These metrics exist across 2 areas:

  1. Resource metrics for Indexers

    • Indexer memory utilization: This shows the latest 90th percentile measurement of the memory utilized by all processes running across the time frame selected for all the indexer hosts
    • Indexer cache churn: This shows the rate at which data is evicted from cache memory to make room for new data. It's a measure of cache downloaded as a percentage of total storage.
    • Indexer CPU utilization: This shows the latest 90th percentile measurement of CPU utilization across all indexers
  2. Resource metrics for Search Heads

    • Search Head memory utilization: This shows the latest 90th percentile measurement of the memory utilized across search heads for the time selected.
    • Search Head CPU utilization: This shows the latest 90th percentile measurement of CPU utilization across all search heads.

Why it Matters

Until today, Splunk Cloud Platform customers measured their capacity using the SVC (Splunk Virtual Compute) metric alone metric. The SVC usage metric on its own cannot efficiently explain why SVC consumption might increase without a corresponding rise in search or ingest activities, or why it might remain constant despite an increase in searches. Resource Metrics in the Workload Dashboard fill that gap and provide clear SVC insights to Admins unlocking control over workload utilization within their deployment.

Key Customer Value

This enhanced visibility empowers customers to:

How can customers access the Resource Metrics?

Customers can access these metrics through the Overview and Workload dashboards.

Within the Overview dashboard, navigate to "Top Metrics" and select any of the five metrics within the "Resource Metrics" category. Customers can view these metrics and click on the drilldown link inside the top metric cards to access the Workload dashboard for further investigation.

The Workload dashboard displays resource metrics by default, complete with thresholds and recommendations for high values. Detailed visualizations are also available directly within the Workload dashboard for deeper analysis.

Interpretation of Resource Metrics

Metric
Threshold Levels
Effect on Stack (when high)
Recommendations when high
Indexer memory utilization
Optimal < 60%
Elevated 60 - 80%
Critical ≥ 80%
  • Data Ingest Latency
  • Out of Memory errors
  1. Reduce search workload by:

    • Reducing search concurrency
    • Reducing search time ranges, which will reduce runtime and might reduce concurrency
    • Simplifying searches
  2. Addressing blocked queues by reducing ingestion rate

  3. Talk to your sales representative about adding more SVCs, which will add additional indexers.

Indexer cache churn
Optimal < 5%
Elevated 5 - 20%
Critical ≥ 20%
  • Slow searches
  • Scheduled search delays
  1. Optimize searches
  2. Talk to your sales representative about adding more SVCs, which will add indexers and increase cache capacity.
Indexer CPU utilization
Optimal < 60%
Elevated 60 - 80%
Critical ≥ 80%
  • Ingest Delays
  • Data Quality Issues
  1. Reduce ingestion rate

  2. Reduce search workload by:

    • Reducing search concurrency
    • Reducing search time ranges, which will reduce runtime and might reduce concurrency
    • Simplifying searches
    • Creating saved searches that summarize data
  3. Talk to your sales representative about adding more SVCs, which will add additional indexers.

Search head memory utilization
Optimal < 60%
Elevated 60 - 80%
Critical ≥ 80%
  • Scheduled search delays
  • Ad hoc search interruptions
  1. Reduce search workload by:

    • Reducing search time ranges, which will reduce data scanned by searches and in turn might reduce memory usage
    • Simplifying searches
  2. Talk to your sales representative about adding more SVCs, which will add additional search heads.

Search head CPU utilization
Optimal < 60%
Elevated 60 - 80%
Critical ≥ 80%
  • Scheduled search delays
  • Ad hoc search interruptions
  1. Reduce search workload by:

    • Reducing search concurrency
    • Reducing search time ranges, which will reduce runtime and might reduce concurrency
    • Simplifying searches
    • Creating saved searches that summarize data
  2. Talk to your sales representative about adding more SVCs, which will add additional search heads.

What's Next?

This release of Resource Metrics marks the exciting beginning of the new Workload Dashboard journey. We've just kick-started this evolution, and you can expect updates that will introduce new visualizations, that enhance visibility into these critical resource metrics, unlocking new ways that customers can monitor, troubleshoot, and predict their Splunk workloads.

Join this journey and share your feedback and ideas with us. Select the feedback button on your Workload dashboard and share your thoughts — we can't wait to hear from you!

To learn more about the Workload dashboard and the Cloud Monitoring Console (CMC), check out our comprehensive documentation.

Related Articles

Announcing the General Availability of Splunk POD: Unlock the Power of Your Data with Ease
Platform
2 Minute Read

Announcing the General Availability of Splunk POD: Unlock the Power of Your Data with Ease

Splunk POD is designed to simplify your on-premises data analytics, so you can focus on what really matters: making smarter, faster decisions that drive your business forward.
Introducing the New Workload Dashboard: Enhanced Visibility, Faster Troubleshooting, and Deeper Insights
Platform
3 Minute Read

Introducing the New Workload Dashboard: Enhanced Visibility, Faster Troubleshooting, and Deeper Insights

Announcing the general availability of the new workload dashboard – a modern and intuitive dashboard experience in the Cloud Monitoring Console app.
Leading the Agentic AI Era: The Splunk Platform at Cisco Live APJ
Platform
5 Minute Read

Leading the Agentic AI Era: The Splunk Platform at Cisco Live APJ

The heart of our momentum at Cisco Live APJ is our deeper integration with Cisco, culminating in the Splunk POD and new integrations, delivering unified, next-generation data operations for every organization.
Dashboard Studio: Token Eval and Conditional Panel Visibility
Platform
4 Minute Read

Dashboard Studio: Token Eval and Conditional Panel Visibility

Dashboard Studio in Splunk Cloud Platform can address more complex use cases with conditional panel visibility, token eval, and custom visualizations support.
Introducing Resource Metrics: Elevate Your Insights with the New Workload Dashboard
Platform
4 Minute Read

Introducing Resource Metrics: Elevate Your Insights with the New Workload Dashboard

Introducing Resource Metrics in Workload Dashboard (WLD) – a modern and intuitive monitoring experience in the Cloud Monitoring Console (CMC) app.
Powering AI Innovation with Splunk: Meet the Cisco Data Fabric
Platform
3 Minute Read

Powering AI Innovation with Splunk: Meet the Cisco Data Fabric

The Cisco Data Fabric brings AI-centric advancements to the Splunk Platform, seamlessly connecting knowledge, business, and machine data.
Remote Upgrader for Windows Is Here: Simplifying Fleet-Wide Forwarder Upgrades
Platform
3 Minute Read

Remote Upgrader for Windows Is Here: Simplifying Fleet-Wide Forwarder Upgrades

Simplify fleet-wide upgrades of Windows Universal Forwarders with Splunk Remote Upgrader—centralized, signed, secure updates with rollback, config preservation, and audit logs.
Dashboard Studio: Spec-TAB-ular Updates
Platform
3 Minute Read

Dashboard Studio: Spec-TAB-ular Updates

Splunk Cloud Platform 10.0.2503 includes a number of enhancements related to tabbed dashboards, trellis for more charts, and more!
Introducing Edge Processor for Splunk Enterprise: Data Management on Your Premises
Platform
2 Minute Read

Introducing Edge Processor for Splunk Enterprise: Data Management on Your Premises

Announcing the introduction of Edge Processor for Splunk Enterprise 10.0, designed to help customers achieve greater efficiencies in data transformation and improved visibility into data in motion.