Splunk Observability at Cisco Live: Agent Evaluation, Tokenomics on AWS, and Simplified Full-Stack Packaging

As AI becomes an integral part of production applications, the engineering teams responsible for those services face urgent, practical questions: Can we tell when an agent behaves unexpectedly? Can we steer its behavior to keep it on the right path? Can we find the logs needed to diagnose a live issue without facing bill shock? And can we connect our AI spend directly to business outcomes to prove these investments are genuinely paying off?

At Cisco Live Melbourne, we’re extending Splunk Observability Cloud and Splunk Agent Observability with new ways to govern AI spend and evaluate agent behavior in live production workflows. These announcements build on foundational capabilities introduced at .conf26, while Splunk Observability Cloud Essentials and Premier Editions provide an easier, more predictable way to adopt and expand full-stack observability.

Together, these updates support our core mission to Optimize AI at Scale: making it easier to troubleshoot applications, manage AI spend, and understand what agents do before their actions reach customers.

Maximize Infrastructure Resilience by Removing Commercial Barriers to Observability

For many organizations, the hardest part of scaling observability across an enterprise isn't the technology-it's the commercial friction. Navigating a maze of separate SKUs, unpredictable usage pricing, and commercial add-ons for emerging capabilities like AI agents makes it difficult and costly for platform leaders to standardize on a single platform.

Splunk Observability Cloud Essentials and Premier Editions solve this by dramatically simplifying how organizations buy, adopt, and expand observability across their entire enterprise:

• Observability Essentials Edition: Establishes a predictable foundation for full-stack visibility-combining Application Performance Monitoring (APM), Infrastructure Monitoring (IM), built-in AI SRE automated troubleshooting, integrated Observability Logs, and out-of-the-box access to Agent Observability without separate licensing hurdles.
• Observability Premier Edition: Scales to complex enterprise environments by adding production-grade Agent Observability (including the new Evaluation Engine and Tokenomics), Digital Experience Monitoring (DEM), Database Monitoring, SecureApp runtime protection, and Business Journeys.

To remove the commercial fear of runaway data costs as you expand, both editions include Observability Logs with tiered storage ($0.60-$0.85 per GB/day). Recent logs stay readily searchable in high-performance storage for live incident troubleshooting, while older logs automatically transition to lower-cost storage where they remain fully queryable. Inside Log Explorer, teams can investigate logs, metrics, and traces side-by-side using point-and-click filters, natural language prompts, or standard SPL-giving teams the freedom to scale telemetry without surprise overages.

Optimize Tokenomics

As enterprises scale generative AI applications on Amazon Web Services, managing the cost and performance of foundation models has quickly become a top operational headache. Building on the Agent Observability innovations introduced at .conf, Tokenomics will soon be available on the AWS Marketplace to provide comprehensive cost governance and performance tracking, giving organizations running AWS-hosted agents a unified way to monitor each agent by its cost and business results.

By pulling telemetry from Amazon Bedrock and CloudWatch, Tokenomics connects raw agent spend directly to business KPIs like NPS and transaction costs to show engineering teams exactly which AWS agents earn their budget, turning opaque token spend into defensible, executive-ready ROI.

Additionally, Tokenomics brings further enhancements, including provider cost settings in Splunk Agent Observability, and availability within Cisco Cloud Control.

Monitor Agent Behavior

Splunk Agent Observability expands its capabilities to observe, evaluate, and guardrail with the new Evaluation Engine, giving teams new ways to verify agent behavior and enforce their own rules. Inside the Evaluation Engine you’ll find the new Eval-G1 Evaluation Models and Evaluation Tuner.

Eval-G1 Evaluation Models evaluate prompts and agent responses inline, before they reach a user or trigger an autonomous action. The new models return evaluations in about 100 milliseconds at 2.5 times faster than traditional large-language-model judges, with 95% benchmark accuracy. This makes it affordable to evaluate 100% of your production traffic for hallucinations, prompt injections, data leakage, and policy violations while drastically reducing the compute cost of continuous production evaluation.

Evaluation Tuner is the workspace where you create and fine-tune evaluator models from your traces and expert feedback in a few clicks. It gives product managers, application owners, and SREs a visual, no-code workspace for creating custom agent checks. Teams can write evaluation criteria in plain language, test them against historical production interactions, adjust sensitivity thresholds with visual sliders, and deploy approved guardrails directly to live workflows. This allows teams to apply their own brand, safety, and compliance rules without building custom Python scripts or machine-learning pipelines.

Together, the new Eval-G1 Evaluation Models and Evaluation Tuner help teams evaluate agent behavior as it happens and apply guardrails that reflect their exact business requirements.

A Clearer Path to Optimizing AI at Scale

For the engineering and platform teams running AI in production, the questions are concrete: Are applications healthy? What is driving AI spend? Are autonomous agents following the rules we set for them?

At Cisco Live Melbourne and KubeCon NA in Salt Lake City, Splunk is adding new ways to evaluate agent behavior, create custom guardrails, manage AI spend, and connect it to business value. We invite you to stop by our booths at Cisco Live and KubeCon (booth #105) for more information on these innovations.

Get Observability Cloud for free or book a demo to see how Splunk can help maximize infrastructure resilience, monitor agent behavior, optimize Tokenomics, and troubleshoot autonomously.