Announcing Splunk Federated Search for Amazon S3 Now Generally Available in Splunk Cloud Platform

Splunk is pleased to announce the general availability of Federated Search for Amazon S3, a new capability that allows customers to search data from their Amazon S3 buckets directly from Splunk Cloud Platform without the need to ingest it.

Enterprises rely heavily on cloud object storage services as the de facto destination for their new data to leverage the cost, compliance, security, scalability and manageability benefits that cloud platforms can offer. Amazon S3 is one of the largest services available today, with over 280 Trillion objects all over the world. However, one of the biggest concerns when using cloud storage solutions is data movement, since it can introduce latency and egress costs when attempting to search that data.

To address this challenge, Splunk users can now search data at rest within their Amazon S3 buckets directly from their Splunk Cloud Platform stack, ideal for investigations that require as-needed access to historical, archival, or low-value data. What’s more, you can still run SPL searches, create dashboards, reports, and correlate data between Amazon S3 and Splunk.

It is important to note that data that requires real-time search performance and high access frequency should still be accessed using Splunk Search on indexed data.

Federated Search for Amazon S3 is supported via an integration with AWS Glue Data Catalog, which provides the schema and metadata necessary to read compatible datasets from Amazon S3. AWS Glue Data Catalog tables provide the necessary schema that Splunk Cloud Platform needs to make sense of the data stored in Amazon S3. This also allows Splunk to search popular data formats such as JSON, CSV, Parquet, XML, ORC and more!

In turn, this integration enables Splunk Admins and users to benefit from the following use cases:

1. Perform forensic investigations directly on historical data stored in Amazon S3 at rest.
2. Run large statistics searches over historical data in Amazon S3.
3. Leverage Amazon S3 as part of their data tiering strategy to store data outside of retention period.

Federated Search for Amazon S3 is available for Splunk Cloud Platform stacks hosted on AWS running on version 9.0.2305. Access to Federated Search for Amazon S3 requires a Data Scan Units license for your Splunk Cloud Platform stack. Contact your Splunk sales representative to learn more about this.

For more about Federated Search for Amazon S3, check out the documentation and release notes, dig into our validated architectures, and tune into our webinar on how to seamlessly search your data with Splunk and AWS.