Innovation and Disruption in the Cloud

It seems like almost everywhere you turn, there’s an announcement of some new cloud technology. According to the marketing of these technologies, all of them are in some way radically "new," "unique," "innovative," etc.

But dig a little deeper and the reality is quite different. Companies are making big investments in the cloud, both in shifting existing workloads to the cloud as well as following a “cloud-first” approach in deploying new applications and projects on cloud infrastructure. So it’s not surprising that just about every software vendor is touting their cloud offerings and capabilities, fighting to get a slice of that customer spend. However, a good number of these are simply “cloud-washing” legacy technologies, i.e. taking existing technology that was designed for non-cloud (e.g. datacenter) environments and deploying it in the cloud. That may clear the minimum bar for saying “cloud”, but by no means does it represent innovation in the cloud.

Separating Innovation from Hype

So what does it mean to innovate in the cloud, and how can you identify solutions that truly innovate vs. ones just trying to leverage marketing hype?

Design for the Cloud

For one, a solution needs to be designed for the cloud. It may be wonderful that vendor X can install its application on the cloud for customers, but that doesn’t do much more than change who manages the infrastructure. Truly designing for the cloud generally requires going all the way down to the fundamental architecture:

• Does the architecture make it possible for the solution to scale resources up and down elastically
• Does it take advantage of both scalable compute services and scalable storage services in the cloud
• Does it utlilize cloud resources in ways that ensure resiliency and reliability
• Can it support large numbers of users and applications transparently and seamlessly in a single service
   

For Streamlio, that meant working with and on Apache Pulsar, technology that had been co-created by the Streamlio team to be able to take full advantage of cloud environments—initially within the Yahoo private cloud, but also within public clouds including the AWS cloud.

Solve a Challenge Specific to Cloud

However, designing for the cloud isn’t by itself enough to make something innovative in the cloud. Another important criteria is whether something addresses a problem or need that is especially important because of cloud. For example, software to generate invoices is needed whether or not cloud is fundamental to your business. Simply offering invoicing software in the cloud is valuable and could even be a financially successful business, but it most likely wouldn’t be a good example of innovation in the cloud unless it solves some invoicing problem that is uniquely important or difficult because of cloud.

In Streamlio’s case, the challenge was that the design patterns that dominate the growing array of cloud applications magnify the importance and challenge of moving and processing data as quickly as possible. Unlike legacy on-premises applications, which are almost always built as monoliths or as a combination of a modest number of weighty components, cloud applications have largely been built on service-based architectures, such as microservices and event-driven architectures, which consist of 10s, 100s or even 1000s of specialized services connected together with data and data flows.

The resulting scale and complexity are beyond the capabilities of legacy messaging and data integration software, which were never designed for them. Streamlio’s solution is built on technology designed to scale to connect 100s, 1000s and even millions of services in a multi-tenant solution.

Be Disruptive

And finally, being innovative in the cloud means doing something new and disruptive. In our case, we didn’t set out to simply deliver yet another messaging technology—there are more than enough of those, most of them hard to distinguish from each other until you get deep into the technical details. What we set out to make possible was a rethinking of data infrastructure used by applications, and to do that with cloud-native technology.

That’s a key part of what is fundamentally innovative about Streamlio—we're delivering a modern data fabric that leverages and complements infrastructure layers like Kubernetes and containers while providing a common interface to data for applications and services running in the cloud. Messaging is a part of that data fabric—it’s what moves and connects data—but it’s complemented by first-class-citizen capabilities for stream processing and stream storage, unlike earlier generations of technology that treated those capabilities as secondary or provided them only through bolt-ons.

We’re honored to be named a cloud innovator alongside other great companies and technologies that are leveraging cloud in new and groundbreaking applications. It will be exciting to see how these and other emerging new solutions will continue to push forward the innovation made possible in the cloud.

Thanks,
Jon Bock