CNCF CloudEvents: A Li’l Message Envelope That Travels Far

Event-drive architectures did not require new messaging systems for their services to interoperate; They just needed little envelopes to pass all the metadata around on top of existing messaging systems This was the brilliance behind the open source project CloudEvents.

Last week, the Cloud Native Computing Foundation has graduated the CloudEvents specification, making it a full-fledged member of the CNCF’s ever-growing cloud native stack.

But the simple format has already found success in many production cloud-level services.

For instance, Microsoft uses CloudEvents to annotate events in the Microsoft 365 productivity suite. It is also the standard event model for Azure cloud services as well as for the DAPR runtime, annotating billions of messages each day. Microsoft enterprise customers are also adopting the specification to tie more closely into the Microsoft infrastructure.

“It’s a simple data model with a focus on describing the context of events that works with numerous popular protocols and data encodings in a uniform way, without trying to invent new protocol features or getting in the way of the protocol features that exist,” enthused Microsoft Principal Architect Clemens Vasters, in a statement.

How CloudEvents Works

The project arose out of the CNCF serverless working group, whose mission was to identify the needs of those adopting serverless technologies.

Serverless is all about carrying out short-lived function calls, which are often embedded in an Event-Driven Architecture (EDA).

EDA requires “a lot of specialized code in the middleware to route the events appropriately,” said Doug Davis, co-chair of CloudEvents and the CNCF Serverless WG, in an interview with The New Stack. This requires a lot of translation (“glue code”) however, as an event provider, such as a Function as a Service, may run on a  different communications bus or use a different protocol than what the rest of the system may use.

Just as with API management, the users needed a higher level of abstraction to communicate “intention” across multiple systems, further elaborated SAP Architect and CloudEvents core contributor Klaus Deißner.

“Short-lived, event-driven processing is driving early adoption and use cases for businesses that expect a high rate of change with unpredictable capacity and infrastructure needs are emerging” -2018 white paper from CNCF Serverless working group

With CloudEvents, when a source generates a message, it encapsulates into the message header a description event. When the message arrives at the destination, this description provides the receiver with more data about the overall event, which would otherwise be lost.

👁 Image

The description can also be used for routing.

For instance, a GitHub Pull Request might be a potential CloudEvent. The event is defined once, and a middleware, using a regular expression parser, can route the request to the correct repository, based on information in the CloudEvents message.

Messages can be appended with the CloudEvents from the event producer itself, or by a simple adapter.

“The point is you don’t need a whole bunch of infrastructure around? Whatever infrastructure you have today should have the ability to do simple things like add an HTTP header, or read an HTTP header on the receiving side to figure out what to do,” Davis said.

What Is the Architecture for CloudEvents ?

One of the chief virtues of CloudEvents is its simplicity, Davis said.

Instead of creating a  new messaging system for cloud native computing, CloudEvents augments existing existing messaging and serialization protocols, simply by augmenting the eventing infrastructure already in place.

“It’s not creating a whole new set of middleware, and we were very conscious not to do that,” Davis said. “We wanted people to slowly adapt to CloudEvents without having to sort of reinvent their entire middleware architecture.”

When using HTTP, for instance, a new header is simply added to the message, either in plain text or in binary-like format.

The full CloudEvents architecture, which can be run on top of different messaging and serialization formats, consists of:

• A base specification defines a set of core and optional attributes, formatted as key-value pairs. This spec also offers associated rules for what constitutes a CloudEvent.
• A format for extensions to add use-case-specific rules, such as support for different tracing standards. User-generated extensions must follow the same naming convention and use the same type system as standard attributes.
• Event format encodings, for mapping the metadata to header and payload elements of an application protocol (i.e. JSON).
• Protocol bindings define how the CloudEvent is bound to an application protocol’s transport frame (i.e. the HTTP message, or WebHooks bindings).

The project has software development kits (SDKs) for nine different programming languages, as well as a robust set of protocol bindings.

Who Uses CloudEvents?

Since its debut in 2018, CloudEvents has been polished by over 340 contributors from 122 different organizations.

On the commercial side, notable users include  Adobe I/O Events, Alibaba Cloud EventBridge, the European Commission, Google Cloud Eventarc, and IBM Cloud Code Engine, among others.

It is also used within many other CNCF projects. Used in Knative Eventing, CloudEvents provides a way for build event-driven applications on Kubernetes, allowing them to easily integrate with Apache Camel, Tekton, and Quarkus.

It is also embedded in CNCF’s Argo, Falco, Harbor, and Serverless Workflow — a related project from the Serverless WG.

The CloudEvents team kicked off a new project called xRegistry to develop a standard set of APIs for registries. The metadata can also be used for subsequent analysis, as well.