Event-driven architecture patterns and when to use them

Event-driven architecture, or EDA, decouples publishers and subscribers so that events – which denote any state change – can flow between the two without either component knowing about the other.

A component “subscribes” to a topic, so that this subscribing component can receive notifications whenever a relevant event occurs without having to connect directly to the event publisher. A mediator – called a broker – facilitates the transmission of events and categorizes them via topics that subscribers subscribe to. 

This structure is foundational to EDA systems but this essential idea translates, depending on the use case, into two primary patterns: Publish/Subscribe (Pub/Sub) and event streaming. Both share elements in common, including asynchronous communication between decoupled publishers and subscribers, but the former is broker-based and the latter is log-based.

In the Publish/Subscribe pattern, publishers and subscribers are decoupled, and communication between the two is asynchronous. Publisher and subscriber components are separated and a broker facilitates the processing of events and the passing of events from publishers to subscribers in realtime. 

Key to the Publish/Subscribe pattern is the shift of architectural complexity from client to publisher. With a Pub/Sub pattern, even complex applications can become as smooth and seamless as smaller, simpler applications. The shift of complexity means the client isn’t stuck polling and waiting and polling again. Events drive processes and with brokers handling message routing, components are free to keep running to support the larger application.

(Source: The Startup)

Enterprises with large codebases tend to benefit the most because asynchronous processes make the overall system more resilient. And because an EDA system separates business logic from communication logic, separate components can keep running even if the communication processes aren’t working. This makes data broadcast features, especially ones using realtime data, much more practical.

Event streaming, like Publish/Subscribe, is an EDA pattern that decouples publishers and subscribers to allow for asynchronous communication. The primary difference is that in an event streaming pattern, publishers create a stream of events that are then written to a log that subscribers can reference asynchronously. 

The benefit to this is that as events are processed, subscribers can keep up with current events in real time and reference events from the past as they were recorded in the log. This design creates “temporal durability,” meaning that messages persist across time. 

Subscribers can access the stream at any time to read recent messages, batch process a series of messages from the last time they accessed the stream, or reference relevant messages from the recent past.

(Source: Redhat)

Event streaming also comes in two models: push and pull. 

• In the pull model, brokers only send subscriber data when subscribers indicate they want to receive events. 

• In the push model, the broker has an internal logic that dictates which subscribers get which updates. 

Kafka, for example, pulls data from brokers whereas SSE and webhooks, for example, push data.

The event streaming pattern is most useful in contexts where applications need access both to event updates in realtime and at least occasional access to past events. A fraud detection service, for example, would benefit from the ability to detect and flag potentially fraudulent activities and then reference previous events that might further evidence fraudulent behavior.

There are three additional patterns that model how events are processed once they reach the subscriber. Each can apply to both Publish/Subscribe and event streaming patterns, but event stream processing – as the name implies – is most common in the event streaming pattern.

• Simple event processing: In its most simple form, event processing involves subscribers capturing and recording events just as they were when they were published.

• Complex event processing: In more complex forms, event processing involves varying levels of analysis on the events as the subscriber captures them. They might, for example, analyze a stream of events to identify patterns. 

• Event stream processing: In this pattern, data streaming platforms, such as Kafka, support an event processing pipeline for streaming and consuming data. Kafka, for example, functions as both a messaging queue and database because companies can insert it to enable event streaming between different components. 

Event-driven architecture provides a range of benefits best seen through examples. 

Publish/Subscribe examples

In the publisher/subscribe pattern, as we discussed above, publishers and subscribers are decoupled and communication between the two is mediated by a broker. The Publish/Subscribe pattern is often best for data broadcast use cases but is also useful in interactive experiences. 

• Dashboards: Urbantz, a logistics platform, uses a Publish/Subscribe pattern to support the broadcast of live delivery updates. With an EDA system supported by Ably, Urbantz can offer a dashboard to clients that provides realtime updates about driver and delivery locations. 

• Interactive learning: MobyMax, an edtech company, relies on a publisher/subscribe pattern to publish millions of updates across a range of live experiences that includes games and team-based activities. With an EDA system supported by Ably, MobyMax can reliably offer these experiences at scale while also operating a live dashboard for monitoring and identifying traffic spikes and bugs.

Event streaming examples

In the event streaming pattern, as written above, publishers and subscribers are similarly decoupled but publishers record events to a log that subscribers can reference to capture events in realtime and to capture events from the recent past.

• IoT: One of the best use cases for IoT is fleet management – particularly in situations where IoT devices, such as small motion and temperature sensors, can help logistics companies monitor the progress and status of their deliveries. Penske, for example, a truck rental company, uses Kafka to stream GPS, speed, mileage, fuel level, and engine temperature events to a central location for analysis.

• Microservices: Before the mid 2000s, most applications were built as monoliths – meaning an entire application was housed as one unit. Due to innovation from companies like Amazon and Netflix, microservices architecture emerged as another option. This architecture design separates and isolates components, turning them into services connected by APIs. In this context, EDA – particularly event streaming – can be a helpful supporting element because it enables faster, more reliable communication between services.

EDA supports a variety of use cases ranging from enterprise-scale microservices to realtime data dashboards supporting essential lines of business applications. 

Particular use cases tend to call for either the Publish/Subscribe pattern or the event streaming pattern. The former is most often best in contexts that require data broadcasts and the latter is often best in contexts that involve connecting and coordinating microservices. 

The use cases above illustrate some of the primary ways either pattern can be used but because of the spreading demand for resiliency and realtime data, there are many more use cases than the four above and many more emerging every day. 

In the next article, you can explore more use cases in greater detail to learn more about why so many companies consider an EDA design essential to their present and future efforts.