What is event-driven architecture?

Event-driven architecture is a form of software design that enables asynchronous event publishing and delivery between decoupled components. With EDA, companies – especially enterprises with large codebases and organizations with complex lines of business applications – can build highly scalable systems that connect and coordinate webs of microservices.

Traditionally, backend systems communicate using API-driven design structured as a series of requests and responses. Each service in an application posts a request and waits for a response before proceeding to the next task. 

In many contexts, the inherent delays between posting a request and receiving a response are fine. If, for example, a customer places an order on Amazon and the system raises an event internally to indicate an order has been placed, small delays rarely matter. A backend process could lie in wait for an order-processed event that will trigger an order confirmation email. The difference between getting a confirmation email instantly or after a few minutes is small.

But EDA is rising in popularity because the use cases for reliably faster responses are spreading and growing. IBM research shows that 68% of organizations not yet using microservices agree that the effort and expense of adopting microservices will be worthwhile. And IDC research shows that 90% of the world’s largest companies will be using realtime data by 2025.

In contexts like these, the inevitable delays caused by request/response systems aren’t tenable, and the value of adopting EDA skyrockets. With EDA, the service posting the event isn't reliant on the other service picking it up, which means the original service will still work without the other service’s response. That results in EDA being more resilient due to decoupling and more responsive due to events driving communication.

In EDA, applications are driven not by services awaiting information but by the events themselves. The focus shifts from services mostly lying in wait and occasionally polling for reasons to repost to events flowing through a proactive system that posts information as soon as events happen.

EDA is an architecture design that structures information flows such that events drive responses through services rather than having services look for events to respond to. 

Within EDA, there are four key concepts to understand: 

1. Publishers are event producers that send event messages when a state change occurs.

2. Events are state changes, which simply mean that something has happened. An event could be anything from a user’s request to reset a password to a warehouse updating its inventory.

3. Subscribers are the recipients of events and in an EDA design, publishers trigger events and communicate the state change to subscribers as “messages” via an “event channel.” 

4. Brokers sit between event producers and subscribers to manage the flow of information. Brokers receive all the events and route messages to subscribers linked to particular events or topics.

Here, brokers introduce flexibility by decoupling the “publishing” parts of an application (the components that produce events) from the “subscription” parts of an application (the components that consume events). 

The benefits, which we’ll cover in greater detail below, emerge primarily from this decoupling. In an EDA design, publishers and subscribers don’t need to be connected or know anything about each other; events drive responses and applications can respond to changes in realtime. At no point is any part of the system frozen, waiting for an update. 

An “event,” in this context, is a deliberately broad term. Events refer to changes in state, meaning that an event can include anything that involves one thing or the status of one thing changing to another. 

At its simplest, an event might be a television set changing from one channel to another. In one moment, the television is storing a state – say, channel three – but after clicking the remote, the television changes to a different channel – channel four. That shift is a state change, qualifying it as an event. 

Events, especially in realtime contexts, tend to be much more complex and numerous. When you’re shopping on an ecommerce platform, for example, a series of events come out of even simple actions, including:

• Confirmation emails triggered by order submissions.

• Mobile device notifications triggered by delivered packages.

• Warehouse inventory updates triggered by fulfilled orders.

Even these kinds of events are relatively easy to observe from the outside in. Within a system, many more subtle state changes can occur and as brokers pass messages from service to service, events can ripple into other events. 

An EDA design can be an essential part of integrating components of an application and connecting microservices, for example, because this approach ensures responsiveness to events. 

In the real world, EDA can be compared to a baker taking cake orders. In a request/response system, customers have to periodically ask the baker if a cake is ready – never knowing whether they’ll get a yes or no and inevitably incurring inefficiencies as they flip between waiting and asking. 

In an EDA system, the baker works with a middle person who either updates customers as cakes are completed or logs completed cakes so customers can see when cakes are ready whenever they want. 

Other examples illustrate the same fundamentals.

• An ecommerce company using an EDA might collect purchase events and push them through an event processing flow designed to catch instances of fraud. With EDA, the purchase event can trigger a broker that orchestrates the delivery of the event message to multiple services, including a fraud detection service. While a traditional system would incur delays, an EDA system can ensure purchase events are sent to fraud detection systems in realtime to ensure fraudulent orders are canceled as soon as possible.
  

• A manufacturer might use a factory equipped with a variety of sensors and monitors so that if important changes occur, such as a significant rise in temperature, brokers can – in realtime – send these events to systems that can make logical executions based on the information. With EDA supporting this process, the manufacturer can be notified instantly when temperatures rise and respond by shutting the system down or flagging operators. 

Here, it’s useful to compare EDA to request/response systems. Without EDA, systems are stateless and handle events as discrete, independent events, whereas with EDA, systems are stateful and maintain awareness of the applications and operations they’re running and supporting. 

The takeaway is that EDA introduces a broker that enables the decoupling of clients and events, ensuring that updates can happen asynchronously and in realtime.

EDA breaks down into two primary patterns: 

1. Publish/Subscribe (Pub/Sub), which is when a broker updates subscribers in realtime.

2. Event streaming, which is when a broker records events in a log that subscribers can check at any time. 

To return to the cake-baking analogy, a Publish/Subscribe pattern means a broker is updating subscribed customers every time a cake is complete and an event streaming pattern means a broker is recording events in a log that subscribed customers can check whenever they’re hungry. 

The same structure applies to our other examples too: An ecommerce company using a Publish/Subscribe pattern would have a fraud detection service receive purchase events via broker as they occurred in realtime whereas an ecommerce company using an event streaming pattern would track all purchase events in a log that the fraud detection service could reference.

And similarly, a manufacturer would use a Publish/Subscribe pattern to learn about temperature updates in realtime whereas a manufacturer using an event streaming pattern would have a log of temperature changes.

Read more about EDA patterns and when to use them.

As we covered in an article about the shift to EDA, software architecture has evolved from processing single subroutines to processing distributed software and executing multiple subroutines via remote calls. The request/response system emerged from this paradigm because software systems were essentially, at this time, a series of separate subroutines that an orchestrator could invoke by request.

But as the cloud and API-driven application design grew in popularity from the 2000s on, more and more applications used APIs to connect to cloud services that enabled them to, for example, rely on Stripe for payments and Twilio for communications. In the early 2010s, pioneering companies like Amazon and Netflix adopted microservices so that they could separate and isolate formerly monolithic applications into services and use APIs to connect those services together.

All the while, the original request/response system has persisted. EDA has a surprisingly long history, having originated from service-oriented architecture ideas (a predecessor of microservices) in the early 2000s but the modern rise of realtime data has made EDA a technology that’s time has come.

The use cases for EDA are the most prominent and pivotal when it’s important for a system to know about and respond to a given event as quickly as possible.

Some of the clearest use cases for EDA include:

• Data replication: A broker can share an event across multiple services that each need copies of the event data.

• Parallel processing: One event can hit an event processor that triggers multiple processes that run concurrently. 

• Microservices: EDA can support a microservices system by ensuring each microservice responds asynchronously and in realtime to messages from other microservices. 

• Connecting disparate devices and experiences: As businesses develop and deploy IoT devices and edge networks, EDA can help systems keep up with event changes even when event producers are distant. 

• Realtime data monitoring and execution: Consumers and businesses alike are increasingly demanding tools and interfaces that process information in realtime – ranging from sports fans wanting up-to-the-second score updates and logistics companies wanting instant shipping updates – and when realtime updates are essential, EDA is necessary to ensure the efficient processing of realtime data. 

Below, we’ll walk through examples that illustrate some of these use cases in real-world situations across both the Publisher/Subscriber and event streaming patterns.

The Publisher/Subscriber and event streaming patterns are similar but tend to suit different use cases. Generally, a Pub/Sub pattern is best when companies are looking to broadcast information in realtime and an event streaming pattern is best when companies are looking to process and analyze data in realtime.

In the Publisher/Subscriber pattern, subscribers and publishers are decoupled and messages flow between them via a broker. This pattern is most suited to use cases where there are high volumes of publishers and subscribers and the amount of each tends to fluctuate.

Research shows that 41% of consumers prefer live chat for customer support but powering this kind of service is difficult, especially scaling it across the globe and maintaining reliability. HubSpot, a SaaS CRM platform, uses Ably and the publisher/subscriber pattern it supports to send 80,000 messages per day.

Notifications – whether they’re about an event on the weekend or about a tool at work – depend on realtime delivery. The longer notifications take to reach an end-user, the more their value decays. Split, a feature delivery platform, uses Ably to publish feature flag update notifications across more than one trillion events per month so that subscribers can stay up-to-date on essential changes.

Similar to notifications, realtime updates also degrade rapidly with time. For sports fans, for example, a realtime application that keeps track of a game’s score can be compelling and exciting, whereas an application that promises the same but delivers updates minutes after a score is made is disappointing. Tennis Australia, the company that manages the Australian Open, uses Ably to broadcast live updates throughout its widely watched matches. During the 2022 tournament, Tennis Australia used Ably to send over nine billion messages to over one million users.

In the event streaming pattern, an implementation of the publisher/subscriber pattern, subscribers and publishers remain decoupled and messages still flow between them via a broker. But with event streaming, subscribers can read events from any point in the event stream and events can persist after they’re processed.

IoT devices are frequently deployed to monitor environmental changes and send information back to a hub for analysis. Logistics companies, for example, might use IoT devices to monitor weather changes that could affect sensitive packages. By using an event streaming pattern, these companies can monitor changes in realtime and if a package arrives damaged, re-examine previous changes to see what might have caused it.

Ecommerce companies lose as much as $48 billion a year to fraud, so it benefits them to find ways to track fraud risks and respond to them as quickly as possible. With event streaming, ecommerce companies can monitor risky shopping behavior and analyze how successful their fraud detection policies are by reviewing previous events.

Traders monitoring stock prices care as much about the retrospective flow of data as the realtime flow. While the realtime flow is essential for making rapid decisions in a similarly rapid market, traders also need to look at previous events to see patterns that can fuel future predictions.

The key benefit of EDA is that it enables clients to subscribe to events and receive updates asynchronously, but if both the client and the event in question are online at the same time, the client can then receive event updates in realtime. The benefits below all flow from that design.

With EDA, businesses can:

• Create fluid, realtime experiences. Because EDA shifts complexity to event producers instead of clients, companies can provide effective, instantaneous-feeling realtime experiences.

• Build flexible systems. EDA decouples event producers and consumers, which means that companies can separate communication logic and business logic. Given that separation, companies can change and iterate different parts of the system without breaking other parts. 

• Deliver messages across devices and platforms. Because EDA uses a broker, publishers and subscribers don’t directly communicate, which means that the broker can push messages to a variety of subscribers across a wide range of devices and platforms.

• Maintain elasticity. With EDA, business logic doesn’t depend on there being a particular amount of publishers and subscribers. Given that decoupling, EDA systems can accommodate surges in users without failure or latency. 

• Increase responsiveness. Because EDA enables asynchronous communication, development teams can program modules separately – all the while knowing that message delivery won’t block the event producers. After message delivery, the event producer returns to its regular work and messages only interrupt subscribers when the message corresponds to a subscribed topic.

Despite its clear advantages over response/request architecture in many contexts, EDA isn’t suitable in every context. Even when companies build EDA for the right reason and use case, issues like scalability, reliability, and latency can make EDA less effective than it can be.

• Excessive complexity. EDA is a rigorous architecture design appropriate for rigorous circumstances. For simple systems and use cases, EDA can be overkill. If static scaling will suffice, for example, a traditional system is likely a better fit.

• Poor fit for media streaming. Both publisher/subscriber and event streaming patterns (despite the name of the latter) are poor models for streaming audio and video content. Media streaming has its own nuanced requirements and systems that provide synchronous point-to-point communication will be more effective.

• Overkill for periodic tasks. EDA is well-suited to asynchronously processing and sending information, so it’s not the best solution for processes that run regular or periodic background tasks. For running regular cron jobs, for example, EDA will be overkill.

• Implementation difficulties. EDA, like realtime updates infrastructure more broadly, lives and dies by how well companies implement it. At its best, EDA helps applications deliver and react to event changes in realtime, but if there are even small issues with scalability, reliability, and latency, the otherwise compelling benefits can evaporate.

Read more about the EDA challenges and how to overcome them.

In an effective event-driven system, the details – particularly as the system scales – make all the difference. Complexity, both from edge cases and from the sheer density of producers, events, and subscribers will chase any EDA project. 

As you build, consider these questions:

• How can you maintain consistently low latencies, especially as you scale?

• How can you increase bandwidth performance?

• How can you preserve data integrity and ensure messages are accurate and delivered without repetition and in the correct order?

• How can you ensure your system consistently remains highly available?

• How can you build a system that is reliable and fault tolerant?

Answering these questions and addressing the challenges they pose involves a range of best practices, including:

• Build for horizontal scalability so that adding components doesn’t require downtime. 

• Choose products and frameworks that support clustering across different regions so that each component remains highly available.

• Choose components that you can separate so that each one is independently scalable and can contribute to greater overall resiliency.

• Build a strategy for exception handling so that your system can log exceptions, retry processing an event a specified amount of times, and move the non-working event to a separate queue. 

• Prioritize security so that your system can maintain transport level security, authorize and control access to events, provide audit trails, and encrypt data.

Many companies can see the benefits of EDA – as well as the benefits of systems EDA can support, such as realtime updates – but not all of them have the resources, time, or niche development skills to build an event-driven system. Especially when you account for the challenges and best practices above, it often makes sense to turn to a provider who specializes in this work.

Ably provides a realtime PaaS that includes APIs and SDKs that developers can use to build and ship realtime features without having to build realtime infrastructure or event-driven systems from scratch. With Ably, companies can depend on publisher/subscriber messaging over wireless WebSockets, multiple protocol capabilities, managed integrations with tools like Kafka, a globally distributed network, and < 65ms round-trip latency.

Max Freiert, Product Group Lead at Hubspot, told us that “Ably makes realtime just work – without it, our product would literally stop working.” At Hubspot, Freiert says, “Ably is now a business-critical part of our organization-wide infrastructure and a key innovation partner that our engineers really like working with.”

Try Ably with a free account.