Pub/Sub pattern architecture

Pub/Sub (or Publish/Subscribe) is an architectural design pattern used in distributed systems for asynchronous communication between different components or services. Although Publish/Subscribe is based on earlier design patterns like message queuing and event brokers, it is more flexible and scalable. The key to this is the fact that Pub/Sub enables the movement of messages between different components of the system without the components being aware of each other’s identity (they are decoupled). 

Learn more about:

• What is Pub/Sub? (extended version)

• When to use the Pub/Sub pattern?

Pub/Sub provides a framework for exchanging messages between publishers (components that create and send messages) and subscribers (components that receive and consume messages). Note that publishers don’t send messages to specific subscribers in a point-to-point manner. Instead, an intermediary is used - a Pub/Sub message broker. 

The message broker groups messages into entities named channels (or topics). When a publisher sends a message on a channel, the Pub/Sub broker routes a copy of the message to each of the components who have subscribed to the respective channel. 

The diagram below shows the high-level architecture of a Pub/Sub system:

The pub/sub pattern

It’s worth pointing out a few things:

• Pub/Sub is a flexible messaging pattern. In theory, you can have any number of publishers sending data to topics/channels. On the other end, you could have any number of subscribers. Note that a component can be both a publisher and a subscriber - think, for example, of chat apps, where you both send (publish) and receive (subscribe to) chat messages. 

• Publishers and subscribers are completely decoupled by the message broker. Publishers send messages without worrying about who or how many subscribers will receive them. Similarly, subscribers consume messages without being concerned about who sent them. 

• Subscribers don’t have to poll to see if any new data is available. Instead, they are notified as soon as it becomes available.

• The message broker is responsible for maintaining a list of subscribers for each channel, and ensuring that messages are delivered to all interested subscribers. 

• Pub/Sub marks a change in the paradigm of communication from point-to-point information exchange to information sharing on a common forum (topics or channels). 

The Pub/Sub pattern brings many benefits to the table, including but not limited to:

• Loose coupling between components, making your system more modular and flexible.

• High scalability (in theory, Pub/Sub allows any number of  publishers to communicate with any number of subscribers).  

• Language-agnostic and protocol-agnostic, which makes it straightforward and fast to integrate Pub/Sub into your tech stack. 

• Asynchronous, event-driven communication that’s ideal for realtime, low-latency apps. 

Learn more about the benefits of Pub/Sub

While Pub/Sub is a useful communication pattern in many scenarios, there are situations where it might not be the best choice. Here are a few examples:

• Overkill for simpler systems and small-scale applications that only need to communicate with a few other components or services.

• Not suitable for audio and video streaming.

• Inappropriate for periodic or background tasks.

• Not ideal for non-event-driven systems and use cases where you need synchronous communication.

Learn more about scenarios where you shouldn’t use Pub/Sub

If you choose the Pub/Sub route, it's essential that you can rely on the tooling you choose. For most teams, that means using a managed pub/sub platform rather than building a solution in-house.

In our own research, we've found that in-house realtime infrastructure projects have a strong tendency to run over budget and over time. And even after all that, they often cost more overall.

If you decide to use a managed Pub/Sub service though, you need to make sure it is fit for purpose. This means evaluating it based on scalability, reliability, performance (including latency), guarantees, ease of development, cost of ownership, and its ability to deliver functionality beyond Pub/Sub.

Thankfully, we have done a review of the five best Pub/Sub solutions to help you to compare their offerings.

Ably is a realtime PaaS. Our APIs and SDKs help developers ship realtime experiences like chat and multiplayer collaboration without having to worry about maintaining and scaling messy infrastructure. 

Key Ably features and capabilities:

• Pub/Sub messaging over serverless WebSockets, with rich features such as message delta compression, automatic reconnections with continuity, user presence, message history, and message interactions.

• Multi-protocol capabilities; in addition to WebSockets, we support protocols like SSE, MQTT, and HTTP/REST. 

• Managed integrations with tools like Kafka, AWS SQS, AWS Kinesis, and RabbitMQ. 

• A globally-distributed network of datacenters and edge acceleration points-of-presence. 

• Guaranteed message ordering and delivery. 

• Global fault tolerance and a 99.999% uptime SLA.

• < 65ms round-trip latency (P99).

• Dynamic elasticity, so we can quickly scale to handle any demand (billions of messages sent to millions of Pub/Sub channels and client connections).