High-scale, high-frequency messaging and burst activity can lead to significant costs and performance challenges for realtime applications. Trying to solve for these spikes places additional cognitive load on development teams, slowing down development times. And because of this, all too often, developers have to shelve adding new features to their product. To solve this issue, we've introduced a powerful and simple-to-use cost optimization feature into Ably Pub/Sub: server-side message batching.
Addressing the N-squared problem
When a large number of users are actively engaged on a channel, such as during a high-impact event, the cost of sending individual chat, reaction, and presence messages to each user can escalate rapidly.
Imagine a live tennis match with 10,000 fans interacting via chat. When one fan posts a message, it needs to be delivered instantly to the other 9,999. Imagine each person doing that; if even 1% of fans then respond, we're already looking at nearly a million message deliveries. Now imagine 100k fans, each posting twice, with 1,000 replies per post – the message volume explodes - we'd see 2 million responses totalling 220 billion message deliveries!
Introducing server-side message batching
With Ably's message batching feature, multiple messages are grouped together before being sent, significantly reducing the number of messages counted - lowering your costs. It’s important to note that all messages are still sent, but you are only charged for the ‘batches’, not the individual messages.
Message batching benefits at a glance:
- Reduces message volume: By combining multiple messages into a single batch, we significantly reduce the number of messages sent over the network.
- Optimizes bandwidth usage: Batching minimizes data transfer, leading to lower bandwidth costs.
- Reduces cognitive load: Developers can focus on the application instead of working through the mechanics of batching.
- Developer velocity: Simple configuration through your Ably dashboard. No code. additional development work. No extra cognitive load.
- Predictable cost reductions: If for example you set a 100ms target batch window, you can expect to receive at most 1 message every 100ms, and thus be billed for at most 10 messages per second.
- Predictable latency: set batching intervals to meet the expected user experience from a low as 20 m/s up to 1 sec.
Ably server-side batching in action
Building on the tennis example, let’s add some detail to get a more exact figure on the number of messages sent with and without batching.
Using Ably, that would see the cost of the event reduce by 84%, from $1,368.20 to $217.25. Try out our consumption estimator tool and see what you could save.
Message batching offers a powerful balance between realtime performance and cost effectiveness. You can customize the batching interval to suit your specific needs, trading off minimal latency for significant cost savings.
With Ably, you don't need to develop and maintain your own complex batching mechanisms - it’s just a simple configuration. Our system intelligently determines the optimal batch size, ensuring efficient delivery without compromising user experience. Whether you're sending hundreds or thousands of messages, Ably Pub/Sub handles the complexities, allowing you to focus on building exceptional realtime applications.
The number of messages you publish and receive, regardless of how they're batched, will always be accurately reflected in your usage statistics. This is unaffected by our internal batching processes.
Message batching use cases
Server-side message batching is most beneficial for high-scale applications with high user concurrency and frequent messaging:
- High-volume chat applications: Chat platforms experience large spikes in message volumes when big news drops, or during live events like sports games or concerts.
- Fan engagement platforms: Sports fan apps see surges in reactions during key moments, like goals, causing spikes in message volume.
- Virtual events: Where there are lots of people concurrently on a channel - both present and subscribing to presence - virtual event platforms experience very high volumes of data being sent to each customer.
- Collaboration: Multi-user, collaborative apps need to handle high-frequency cursor updates when a number of users are collaborating together without compromising performance.
Get started with server-side batching
It’s quick and easy to start batching messages. Head to our server-side messaging batching docs, which will take you through the simple configuration. If you have any questions, please get in touch - we’ll be happy to help.
