We’re excited to announce that Ably LiveSync is now in public alpha!
LiveSync is a state synchronisation product that allows you to watch changes to your database and publish them reliably to millions of frontend clients, enabling those clients to stay up to date with the latest changes in realtime.
LiveSync at a glance
LiveSync is designed to work with the tech stack of your choice and protects your application from data inconsistencies caused by dual-writes, as well as scaling problems caused by thundering herds, which cause a huge load on your database.
LiveSync has three key components:
- A Database Connector that monitors specific changes in your database and publishes those changes;
- Ably Pub/Sub Channels, which are the mechanism behind-the-scenes to broadcast those changes at scale; and
- A Models SDK that helps the front end clients consume those changes and merge them correctly with the overall application state. Additionally, the Models SDK offers a way to enable optimistic updates to deliver instant updates for a snappy user experience.
In the alpha release, you get two options to host the Database Connector: either host it yourself using the Docker image or host it easily on Ably’s infra by setting up an integration rule for PostgreSQL on the dashboard. LiveSync is designed to be database agnostic with support for PostgreSQL in this version. If you would like support for other databases, please reach out.
Addressing the challenge of realtime data synchronization - no trade-offs
LiveSync is designed to overcome the challenges posed by existing data synchronization solutions, and the trade-offs that they make. Periodic polling leads to inefficient backend load and outdated data, while alternatives like setting up CDC and stream processing systems add a lot of infrastructure management overhead. Let’s specifically take a look at two key problems that make it hard for developers to stream database updates at scale:
Thundering herd problem
This issue has two sides. The first is polling. Many “realtime” frontend apps still rely on periodic database polling to maintain an up-to-date state. But polling is tricky. If you want the most up-to-date data you have to poll more frequently, which increases the load on your backend. And this only gets worse as you get more users.
The second side of the problem is seen when a pub/sub mechanism is used to notify all clients that they need to request the latest data. This creates a stampede where all clients request the data at the exact same time.
Both of these problems can lead to a sudden surge in traffic, overwhelming the database's capacity to respond effectively.
Dual-write problem
The dual-write problem happens when you need to change a piece of data in two places, but the change should happen either in both places, or neither. Imagine if data is successfully written to your database but fails to write to Ably Channels. This discrepancy, especially when those writes are happening concurrently, can lead to inconsistencies - resulting in incorrect or corrupted data, or system failures. When writing to more than one system, there’s also no guarantee of the ordering of those writes, which is a real nightmare when your business logic depends on the sequence of events.
Addressing the thundering herd and dual-write problems with LiveSync
LiveSync eliminates these complexities and inefficiencies, offering a seamless database to frontend synchronisation while allowing you to maintain control over your tech stack. The key benefits being:
- Single source of truth - With LiveSync, you can ensure that your database is truly the single source of truth with the changes synced across the client devices in realtime in the same order as the database.
- Event driven architectures and realtime updates - There is no need to poll for updates, subscribers are automatically notified of new data as soon as it becomes available in your database.
- No more dual-writes - LiveSync effectively enables you to transactionally publish changes on Ably Pub/Sub Channels as you write to your database thus removing the possibility of eliminating inconsistencies caused due to potential failures. You can ensure writes are always published in the right order and delivered exactly once to all the clients.
- State management supporting the frontend - With the Models SDK, you get the design patterns to merge new changes with your existing application state, removing the guesswork and unforeseen errors with realtime state management.
- Database-agnostic design - LiveSync is designed to work with your existing database and currently supports PostgreSQL in the alpha version.
- Live collaboration with a snappy UX - No more refreshing your page or watching the wheel spin. Multiple users can collaborate on a single page with optimistic updates reducing perceived latency for users.
- Highly reliable & scalable infrastructure by default - LiveSync leverages Ably’s existing Pub/Sub platform with proven scale to broadcast updates to clients. Change events are distributed through a global system with no single point of failure and a 99.999% uptime SLA. And thanks to autoscaling systems that actively manage capacity, we can absorb very large spikes in concurrent users or interactive actions.
Get started with LiveSync now
Sign-up for a free account and dive into our docs to give LiveSync a try. If you are interested in being an alpha tester, or would like to provide feedback on the product, please get in touch - we’d love to collaborate!
