- Topics
- /
- /
- Centrifugo reliability
Centrifugo reliability: what it does and doesn't guarantee
Most of the time, Centrifugo delivers messages without trouble. A healthy cluster gets publications to connected clients quickly, and history and recovery fill short gaps when clients reconnect.
But that's expected behavior, not a guarantee. Centrifugo's delivery is at-most-once by default, so a message can be lost; history and recovery raise that to at-least-once only within a bounded window, and exactly-once isn't offered at any tier. That gap between what usually happens and what's guaranteed gets overlooked more often than it should.
This page works through exactly what Centrifugo commits to for message delivery, ordering, latency, and uptime. It also covers what's left for your team to build if you need more than that, and what changes when a managed platform states those same properties as published commitments instead.
Key takeaways
Centrifugo guarantees ordering only when you publish to a channel sequentially or in a single request. Concurrent publishes to the same channel carry no ordering guarantee.
Centrifugo's recovery only covers short gaps. When a client is gone longer than the history window, it reports recovered: false, and your application has to reload state from its own database.
Reliable delivery at scale means building your own deduplication, delivery acknowledgment, and state-fallback layers on top of Centrifugo.
Centrifugo runs as a cluster in whichever region you deploy it to. Serving users worldwide at low latency and staying up through a regional outage both take a globally distributed system. With Centrifugo, your team has to build it.
What reliability means for a realtime system
"Reliable" gets used to describe four distinct properties:
Delivery: at-most-once (a message might be lost), at-least-once (retried until confirmed, so it might arrive more than once), or exactly-once (arrives precisely once, no drops and no duplicates).
Ordering: whether messages arrive in the sequence they were published, which matters most when state depends on that sequence rather than treating each message as independent.
Latency: how long delivery takes, from publish to receipt.
Uptime: how often the system is available to deliver anything at all, usually stated as a percentage over a year.
Global distribution is a related but distinct measure, and it's worth flagging here because of its impact on reliability. Rather than asking whether the system is up, it asks whether the system runs close to users wherever they are, and keeps working if an entire region fails. Without a globally distributed system, users far from your deployment get slower delivery, and a regional outage takes delivery, ordering, latency, and uptime down together.
A system can be strong on one of these properties, and weak on another, so being classed as "reliable" doesn't say much.
The importance of each property also depends on the application, not just the vendor. A live cursor or a "now playing" widget can usually tolerate a dropped update, since the next one supersedes it. A counter, a synced document, or a financial transaction can't tolerate that. A duplicate or a missed update corrupts application state rather than leaving it briefly out of date.
The rest of this page looks at what Centrifugo commits to on each of the four, and global distribution.
What does Centrifugo provide when it comes to reliability?
Message delivery: at-most-once by default, at-least-once with history and recovery
By default, Centrifugo's delivery model is at-most-once: Centrifugo can lose a message on its way to a client. The application is expected to tolerate that loss.
And although Centrifugo's recovery feature gets you to at-least-once, it only does so within limits. History is a bounded, ephemeral cache, not durable storage. It holds a fixed number of recent messages for a fixed retention window. That's enough to survive a short reconnect, such as a balancer reload or a brief network blip. It isn't enough to catch a client up after a much longer absence.
When a client reconnects inside that window, Centrifugo replays exactly what it missed and reports recovered: true. When Centrifugo can't do that, because the gap is too large or the stream itself was lost, it reports recovered: false. The application then has to reload its state from its own database instead.
Neither tier, OSS or PRO, offers exactly-once delivery. Centrifugo can de-duplicate retried publishes with an idempotency key, but it doesn't guarantee each client receives a message exactly once, and its docs advise tolerating an occasional repeat after recovery. Closing that gap means building a deduplication layer in your application, which is real engineering work your team has to take on.
Message ordering: guaranteed for sequential publishes, not concurrent ones
Centrifugo guarantees message order within a channel, but only when messages are published one after another or together in a single request. Publish one message, wait for it to complete, then publish the next, and subscribers receive them in the order you sent them.
Publish two messages to the same channel at the same time, and that guarantee disappears. Subscribers will all still see the messages in the same order as each other, just not necessarily the order you actually sent them in.
This matters most when your application's state depends on the order of events. In a presence feed, a "leave" has to be processed after the matching "join", and in a collaborative document, edits have to apply in the order they were made. If your application publishes to the same channel concurrently, your team has to enforce that order, because Centrifugo won't.
And if your backend publishes through a message queue instead of calling Centrifugo directly, Centrifugo can only keep messages in the order the queue hands them over. Kafka and a PostgreSQL outbox table can split messages across partitions, and only keep order inside each one. If two messages for the same channel land in different partitions, subscribers can receive them in either order. With NATS JetStream, order isn't preserved once more than one Centrifugo instance reads from the stream.
Uptime and latency: down to how you run it
Because Centrifugo runs on your own infrastructure, its uptime and latency are outcomes of how your team sizes, runs, and monitors it. No vendor can commit to those numbers for software it doesn't operate, and Centrifugo's documentation doesn't state an uptime figure or a latency target for either the open source or PRO edition.
Support doesn't change that. Centrifugo OSS is a community project, with no support contract. Centrifugo PRO's license includes a year of updates and prioritized support, which means your requests are handled ahead of community channels, but it doesn't guarantee a response time.
Centrifugo doesn't commit to how many connections a node can handle either. Its FAQ gives only a ballpark of 50,000 to 100,000, and tells you to load-test your own setup to find the real number. If that number is wrong, you find out at peak, when a traffic spike or mass reconnect pushes nodes past their limit, and increased latency and dropped connections follow. Finding the right number, keeping it right as your traffic changes, and recovering when it's wrong are all your team's job.
And there's a practical reason faster, guaranteed support would be hard to offer. Centrifugo's own contributor history shows one person accounting for the large majority of commits to its server repository, with only a handful of people ever having made more than a handful of commits. That's a very different setup from a managed platform with its own 24/7 engineering on-call rotation.
A globally distributed system: yours to build
A Centrifugo cluster runs in whichever region you deploy it to. Centrifugo has no built-in way to route clients to their nearest location, replicate messages between regions, or fail over automatically when a region goes down. So users far from your cluster get higher latency than users close by, and a cloud provider outage in that region takes the deployment down entirely.
If your product needs to serve users worldwide at low latency and stay up through a regional outage, you need a globally distributed system: clusters in several regions, a way to route each client to the nearest one, messages and history kept in sync between them, and failover you build and test. That's a distributed systems project in its own right, on top of the delivery, ordering, and support gaps covered earlier.
What you'd need to build to run Centrifugo reliably in production
Everything that Centrifugo provides is a primitive, not a finished guarantee. To get guarantees you can rely on in production, your team has to build the rest:
What you'd build | Why Centrifugo needs it | What it involves |
Deduplication layer | No exactly-once delivery to clients, and recovery can occasionally redeliver a message | A stable ID per message and logic to discard repeats |
Delivery acknowledgment | No built-in API to confirm a message reached a client | Clients send an acknowledgment back (for example, over RPC), and your backend tracks it |
Backend-state fallback | Recovery returns recovered: false when it can't fill a gap, which is expected, not an edge case | Your own source of truth for clients to reload from |
Ordering safeguards | No ordering guarantee for concurrent publishes to a channel | A queue or serialization layer in front of Centrifugo |
Globally distributed system | No built-in routing to the nearest region, cross-region replication, or failover | Clusters in several regions, routing each client to the nearest one, cross-region message sync, and failover you build and test |
Broker redundancy | History and recovery depend on the broker (Redis or PostgreSQL), and a broker outage loses streams that recovery can't fill | Redis replication with automatic failover (Sentinel or Cluster), or a managed high-availability broker, plus regular failover tests |
Capacity headroom | No committed connections-per-node figure, only a ballpark, and connections stay on the node they joined | Load-testing peak and mass-reconnect scenarios, then running spare capacity before you need it |
Monitoring and alerting | No published latency or delivery baseline to alert against | Measuring your own baselines and alerting against them |
None of this is unusual engineering, but once it's built, it's yours. Every layer needs to be maintained through Centrifugo upgrades, retesting as traffic grows, and someone on call who understands how it works.
Why the difference between "possible" and "guaranteed" matters in production
Centrifugo's recovery is built for one common situation: a deploy or load balancer reload that disconnects every client at once, after which they all try to reconnect within seconds. Within certain limits, Centrifugo commits to replaying whatever each client missed, so most users never notice anything happened.
Those limits are:
The client reconnects within the history window your team configured, which is usually sized for a brief reconnect, not a longer outage.
It missed no more than 300 messages. That's the default limit per recovery attempt, which you can change, and it applies however long your window is.
The message history survived the disconnect: it wasn't evicted or lost, for example in a broker restart without persistence.
Outside those limits, there's no commitment at all. Centrifugo tells the client recovery didn't work (recovered: false), and nothing else steps in, so your application has to fill the gap itself.
Example: a deploy on a chat app
The deploy disconnects every client, and they all reconnect within a few seconds.
Most reconnect inside the history window and recover cleanly, with no lost messages and no visible gap.
One user on a slow mobile connection reconnects late, outside the window, and gets recovered: false.
Unless your app reloads their messages from its own database, that user is missing messages and has no way of knowing.
So recovery makes a clean reconnect likely, not certain. Your application has to catch the cases it misses, and when one of those turns into an incident, there's no vendor to escalate to.
What a managed platform guarantees
A managed realtime platform takes on the four reliability properties (delivery, ordering, uptime, and latency) as part of the service. Instead of your team building deduplication and failover and proving they work, the provider states what it commits to, measures it, and is accountable if it falls short. How strong those commitments are varies by provider, so it's worth checking which are contractual and which are published targets.
Ably is one example. Here's what it publishes for each of these four properties, plus global distribution:
Delivery: exactly-once delivery through idempotent publishing, with each message stored redundantly across isolated datacenters before it's acknowledged, on infrastructure designed for 99.999999% message survivability.
Ordering: each publisher's messages arrive in order at every subscriber, built into the platform rather than something the application has to add.
Uptime: a 99.999% uptime SLA, which allows about five minutes and 15 seconds of downtime a year. Ably has gone more than seven years without a global outage and regional incidents are routed around automatically, so global service continues.
Latency: 6.5ms message delivery latency within a region, measured at the edge of Ably's network, and a 37ms global median round-trip latency, measured from real browsers. Both are published performance figures, not guarantees.
Global distribution: 12 globally distributed regions and 750+ edge locations through AWS CloudFront, with each client routed to its nearest region. Regional incidents are routed around automatically, so service continues without failover your team has to build and test.
Compared with self-hosting, most of the reliability work stops being yours to build, maintain, and staff on-call for. Deduplication, ordering, a globally distributed system, broker redundancy, and capacity sit with the platform, backed by stated commitments and a provider that's accountable if they fall short. What stays with your application is its own source of truth, any per-user delivery confirmation it needs, and monitoring of its own behavior.
What migrating off Centrifugo involves
A managed platform's commitments on delivery, ordering, uptime, and latency are only worth having if moving to it doesn't turn into a project of its own. Migrating isn't a rip-and-replace. Most teams move one channel or workload at a time, running both systems in parallel until behavior is validated. That doesn't mean it has to be a long project. How long it takes depends mostly on how much of your application relies on Centrifugo-specific behavior, rather than on general pub/sub concepts that carry over.
On the reliability side, two things need to be reworked rather than reconfigured.
Recovery: In Centrifugo, how long a client can be disconnected and still catch up depends on the history window your team configures. In Ably, it's a fixed two minutes, whether the same client reconnects or a new one starts up, such as after a page reload. After two minutes, Ably won't replay missed messages automatically, so when a client comes back after a longer gap, your app needs to request them from Ably's history API.
Ordering: In Centrifugo, a channel's history is a single stream, so every client sees it in the same order. Ably works differently when a client loads recent history to catch up with a live channel. Messages from the last two minutes can be ordered differently depending on the client's region, while older messages (with persisted history enabled) come back in one global order. This only affects messages published at the same time from different regions, since each publisher's messages still arrive in order at every subscriber. If your application depends on a single order across regions, either publish each channel's messages from one publisher, or have clients sort by a sequence number you include in each message.
Presence, authentication, connection handling, and backend integration all carry over as concepts, so most of the work is mapping how each one behaves on Ably rather than rebuilding it. Our team can help you scope that mapping for your setup.
Ready to move off Centrifugo?
Ably runs this infrastructure for you, so these commitments come by default rather than as something your team has to build toward. When you're ready, our team can walk through exactly what migrating to Ably looks like for your specific setup.
Does Centrifugo guarantee message delivery?
Not by default, and no tier changes that. Centrifugo's default delivery model is at-most-once; enabling history and recovery gets you to at-least-once within a bounded retention window. Even Centrifugo PRO, the paid tier, adds prioritized support rather than a delivery guarantee, and exactly-once isn't available at either tier.
Who is responsible for uptime when you self-host Centrifugo?
Your team is. Because Centrifugo runs on your infrastructure, its uptime and latency depend on how you size, run, and monitor it, and no vendor can commit to them on your behalf. A managed platform runs the infrastructure itself, so it can make those commitments.
What's the difference between Centrifugo's recovery feature and a delivery guarantee?
Recovery replays messages that a reconnecting client missed, but only if the gap fits within a bounded history window. Outside that window, recovery reports recovered: false rather than guaranteeing delivery, and the application has to reload state from its own backend.
Does Centrifugo guarantee message ordering?
Only for publishes to the same channel that are sequential, where one completes before the next begins, or sent together in a single request; concurrent publishes carry no ordering guarantee, though all subscribers still see the same order as each other. If you feed publications in through Centrifugo's async consumers, ordering also depends on that pipeline: Kafka and a PostgreSQL outbox keep order within a partition, but NATS JetStream doesn't once more than one Centrifugo instance reads from it.
What does Centrifugo PRO's priority support include?
A year of updates, and your requests to Centrifugal Labs are handled ahead of community channels. It doesn't guarantee a response time, and it doesn't change who runs your deployment or is responsible when it fails: that's still your team.
Can Centrifugo run as a globally distributed system?
Not out of the box. A Centrifugo cluster runs in the region you deploy it to, so serving users worldwide and surviving a regional outage means running clusters in several regions, routing clients between them, keeping them in sync, and building failover yourself, not flipping a config setting. That's one of the costs that's easy to underestimate when comparing a Centrifugo bill against a managed subscription that includes it.