Scaling Socket.IO in production, and for AI workloads

• Socket.IO's often-cited 10,000 to 30,000 connection ceiling per instance is mostly an OS default (open file limits, local port range), not a hard architectural limit. But the coordination overhead of scaling past it is real.

• Sticky sessions and in-memory state mean a crashed instance loses more than a connection. For a long-running AI agent task, the crash wipes out whatever progress that task had made.

• AI chat and agent features change what scaling Socket.IO requires. Connections stay open longer, sessions need to survive device switches, and human handoff needs full context, not a summary.

Running Socket.IO on a single instance is straightforward. Every server added past the first instance turns the deployment into a distributed system. That system needs a cluster to coordinate, a load balancing layer, and Redis or an equivalent to keep those servers in sync.

The cluster, load balancer, and Redis layer all have to hold up under conditions you don't control. A live sports event or a product launch can take you from thousands to millions of concurrent connections within minutes. Your infrastructure needs to add capacity automatically, and fast enough that users don't see dropped connections at the exact moment engagement peaks. Socket.IO is also single-region by design, so failing over across regions is something you build yourself, not something the library gives you.

Connection shedding, backpressure handling, load balancing strategy, heartbeat frequency, and connection monitoring aren't decisions you make once and move past. They need revisiting as traffic patterns and the application change, and someone has to own that ongoing tuning.

Three mechanisms account for most of the operational burden of scaling Socket.IO yourself. The first is how many connections a single instance can hold. The second is how session state is tied to a server. The third is what it takes to add more servers without losing that state.

In most Node.js environments, Socket.IO's practical connection limit sits in the range of 10,000 to 30,000 concurrent connections per instance. That range comes mostly from two OS defaults. The first is the maximum number of open file descriptors, 1,024 by default on many systems. The second is the local port range available for outbound connections, roughly 28,000 by default. Socket.IO's own performance tuning guide confirms both can be tuned well past 30,000 connections per IP.

But tuning the OS defaults doesn't remove the deeper problem. Event loop contention, memory pressure, and garbage collection start degrading performance in less predictable ways as connection count climbs. That happens regardless of how high the file descriptor and port limits are set.

AI chat and agent sessions reach the 10,000-to-30,000 connection ceiling faster than typical chat traffic does. A support conversation between two people might hold a connection open for a couple of minutes. An AI agent might stream a multi-step response, wait on a tool call, or run a research task. Each of those can hold that same connection open for minutes at a time. Fewer concurrent users reach the same per-instance ceiling as a result.

Socket.IO usually keeps connection state, such as subscriptions and user context, in memory on a single instance. A client has to keep reconnecting to that same instance, or that context is gone.

Enforcing this client-to-instance pinning at the load balancer, using IP or cookie-based affinity, comes with three costs. This pinning works against elastic autoscaling, since a new instance can't absorb load from sessions pinned to another one. It also increases blast radius, since a crashed server wipes out the state of every client pinned to it, not just their connections. And it couples clients to the underlying infrastructure, making the whole system harder to change.

A crashed instance costs an AI agent session far more than it costs a single chat message. If the instance holding a two-minute agent task crashes, the accumulated state of that task (potentially a dozen completed steps) is gone with it. The user has no way to resume without starting over.

A single Socket.IO server works for small applications. Past a few thousand concurrent users, multiple regions, or realtime collaboration at scale, it becomes the bottleneck, so most teams add servers.

Scaling out means adding two components. The first is a load balancing layer. HAProxy, Traefik, and NGINX all support Socket.IO. The second is a way to pass events between servers, since Socket.IO instances don't communicate with each other directly. Socket.IO now ships eight official adapters, including Redis, Redis Streams, MongoDB, Postgres, Cluster, Google Cloud Pub/Sub, AWS SQS, and Azure Service Bus. The Redis adapter remains the most common choice.

The Redis adapter works through Redis Pub/Sub. Each server publishes messages to a channel, and every other server subscribed to that channel forwards them to its own connected clients. A message published by one server only reaches clients connected to another server because both servers are subscribed to the same Redis channel.

This Redis pub/sub mechanism solves room-level messaging across servers. It doesn't unify connection state, and it doesn't remove the need for sticky sessions.

Redis adds latency and becomes a single point of failure. Connection state stays local to each instance, and sticky sessions are still required to preserve client-server affinity. Failure recovery and state synchronization remain your responsibility. The adapter doesn't handle them for you.

Socket.IO supports horizontal scaling in principle, but not natively: you're building and maintaining the coordination logic yourself. That's a reasonable trade for a narrow, well-understood use case. Doing this yourself gets expensive fast as usage and criticality grow. AI features tend to push both at once: more concurrent sessions, and each one worth more to keep running correctly. Only vertical scaling avoids horizontal scaling entirely. Every other strategy, including the sticky-sessions-and-load-balancer approach and the Redis pub/sub adapter just covered, is a different way of doing it.

There's no single path to scaling Socket.IO. Each option comes with trade-offs in complexity, cost, and operational risk. Most of them are different ways of scaling horizontally; only vertical scaling avoids that entirely. Here's a deeper look at the main strategies, and how they stack up in real-world deployments:

Socket.IO's scaling limits aren't hypothetical. A few teams have published exactly where they hit them, before AI workloads were part of the picture.

Trello ran into Socket.IO's connection limits early, using a modified version of the Socket.IO client and server libraries. That modified setup started struggling above roughly 10,000 simultaneous connections per process, even after scaling out to multiple processes and adding a Redis store. After launching at TechCrunch Disrupt, Trello's WebSocket implementation behaved unpredictably under the sudden load. The team fell back to plain polling while tuning server performance, scaling from 300 to 50,000 users within a week. Trello has since been widely reported to have moved to a custom WebSocket setup as usage grew further. The specifics of that later architecture can't currently be verified against a live, citable source.

JioHotstar (formerly Disney+ Hotstar, following its 2025 merger with JioCinema) evaluated Socket.IO, NATS, and MQTT for a realtime social feed running alongside live sports video. The team chose an MQTT broker. Caching wasn't an option because the content was tied to live match events. The team needed a protocol that could handle broadcast, fan-in, and per-user messaging patterns at concurrency reaching tens of millions of connections.

Socket.IO's core limits show up at scale like Trello's and JioHotstar's, even without AI in the picture. AI chat and agent features push those limits harder.

AI chat and agent products place different demands on a realtime layer than typical chat traffic. A conversation can run for minutes rather than seconds. It can hold a connection open the whole time it's waiting on a model or a tool call. It may need to survive a user switching from a laptop to a phone mid-task. It may also need to hand off from an AI agent to a human without the customer repeating themselves. None of this is an edge case in an AI product: it's the normal shape of the traffic.

Socket.IO's architecture wasn't built around long-lived connections, multi-device continuity, or AI-to-human handoff. The gaps line up with the connection ceiling, sticky sessions, and coordination overhead. An AI feature landing on top of a Socket.IO deployment that already exists for something else hits these gaps, and so does one built as part of a new product from the start. Either way, the gaps show up the same way.

• Long-lived, stateful connections reach Socket.IO's connection ceiling with fewer concurrent users than typical chat traffic needs. This is because each AI session holds its connection open far longer than a chat message does.

• Sessions don't survive a device switch. Socket.IO's model ties one connection to one device, with session state held in memory on whichever instance the client is connected to. If a user starts a conversation on a laptop and picks it up on a phone, the session doesn't follow. The state never left that first instance.

• Human handoff needs full context, not a summary. The pattern that works for AI-powered customer support has three steps. The AI handles the routine query, escalates to a human with full context intact, and hands back to the AI once resolved. Socket.IO has no session model built for that handoff. A team has to build the context transfer, the channel handoff, and the state reconciliation themselves, from scratch.

• Retries and reconnects can duplicate or corrupt a response. Without built-in delivery guarantees, a dropped connection during a stream followed by a retry can produce duplicate tokens or a corrupted UI. Socket.IO's own delivery guarantees documentation covers message ordering, but deduplication across a reconnect during an active stream is left to the application to build.

• Silent failures are hard to catch. An agent that has crashed mid-task looks, from a client's perspective, identical to a healthy agent that's taking a long time to respond. Socket.IO has no built-in presence or heartbeat mechanism designed to tell the two apart, so teams fall back to polling or timeout heuristics. A Kubernetes-specific scaling issue logged against Socket.IO shows this coordination gap is an active, unresolved friction point for teams running it today, not a hypothetical.

None of this (the connection ceiling, session continuity, handoff, retries, or silent failures) means Socket.IO is the wrong choice for every AI feature. A single-turn AI response with no handoff, no multi-device requirement, and modest concurrency can work within Socket.IO's connection-ceiling and sticky-session limits. What changes is the point where longer-running agent tasks, human-in-the-loop escalation, or devices switching mid-conversation push past what those limits can absorb.

AWS's Application Load Balancer is one example of an infrastructure default that makes this worse. It times out idle connections after 60 seconds by default, a setting built for short-lived HTTP requests, not a multi-minute agent task. Many teams don't discover this until a long-running stream drops in production.

A few signals, on their own or together, suggest the effort of running Socket.IO yourself is outweighing the benefit of keeping it in-house:

• Uptime incidents tied directly to connection scale, not application bugs.

• Constant firefighting around sticky session logic, especially after deploys or instance restarts.

• Infrastructure and DevOps costs that keep climbing faster than usage does.

• Engineering time going into transport plumbing (reconnection logic, deduplication, presence) instead of the product itself.

For AI features specifically, watch for a few more:

• Agents losing accumulated task state whenever an instance restarts or redeploys.

• Streaming costs that scale with connection count rather than actual engagement. A stream running to a tab nobody's watching costs the same as one being watched.

• A growing list of edge cases around retries, ordering, and reconnects that never quite gets shorter, no matter how many get fixed.

Whether you're deciding this for an existing Socket.IO deployment or before writing a line of a new build, the alternative is a managed realtime platform. Ably is one such platform: a globally distributed Pub/Sub platform designed for elastic scale, with connection state recovery, guaranteed message ordering, and edge acceleration built in. That's different from assembling the equivalent yourself out of a load balancer, an adapter, and a self-managed Redis instance.

Compared to a self-hosted Socket.IO deployment, a platform like Ably solves the connection ceiling, sticky sessions, and horizontal-scaling coordination directly:

• No sticky sessions: connection state isn't tied to a single instance, so autoscaling and failover work the way they're supposed to.

• Connection state recovery: a client that reconnects within two minutes gets every missed message replayed in order, with exactly-once delivery, without custom catch-up logic. Message history stays available for up to 72 hours beyond that.

• Global scalability and a 99.999% uptime SLA, without managing the cluster yourself.

• SDKs across 20+ languages and platforms, plus a documented migration path from Socket.IO. The two APIs are similar enough that this doesn't require native support for the Socket.IO wire protocol itself. For teams already running Socket.IO, most of the migration work sits at the connection layer, not in application logic. Event handlers and room or channel logic typically carry over with minimal changes.

AI chat and agent features need connections that run long, sessions that survive a device switch, and handoff to a human mid-conversation that preserves context. Ably AI Transport is a purpose-built layer on the same platform for teams building those features. Colin Kennedy, Principal Product Engineer at Fin (formerly Intercom), described the move this way: "Ably gives us the reliable, low-latency AI transport we need for Messenger and Fin. No polling, no dropped messages, just a platform we can finally build next-generation AI experiences on."

See a fuller comparison of Socket.IO and Ably, or look at other alternatives to Socket.IO if a different trade-off fits better. Ably's free plan covers enough concurrent connections to test connection recovery and scaling behavior against real traffic.

Scaling Socket.IO yourself means continuing to own the connection ceiling, sticky sessions, and the coordination work of horizontal scaling. No amount of engineering effort makes those go away. A managed platform removes that burden by handling connection recovery, ordering, and elastic scale directly. That's the choice, whether it's for a new build or for something already in production.

Partly, and only for the same client reconnecting quickly. Socket.IO's connection state recovery feature restores a socket's id, rooms, and any missed packets after a brief, unexpected disconnection. You configure the recovery window yourself. Two minutes is a common choice.

Connection state recovery doesn't cover a session picked up from a different device. It's also not compatible with the standard Redis adapter. Only the in-memory, Redis Streams, or MongoDB adapters support it. Most horizontally scaled Socket.IO deployments use the standard Redis adapter, so most teams running one don't get this recovery feature at all.

Yes. The official Socket.IO server runs on Node.js. It can also run on Bun, via a dedicated engine package, but there's no official server runtime outside the JavaScript ecosystem. The OS-level defaults behind the connection ceiling apply wherever the server process runs, so the numbers hold regardless of which of those runtimes you pick.

For most teams, no. If you're comfortably inside the 10,000-to-30,000 connection ceiling per instance and don't need multi-region redundancy, vertical scaling gets you further with far less operational overhead. The trade-off changes once you're consistently pushing past that ceiling or need failover across regions. The same shift happens once you're running AI features that hold connections open for minutes instead of seconds.

Not natively. Every horizontal-scaling approach in the "Socket.IO scaling strategies and trade-offs" table, including the adapters, still needs session affinity at the load balancer. Socket.IO requires it because it keeps connection state in memory on whichever instance the client is attached to. The only way to avoid that requirement is to move the state into an external store that the application checks on every request. Doing so adds the latency and consistency work described in "What scaling across multiple servers doesn't solve."

No. More instances add capacity, but they don't touch sticky sessions, multi-device continuity, or AI-to-human handoff. A larger cluster hits the same connection-ceiling math sooner with AI traffic. Sticky sessions, human handoff, and the retry and silent-failure gaps still require the same manual work, just at a bigger scale.