History

While, by default, messages are only stored for two minutes on Ably’s servers, these messages can be retrieved in the event of a lost connection using the History API. There are several options available to increase this storage time. One option is to enable persistence of all messages, for typically 24 hours. These messages can also be retrieved using the History API. Persistence of the last message is also available, although this is retrieved using the rewind channel option. Finally, it is possible to persist messages to external storage services, using the various integrations that are available.

By default, all messages sent on Ably will be stored for 2 minutes on our servers. This allows for clients who disconnect for less than 2 minutes to recover any messages they might have missed, through our History API. The recovering client will receive these messages in the original order they were sent, and this is applicable to both regular channel messages and presence messages.

The following diagram illustrates the default persistence of messages:

If your use case requires longer retention of messages, that is, longer than the default two minutes, you can enable “persisted history” by setting a channel rule. If persisted history is enabled for a channel, its messages will typically be stored for 24 – 72 hours on disk.

In this scenario, the History API can be used to retrieve the persisted messages. This is illustrated in the following diagram:

Every message that is persisted to, or retrieved from, disk counts as an extra message towards your monthly quota. For example, for a channel that has persistence enabled, if a message is published, two messages will be deducted from your monthly quota. If the message is later retrieved from history, another message will be deducted from your monthly quota.

To enable history on a channel, it is necessary to add a persist all messages channel rule in the settings of your Ably dashboard.

It is also possible to persist last message on a channel, which preserves the last message published on the channel for one year.

This last message is retrievable using the channel rewind mechanism by attaching to the channel with rewind=1. Only regular channel messages are stored, not presence messages.

The persisted last message is not accessible using the normal History API, only through rewind.

This option for persistence is illustrated in the following diagram:

In the previous diagram, the channel rewind facility could be used to retrieve the last message, message 4 in this case.

To enable persist last message on a channel, it is necessary to add a persist last message channel rule in the settings of your Ably dashboard.

The order in which historical messages are returned by the History API is based on the message timestamp that was assigned by the channel in the region that the message was published in. This ordering is what Ably calls the canonical global order.

It is important to note that this is not necessarily the order that messages were received by a realtime client. The order in which each realtime client receives a message depends on which region the client is in.

Ably preserves ordering for a specific publisher on a specific channel but, for example, if two publishers in regions A and B publish “message 1″ and “message 2″ simultaneously, then it is very possible that a subscriber in region A will receive “message 1″ before “message 2″, but that a subscriber in region B will receive “message 2″ before “message 1″.

There are instances where messages will not be in canonical global order:

• Recent messages (less than two minutes old) are retrieved from live ephemeral storage and are still ordered by region. They only appear in the canonical global order if you enable message persistence, which also prevents duplication and missing messages.
• You choose to retrieve historical messages only up to the point at which a client attaches to a channel. You would typically use this approach to bring a subscriber up to date as part of connection state recovery.

• History API – Realtime SDK
• History API – REST SDK
• Channel rules
• Channel rewind
• Tutorial on retrieving message history
• Tutorial on persisting messages to AWS S3