Firehose
Ably’s Firehose can stream your realtime data published within the Ably platform directly to another streaming or queueing service. For example, all messages published by any device on a channel could be immediately streamed to Amazon Kinesis allowing you to process this data in realtime. Firehose is offered as part of Ably Integrations.
Using configurable rules, you can stream various data-sources, including messages, presence events, occupancy, and channel lifecycle events.
Unlike channels, which follow a pub/sub pattern, where each message is delivered to any number of subscribers, Firehose operates such that each message is delivered once to your streaming or queueing server.
As each message is delivered once to your streaming or queueing server, this design is commonly used to process realtime data published by Ably asynchronously. For example, using workers consuming data from your stream or queue, you could persist each message of a live chat to your own database, start publishing updates once a channel becomes active, or trigger an event if a device has submitted a location that indicates that it has reached its destination.
Find out why Ably thinks streams and message queues help solve many of the challenges associated with consuming pub/sub data server-side in the article: Message queues — the right way to process and work with realtime data on your servers.
Note that if you want to consume realtime data from a queue, you should take a look at Ably Queues. They provide a simple and robust way to consume realtime data from your worker servers without having to worry about queueing infrastructure.
Data sources
Ably currently supports the following sources of data that can be streamed by Firehose:
- Messages – messages are streamed as soon as they are published on a channel.
- Presence – when clients enter, update their data, or leave channels, the presence event is streamed.
- Channel lifecycle – get notified when a channel is created (following the first client attaching to this channel) or discarded (when there are no more clients attached to the channel).
- Occupancy – when a channel is created (following the first client attaching to this channel) or discarded (when there are no more clients attached to the channel), the lifecycle event is streamed.
Firehose rules
You can create Firehose rules for the following supported services:
AWS Authentication
For details of authenticating AWS services with Ably, see the AWS authentication documentation.