• Blog:

  • Home
  • Ably News
  • Ably Engineering
  • Developers
  • Industry Trends
  • Hardest Aspects of Realtime Engineering
  • Implementing a simple WebRTC signaling mechanism with FSharp, Fable, and Ably

    By: Michał Niegrzybowski 10 min read

    For some time now, I’ve been trying to combine Fable and WebRTC in my daily work. In this article I am going to describe the process of implementing a signaling mechanism for WebRTC.

    What is WebRTC?

    But first, what is WebRTC? WebRTC is a browser technology that allows exchanging data among N peers via either the UDP or TCP protocol. In this exchange peers talk to each other without a server in the middle (except when Relay connection is used). WebRTC is a native API available in all modern browsers. One of the things that are not ready by default is the signaling process. That's what this article is about.

    WebRTC signaling

    Before we go into details of the implementation, let’s get some basic concepts around signaling straight. To establish a connection between two peers they need to exchange messages with SDP (Session Description Protocol) information. The number of those messages depends on:

    • the number of STUN/TURN server
    • type of connection (P2P/Relay)

    SDP contains a lot of information needed to start a connection. Messages to be exchanged are called Offer, Answer, and Candidate. We will use a Trickle-Ice mechanism. It means that each Offer, Answer, and Candidate will be sent separately in an asynchronous (async) manner. A Candidate has information about how to connect to a peer which creates them (more about them here). There is also a simpler way of establishing a connection where Answer and Offer would contain every needed Candidate. But I want to focus on an async version — we are using it in our product because of better performance.

    Implementation

    Switching messages as described above are named "signaling". When I was looking for a out-of-the-box solution for signaling I came across Ably. I decided to give it a try for my base application, in which I want to send a file from one peer to another.

    When I was reading the Ably documentation, I realized that the only functions that I will need would be publish and subscribe. These are used in a message channel context, so I created an Ably interface in F# which is a port of JS interfaces from their official lib. To get started, you will need an Ably account.

    module AblyMapping =
      type IMessage =
        abstract name: string
        abstract data: string
    
      type IChannel =
        abstract publish: string * string -> unit
        abstract subscribe: string * (IMessage -> unit) -> unit
    
      type IChannels =
        abstract ``get``: string -> IChannel
    
      type IRealTime =
        abstract channels: IChannels
    
      type IAbly =
        abstract Realtime: (string) -> IRealTime
    
      let Ably: IAbly = importAll "Ably"
    

     

    Changes in packages.json.

    {
        …
        "ably": "^1.2.4"
    }
    

    (The code above is self-explanatory when we look at the JS example from the Ably site.)

    var ably = new Ably.Realtime('apikey');
    var channel = ably.channels.get('gut-tub');
    
    // Publish a message to the gut-tub channel
    channel.publish('greeting', 'hello');
    
    var ably = new Ably.Realtime('apikey');
    var channel = ably.channels.get('gut-tub');
    
    // Subscribe to messages on channel
    channel.subscribe('greeting', function(message) {
      alert(message.data);
    });
    

     

    Initialization of Ably can be found in the init method.

    let init (): Model * Cmd<Msg> =
      let channel =
        AblyMapping.Ably.Realtime “api-key”
        |> fun x -> x.channels.get “channel name"
      let model =
        {
            Role = None
            Channel = channel
            ConnectionState = WebRTC.ConnectionState.NotConnected
            WaitingCandidates = []
        }
      model, Cmd.none
    

     

    At this point, we could start sending and receiving messages via Ably, but I want to start by first installing the WebRTC-adapter library. This library gives me confidence that what I want to achieve with the native API would work the same way in every browser. Since there are some slight differences between implementations, I add the following line to the packages.json with the webrtc-adapter library.

    {
      …
      "webrtc-adapter": "^7.7.0"
    }
    

     

    Usage in F# code.

    module WebRTC =
      let private adapter: obj = importAll "webrtc-adapter"
    

     

    The above ensures the interfaces are compatible across all supported browsers. Now we can switch to the code responsible for handling WebRTC. The code snippet is big because we need to make a distinction between the sender and the receiver of a file. This is because we don’t have an application on the server-side. I started with the creation of a PeerConnection object with a communication server address.

    let create () =
      let conf =
        [| RTCIceServer.Create( [| "turn:location:443" |], "user", "pass", RTCIceCredentialType.Password ) |]
        |> RTCConfiguration.Create
        |> fun x ->
          x.iceTransportPolicy <- Some RTCIceTransportPolicy.Relay
          x
    
        let pc = RTCPeerConnection.Create(conf)
    
        { Peer = pc; Channel = None }
    

     

    I test everything locally, so I force using a Relay connection. Otherwise, communication would be done via local network. That would result in skipping the signaling phase (host candidates would be used).

    Note: Using public STUN/TURN servers is not recommended. We are not sure what the configuration looks like or who owns them.

    We create a LifecycleModel object which contains a PeerConnection and RTCDataChannel inside, and then create an instance of it as a mutable field instead of part of the Elmish model. This is the simplest approach: the default Thoth serializer couldn’t handle serializing PeerConnection; also passing the whole big object wouldn’t be something that we want to do.

    type LifecycleModel = {
      Peer: RTCPeerConnection
      Channel: RTCDataChannel option
    }
    
    ...
    
    let mutable peer: LifecycleModel = Unchecked.defaultof<_>
    
    ...
    
    peer <- WebRTC.create ()
    

     

    After the creation of the PeerConnection object, we see that in the UI we have a possibility to connect as sender or receiver.

    WebRTC adapter UI showing how to initialize as a sender or a receiver

    This differentiates the different ways that WebRTC works, and shows a different value for a Role field in a model. When we look at how the sender logic is implemented we start with the creation of RTCDataChannel, which would then be used as a transfer channel between users. The creation of a channel is located in the createChannel function.

    let createChannel pc name =
      let channel = pc.Peer.createDataChannel name
      { pc with Channel = Some channel}
    

     

    Moving on to the configuration of messages sent and received via Ably, when we are a sender we want to listen to Answer and Candidate(from the receiver) messages. You can see how to achieve this in the following code snippet.

    let subs =
      [
          "answer", fun (msg: AblyMapping.IMessage) -> dispatch (Msg.WebRTC (WebRTC.Msg.Answer msg.data))
          "receiver-candidate", fun (msg: AblyMapping.IMessage) -> dispatch (Msg.WebRTC (WebRTC.Msg.Candidate msg.data))
      ] |> Map.ofList
    

     

    The combination with the Ably channel is visible in the init method from the Signaling module.

    let init (channel: IChannel) (subscribers: Map<string, IMessage -> unit>) =
      subscribers
      |> Map.iter (fun subscriberKey subscriberFunc ->
          channel.subscribe (subscriberKey, subscriberFunc)
      )
      channel
    

     

    We thus subscribe to all messages with the given keys answer and receiver-candidate. When they occur we propagate Elmish messages that would be handled in the update method.

    In the receiver scenario the difference is that we don’t create RTCDataChannel (this is why it is marked as option in LifecycleModel). We gather it when the connection would be in a connecting phase. If it were created on its own we would receive an invalid state error. This is why we only subscribe to messages Offer and Candidate (from Sender).

    When a subscription to Ably messages is ready we send an Elmish message Signaling with a ready channel which is updated in the application model. Meanwhile the WebRTC configuration is updated with callbacks to functions that need to be handled in a connection/data transfer process.

    Assignment of those callbacks and how they are handled is visible in a subscribe function. This should do the following.

    1. Initialize RTCDataChannel through which data (file) would be transferred.
    let private initReceiverChannel (pc: RTCPeerConnection) msgHandler dispatch =
      let mutable updatedChannel: RTCDataChannel = Unchecked.defaultof<_>
      let callback (ev: RTCDataChannelEvent) =
        let receiveChannel = subscribeChannel ev.channel msgHandler dispatch
        internalLog (sprintf "updating channel: %A" ev.channel.id)
        peer <- { peer with Channel = Some receiveChannel }
    
      pc.ondatachannel <- callback
      updatedChannel
    
    let updatedChannel =
      if role = Role.Sender then
        match channel with
        | Some c ->
          internalLog (sprintf "initialize channel for: %A" role)
          subscribeChannel c msgHandler dispatch
        | None -> failwith "Channel is not initilized for sender"
      else
          internalLog (sprintf "initialize channel2 for: %A" role)
          initReceiverChannel pc msgHandler dispatch
    

     

    1. Handle connection state (only for diagnostic purposes).
    pc.oniceconnectionstatechange <-
      fun _ ->
        internalLog (sprintf "Connection state changed to: %A" pc.iceConnectionState)
    

     

    1. Handle exchange of candidates.
    pc.onicecandidate <-
      fun e ->
        match e.candidate with
        | None -> internalLog "Trickle ICE Completed"
        | Some cand ->
          cand.toJSON()
          |> Base64.``to``
          |> Signaling.WebRTCCandidate.Candidate
          |>
            if role = Role.Sender then
                Signaling.Notification.SenderCandidate
            else Signaling.Notification.ReceiverCandidate
          |> onSignal
    

     

    Now the configuration of WebRTC is ready on the sender and receiver side.

    Configuration of WebRTC is ready to connect

    We can initiate a connection by clicking the Connect button, after which the init method from the WebRTC module is called.

    let init connection onSignal =
      let pc, channel = connection.Peer, connection.Channel
      pc.createOffer().``then``(fun desc ->
        pc.setLocalDescription (desc) |> Promise.start
        if isNull desc.sdp then
          internalLog "Local description is empty"
        else
          desc
          |> Base64.``to``
          |> Signaling.WebRTCOffer.Offer
          |> Signaling.Notification.Offer
          |> onSignal)
    
      |> Promise.catch (sprintf "On negotation needed return error: %A" >> internalLog)
      |> Promise.start
    
      { Peer = pc
        Channel = channel }
    

     

    We send Offer via the Ably channel and set it as LocalDescription via setLocalDescription method on the PeerConnection object. Right now the application flow is not discernible from the code at first glance. The other side of communication should receive Offer via the Ably channel which would then be propagated as an Offer Elmish message and handled in the update method.

    let setOffer (lifecycle: LifecycleModel) onSignal remote =
      try
        let desc = rtcSessionDescription remote
        internalLog (sprintf "setting offer: %A" desc)
        lifecycle.Peer.setRemoteDescription desc
        |> Promise.catch (sprintf "Failed to set remote description: %A" >> internalLog)
        |> Promise.start
        lifecycle.Peer.createAnswer().``then``(fun desc ->
          lifecycle.Peer.setLocalDescription (desc) |> Promise.start
          if isNull desc.sdp then
            internalLog "Local description is empty"
          else
            internalLog (sprintf "sending answer: %A" desc)
            desc
            |> Base64.``to``
            |> Signaling.WebRTCAnswer.Answer
            |> Signaling.Notification.Answer
            |> onSignal)
    
        |> Promise.catch (sprintf "On negotation needed errored with: %A" >> internalLog)
        |> Promise.start
      with e ->
        internalLog (sprintf "Error occured while adding remote description: %A" e)
    

     

    This should set Offer from a sender as a RemoteDescription on a receiver side and, in case of success, it should generate Answer. It then gets sent to the sender via Ably. The important thing to mention here is that Candidates are generated after the creation of Offer/Answer. They shouldn’t be set on the PeerConnection object before setting Local and Remote descriptions. Because of this, the Elmish model has a buffer for Candidates that could be received before setting Remote/Local descriptions.

    Buffer handling:

    if model.WaitingCandidates.Length > 0 then
      model.WaitingCandidates
      |> List.iter (WebRTC.setCandidate peer )
      { model with
                  ConnectionState = WebRTC.ConnectionState.Connecting
                  WaitingCandidates = [] }, Cmd.none
    else
      { model with ConnectionState = WebRTC.ConnectionState.Connecting }, Cmd.none
    

     

    Handling a message that contains Candidates looks like this.

    if model.ConnectionState <> WebRTC.ConnectionState.NotConnected then
      WebRTC.setCandidate peer candidate
      model, Cmd.none
    else
      { model with WaitingCandidates = candidate::model.WaitingCandidates }, Cmd.none
    

     

    ConnectionState is set when Offer or Answer are received or when the DataChannel is open.

    Handling Answer.

    | WebRTC (WebRTC.Msg.Answer answer) ->
      WebRTC.setAnswer peer answer
      { model with ConnectionState = WebRTC.ConnectionState.Connecting }, Cmd.none
    

     

    On to the setAnswer implementation.

    let setAnswer (lifecycle: LifecycleModel) remote =
      try
        let desc = rtcSessionDescription remote
        internalLog (sprintf "setting answer: %A" desc)
        lifecycle.Peer.setRemoteDescription desc
        |> Promise.catch (sprintf "Failed to set remote description: %A" >> internalLog)
        |> Promise.start
      with e ->
        internalLog (sprintf "Error occured while adding remote description: %A" e)
    

     

    Here we only set RemoteDescription on a PeerConnection object.

    Right now, if everything works, we should have a WebRTC connection ready, with DataChannel open between our peers. We could move on to the code which is responsible for sending files.

    In UI, there should be a textBox that reacts on a file drop.

    UI that reacts on a file drop

    This is handled as follows.

    Field.div [
      Field.IsGrouped ] [
      Control.p [ Control.IsExpanded ] [
        div [
          Class "card border-primary"
          Draggable true
          Style [ Height "100px" ]
          OnDragOver ( fun e ->
            e.preventDefault()
          )
          OnDrop ( fun e ->
            e.preventDefault()
            if e.dataTransfer.files.length > 0 then
              let file = e.dataTransfer.files.[0]
              { Name = file.name; Data = file.slice () }
              |> SendFile
              |> dispatch
          )
        ] []
      ]
    ]
    

     

    SendFile message handling.

    | SendFile file ->
      MessageTransfer.send peer (ChannelMessage.File file.Data)
      model, Cmd.none
    

     

    Method send in MessageTransfer.

    let send (lifecycle: LifecycleModel) (msg: ChannelMessage) =
      match msg, lifecycle.Channel with
      | File blob, Some channel ->
        blob
        |> U4.Case2
        |> channel.send
      | _, _ -> log "MessageTransfer" (sprintf "Unable to process: %A channel is: %A" msg lifecycle.Channel)
    

     

    On the receiver side onmessage handling on DataChannel object looks as follows.

    channel.onmessage <-
      fun e ->
        internalLog (sprintf "received msg in channel: %A" e.data)
        let blob = e.data :?> Blob
        msgHandler blob
    

     

    For simplicity, we assume that only a JavaScript blob would be sent via DataChannel. We also pass to there a msgHandler which is implemented in the following way.

    let download (data: Blob) =
      let url = Browser.Url.URL.createObjectURL data
      let mutable anchor = document.createElement "a"
      anchor.hidden <- true
      anchor.setAttribute ("href", url)
      anchor.setAttribute ("download", "image.png")
      document.body.appendChild anchor |> ignore
      anchor.click ()
      document.body.removeChild anchor |> ignore
    

     

    Its responsibility is to receive a FileBlob and immediately download it. We assume that the file would be always a png with the name image.

    Conclusion

    To sum up, thanks to Ably I was able to implement signaling in a simple way. In my daily work I am always seeking to achieve signalling in the most performant way, and when making a decision, I now have to seriously consider how Ably does it. During some initial tests to compare it with standard HTTP request/response mechanisms, it looks promising.

    Another interesting thing is that a user can see how the messages/connections work and flow right on the Ably dashboard, which also includes information about used quota. That's a nice bonus.

    Ably dashboard with usage statistics

    Ably dashboard with quota and usage statistics

    Because there are no built-in signaling solutions, Ably is for sure a good out-of-the-box option to consider for scenarios such as what I outlined above. In my next articles, I hope to compare it with other ways to do signaling.

    I hope you enjoy it. Thanks for reading!

    Click here for the code repository.

    WebRTC resources


    Guest blog by Michał Niegrzybowski and originally published at his blog. Michał currently works for Datto as a senior software engineer, he is a Software Craftsman, big fan of functional programming, TDD, DDD, and new fancy libraries/frameworks. You can connect with him on Twitter or LinkedIn.