DCAS handbook

Webrtc

At the start client only has Gateway Socket connection.

Gateway Websocket

Client sends Update Voice State over Gateway Socket;

Server broadcasts VOICE_STATE_UPDATE event to notify client and other guild users that request has been acknowledged and user is now considered to be in voice channel;

Client shows “Awaiting endpoint” status:

Server generates voice server token for our user.

Server picks voice server instance with lowest load, and notifies it that user with given user id and voice server token will soon connect to it.

Server sends VOICE_SERVER_UPDATE message to user (only to him). This message contains Voice Websocket address and token;

#![allow(unused)]
fn main() {
pub struct VoiceServerUpdate {
    /// Voice Websocket address, in format host:port
    pub endpoint: Option<String>,

    /// Token user should use to identify himself to Voice Websocket
    pub token: String,

    pub guild_id: Option<Snowflake>,
    pub channel_id: Option<Snowflake>,
}
}

Client now shows “Connecting” status:

Voice Websocket

Now user connects to Voice Websocket and sends Identify message:

{
    "op": 0,
    "d": {
        "server_id": "REDACTED",
        "user_id": "REDACTED",
        "session_id": "REDACTED",
        "token": "REDACTED",
        "max_dave_protocol_version": 0,
        "video": true,
        "streams": [ { "type": "video", "rid": "100", "quality": 100 } ]
    }
}

Voice server validates user id - token pair.

Then it picks random (?) SSRC.

Voice server responds with Ready message.

This message contains:

  • ip and port of Voice Data server;
  • ssrc and rtx_ssrc (retransmission) of voice and video streams;
  • encryption modes Voice Data server supports;

The tricky part here is that ip address should be non-local (not localhost nor 127.0.0.1), otherwise client’s browser would refuse to make next steps. Something like 192.168.0.101 is fine.

{
    "op": 2,
    "d": {
        "ssrc": 627362,
        "ip": "REDACTED",
        "port": 50001,
        "modes": [
            "aead_aes256_gcm_rtpsize",
            "aead_aes256_gcm",
            "aead_xchacha20_poly1305_rtpsize",
            "xsalsa20_poly1305_lite_rtpsize",
            "xsalsa20_poly1305_lite",
            "xsalsa20_poly1305_suffix",
            "xsalsa20_poly1305"
        ],
        "streams": [
            {
                "type": "video",
                "ssrc": 627363,
                "rtx_ssrc": 627364,
                "rid": "100",
                "quality": 100,
                "active": false
            }
        ],
        "experiments": [ "fixed_keyframe_interval" ],
    }
}

Client changes its status to “RTC Connecting”:

SDP

Client generates DTLS certificate and random username-password pair, and then sends SelectProtocol message:

{
    "op": 1,
    "d": {
        "protocol": "webrtc",
        "data": "a=fingerprint:sha-256 66:68:CB:88:C2:70:3C:75:FE:33:DE:09:9C:0D:A3:FC:1D:BD:8F:27...",
        "sdp": "a=fingerprint:sha-256 66:68:CB:88:C2:70:3C:75:FE:33:DE:09:9C:0D:A3:FC:1D:BD:8F:27...",
        "codecs": [
            { "name": "opus", "type": "audio", "priority": 1000, "payload_type": 109, "rtx_payload_type": null },
            { "name": "H264", "type": "video", "priority": 1000, "payload_type": 126, "rtx_payload_type": 127  },
            { "name": "VP8",  "type": "video", "priority": 2000, "payload_type": 120, "rtx_payload_type": 124  },
            { "name": "VP9",  "type": "video", "priority": 3000, "payload_type": 121, "rtx_payload_type": 125  }
        ],
        "rtc_connection_id": "7ddd3969-1378-4600-bb86-c509754a1cf3"
    }
}

data and sdp fields are the same for (at least) browser connection. These fields contains a short form of SDP request (fields reordered for readability):

a=ice-options:trickle

a=fingerprint:sha-256 66:68:CB:88:C2:70:3C:75:FE:33:DE:09:9C:0D:A3:FC:1D:BD:8F:27:C8:05:26:DE:28:A1:C7:6B:BD:97:A4:65
a=ice-pwd:8dffaa0ae878690daa979544dfe5b16b
a=ice-ufrag:1de8e173

a=rtpmap:109 opus/48000/2
a=rtpmap:120 VP8/90000
a=rtpmap:124 rtx/90000

a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2/recvonly urn:ietf:params:rtp-hdrext:csrc-audio-level
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:5 urn:ietf:params:rtp-hdrext:toffset
a=extmap:6/recvonly http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:7 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
  • trickle means that this is a short form of SDP;
  • fingerprint is a hash of client’s certificate to be used in DTLS setup
    • client generates DTLS certificate himself;
    • certificate type is pkcs12;
    • the name of fingerprint thing is certificate identity;
  • ice-ufrag/ice-pwd is a login-password pair to be used in STUN request;
  • rtpmap maps media stream types to client-specific payload type ids;
  • extmap is a list of supported extensions;

codecs json field and sdp’s rtpmap contains the same data, just in different formats. Meaning of this data is as follows: “dear server, if you want to send me RTP frames of such media type, please assign such payload type in a RTP header”.

Server generates its own DTLS certificate and username-password pair and responds with SessionDescription message:

{
    "op": 4,
    "d": {
        "audio_codec": "opus",
        "video_codec": "H264",
        "media_session_id": "c53b3ba7aca8f2ab65e9251a51924e4c",
        "sdp": "m=audio 50001 ICE/SDP\na=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A...",
    }
}

The sdp is as follows:

m=audio 50001 ICE/SDP
a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87
c=IN IP4 192.168.0.101
a=rtcp:50001
a=ice-ufrag:Jkcq
a=ice-pwd:tRwkG1TVS/b7F1mkps9ZQB
a=fingerprint:sha-256 4A:79:94:16:44:3F:BD:05:41:5A:C7:20:F3:12:54:70:00:73:5D:33:00:2D:2C:80:9B:39:E1:9F:2D:A7:49:87
a=candidate:1 1 UDP 4261412862 192.168.0.101 50001 typ host
  • 192.168.0.101 is and IP and 50001 is a port of Voice Data server;
  • fingerprint, ice-ufrag and ice-pwd has the same meaning as in SDP request;
  • for candidate syntax, refer to RFC 5245;

Client changes its status to “Checking route” indicating that now it is ready to poke Data server:

STUN

Client sends STUN request to Voice Data server.

The message itself is a regular STUN (RFC 8489) message – a bunch of bytes sent via UDP.

Client request contains Username and MessageIntegrity attributes among the others:

  • Username contains the exact ice-ufrag value client sent via SDP;
  • MessageIntegrity is HMAC-SHA1 seeded with ice-pwd from SDP;

Voice Data server validates message fields and “remembers” client identity - client IP address/port pair.

Voice Data server responds with STUN Success/Binding message – just like a normal STUN server. The message contains the following attributes:

  • XorMappedAddress with client’s address;
  • MessageIntegrity seeded with ice-pwd from server’s SDP;
  • Fingerprint (crc32, as described in RFC 8489).

DTLS

Client performs SRTP/DTLS connection to the Voice Data server. Yup, you have to keep STUN and SRTP server on the same port – thats how NAT works.

During DTLS client and server mutually verifies each other’s certificates generated at SDP step.

At the end of this step client finally shows green status:

Data exchange

Client sends SRTP packets w/ RTP Payload type set to client-specific payload type integer. For normal voice (non-steaming, non-video) communications no matter what payload type is, de-facto it is 48kHz 2-channel opus-encoded audio.

Data server decrypts voice data, and for each listening client (i.e non-sender that is also non-deafen) changes packet’s RTP Payload type to the type client expects to see, re-encrypts voice data and sends packet.

Further reading