Android WebRTC signaling design and handshake

Estimated read time 5 min read

WebRTC supports many-to-many communication. How to complete signaling design and media negotiation network negotiation is the key.

Signaling service design

Why do you need a signaling server?

Because WebRTC requires media negotiation and network negotiation before formal communication. The so-called media negotiation can be understood as the exchange of media information between devices (including codec format, resolution and other information). The network negotiation of WebRTC is commonly known as “hole digging” and requires Know a series of information such as TCP/IP to ensure that the network between the two devices is reachable, so the signaling server is a necessary prerequisite.

We use WebSocket for the signaling service based on the Mesh solution, which is suitable for local area networks and is lightweight and low-cost.

The Mesh solution supports many-to-many. One device can accept communication transmissions from multiple devices (such as video conferencing Jiugongge), and one device can also transmit collected data to multiple devices. That is to say, each device must know the existence of all devices. In other words, each device needs to know the login and logout status of all devices.

Design

Continuing from the above: Assume there are 5 devices A/B/C/D/E. How does each device know the status of all devices?

There is no master-slave concept for communication between WebRTC devices , but there is a signaling server WebSocket, so if A is the server. B/C/D/E are clients. There is no way for B/C/D/E clients to communicate directly. So how do they know the existence of each other? Now they can only forward through the server. The way:

A is the master and B is already logged in. At this time, C logs in. A receives C’s login status and forwards the status to all devices. In this way, A/B/C all know each other’s existence and can perform subsequent media negotiation and other operations.

/**
 * Send a text to all connected endpoints
 *
 * @param text the text to send to the endpoints
 */
public void broadcast(String text) {
  broadcast(text, connections);
}

As it happens, WebSocket Server also provides a broadcast interface to facilitate us to forward notifications to all devices.

WebSocket implementation example

implementation 'org.java-websocket:Java-WebSocket:1.5.6'

clinet




public class SignalingWebSocketClient extends WebSocketClient {

    public static final String TAG = SignalingWebSocketClient.class.getSimpleName();

    public SignalingWebSocketClient(URI serverUri) {
        super(serverUri);
    }

    @Override
    public void onOpen(ServerHandshake handshakedata) {
        //socket 已经连接成功 在此做一些初始化工作
        Log.d(TAG,"onOpen");
    }

    @Override
    public void onMessage(String message) {
        //处理来自服务端的消息
    }

    @Override
    public void onClose(int code, String reason, boolean remote) {
        //连接已经关闭 在此做一些释放工作
        Log.d(TAG,"onClose");
    }

    @Override
    public void onError(Exception ex) {
        //在此做一些错误处理
        Log.d(TAG,"onError");
    }
}

server

import org.java_websocket.WebSocket;
import org.java_websocket.handshake.ClientHandshake;
import org.java_websocket.server.WebSocketServer;

import java.net.InetSocketAddress;


public class SignalingWebSocketServer extends WebSocketServer {

    public static final String TAG = SignalingWebSocketServer.class.getSimpleName();

    private String hostname;

    private Map<String,WebSocket> webSocketList;

    public SignalingWebSocketServer(String hostname, int signalingPort) {
        super(new InetSocketAddress(hostname,signalingPort));
        setReuseAddr(true);
        this.hostname = hostname;
        webSocketList = new HashMap<>();
    }

    @Override
    public void onOpen(WebSocket conn, ClientHandshake handshake) {
        InetSocketAddress local = conn.getLocalSocketAddress();
        InetSocketAddress remote = conn.getRemoteSocketAddress();
        Log.d(TAG, "WebSocket open server:" + local + " client:" + remote);
        webSocketList.put(remote.getAddress().getHostAddress(),conn);
        //服务端存储多个客户端的连接 websocket 实例
    }

    @Override
    public void onClose(WebSocket conn, int code, String reason, boolean remote) {
        Log.d(TAG, "webServer close: " + code + ", reason: " + reason);
        String remoteIp = conn.getRemoteSocketAddress().getAddress().getHostAddress();
        webSocketList.remove(remoteIp);
    }

    @Override
    public void onMessage(WebSocket conn, String message) {

    }

    @Override
    public void onError(WebSocket conn, Exception ex) {
        Log.d(TAG, "webServer onError ex: " + ex.getMessage());
        String remoteIp = conn.getRemoteSocketAddress().getAddress().getHostAddress();
        webSocketList.remove(remoteIp);
    }

    @Override
    public void onStart() {

    }

    /**
     * 向指定客户端发送消息
     * @param clientIp
     * @param msg
     */
    public void sendMessage(String clientIp,String msg){
        if (webSocketList.containsKey(clientIp)) {
            webSocketList.get(clientIp).send(msg);
        }
    }

    /**
     * 像所有在线的客户端广播消息
     * @param msg
     */
    public void broadcastMessage(String msg){
        broadcast(msg);
    }

}

start&connect

if (XXX.isClient) {
    try {
        URI serverUri = new URI("ws://" + apIp + ":" + AppConstants.SIGNALING_PORT);
         client = new SignalingWebSocketClient(serverUri);
        client.connect();
        Log.d(TAG,"client connect ws url : " + serverUri);
    } catch (URISyntaxException e) {
        throw new RuntimeException(e);
    }
} else {
    server = new SignalingWebSocketServer(apIp,AppConstants.SIGNALING_PORT);
    server.start();
    Log.d(TAG,"server start ip + port " + apIp + ":" + AppConstants.SIGNALING_PORT);
}

If the client needs to call the connect method, the server needs to call start. Note that the server needs to be started first. ws:// is the WebSocket protocol.

media consultation

The so-called media negotiation is to exchange the above information through the signaling server built above to complete upper-layer communication.

createOffer & offer

A is the server and B is the client.

B logs in to A and receives B’s login information. A forwards the broadcast to B. At this time, the createOffer action can be performed. If createOffer is successful, B will send the offer to A.

peerConnection.createOffer(object : XxSdpObserver() {
    override fun onCreateSuccess(desc: SessionDescription?) {
        super.onCreateSuccess(desc)
        var sdp = desc.description
        sdp = RTCUtils.preferCodec(sdp, RTCUtils.getSdpVideoCodecName(RTCUtils.VIDEO_CODEC_H264_BASELINE), false)
        val newDesc = SessionDescription(desc.type, sdp)
        peerConnection.setLocalDescription(object : XxSdpObserver() {
            override fun onSetSuccess() {
                super.onSetSuccess()
                val data = TcpDataModel(
                    TcpDataModel.DataModelType.Offer,
                    username,
                    target,
                    newDesc.description
                )
                listener.onTransferEventToSocket(data)
            }
        }, newDesc)
    }
}, mediaConstraint)

createAnswer & answer

A receives the offer signaling sent by B and creates an Answer. If the creation is successful, the Answer is sent to B.

peerConnection.createAnswer(object : XxSdpObserver() {
    override fun onCreateSuccess(desc: SessionDescription?) {
        super.onCreateSuccess(desc)
        var sdp = desc.description
        sdp = RTCUtils.preferCodec(sdp, RTCUtils.getSdpVideoCodecName(RTCUtils.VIDEO_CODEC_H264_BASELINE), false)
        val newDesc = SessionDescription(desc.type, sdp)
        peerConnection.setLocalDescription(object : XxSdpObserver() {
            override fun onSetSuccess() {
                super.onSetSuccess()
                val data = TcpDataModel(
                    TcpDataModel.DataModelType.Answer,
                    username,
                    target,
                    newDesc.description
                )
                listener.onTransferEventToSocket(data)
            }
        }, newDesc)
    }
}, mediaConstraint)

ice

peerConnection.addIceCandidate(iceCandidate)


public boolean addIceCandidate(IceCandidate candidate) {
  return nativeAddIceCandidate(candidate.sdpMid, candidate.sdpMLineIndex, candidate.sdp);
}

After the ice exchange is completed, the media negotiation is completed and WebRTC audio and video communication can be carried out.

You May Also Like

More From Author

+ There are no comments

Add yours