前言
本文主要以“代码是最好的注释”为基点，介绍在处理iOS端多人音视频的建立流程。 在看本篇前建议先了解一下多人音视频通讯现在的常用架构，参考《WebRTC多人音视频聊天架构与实战》。 本方案使用的是当中提到的第一种架构：Mesh 架构
六大事件
- join : 加入房间
- offer : 本端
群发offer及接收对端offer的处理 - answer : 本端发送answer及接收对端发来answer的处理
- ice_candidate ： 对端的网络地址通过 socket 转发给本端
- new_peer: 成员进入
- remove_peer: 成员离开
join
- 首先，多人音视频的发起者，通过信令发送’
__join’ 加入房间消息。 - 信令服务器收到消息后，会根据用户所加入的房间, 向房间里的其他用户发送’
_peers’ 消息 ,信令中包含聊天室中其他用户的信息,客户端根据信息来逐个构建与其他用户的点对点连接
offer
群发offer
上面说到，信令服务器在接收到’__join’ 消息后， 转而发送包含房间内其他用户信息的 ‘__peer’ 消息。 那么其他端在收到’__peer’ 消息后该如何处理呢 ？
没错，在收到’_peer'消息后， 本端需要向房间里的其他用户发送offer 。 发送offer的过程如下：
- 记录所有对端连接的ID
- (void)_addToConnectionIDS:(NSArray <NSString *>*)connections
{
if (connections.count == 0) return;
[connections enumerateObjectsUsingBlock:^(id _Nonnull obj, NSUInteger idx, BOOL * _Nonnull stop) {
NSLog(@"%@", obj);
}];
[_peerConnectionIDS addObjectsFromArray:connections];
}
其中数组
connections为__peer信令返回的房间内其他用户的连接ID, 本端维护一个对端各连接点ID的数组_peerConnectionIDS
- 建立本地流信息
- (void)_setupForLocalStream
{
if (!_localStream) {
// 设置点对点工厂
[self _setupForFactory];
//本地流 通过工厂来创建
_localStream = [_factory mediaStreamWithStreamId:@"ARDAMS"];
//音频轨对象 通过工厂 来创建
RTCAudioTrack *audioTrack = [_factory audioTrackWithTrackId:@"ARDAMSa0"];
// 将音频轨迹添加到本地流媒体
[_localStream addAudioTrack:audioTrack];
AVAuthorizationStatus authStatus = [AVCaptureDevice authorizationStatusForMediaType:AVMediaTypeVideo];
NSArray<AVCaptureDevice *> *devices;
if(authStatus == AVAuthorizationStatusRestricted || authStatus == AVAuthorizationStatusDenied) { // 摄像头权限
if ([_delegate respondsToSelector:@selector(webRTCHelper:setLocalStream:)]) {
[_delegate webRTCHelper:self setLocalStream:nil];
}
}
else {
#if __IPHONE_OS_VERSION_MIN_REQUIRED < 100000
if ([AVCaptureDeviceDiscoverySession class]) {
AVCaptureDeviceDiscoverySession *deviceDiscoverySession = [AVCaptureDeviceDiscoverySession discoverySessionWithDeviceTypes:@[AVCaptureDeviceTypeBuiltInWideAngleCamera] mediaType:AVMediaTypeVideo position:AVCaptureDevicePositionFront];
devices = [deviceDiscoverySession devices];
}
else {
devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo];
}
#else
AVCaptureDeviceDiscoverySession *deviceDiscoverySession = [AVCaptureDeviceDiscoverySession discoverySessionWithDeviceTypes:@[AVCaptureDeviceTypeBuiltInWideAngleCamera] mediaType:AVMediaTypeVideo position:AVCaptureDevicePositionFront];
devices = [deviceDiscoverySession devices];
#endif
AVCaptureDevice *device = [devices lastObject];
if (device) {
RTCAVFoundationVideoSource *videoSource = [_factory avFoundationVideoSourceWithConstraints:[self _setupForLocalVideoConstraints]];
[videoSource setUseBackCamera:NO];
//视频轨对象 也是通过工厂来创建
RTCVideoTrack *videoTrack = [_factory videoTrackWithSource:videoSource trackId:@"ARDAMSv0"];
// 添加视频轨迹
[_localStream addVideoTrack:videoTrack];
if ([_delegate respondsToSelector:@selector(webRTCHelper:setLocalStream:)]) {
[_delegate webRTCHelper:self setLocalStream:_localStream];
}
}
else {
if ([_delegate respondsToSelector:@selector(webRTCHelper:setLocalStream:)]) {
[_delegate webRTCHelper:self setLocalStream:nil];
}
}
}
}
}
本地流的创建是通过点对点工厂
RTCPeerConnectionFactory来创建。 本地流创建出来后，需要将本端的音频轨及视频轨添加到流媒体中。本地流创建成功后，可以通过回调方法，回调给使用者用来显示本端视频
- 建立所有连接
- (void)_setupForConnections
{
[_peerConnectionIDS enumerateObjectsUsingBlock:^(NSString * _Nonnull obj, NSUInteger idx, BOOL * _Nonnull stop) {
RTCPeerConnection *connection = [self _createConnection:obj];
[_peerConnections setObject:connection forKey:obj];
}];
}
- (RTCPeerConnection *)_createConnection:(NSString *)connectionID
{
[self _setupForFactory];
[self _setupForIceServers];
RTCConfiguration *configuration = [[RTCConfiguration alloc] init];
[configuration setIceServers:_iceServers];
return [_factory peerConnectionWithConfiguration:configuration constraints:[self _setupForPeerVideoConstraints] delegate:self];
}
如果要与对端所有人进行通讯，必须要先和对端所有人建立连接。 上面维护了一个对端各连接点ID的数组
_peerConnectionIDS， 通过遍历这个数组创建对端各点对点的连接。 接着，本地再维护一个字典_peerConnections， key为连接点ID, value为点对点连接的对象RTCPeerConnection
- 所有连接添加本地流信息
- (void)_addLocalStreamToConnections
{
[_peerConnections enumerateKeysAndObjectsUsingBlock:^(NSString * _Nonnull key, RTCPeerConnection * _Nonnull obj, BOOL * _Nonnull stop) {
[self _setupForLocalStream];
[obj addStream:_localStream];
}];
}
遍历本地维护的所有对端点对点连接的字典
_peerConnections， 让对端所有连接都添加本地流信息。
- 所有连接发送 SDP offer
- (void)_sendSDPOffersToConnections
{
[_peerConnections enumerateKeysAndObjectsUsingBlock:^(NSString * _Nonnull key, RTCPeerConnection * _Nonnull obj, BOOL * _Nonnull stop) {
[self _sendSDPOfferToConnection:obj];
}];
}
- (void)_sendSDPOfferToConnection:(RTCPeerConnection *)connection
{
_role = RoleCaller;
/** Generate an SDP offer. */
[connection offerForConstraints:[self _setupForOfferOrAnswerConstraint] completionHandler:^(RTCSessionDescription * _Nullable sdp, NSError * _Nullable error) {
if (error) {
NSLog(@"offerForConstraints error");
return;
}
if (sdp.type == RTCSdpTypeOffer) {
__weak __typeof(connection) wConnection = connection;
/** Apply the supplied RTCSessionDescription as the local description. */
// A:设置连接本端 SDP
[connection setLocalDescription:sdp completionHandler:^(NSError * _Nullable error) {
if (error) {
NSLog(@"setLocalDescription error");
return;
}
[self _didSetSessionDescription:wConnection];
}];
}
}];
}
此处是关键点，上面所有的操作都在为此做准备。发送offer的过程如下
- 通过 RTCPeerConnection 对象的
offerForConstraints:生成sdp offer- 将拿到的sdp 通过RTCPeerConnection 对象的
setLocalDescription:设置为本地sdp- 本地sdp设置成功后需要通过信令将本端sdp发送到对端
- (void)_didSetSessionDescription:(RTCPeerConnection *)connection
{
NSLog(@"signalingState:%zd role:%zd", connection.signalingState, _role);
NSString *connectionID = [self _findConnectionID:connection];
if (connection.signalingState == RTCSignalingStateHaveRemoteOffer) { // 新人进入房间就调(远端发起 offer)
/** Generate an SDP answer. */
[connection answerForConstraints:[self _setupForOfferOrAnswerConstraint] completionHandler:^(RTCSessionDescription * _Nullable sdp, NSError * _Nullable error) {
if (error) {
NSLog(@"answerForConstraints error");
return;
}
if (sdp.type == RTCSdpTypeAnswer) {
__weak __typeof(connection) wConnection = connection;
// B:设置连接本端 SDP
[connection setLocalDescription:sdp completionHandler:^(NSError * _Nullable error) {
if (error) {
NSLog(@"setLocalDescription error");
return;
}
[self _didSetSessionDescription:wConnection];
}];
}
}];
}
else if (connection.signalingState == RTCSignalingStateHaveLocalOffer) { // 本地发送 offer
if (_role == RoleCaller) {
NSDictionary *dic = @{@"eventName": @"__offer",
@"data": @{@"sdp": @{@"type": @"offer",
@"sdp": connection.localDescription.sdp
},
@"socketId": connectionID
}
};
RTMMessage *message = [RTMMessage createChannelMessageWithChannelId:self.channelId data:[dic mj_JSONString]];
[_channel sendMessage:message completion:^(RTMSendChannelMessageStatusCode code) { }];
}
}
else if (connection.signalingState == RTCSignalingStateStable) { // 本地发送 answer
if (_role == RoleCalled) {
NSDictionary *dic = @{@"eventName": @"__answer",
@"data": @{@"sdp": @{@"type": @"answer",
@"sdp": connection.localDescription.sdp
},
@"socketId": connectionID
}
};
RTMMessage *message = [RTMMessage createChannelMessageWithChannelId:self.channelId data:[dic mj_JSONString]];
[_channel sendMessage:message completion:^(RTMSendChannelMessageStatusCode code) {}];
}
}
}
这个方法内汇总了各情况下处理，此处我们只需要看这里：
else if (connection.signalingState == RTCSignalingStateHaveLocalOffer) { // 本地发送 offer
if (_role == RoleCaller) {
NSDictionary *dic = @{@"eventName": @"__offer",
@"data": @{@"sdp": @{@"type": @"offer",
@"sdp": connection.localDescription.sdp
},
@"socketId": connectionID
}
};
RTMMessage *message = [RTMMessage createChannelMessageWithChannelId:self.channelId data:[dic mj_JSONString]];
[_channel sendMessage:message completion:^(RTMSendChannelMessageStatusCode code) { }];
}
}
通过
signalingState属性可以拿到当前RTCPeerConnection对象的信令状态，当状态值为RTCSignalingStateHaveLocalOffer时表示本地发送offer。 在构造好约定的传输格式@{@"type": @"offer",@"sdp": connection.localDescription.sdp }及信令eventName '__offer'后，通过信令服务器发出。
接收offer
/**
远端发来 offer
*/
- (void)_receiveOffer:(NSMutableDictionary *)resultDic
{
// 设置当前角色状态为被呼叫,(被发offer）
_role = RoleCalled;
NSDictionary *dataDic = resultDic[@"data"];
NSDictionary *sdpDic = dataDic[@"sdp"];
// 拿到SDP
NSString *sdp = sdpDic[@"sdp"];
NSString *type = sdpDic[@"type"];
NSString *connectionID = dataDic[@"socketId"];
RTCSdpType sdpType = [self _typeForString:type];
// 拿到这个点对点的连接
RTCPeerConnection *connection = [_peerConnections objectForKey:connectionID];
// 根据类型和SDP 生成SDP描述对象
RTCSessionDescription *remoteSdp = [[RTCSessionDescription alloc] initWithType:sdpType sdp:sdp];
if (sdpType == RTCSdpTypeOffer) {
// 设置给这个点对点连接
__weak __typeof(connection) wConnection = connection;
// B:设置连接对端 SDP
[connection setRemoteDescription:remoteSdp completionHandler:^(NSError * _Nullable error) {
if (error) {
NSLog(@"setRemoteDescription error");
}
[self _didSetSessionDescription:wConnection];
}];
}
}
在接收到对端发送的’
__offer'消息后， 本端需要做如下处理：
- 解析接收到的消息结构，解析出当前的sdp类型及sdp信息，并构造
RTCSessionDescription对象- 根据解析的connectId 获取到点对点连接
RTCPeerConnection对象- 将接收到的对端sdp通过点对点连接对象的
setRemoteDescription:设置为远端sdp- 设置成功后，本端通过信令服务器发送’
__answer'消息来响应
answer
收到远端offer后主动发送answer
接上面：在收到对端’__offer‘消息后，先设置remote sdp, 接着主动发送’__answer'消息响应
_didSetSessionDescription:方法中的处理：
else if (connection.signalingState == RTCSignalingStateStable) { // 本地发送 answer
if (_role == RoleCalled) {
NSDictionary *dic = @{@"eventName": @"__answer",
@"data": @{@"sdp": @{@"type": @"answer",
@"sdp": connection.localDescription.sdp
},
@"socketId": connectionID
}
};
RTMMessage *message = [RTMMessage createChannelMessageWithChannelId:self.channelId data:[dic mj_JSONString]];
[_channel sendMessage:message completion:^(RTMSendChannelMessageStatusCode code) {}];
}
}
构造好约定的传输格式
@{@"type": @"answer",@"sdp": connection.localDescription.sdp }及信令eventName '__answer'后，通过信令服务器发出。
接收远端answer
/**
远端发来 answer
*/
- (void)_receiveAnswer:(NSMutableDictionary *)resultDic
{
NSDictionary *dataDic = resultDic[@"data"];
NSDictionary *sdpDic = dataDic[@"sdp"];
NSString *sdp = sdpDic[@"sdp"];
NSString *type = sdpDic[@"type"];
NSString *connectionID = dataDic[@"socketId"];
RTCSdpType sdpType = [self _typeForString:type];
RTCPeerConnection *connection = [_peerConnections objectForKey:connectionID];
RTCSessionDescription *remoteSdp = [[RTCSessionDescription alloc] initWithType:sdpType sdp:sdp];
if (sdpType == RTCSdpTypeAnswer) {
__weak __typeof(connection) wConnection = connection;
/** Apply the supplied RTCSessionDescription as the remote description. */
// A:设置连接对端 SDP
[connection setRemoteDescription:remoteSdp completionHandler:^(NSError * _Nullable error) {
if (error) {
NSLog(@"setRemoteDescription error");
}
[self _didSetSessionDescription:wConnection];
}];
}
}
接收到对端的’__answer’消息后，解析出消息的结构。 将对端的sdp信息设置为本端的
remote sdp。 这时的代码处理如下：
if (connection.signalingState == RTCSignalingStateHaveRemoteOffer) { // 新人进入房间就调(远端发起 offer)
/** Generate an SDP answer. */
[connection answerForConstraints:[self _setupForOfferOrAnswerConstraint] completionHandler:^(RTCSessionDescription * _Nullable sdp, NSError * _Nullable error) {
if (error) {
NSLog(@"answerForConstraints error");
return;
}
if (sdp.type == RTCSdpTypeAnswer) {
__weak __typeof(connection) wConnection = connection;
// B:设置连接本端 SDP
[connection setLocalDescription:sdp completionHandler:^(NSError * _Nullable error) {
if (error) {
NSLog(@"setLocalDescription error");
return;
}
[self _didSetSessionDescription:wConnection];
}];
}
}];
}
通过RTCPeerConnection对象的
answerForConstraints:生成响应端（本端）的sdp. 并通过setLocalDescription:设置为本端的sdp
未完待续