ios - WebRTC + IOS + Freeswitch : Can't hear audio

Question

我正在尝试实现 mod_verto在 IOS 上(从 iPhone 调用到桌面)。我正在使用 Google's libjingle library对于 RTC 端，使用 this excellent tutorial 启动并运行它.

• 从我的 iPhone 调用电话时，我在桌面浏览器上使用 Verto Communicator 接听电话(已下载并在我的本地计算机上运行)。
• 在 iPhone 端，我可以听到桌面上的音频，但在桌面端听不到任何声音。

• 如果我使用 2 个浏览器窗口(使用 Verto Communicator)进行调用，一切正常。

• 完全披露，我正在使用 ws:// 不安全的 websocket 连接FreeSwitch

这是我的 JSONRPC 日志:

---

发送登录请求:

{"jsonrpc":"2.0","method":"login","id":1,"params":{"login":"1000@MY-IP-ADDRESS","loginParams":{},"userVariables":{},"passwd":"1234","sessid":"53FB0781-B586-4CDA-98C6-558680663B46"}}

登录响应:

{"jsonrpc":"2.0","id":1,"result":{"message":"logged in","sessid":"53FB0781-B586-4CDA-98C6-558680663B46"}}

verto.invite(包括iPhone sdp):

{"jsonrpc":"2.0","method":"verto.invite","id":2,"params":{"dialogParams":{"remote_caller_id_number":"1008","useVideo":false,"useMic":"any","useStereo":false,"tag":"webcam","login":"1000@159.203.164.7","useCamera":"any","videoParams":{"minFrameRate":30,"minWidth":"1280","minHeight":"720"},"destination_number":"1008","screenShare":false,"caller_id_name":"FreeSWITCH User","caller_id_number":"1000","callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800","remote_caller_id_name":"Outbound Call","useSpeak":"any"},"sessid":"53FB0781-B586-4CDA-98C6-558680663B46","sdp":"v=0\r\no=- 8564086442942257834 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE audio video\r\na=msid-semantic: WMS\r\nm=audio 58157 UDP\/TLS\/RTP\/SAVPF 111 103 104 9 102 0 8 106 105 13 127 126\r\nc=IN IP4 82.166.93.197\r\na=rtcp:52576 IN IP4 82.166.93.197\r\na=candidate:3168280865 1 udp 2122260223 11.0.0.244 58157 typ host generation 0\r\na=candidate:1260196625 1 udp 2122194687 10.134.172.254 58951 typ host generation 0\r\na=candidate:3168280865 2 udp 2122260222 11.0.0.244 52576 typ host generation 0\r\na=candidate:1260196625 2 udp 2122194686 10.134.172.254 58945 typ host generation 0\r\na=candidate:4066106833 1 tcp 1518280447 11.0.0.244 60562 typ host tcptype passive generation 0\r\na=candidate:94302177 1 tcp 1518214911 10.134.172.254 60563 typ host tcptype passive generation 0\r\na=candidate:4066106833 2 tcp 1518280446 11.0.0.244 60564 typ host tcptype passive generation 0\r\na=candidate:94302177 2 tcp 1518214910 10.134.172.254 60565 typ host tcptype passive generation 0\r\na=candidate:1610196941 1 udp 1686052607 82.166.93.197 58157 typ srflx raddr 11.0.0.244 rport 58157 generation 0\r\na=candidate:1610196941 2 udp 1686052606 82.166.93.197 52576 typ srflx raddr 11.0.0.244 rport 52576 generation 0\r\na=candidate:2274372738 2 udp 1685987070 176.13.15.205 5834 typ srflx raddr 10.134.172.254 rport 58945 generation 0\r\na=candidate:2274372738 1 udp 1685987071 176.13.15.205 5840 typ srflx raddr 10.134.172.254 rport 58951 generation 0\r\na=ice-ufrag:g8lHDtPwH7m5xRex\r\na=ice-pwd:Q6jcBJNTWAyu0JTuIaQAeNI3\r\na=fingerprint:sha-256 0F:A1:68:51:87:3E:B4:C1:0D:33:97:40:78:22:2A:8C:D2:B6:46:23:F5:99:C9:88:5D:34:DB:E2:C5:94:B3:DD\r\na=setup:actpass\r\na=mid:audio\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:3 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/abs-send-time\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:111 opus\/48000\/2\r\na=fmtp:111 minptime=10; useinbandfec=1\r\na=rtpmap:103 ISAC\/16000\r\na=rtpmap:104 ISAC\/32000\r\na=rtpmap:9 G722\/8000\r\na=rtpmap:102 ILBC\/8000\r\na=rtpmap:0 PCMU\/8000\r\na=rtpmap:8 PCMA\/8000\r\na=rtpmap:106 CN\/32000\r\na=rtpmap:105 CN\/16000\r\na=rtpmap:13 CN\/8000\r\na=rtpmap:127 red\/8000\r\na=rtpmap:126 telephone-event\/8000\r\na=maxptime:60\r\nm=video 61966 UDP\/TLS\/RTP\/SAVPF 100 101 116 117 96\r\nc=IN IP4 82.166.93.197\r\na=rtcp:63816 IN IP4 82.166.93.197\r\na=candidate:3168280865 1 udp 2122260223 11.0.0.244 61966 typ host generation 0\r\na=candidate:1260196625 1 udp 2122194687 10.134.172.254 50435 typ host generation 0\r\na=candidate:3168280865 2 udp 2122260222 11.0.0.244 63816 typ host generation 0\r\na=candidate:1260196625 2 udp 2122194686 10.134.172.254 63396 typ host generation 0\r\na=candidate:4066106833 1 tcp 1518280447 11.0.0.244 60566 typ host tcptype passive generation 0\r\na=candidate:94302177 1 tcp 1518214911 10.134.172.254 60567 typ host tcptype passive generation 0\r\na=candidate:4066106833 2 tcp 1518280446 11.0.0.244 60568 typ host tcptype passive generation 0\r\na=candidate:94302177 2 tcp 1518214910 10.134.172.254 60569 typ host tcptype passive generation 0\r\na=candidate:1610196941 1 udp 1686052607 82.166.93.197 61966 typ srflx raddr 11.0.0.244 rport 61966 generation 0\r\na=candidate:1610196941 2 udp 1686052606 82.166.93.197 63816 typ srflx raddr 11.0.0.244 rport 63816 generation 0\r\na=candidate:2274372738 1 udp 1685987071 176.13.15.205 5879 typ srflx raddr 10.134.172.254 rport 50435 generation 0\r\na=candidate:2274372738 2 udp 1685987070 176.13.15.205 5860 typ srflx raddr 10.134.172.254 rport 63396 generation 0\r\na=ice-ufrag:g8lHDtPwH7m5xRex\r\na=ice-pwd:Q6jcBJNTWAyu0JTuIaQAeNI3\r\na=fingerprint:sha-256 0F:A1:68:51:87:3E:B4:C1:0D:33:97:40:78:22:2A:8C:D2:B6:46:23:F5:99:C9:88:5D:34:DB:E2:C5:94:B3:DD\r\na=setup:actpass\r\na=mid:video\r\na=extmap:2 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:3 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/abs-send-time\r\na=extmap:4 urn:3gpp:video-orientation\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:100 VP8\/90000\r\na=rtcp-fb:100 ccm fir\r\na=rtcp-fb:100 nack\r\na=rtcp-fb:100 nack pli\r\na=rtcp-fb:100 goog-remb\r\na=rtcp-fb:100 transport-cc\r\na=rtpmap:101 VP9\/90000\r\na=rtcp-fb:101 ccm fir\r\na=rtcp-fb:101 nack\r\na=rtcp-fb:101 nack pli\r\na=rtcp-fb:101 goog-remb\r\na=rtcp-fb:101 transport-cc\r\na=rtpmap:116 red\/90000\r\na=rtpmap:117 ulpfec\/90000\r\na=rtpmap:96 rtx\/90000\r\na=fmtp:96 apt=100\r\n"}}

调用创建响应:

{"jsonrpc":"2.0","id":2,"result":{"message":"CALL CREATED","callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800","sessid":"53FB0781-B586-4CDA-98C6-558680663B46"}}

verto.media 调用:

{"jsonrpc":"2.0","method":"verto.media","id":637,"params":{"sdp":"v=0\no=FreeSWITCH 1457232832 1457232833 IN IP4 159.203.164.7\ns=FreeSWITCH\nc=IN IP4 159.203.164.7\nt=0 0\na=msid-semantic: WMS TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub\nm=audio 30784 UDP/TLS/RTP/SAVPF 111 126\na=rtpmap:111 opus/48000/2\na=fmtp:111 useinbandfec=1; minptime=10\na=rtpmap:126 telephone-event/8000\na=silenceSupp:off - - - -\na=ptime:20\na=sendonly\na=fingerprint:sha-256 FE:CD:54:3E:2A:D7:DB:00:57:B7:D4:55:A8:EB:79:08:16:BB:B0:EA:43:44:42:9A:90:01:49:37:7B:31:48:F8\na=setup:active\na=rtcp-mux\na=rtcp:30784 IN IP4 159.203.164.7\na=ice-ufrag:qLh1zzclxONPNyQO\na=ice-pwd:G7g4Drkist37beYsP5jfvlqS\na=candidate:9922185636 1 udp 659136 159.203.164.7 30784 typ host generation 0\na=ssrc:1323504502 cname:bhqCyFkpPbjUPSk0\na=ssrc:1323504502 msid:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub a0\na=ssrc:1323504502 mslabel:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub\na=ssrc:1323504502 label:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Uba0\nm=video 31380 UDP/TLS/RTP/SAVPF 100\na=rtpmap:100 VP8/90000\na=sendonly\na=fingerprint:sha-256 FE:CD:54:3E:2A:D7:DB:00:57:B7:D4:55:A8:EB:79:08:16:BB:B0:EA:43:44:42:9A:90:01:49:37:7B:31:48:F8\na=setup:active\na=rtcp-mux\na=rtcp:31380 IN IP4 159.203.164.7\nb=AS:1024\na=rtcp-fb:100 ccm fir\na=rtcp-fb:100 nack\na=rtcp-fb:100 nack pli\na=ssrc:594893571 cname:bhqCyFkpPbjUPSk0\na=ssrc:594893571 msid:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub v0\na=ssrc:594893571 mslabel:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub\na=ssrc:594893571 label:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ubv0\na=ice-ufrag:2KDK4wDMYuAuVdAZ\na=ice-pwd:YTpxObqpLuBEfig7TKHN6bqU\na=candidate:7508673635 1 udp 659136 159.203.164.7 31380 typ host generation 0\n","callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800"}}

verto.answer 被调用:

{"jsonrpc":"2.0","method":"verto.answer","id":638,"params":{"callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800"}}

---

问:我缺少什么才能在浏览器端听到音频？
任何信息表示赞赏:)

---

更新，添加了freeswitch log

更新 2 IOS:音频流代码

...
let audioTrack = self.factory.audioTrackWithID("Local-Audio")
self.localMediaStream?.addAudioTrack(audioTrack);
self.peerConnection!.addStream(self.localMediaStream)
...

更新 3 - 部分解决方案在检查我的代码时，我发现了用于将视频轨道添加到我的本地媒体流的旧代码，禁用此部分可以解决音频问题，但是为什么？该代码有什么问题？

P.S Promise 类是 friend 创建的，模仿了 JS Promise 方法。

func getUserMedia(mediaOptions:Dictionary<String , Any>? = nil) -> Promise<RTCMediaStream>{
    return Promise<RTCMediaStream>(executor: { (resolve, reject) -> () in
        var cameraID:String?
        self.localMediaStream = self.factory.mediaStreamWithLabel("Local-Meida")

        //if video option is enabled (default true)

        //-------------- Disabling this section solves the audio issues --------------
        if(mediaOptions?["video"] as? Bool ?? true){
            for captureDevice in AVCaptureDevice.devicesWithMediaType(AVMediaTypeVideo){
                if (captureDevice.position == mediaOptions?["devicePosition"] as? AVCaptureDevicePosition ?? AVCaptureDevicePosition.Front){
                    cameraID = captureDevice.localizedName
                    break
                }
            }

            if(cameraID == nil){
                reject(NSError(domain: "No cammera detected", code: 0, userInfo: nil))
            }

            let capturer = RTCVideoCapturer.init(deviceName: cameraID)

            let videoSource = self.factory.videoSourceWithCapturer(capturer, constraints: mediaOptions?["constraints"] as? RTCMediaConstraints ?? nil)

            if let localVideoTrack = self.factory.videoTrackWithID("Local-Video", source: videoSource){
                //!!!! THIS IS THE PROBLEMATIC LINE !!!!
                self.localMediaStream?.addVideoTrack(localVideoTrack)
            }else{
                reject(NSError(domain: "No Video track", code: 0, userInfo: nil))
            }
        }
        //-------------- Disabling this section solves the audio issues --------------

        if(mediaOptions?["audio"] as? Bool ?? true){
            let audioTrack = self.factory.audioTrackWithID("Local-Audio")
            self.localMediaStream?.addAudioTrack(audioTrack);
        }
        self.peerConnection!.addStream(self.localMediaStream)

        resolve(self.localMediaStream!)
    })
}

在有问题的行调试

Answer 1

更新

除非检查媒体服务器、Web 客户端 和iOS 客户端，否则很难理解 WebRTC 实现出了什么问题在我们的案例中。

您的情况是音频通话，因此您的localStream 中不需要包含任何视频流，但如果您仔细观察，您会发现< em>你实际上是在你的移动流上添加一个 videoTrack:

 if(mediaOptions?["video"] as? Bool ?? true){
        for captureDevice in AVCaptureDevice.devicesWithMediaType(AVMediaTypeVideo){
            if (captureDevice.position == mediaOptions?["devicePosition"] as? AVCaptureDevicePosition ?? AVCaptureDevicePosition.Front){
                cameraID = captureDevice.localizedName
                break
            }
        }

        if(cameraID == nil){
            reject(NSError(domain: "No cammera detected", code: 0, userInfo: nil))
        }

        let capturer = RTCVideoCapturer.init(deviceName: cameraID)

        let videoSource = self.factory.videoSourceWithCapturer(capturer, constraints: mediaOptions?["constraints"] as? RTCMediaConstraints ?? nil)

        if let localVideoTrack = self.factory.videoTrackWithID("Local-Video", source: videoSource){
            //!!!! THIS IS THE PROBLEMATIC LINE !!!!
            self.localMediaStream?.addVideoTrack(localVideoTrack)
        }else{
            reject(NSError(domain: "No Video track", code: 0, userInfo: nil))
        }
    }

因此导致问题的行是:self.localMediaStream?.addVideoTrack(localVideoTrack)，因为您将视频附加到 localStream。

意见

我们可能会遇到不同的场景，正如我所说的，这里我根据我在构建类似系统时的经验列出一些意见:

1. 您的 MediaServer 可能没有可以在成功状态下重定向和处理您的调用的实现，因为当您附加视频时会添加其他内容(请查看您的 session 描述您实际发送的内容那里)，它只是拒绝创建调用。
2. 即使您的 MediaServer 可以处理这种情况，这也将包括 Client 的正确实现(Desktop 和 Mobile) 以符合其协议(protocol)的信号。
3. 您通过了所有测试，现在您正在添加视频和音频，因此您正在从移动设备启动 localStream，同样需要以其他方式创建。然后，当您通过 websockets 添加流、删除流和其他内容时，您需要处理事件。

本例解决方案

去掉localStream里面添加localTrack的部分，这样即使有错误也不是你创建localStream引起的，所以这一步目前已经解决了。

---

原始答案

我这里有一个我的工作版本，但由于您只使用音频，因此可以根据您的需要进行调整。

创建和设置 peerConnection (localSide)

// Connecting to the socket
    .........

// Create PeerConnectionFactory
self.peerConnectionFactory = [[RTCPeerConnectionFactory alloc] init];

RTCMediaConstraints *constraints = [self defaultPeerConnectionConstraints];

// Initialize peerConnection based on a list of ICE Servers
self.peerConnection = [self.peerConnectionFactory peerConnectionWithICEServers:[self getICEServers] constraints:constraints delegate:self];

// Create the localStram which contains the audioTrack
RTCMediaStream *localStream = [self createLocalMediaStream];

// Add this stream to the peerConnection
[self.peerConnection addStream:localStream];

// Please be aware here that I am using blocks, as I created a wrapper for easier maintenance, but you can use createOfferWithDelegate: which will go back at your delegation
NSLog(@"Creating peer offer");
RTCManager *strongSelf = self;
[strongSelf.peerConnection createOfferWithCallback:^(RTCSessionDescription *sdp, NSError *error) {
    if (!error) {
        dispatch_async(dispatch_get_main_queue(), ^{
            NSLog(@"Success at creating offer, now setting local description");
            [strongSelf.peerConnection setLocalDescriptionWithCallback:^(NSError *error) {
                if (!error) {
                    dispatch_async(dispatch_get_main_queue(), ^{
                        NSLog(@"Success at setting local description");
                        // On my type of signalization here I am connected, but yours is based on what type of signalization requires
                    });
                }

            } sessionDescription:sdp];
        });
    }
} constraints:[strongSelf defaultPeerConnectionConstraints]];

助手

// Now here we create the stream which contains the audio (Please note the ID)
- (RTCMediaStream *)createLocalMediaStream {
    RTCMediaStream *localStream = [self.peerConnectionFactory mediaStreamWithLabel:@"ARDAMS"];
    [localStream addAudioTrack:[self.peerConnectionFactory audioTrackWithID:@"ARDAMSa0"]];

    return localStream;
}

- (RTCMediaConstraints *)defaultPeerConnectionConstraints {
    // DtlsSrtpKeyAgreement is required for Chrome and Firefox to interoperate.
    NSArray *optionalConstraints = @[[[RTCPair alloc] initWithKey:@"DtlsSrtpKeyAgreement" value:@"true"]];

    RTCMediaConstraints *constraints = [[RTCMediaConstraints alloc] initWithMandatoryConstraints:nil optionalConstraints:optionalConstraints];
    return constraints;
}

请注意，您的问题也可能是由于未在主线程 上调用内容引起的。