Copy link to clipboard
Copied
Just wanted to say a massive thanks to Chris and the Air dev team for enabling echo cancellation on mobile in Air 22!
We've waited for this and cannot say just how exciting this is and how huge the impact will be for our business and our clients.
AIR 22 is a HUGE leap forward!
THANKS!
Copy link to clipboard
Copied
Agreed, we have waited a long time for this and it's finally here - thanks!
However, I am having a few problems. When I compile using SDK22 for iOS using mic = Microphone.getMicrophone(), all works fine - I am testing using win/mac/android/ios voice via rtmp, using both nellymoser and speex codecs. But when I use mic = Microphone.getEnhancedMicrophone() on iOS (everything else the same) it complies and runs on iOS, but the incoming audio is garbled and has delay, the outgoing audio is also garbled/studder which you can barely make it out (tested using both codecs). When you turn on spearkerphone, without the enhanced mic you get huge echo, with enhanced mic you do not get echo (which is great) but the sound is garbled.
I am wondering if there are any settings that need to be set, or codec restrictions, or?
Thanks.
Copy link to clipboard
Copied
Hi,
Which device you are using at your end?
And could you please try by setting different microphone properties in your action script code as mic.rate() and mic.gain()
You will get the detailed description of microphone properties at below link:
Microphone - Adobe ActionScriptĀ® 3 (AS3 ) API Reference
Thanks,
Adobe AIR Team
Copy link to clipboard
Copied
We have waited it for many years!
Thanks for Adobe AIR Team.
And I have the same issue.I tested it on iphone 4s.
It works fine on Android.
The issue is same as above:
However, I am having a few problems. When I compile using SDK22 for iOS using mic = Microphone.getMicrophone(), all works fine - I am testing using win/mac/android/ios voice via rtmp, using both nellymoser and speex codecs. But when I use mic = Microphone.getEnhancedMicrophone() on iOS (everything else the same) it complies and runs on iOS, but the incoming audio is garbled and has delay, the outgoing audio is also garbled/studder which you can barely make it out (tested using both codecs). When you turn on spearkerphone, without the enhanced mic you get huge echo, with enhanced mic you do not get echo (which is great) but the sound is garbled.
Copy link to clipboard
Copied
chris.campbellā What about AEC support for Android? Is it possible to get this feature?
Copy link to clipboard
Copied
Tried again with the latest beta sdk 22 (may 11). Devices tested: 4s, 5, 5s and ipad. All have the same results, as noted above - inbound audio has delay and is garbled, outgoing mic audio is garbled/studder. I again tested this with incoming codecs nellymoser and speex, same results. Note that when in a call, if I mute the mic (mic = null) on the ios side, the incoming audio comes in very clear no delay, but once I reactivate the mic, the issues return (it affects outbound and inbound audio).
Is AEC available on Android?
Copy link to clipboard
Copied
Hi,
This problem can be addressed by tuning up the outbound and inbound audio using different Microphone properties. Try increasing the mic.gain() and mic.rate() slightly, to get clear audio on both sides.
Also the Performance of this feature will be varying depending upon the iOS version and the device type. So it is possible that you may see different performance on low end devices like 4s.
If It still doesn't help after setting up higher mic.gain and mic.rate properties, please share a sample project so that we can reproduce the bug at our end.
No, AEC is not available on Android. For now, we are investigating on it as Android fragmentation is making it a bit difficult to bring out a consistence experience.
Thanks,
Adobe AIR Team
Copy link to clipboard
Copied
Hi,
Perhaps you can post the gain and rate settings you have used in your labs that work?
Just tried mic.gain at 50,60,70,80,90,100 and, although its louder, it is still garbled/studder on the other end. Tried changing the rate, but it seems to make it worse.
Just to be clear, changing these settings on the ios side running sdk 22.
Thanks.
Copy link to clipboard
Copied
Hi,
I am trying to reproduce the bug using a loopback scenario mentioned in below lines of code:
public function TestCase()
{
mic = Microphone.getEnhancedMicrophone();
mic.rate = 48;
mic.setLoopBack(true);
mic.addEventListener(StatusEvent.STATUS, mic_status);
}
But i am not observing any garbled voice.
Could you please provide us a sample project so that we can reproduce the bug at our end.
We will further look into it.
Thanks,
Adobe AIR Team
Copy link to clipboard
Copied
You can download the sample for test from here.
http://115.28.8.57:8080/videoChat.rar
1.put main.asc to ams application folder.
2.modify videoChat.as and publish to a .ipa file. and a .swf file.
3.install .ipa on iphone A and connect to fms.
4.install .swf on PC B and connect to fms.
5.double click B in userlist on A to open B's audio.
6.double click A in userlist on B to open A's audio.
7.will hear B's audio is garbled/studder on iphone A.
Copy link to clipboard
Copied
FMSer.cn thanks for project. I also tried changing the mic.rate up and down with predictable results: at 48 the sound is sped up so the voice sounds like a garbled mickey mouse (well sort of). In our case, an example is as follows:
1) create a netconnection to an rmtp server (media server)
2) once connected, create a netstream, attached to the above netconnection
3) get mic (enhanced mic)
4) attach mic to netstream above
5) publish stream to netstream above
6) play stream from other user
Do this on both devices.
Note using ios on one side with enhanced mic, android/win/mac on the other without enhanced mic - all configurations result in the same garbled/studder sound, and add delay on inbound on the ios side. On ios side, if mute mic (mic=null) while in a call, the incoming sound returns to normal without garble/delay - unmute the mic (mic=enhancedmic) incoming sound becomes garbled/delay.
I have not tried this with setLoopBack(true) as I only use that mode for user to make sure they can hear themselves as a test - not during a call which is where I need AEC.
Copy link to clipboard
Copied
Hi,
I tried reproducing the issue at my end using the sample project placed at below link:
Could you please try with this project at your end and share the results.
Please don't forget to change the IP Address in the .as file.
Device used : iPhone 6 with OS Version 8.1.3
Thanks,
Adobe AIR Team
Copy link to clipboard
Copied
Hi deesharm
I have tested your sample on iphone4s with OS version 8.2,the result is same as before,when I speak,I can hear my sound back from fms garbled/studder like a garbled mickey mouse.
Copy link to clipboard
Copied
Hi,
The Performance of this feature will be varying depending upon the iOS version and the device type. So it is possible that you may see different performance on low end devices like 4s.
Could you please try the same code on some high end device like iPhone 6 and please share the result?
Thanks,
Adobe AIR Team
Copy link to clipboard
Copied
Have the same issue with studder sound on iphone 5. For streaming I use wowza media server, and got the same results ((( too bad
Problem is not in devices ) You guys made a big job, thank you for that! But unfortunately this feature doesn't work well )
Copy link to clipboard
Copied
Hi Alexejs,
I tried reproducing the problem using the sample project that you shared :
I established the connection between iPhone6 (8.1.3) and iPhone 6+ (9.3.2) and tried with another combination iPhone 5s (9.2.1) and iPhone6(8.1.3) using below code snippet:
package
{
import flash.display.Sprite;
import flash.display.StageAlign;
import flash.display.StageScaleMode;
import flash.events.AsyncErrorEvent;
import flash.events.NetStatusEvent;
import flash.events.TimerEvent;
import flash.media.Microphone;
import flash.net.NetConnection;
import flash.net.NetStream;
import flash.utils.Timer;
public class EchoCancel_RTMPTestNotWorking extends Sprite
{
public var ns:NetStream;
public var nc:NetConnection;
public var mic:Microphone;
public var ns2:NetStream;
public var nc2:NetConnection;
public var delay:Number = 10000;
private var PUBLISH_STREAM_NAME:String = "";
private var PLAY_STREAM_NAME:String = "";
private var USER_NAME:String = "";
// Define before compilation who we are first phone or second phone
private var firstPhone:Boolean = true;
public function EchoCancel_RTMPTestNotWorking()
{
super();
if (firstPhone) {
PUBLISH_STREAM_NAME = "streamOne";
PLAY_STREAM_NAME = "streamTwo";
USER_NAME = "userOne";
}else {
PUBLISH_STREAM_NAME = "streamTwo";
PLAY_STREAM_NAME = "streamOne";
USER_NAME = "userTwo";
}
test();
}
public function test():void
{
nc= new NetConnection();
nc.addEventListener(NetStatusEvent.NET_STATUS, netStatusHandler);
nc.addEventListener(AsyncErrorEvent.ASYNC_ERROR , onAsyncErrorEventHandler);
var clientObj:Object = new Object();
nc.client = clientObj;
nc.connect("rtmp://IPADDRESS/live",USER_NAME);
}
private function onAsyncErrorEventHandler(e:AsyncErrorEvent):void
{
trace("e"+e);
}
private function netStatusHandler(event:NetStatusEvent):void {
trace(event.info.code + " | " + event.info.description);
switch (event.info.code) {
case "NetConnection.Connect.Success":
ns = new NetStream(nc);
ns.publish(PUBLISH_STREAM_NAME,"live");
enableMic();
ns2 = new NetStream(nc);
ns2.play(PLAY_STREAM_NAME, -1);
break;
case "NetStream.Publish.Start":
break;
}
}
public function enableMic():void {
//ns.attachAudio(null);
mic= Microphone.getEnhancedMicrophone();
mic.codec = "speex";
ns.attachAudio(mic);
}
But I didn't observe any garbled/stutter voice. I am using Adobe Media Server at my end and compiling the swf using AIR 22.0.0.129.
If possible, Please share an audio or video file to dictate the exact issue that you are facing.
Thanks,
Adobe AIR Team
Copy link to clipboard
Copied
Dear deesharm,
When I connect to FMS from ios9,it's fail,but it's ok on ios4.Should I modify app.xml to add something such as:
<InfoAdditions>
<![CDATA[
<key>NSAppTransportSecurity</key>
<dict>
<key>NSAllowsArbitraryLoads</key><true/>
</dict>
]]>
</InfoAdditions>
Thanks.
Copy link to clipboard
Copied
Hi Firefox hdt,
No, we don't need to add any extra tags in app.xml to make Acoustic Echo Cancellation works on iOS.
Could you please tell us which device you are using for iOS 4. And it would be helpful if you can share the code snippet and the information about media server that you are using at your end?
Are you testing a loopback scenario or end to end device connection?
Thanks,
Adobe AIR Team
Copy link to clipboard
Copied
I don't think that AEC is working on iOS. My device is New iPad 3rd generation (A1416) and iOS version is 9.3.2.
For microphone, I did code like this.
----------------------------------------------------------
var _mic:Microphone;
...
this._mic = Microphone.getEnhancedMicrophone();
var option:MicrophoneEnhancedOptions = new MicrophoneEnhancedOptions();
option.mode = MicrophoneEnhancedMode.FULL_DUPLEX;
option.autoGain = false;
option.echoPath = 128;
option.nonLinearProcessing = true;
this._mic.enhancedOptions = option;
this._mic.noiseSuppressionLevel = -30;
this._mic.codec = "speex";
this._mic.encodeQuality = 5;
this._mic.framesPerPacket = 1;
//this._mic.setUseEchoSuppression( true );
this._mic.setLoopBack( true );
this._mic.rate = 48;
this._mic.gain = 50;
this._mic.setSilenceLevel( 0, 200 );
----------------------------------------------------------
Well, I might say one thing. I use RTMP protocol. In RTMP, is AEC impossible?
Copy link to clipboard
Copied
Hi Joonsoo Kim,
We tried with a sample application made using the code snippet you provided to us. Yes, we do observe the echo occurring in loopback scenario but it works completely fine if we include the same mic settings in a sample application that establish a VOIP connection between two iPhone devices.
Could you please try it with an application where two devices are interacting with each other.
Please refer to the below code snippet :
package
{
import flash.display.Sprite;
import flash.display.StageAlign;
import flash.display.StageScaleMode;
import flash.events.AsyncErrorEvent;
import flash.events.NetStatusEvent;
import flash.events.TimerEvent;
import flash.media.Microphone;
import flash.media.MicrophoneEnhancedOptions;
import flash.net.NetConnection;
import flash.net.NetStream;
import flash.utils.Timer;
public class EchoCancel_RTMPTestWorking extends Sprite
{
public var ns:NetStream;
public var nc:NetConnection;
public var mic:Microphone;
public var ns2:NetStream;
public var nc2:NetConnection;
public var delay:Number = 10000;
private var PUBLISH_STREAM_NAME:String = "";
private var PLAY_STREAM_NAME:String = "";
private var USER_NAME:String = "";
// Define before compilation who we are first phone or second phone. set "firstPhone:Boolean = false" when launching the code on second phone.
private var firstPhone:Boolean = true;
public function EchoCancel_RTMPTestWorking()
{
super();
// support autoOrients
stage.align = StageAlign.TOP_LEFT;
stage.scaleMode = StageScaleMode.NO_SCALE;
if (firstPhone) {
PUBLISH_STREAM_NAME = "streamOne";
PLAY_STREAM_NAME = "streamTwo";
USER_NAME = "userOne";
}else {
PUBLISH_STREAM_NAME = "streamTwo";
PLAY_STREAM_NAME = "streamOne";
USER_NAME = "userTwo";
}
test();
}
public function test():void
{
nc= new NetConnection();
nc.addEventListener(NetStatusEvent.NET_STATUS, netStatusHandler);
nc.addEventListener(AsyncErrorEvent.ASYNC_ERROR , onAsyncErrorEventHandler);
var clientObj:Object = new Object();
nc.client = clientObj;
trace("even listner called");
nc.connect("rtmp://IPADDRESS/live",USER_NAME);
trace("connection success");
}
/**
* ON ASYNC ERROR
* @param e
*/
private function onAsyncErrorEventHandler(e:AsyncErrorEvent):void
{
trace("e"+e);
}
/**
* ON NET STATUS
* @param event
*/
private function netStatusHandler(event:NetStatusEvent):void {
trace(event.info.code + " | " + event.info.description);
switch (event.info.code) {
case "NetConnection.Connect.Success":
trace("Connection Sucseed Publish My and Play Others stream");
//PUBLISH MY STREAM
ns = new NetStream(nc);
ns.publish(PUBLISH_STREAM_NAME,"live");
enableMic();
// PLAY OTHER STREAM
ns2 = new NetStream(nc);
ns2.play(PLAY_STREAM_NAME, -1);
break;
case "NetStream.Publish.Start":
break;
}
}
public function enableMic():void {
//ns.attachAudio(null);
mic= Microphone.getEnhancedMicrophone();
var option:MicrophoneEnhancedOptions = new MicrophoneEnhancedOptions();
//option.mode = MicrophoneEnhancedMode.FULL_DUPLEX;
option.autoGain = false;
option.echoPath = 128;
option.nonLinearProcessing = true;
mic.enhancedOptions = option;
mic.noiseSuppressionLevel = -30;
mic.codec = "speex";
mic.encodeQuality = 5;
mic.framesPerPacket = 1;
mic.setUseEchoSuppression(true);
mic.rate = 48;
mic.gain = 50;
mic.setSilenceLevel(0,200);
ns.attachAudio(mic);
}
}
}
Thanks,
Adobe AIR Team
Copy link to clipboard
Copied
Thank you very much for your reply. After your reply, I implemented your code and added more code to get log.
That was "mic.enhancedOptions == null". Before testing 2 devices, in 1 device, it's already "true" in iPad
(New iPad 3rd generation (A1416) and iOS version is 9.3.2).
In PC debugging, it's false. I think it's right.
Do you think is it right? I thought if AEC might support in iOS, i had to get "false".
I think why I get gabble voice is microphone could not implement enhancedOptions in iOS...
Copy link to clipboard
Copied
Maybe, on iOS, enhancedOptions is null.
I read "Release Note ( Flash Player Help | Release Notes | Flash PlayerĀ® 22 AIRĀ® 22 )" and found "The API MicrophoneEnhancedOptions will be a no-op on AIR for iOS.".
Does it mean enhancedOptions is null...?
Copy link to clipboard
Copied
Hi, just tried this again with the latest prod air sdk. I have tried with Android/Mac/Win on one end, ios on the other, and ios to ios. Am using ios 9.3.2 on 4s. 5, 5s, 6 and ipad. Using rtmp. Have set mic codec to speex using enhancedmic - still get garbled/delay voice out and in as above. If I mute mic on the ios side, audio in on ios is fine. I don't think the MicrophoneEnhancedOptions works on ios, but could be wrong. In any case, still having issues.
Copy link to clipboard
Copied
Hi, just tried again using your posted code above (mic, ns) over an rtmp live connection using 2 ios devices (iphone/ipad) and observed the following:
- audio played on both devices is garbled and delayed
- if I mute one of the mics, its inbound audio is ok (although it sounds a bit electronic), and if I then unmute, the inbound audio returns to garbled and delayed.
mic= Microphone.getEnhancedMicrophone();
var option:MicrophoneEnhancedOptions = new MicrophoneEnhancedOptions();
//option.mode = MicrophoneEnhancedMode.FULL_DUPLEX;
option.autoGain = false;
option.echoPath = 128;
option.nonLinearProcessing = true;
mic.enhancedOptions = option;
mic.noiseSuppressionLevel = -30;
mic.codec = "speex";
mic.encodeQuality = 5;
mic.framesPerPacket = 1;
mic.setUseEchoSuppression(true);
mic.rate = 48;
mic.gain = 50;
mic.setSilenceLevel(0,200);
Copy link to clipboard
Copied
Hi,
For further investigation we would like to know how are you testing the connection between two devices.
It might be the case if both the devices are kept in close vicinity, it can result in garbled voice.
Thanks,
Adobe AIR Team