Inspiring

Question

Acoustic echo cancellation on mobile!

Forum|Forum|9 years ago
May 4, 2016
8 replies
4215 views

Just wanted to say a massive thanks to Chris and the Air dev team for enabling echo cancellation on mobile in Air 22!

We've waited for this and cannot say just how exciting this is and how huge the impact will be for our business and our clients.

AIR 22 is a HUGE leap forward!

THANKS!

This topic has been closed for replies.

S

Spider22

Inspiring

Hi,

Testing for mobile to mobile is 2 devices in 2 separate rooms. Testing for computer to mobile, use headsets on the computer.

It is not just the garbled voice, but the delay that is introduced is very long, and it even seems to lengthen the voice itself when aec is active and the mic is on. If the mic is muted, the inbound audio is fine.

Not sure if that helps?

Thanks, trying to get this to work.

S

Spider22

Inspiring

Hi, just tried again using your posted code above (mic, ns) over an rtmp live connection using 2 ios devices (iphone/ipad) and observed the following:

- audio played on both devices is garbled and delayed

- if I mute one of the mics, its inbound audio is ok (although it sounds a bit electronic), and if I then unmute, the inbound audio returns to garbled and delayed.

mic= Microphone.getEnhancedMicrophone();

var option:MicrophoneEnhancedOptions = new MicrophoneEnhancedOptions();

//option.mode = MicrophoneEnhancedMode.FULL_DUPLEX;

option.autoGain = false;

option.echoPath = 128;

option.nonLinearProcessing = true;

mic.enhancedOptions = option;

mic.noiseSuppressionLevel = -30;

mic.codec = "speex";

mic.encodeQuality = 5;

mic.framesPerPacket = 1;

mic.setUseEchoSuppression(true);

mic.rate = 48;

mic.gain = 50;

mic.setSilenceLevel(0,200);

deesharm

Adobe Employee

Hi,

For further investigation we would like to know how are you testing the connection between two devices.

It might be the case if both the devices are kept in close vicinity, it can result in garbled voice.

Thanks,

Adobe AIR Team

S

Spider22

Inspiring

Hi, just tried this again with the latest prod air sdk. I have tried with Android/Mac/Win on one end, ios on the other, and ios to ios. Am using ios 9.3.2 on 4s. 5, 5s, 6 and ipad. Using rtmp. Have set mic codec to speex using enhancedmic - still get garbled/delay voice out and in as above. If I mute mic on the ios side, audio in on ios is fine. I don't think the MicrophoneEnhancedOptions works on ios, but could be wrong. In any case, still having issues.

J

joonsook72856968

Participant

I don't think that AEC is working on iOS. My device is New iPad 3rd generation (A1416) and iOS version is 9.3.2.

For microphone, I did code like this.

----------------------------------------------------------

var _mic:Microphone;

...

this._mic = Microphone.getEnhancedMicrophone();

var option:MicrophoneEnhancedOptions = new MicrophoneEnhancedOptions();

option.mode = MicrophoneEnhancedMode.FULL_DUPLEX;

option.autoGain = false;

option.echoPath = 128;

option.nonLinearProcessing = true;

this._mic.enhancedOptions = option;

this._mic.noiseSuppressionLevel = -30;

this._mic.codec = "speex";

this._mic.encodeQuality = 5;

this._mic.framesPerPacket = 1;

//this._mic.setUseEchoSuppression( true );

this._mic.setLoopBack( true );

this._mic.rate = 48;

this._mic.gain = 50;

this._mic.setSilenceLevel( 0, 200 );

----------------------------------------------------------

Well, I might say one thing. I use RTMP protocol. In RTMP, is AEC impossible?

deesharm

Adobe Employee

Hi Joonsoo Kim,

We tried with a sample application made using the code snippet you provided to us. Yes, we do observe the echo occurring in loopback scenario but it works completely fine if we include the same mic settings in a sample application that establish a VOIP connection between two iPhone devices.

Could you please try it with an application where two devices are interacting with each other.

Please refer to the below code snippet :

package

{

import flash.display.Sprite;

import flash.display.StageAlign;

import flash.display.StageScaleMode;

import flash.events.AsyncErrorEvent;

import flash.events.NetStatusEvent;

import flash.events.TimerEvent;

import flash.media.Microphone;

import flash.media.MicrophoneEnhancedOptions;

import flash.net.NetConnection;

import flash.net.NetStream;

import flash.utils.Timer;

public class EchoCancel_RTMPTestWorking extends Sprite

{

public var ns:NetStream;

public var nc:NetConnection;

public var mic:Microphone;

public var ns2:NetStream;

public var nc2:NetConnection;

public var delay:Number = 10000;

private var PUBLISH_STREAM_NAME:String = "";

private var PLAY_STREAM_NAME:String = "";

private var USER_NAME:String = "";

// Define before compilation who we are first phone or second phone. set "firstPhone:Boolean = false" when launching the code on second phone.

private var firstPhone:Boolean = true;

public function EchoCancel_RTMPTestWorking()

{

super();

// support autoOrients

stage.align = StageAlign.TOP_LEFT;

stage.scaleMode = StageScaleMode.NO_SCALE;

if (firstPhone) {

PUBLISH_STREAM_NAME = "streamOne";

PLAY_STREAM_NAME = "streamTwo";

USER_NAME = "userOne";

}else {

PUBLISH_STREAM_NAME = "streamTwo";

PLAY_STREAM_NAME = "streamOne";

USER_NAME = "userTwo";

}

test();

}

public function test():void

{

nc= new NetConnection();

nc.addEventListener(NetStatusEvent.NET_STATUS, netStatusHandler);

nc.addEventListener(AsyncErrorEvent.ASYNC_ERROR , onAsyncErrorEventHandler);

var clientObj:Object = new Object();

nc.client = clientObj;

trace("even listner called");

nc.connect("rtmp://IPADDRESS/live",USER_NAME);

trace("connection success");

}

/**

* ON ASYNC ERROR

* @param e

*/

private function onAsyncErrorEventHandler(e:AsyncErrorEvent):void

{

trace("e"+e);

}

/**

* ON NET STATUS

* @param event

*/

private function netStatusHandler(event:NetStatusEvent):void {

trace(event.info.code + " | " + event.info.description);

switch (event.info.code) {

case "NetConnection.Connect.Success":

trace("Connection Sucseed Publish My and Play Others stream");

//PUBLISH MY STREAM

ns = new NetStream(nc);

ns.publish(PUBLISH_STREAM_NAME,"live");

enableMic();

// PLAY OTHER STREAM

ns2 = new NetStream(nc);

ns2.play(PLAY_STREAM_NAME, -1);

break;

case "NetStream.Publish.Start":

break;

}

public function enableMic():void {

//ns.attachAudio(null);

mic= Microphone.getEnhancedMicrophone();

var option:MicrophoneEnhancedOptions = new MicrophoneEnhancedOptions();

//option.mode = MicrophoneEnhancedMode.FULL_DUPLEX;

option.autoGain = false;

option.echoPath = 128;

option.nonLinearProcessing = true;

mic.enhancedOptions = option;

mic.noiseSuppressionLevel = -30;

mic.codec = "speex";

mic.encodeQuality = 5;

mic.framesPerPacket = 1;

mic.setUseEchoSuppression(true);

mic.rate = 48;

mic.gain = 50;

mic.setSilenceLevel(0,200);

ns.attachAudio(mic);

}

Thanks,

Adobe AIR Team

J

joonsook72856968

Participant

Thank you very much for your reply. After your reply, I implemented your code and added more code to get log.

That was "mic.enhancedOptions == null". Before testing 2 devices, in 1 device, it's already "true" in iPad

(New iPad 3rd generation (A1416) and iOS version is 9.3.2).

In PC debugging, it's false. I think it's right.

Do you think is it right? I thought if AEC might support in iOS, i had to get "false".

I think why I get gabble voice is microphone could not implement enhancedOptions in iOS...

S

Spider22

Inspiring

FMSer.cn thanks for project. I also tried changing the mic.rate up and down with predictable results: at 48 the sound is sped up so the voice sounds like a garbled mickey mouse (well sort of). In our case, an example is as follows:

1) create a netconnection to an rmtp server (media server)

2) once connected, create a netstream, attached to the above netconnection

3) get mic (enhanced mic)

4) attach mic to netstream above

5) publish stream to netstream above

6) play stream from other user

Do this on both devices.

Note using ios on one side with enhanced mic, android/win/mac on the other without enhanced mic - all configurations result in the same garbled/studder sound, and add delay on inbound on the ios side. On ios side, if mute mic (mic=null) while in a call, the incoming sound returns to normal without garble/delay - unmute the mic (mic=enhancedmic) incoming sound becomes garbled/delay.

I have not tried this with setLoopBack(true) as I only use that mode for user to make sure they can hear themselves as a test - not during a call which is where I need AEC.

deesharm

Adobe Employee

Hi,

I tried reproducing the issue at my end using the sample project placed at below link:

Dropbox - Echo Cancellation

Could you please try with this project at your end and share the results.

Please don't forget to change the IP Address in the .as file.

Device used : iPhone 6 with OS Version 8.1.3

Thanks,

Adobe AIR Team

F

FMSer_cn

Participating Frequently

Hi deesharm

I have tested your sample on iphone4s with OS version 8.2,the result is same as before,when I speak,I can hear my sound back from fms garbled/studder like a garbled mickey mouse.

S

Spider22

Inspiring

Hi,

Perhaps you can post the gain and rate settings you have used in your labs that work?

Just tried mic.gain at 50,60,70,80,90,100 and, although its louder, it is still garbled/studder on the other end. Tried changing the rate, but it seems to make it worse.

Just to be clear, changing these settings on the ios side running sdk 22.

Thanks.

deesharm

Adobe Employee

Hi,

I am trying to reproduce the bug using a loopback scenario mentioned in below lines of code:

public function TestCase()

{

mic = Microphone.getEnhancedMicrophone();

mic.rate = 48;

mic.setLoopBack(true);

mic.addEventListener(StatusEvent.STATUS, mic_status);

}

But i am not observing any garbled voice.

Could you please provide us a sample project so that we can reproduce the bug at our end.

We will further look into it.

Thanks,

Adobe AIR Team

F

FMSer_cn

Participating Frequently

You can download the sample for test from here.

http://115.28.8.57:8080/videoChat.rar

1.put main.asc to ams application folder.

2.modify videoChat.as and publish to a .ipa file. and a .swf file.

3.install .ipa on iphone A and connect to fms.

4.install .swf on PC B and connect to fms.

5.double click B in userlist on A to open B's audio.

6.double click A in userlist on B to open A's audio.

7.will hear B's audio is garbled/studder on iphone A.

S

Spider22

Inspiring

Tried again with the latest beta sdk 22 (may 11). Devices tested: 4s, 5, 5s and ipad. All have the same results, as noted above - inbound audio has delay and is garbled, outgoing mic audio is garbled/studder. I again tested this with incoming codecs nellymoser and speex, same results. Note that when in a call, if I mute the mic (mic = null) on the ios side, the incoming audio comes in very clear no delay, but once I reactivate the mic, the issues return (it affects outbound and inbound audio).

Is AEC available on Android?

deesharm

Adobe Employee

Hi,

This problem can be addressed by tuning up the outbound and inbound audio using different Microphone properties. Try increasing the mic.gain() and mic.rate() slightly, to get clear audio on both sides.

Also the Performance of this feature will be varying depending upon the iOS version and the device type. So it is possible that you may see different performance on low end devices like 4s.

If It still doesn't help after setting up higher mic.gain and mic.rate properties, please share a sample project so that we can reproduce the bug at our end.

No, AEC is not available on Android. For now, we are investigating on it as Android fragmentation is making it a bit difficult to bring out a consistence experience.

Thanks,

Adobe AIR Team

S

Spider22

Inspiring

Agreed, we have waited a long time for this and it's finally here - thanks!

However, I am having a few problems. When I compile using SDK22 for iOS using mic = Microphone.getMicrophone(), all works fine - I am testing using win/mac/android/ios voice via rtmp, using both nellymoser and speex codecs. But when I use mic = Microphone.getEnhancedMicrophone() on iOS (everything else the same) it complies and runs on iOS, but the incoming audio is garbled and has delay, the outgoing audio is also garbled/studder which you can barely make it out (tested using both codecs). When you turn on spearkerphone, without the enhanced mic you get huge echo, with enhanced mic you do not get echo (which is great) but the sound is garbled.

I am wondering if there are any settings that need to be set, or codec restrictions, or?

Thanks.

deesharm

Adobe Employee

Hi,

Which device you are using at your end?

And could you please try by setting different microphone properties in your action script code as mic.rate() and mic.gain()

You will get the detailed description of microphone properties at below link:

Microphone - Adobe ActionScript® 3 (AS3 ) API Reference

Thanks,

Adobe AIR Team

F

FMSer_cn

Participating Frequently

We have waited it for many years!

Thanks for Adobe AIR Team.

And I have the same issue.I tested it on iphone 4s.

It works fine on Android.

The issue is same as above:

However, I am having a few problems. When I compile using SDK22 for iOS using mic = Microphone.getMicrophone(), all works fine - I am testing using win/mac/android/ios voice via rtmp, using both nellymoser and speex codecs. But when I use mic = Microphone.getEnhancedMicrophone() on iOS (everything else the same) it complies and runs on iOS, but the incoming audio is garbled and has delay, the outgoing audio is also garbled/studder which you can barely make it out (tested using both codecs). When you turn on spearkerphone, without the enhanced mic you get huge echo, with enhanced mic you do not get echo (which is great) but the sound is garbled.

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded