QR Code Decoding/Reading and iOS

[K2xL_com]

Okay, this is driving me... insane.

I'm trying to implement a QR code reader in my iOS application (specifically for IPhone 4). I have tried THREE count em THREE different libraries and each one does not work. THEY ALL WORK PERFECTLY IN SIMULATOR BUT DO NOT WORK ON MOBILE DEVICE

1. I tried zxing bar code library which has an actionscript port.

On IPHONE 4: The application freezes (no error is thrown on remote debugging). I tried clipping the video source so not as big of a bitmap data is processed and it still freezes! (Actually the frames update maybe every 7-8 seconds)

2. I tried libspark's qr decode

Works perfectly in emulator.

On IPHONE 4: Same as Zxing... freezes (no error is thrown in remote debuggin).

3. Someone used Quasimondo's adaptive filter library to extend libspark's qr code library. I tried that and clipped the rectangle

On IPHONE 4: An error is thrown (via remote debugging) - one that isn't thrown in the emulator (it works perfectly in emulator). It APPEARS (although it's not easy to investigate) that the error is stemming from the air export not supporting pbj files (pixel bender/shaderjob stuff) - is that not supported in IPhone 4?

So it appears that there is a problem when you try and process bitmapdata on an IPhone... I've tried different render modes (gpu) and it didn't do anything for me.

DOES ANYONE know of any libraries that read QR codes? I really love developing ios stuff with AS3... Has anyone managed to use one of these libaries ??

[Innovatology · Accepted solution]

Well, if all 3 libraries fail, then maybe you are doing something wrong... 😉

Probably not good news for you, but I've used the zxing library on both Android and BlackBerry with no problem at all.

Are you reducing the camera resolution to a sensible level, e.g. 320x240 or 640x480? If you're trying to process a  5-megapixel image every cycle, that could cause a problem.

Also, snap a bitmap from your video feed every second or so with a timer, then and analyze that. You really don't need to try to decode 25 barcodes a second.

Otherwise: post a code snippet, so we can see what you're doing.

[K2xL_com]

Thanks for the response.

At first I looked at my code and i was just doing a 320,240... but then i realized i added it to a movieclip and stretched that. Instead of passing the original 320,240 clip i was passing in the stretched clip (doh).

I fixed it and trying again and will let you know what happens.

[K2xL_com]

So there is not freezing anymore, but I can't get the Iphone to read qr code :-(. Reads it fine on emulator but it just can't detect it on iphone

cam = Camera.getCamera();

cam.setMode(320, 240, 20, false);

cam.setQuality(0, 100);

cameraVideo.attachCamera(cam);

cameraVideo.width = cam.width;

cameraVideo.height = cam.height;

cameraVideo.cacheAsBitmap = true;

cameraVideoContainer.addChild(cameraVideo); // for actually what appears on screen

cameraVideoContainer.width = stage.stageWidth;

cameraVideoContainer.height = stage.stageHeight;

bitmapDataSnapshot = new BitmapData(cameraVideo.width, cameraVideo.height);

And the way I do it is i have a timer run every 250 milliseconds and take a snapshot.

bitmapDataSnapshot.draw(cameraVideo, null, null, null, null,false);

and then pass it into

public function decodeBitmapData(bmpd : BitmapData) : void {

var lsource : BufferedImageLuminanceSource = new BufferedImageLuminanceSource(bmpd);

var bitmap : BinaryBitmap = new BinaryBitmap(new GlobalHistogramBinarizer(lsource));

var ht : HashTable = null;

ht = this.getAllHints();

var res : Result = null;

try {

res = qrReader.decode(bitmap, ht);

} catch(e : Error) {

// trace(e);

}

if (res != null) {

var parsedResult : ParsedResult = ResultParser.parseResult(res);

trace("Result::: " + parsedResult.getDisplayResult());

onQrDecodeComplete(parsedResult.getDisplayResult());

}

}

[Innovatology]

cam.setMode(320, 240, 20, false);

If you're decoding 4 QR decodes per second, there is no need to use 20fps for the camera. Consider reducing that to 4.

cameraVideo.cacheAsBitmap = true;

Why????? This could very well be the cause your problem! As the video feed changes every frame, why cache it?!

try {
  res = qrReader.decode(bitmap, ht);
} catch(e : Error) {
  // trace(e);
}

You're suppressing all possible error messages here. Try removing the // to trace whether any errors are occurring (QRReader may also raise an exception when no QR is detected, so just ignore those).

var ht : HashTable = null;

ht = this.getAllHints();

If you are only interested in QR codes and not in other barcode types, consider using this instead:

var ht:HashTable = new HashTable();

ht.Add(DecodeHintType.POSSIBLE_FORMATS,BarcodeFormat.QR_CODE);

and possibly add this to increase accuracy at the cost of speed:

ht.Add(DecodeHintType.TRY_HARDER, true); 

[K2xL_com]

I should've been a bit more clear about my timer before. I try 4 times a second but I stop the timer before it processess and then restart it after (to prevent a bunch of callbacks being cued up)

I had the cacheasbitmap turned on only because things were going slow - and i saw that in an example someone had it turned on.  I turned it off now.

My this.getAllHints() already does what you suggest

public function getAllHints() : HashTable {

var ht : HashTable = new HashTable;

ht.Add(DecodeHintType.POSSIBLE_FORMATS, BarcodeFormat.QR_CODE);

return ht;

}

As for the try catch, that is actually also from sample code but when i was debugging i also had the same thought that it was a big performance hit. There is an error being thrown every time it tries to process. What I tried doing is going inside the class and having it return null rather than throw an error and bubble the code all the way up doing if (x == null) checks, but then I realized a different error is sometimes thrown depending on what "hints" it detects.

However, I had not tried ht.Add(DecodeHintType.TRY_HARDER, true);

But anyway, it doesn't matter because  there is not really any performance issues anymore (unless i do remote debugging which i realize now is incredibly slower than release) but after added the TRY_HARDER it still doesn't seem to detect no matter what angle :-(. Again on the iphone simulator it detects perfectly (and very quickly)

[Innovatology]

Have you tried putting your bitmapDataSnapshot into the source of a (temporary) <s:BitmapImage> to check whether you are capturing the QR image correctly?

I know AIR thinks its being clever with it's auto-orientation, but sometimes gets it wrong where the camera is concerned (rotating twice). I think the QRReader needs the QR code to be "the right way up", it won't work if the input is rotated.

[K2xL_com]

That was a great suggestion. I added it to a bitmap and this is what I got:

In emulator: http://i54.tinypic.com/2v7uzbk.png

Snapshot taken when on IPhone: http://i52.tinypic.com/2ppyjbd.png

Notice the difference? So strange... Thankfully it processes the qr code wonderfully when i match it up in the bitmapdata.

I wonder if the camera's width (which i use for size of bitmapdata) is not returning the right value when in landscape mode in iphone...

will debug and see what's up

[K2xL_com]

Yup some weirdness is definitely going on.

Camera's width and height on Emulator: 320,240

Camera's width and height on IPhone: 192,144

Whaaat... it reduces it by 60%... I don't quite understand what's going on...

cam = Camera.getCamera();

cam.setMode(320, 240, 20, false);

cam.setQuality(0, 100);

cameraVideo.attachCamera(cam);

cameraVideo.width = cam.width;

cameraVideo.height = cam.height;

In emulator: cameraVideo.width = 320

In IPhone: 192

[Innovatology]

This rather old doc might help:

http://flash-communications.net/technotes/setMode/index.html

The iPhone doesn't support 320x240, so Flash will have to scale it up or down. In fact, IOS adds its own logic behind Flash's logic. which may screw things up. Scroll down to the last table in:

http://developer.apple.com/library/ios/#documentation/AudioVideo/Conceptual/AVFoundationPG/Articles/...

Also, I have noticed that

• In addition to pixel count, Flash tries take the aspect ratio of the requested resolution amd the available camera modes into account. It may influence the decision, and may even result in clipping if you request a 4:3 image but the camera is wider.
• .setQuality() can  sometimes not only change the bandwidth or framerate, but also the selected mode! You may want to move setQuality() before setMode(), or even remove it completely.
• a lot of peope use setMode(360,360, fps, area) for some reason. Probably something to do with portrait/landscape.

[Joe ... Ward]

Here's an example code reader by Brian Rinaldi. I specifically asked him about performance and he claimed it worked fine. (I haven't tested it myself, but I have faith in Brian.) http://remotesynthesis.com/post.cfm/adding-a-qr-code-reader-in-flex-on-android

He is using the ZXing library.

[K2xL_com]

Thanks,

I actually submitted a bug to Adobe and then received a response back.

I posted it on my blog for easier reference: http://k2xl.com/wordpress/2011/08/27/camera-quality-on-ios-for-air/

So this explains a LOT... unfortuantely there doesn't seem to be an easy way to access the available resolutions for the device... so I submitted a feature request to the air team to include a way in the API to handle that.

Anyway, this explains a lot.

With Zxing library:

192x144 - No freezing

640x320 - Freezing

So sadly, it turns out the Air export for iOS is really really bad at looping through bitmapdata. I have a native QR code scanner application and it uses a very high resolution and works wonderfully... I'll do some testing on my friends android to see if I have same problems (I have a feeling it's related only to iOS).

192x144 is fine response wise, but the quality is so bad that getting the a qr code read is incredibly difficult.

[Guest]

I actually tried making an AIR app using zxing library on iOS and the application is able to detect QR codes quite well when run on iPhone and iPod. I used 640x320 mode for Camera.

But as you said the decoding function takes too long for some of the images. I used getTimer() and measured the time taken by decodeBitmap function :

var t=getTimer();

decodeBitmapData(bmd, 300, 300);

trace("Time taken by decoding function : "+getTimer()-t);

And it turns out that depending on the image it took from 20 ms to 2300 ms to process an image. This may cause the application look like freezing in between.

But I have seen many code scanning applications avoid this by letting the user first select the area where the QR code is and then press a button to actually process the code rather than scanning and decoding each and every image that is being captured by camera stream. If we go by the former approach then I dont think that the time taken by decoding function is really a problem.

Thanks,

Meet

Message was edited by: meetshah4288

[mspecht001]

I tried to use the QR code reader for a test app on my iPhone 4, but it did not work. I read this thread twice but still cannot figure out were the problem occurs. Here is the sample code which should bethe same as psoted several times here:

protected function button1_clickHandler(event:MouseEvent):void

                              {

                                        if (!cameraStarted) {

                                                  if (Camera.isSupported)

                                                  {

                                                            camera=Camera.getCamera();

                                                            camera.setMode(640, 480, 24);

                                                            videoDisplay.x = 360;

                                                            sv.addChild(videoDisplay);

                                                            videoDisplay.attachCamera(camera);

                                                            videoDisplay.rotation=90;

                                                            qrReader=new QRCodeReader;

                                                            btn.label = "Scan Now";

                                                            lbl.text = "";

                                                            cameraStarted = true;

                                                  }

                                                  else {

                                                            lbl.text = "no camera found";

                                                  }

                                        }

                                        else {

                                                  decodeSnapshot();

                                        }

                              }

                              public function decodeSnapshot():void

                              {

                                        lbl.text="checking...";

                                        bmd=new BitmapData(300, 300);

                                        bmd.draw(videoDisplay, null, null, null, null, true);

//                                        videoDisplay.cacheAsBitmap=true;

                                        videoDisplay.cacheAsBitmapMatrix=new Matrix;

                                        decodeBitmapData(bmd, 300, 300);

                                        bmd.dispose();

                                        bmd=null;

                                        System.gc();

                              }

                              public function decodeBitmapData(bmpd:BitmapData, width:int, height:int):void

                              {

                                        var lsource:BufferedImageLuminanceSource=new BufferedImageLuminanceSource(bmpd);

                                        var bitmap:BinaryBitmap=new BinaryBitmap(new GlobalHistogramBinarizer(lsource));

                                        var ht:HashTable=null;

                                        ht=this.getAllHints();

                                        var res:Result=null;

                                        try {

                                                  res=qrReader.decode(bitmap, ht);

                                        }

                                        catch (event:Error) {

                                                  trace(event);

                                        }

                                        if (res == null) {

                                                  videoDisplay.clear();

                                                  lbl.text="nothing decoded";

                                        }

                                        else {

                                                  var parsedResult:ParsedResult=ResultParser.parseResult(res);

                                                  lbl.text=parsedResult.getDisplayResult();

                                                  sv.removeChild(videoDisplay);

                                                  cameraStarted = false;

                                                  btn.label = "Start Camera";

                                        }

                              }

                              public function getAllHints():HashTable

                              {

                                        var ht:HashTable=new HashTable;

                                        ht.Add(DecodeHintType.POSSIBLE_FORMATS, BarcodeFormat.QR_CODE);

                                        return ht;

                              }

Can anybody please give me a hint what might be the problem?

[brad5151]

Here is a QR Reader ANE for iOS only. It worked great for me. Very speedy. It comes with a Flex sample project.

Its very easy to translate to Flash Pro but still if it can save anyone some time, leave your email and i'll send you the sample Flash Pro project I created.

Now I just need an android ANE!

http://code.google.com/p/qr-zbar-ane/

[mdikici]

Hello Brad5151,

I tried to implement this native extension to my project. But i couldn't compile it. Either with Flash CS6 and with command line it was not possible. I am using AIR 3.3.0.3650 on the actionscript settings on Flash Pro and i am using the AIR SDK 3.3. Could you please send me the Flash Pro example and let me know which command prompt i can use to create the ipa file. My e-mail is: murat.dikici@mspp.com

Thank you very much,

Murat Dikici

[mdikici]

Hello Brad5151, you helped me a lot. It works now as i need and it is too fast. Thank you very much.

[marciofermino]

Hello friends, please if anyone can help.

I really need to read barcodes EAN 13 and others.

In ActionScript (Flex or Air Mobile)

Thank you.

[sbhave]

I have written ANE version for ZBar [I have extended http://code.google.com/p/qr-zbar-ane/ to add more features and to support Android as well] that works on Android and iOS over same ActionScript API. It works similar to CameraUI i.e. I open Native Activity/ViewController that scans and sends the data to AIR app as an async event.

I have developed this for one of my projects. If anyone is interested revert back on this thread itself.

Regards,

sbhave

[aboutveniceY]

hello sbhave

thanks for create this ANE,it's fast and useful!

And I am wondering to know is this possible to lauch the scan without the native scan UI (the UI with a cancel button) ?

thanks for it.

Best

Yang,HS

[Guest]

Hello Sbhave,  I am also working on the similar project, would you mind to share your code for reference. Many thanks.

Best Regards

Alice

[Douglas McCarroll]

I'm also interested.