Performance issue using copypixels?

[xTLS]

I am working on an app for the iPad 3 (but this is not an iOS specific problem so I'm posting it here) which includes a fullscreen gallery in one area. The gallery mimics the iPad's native photo gallery in function; the user can swipe left and right between images.

I had noticed that running on the device, the app would freeze for a moment as it reached the edge of each photo - a small amount of lag as it loaded the next 2048*1536 photo into the display. I figured I could solve this by cutting the photos into smaller pieces with code, and that works:

for (var j:int = 0; j < parts; j++) {

          bmpD = new BitmapData(bmpWidth/parts, bmpHeight, false, 0x333333);

          rect = new Rectangle((bmpWidth / parts)*j, 0, bmpWidth / parts, bmpHeight);

          bmpD.copyPixels(bmp, rect, point);

          bitmap = new Bitmap(bmpD);

          bitmap.x = (bmpWidth / parts) * j;

          bitmap.y = (stageHeight - bmpHeight) / 2;

          mc.addChild(bitmap);

}

However, running this loop when the gallery opens causes a ~5 second delay in the loading of the gallery (made up of 8 images), vs less than a second without the loop. It causes a ~1 second delay on my PC, vs no delay without the loop. This happens regardless of how many loop iterations occur. There is no noticable difference in load time between parts = 2 and parts = 128.

I had gotten the impression copyPixels is pretty fast, but this loop takes a horrendously long time to execute. It also doesn't make sense to me that the length of time required to execute the loop wouldn't be affected by the number of iterations. Am I missing something? Or is the problem just the volume of pixels that have to be copied, regardless of how many are copied at once?

[kglad]

the speed of copyPixels is inversely related to the number of pixels copied.

[xTLS]

All right. I had thought copyPixels was fast enough to carry this out, since it is used in blitting, but I guess this is too much pixel data to work with. \

I read, and thought it made sense, that for a large graphic it's faster to copy it in pieces than all at once. However, using getTimer() to time the loop now, I see that it is not the case; the more pieces used the slower the loop executes. I get a ~1% slowdown for every ten pieces.

I don't suppose there's another, faster way of doing this?

[kglad]

there shouldn't be any need for optimization steps to display a fullscreen iPad gallery whose only transition involves x,y changes, as long as the images are bitmaps or displayobjects whose cacheAsBitmap property is enabled.

[xTLS]

Whether there "shouldn't be" or not, the issue occurs with complete consistency. I have eight 2048*1536 jpegs loaded from the libary as BitmapData and added to Bitmaps that are added to the stage. The Bitmaps are only translated horizontally. When swiping the screen between photos, on the last frame before the next photo is displayed, the app tends to freeze noticably.

It appears to be a caching problem, as it generally won't happen in galleries of four images or less, or while swiping back and forth between the same three images or so. The app is running in GPU mode, if that makes any difference. It looks to me like either AIR or the iPad 3 can't reliably cache more than about four 2048*1536 images at once; perhaps a problem feeding the image data from memory to the GPU?

[kglad]

that's 96mb of ram for those 8 images.  what ram is the rest of your app using? and are you "disposing" of bitmapdata that no longer needs to be in ram?

[xTLS]

Yes, I've been managing memory very carefully, as there is a large amount of high-res imagery in the app. I've made surel to dispose bitmaps, disable events, and do all necessary preparation for garbage collection; however, this issue occurs even when the app is first opened, where only the main menu graphics have been loaded. There are no stutters while swiping once the gallery photos are cut into pieces, but the 5 second loading time is unacceptable.

I apparently can't profile the app in Flash Builder because I used CameraRoll, and no one will explain how to get around that, so I don't know the exact memory usage. There are no performance issues anywhere else.

[sinious]

You could try to get into the Adobe Pre-release team for Project Monocle (a new high end profiler used in FB4.7):

https://prerelease.adobe.com/callout/?callid=59A4F95D908A4D2FB9F302729FD0D87F

Aside that I still use blitting with copyPixels for some older AIR apps prior to Stage3D. I don't admire your position due to RAM. You definitely should be using a Vector for efficiency to hold cached images and never cache more than what you need. Note that the iPad/iPhone gallery stops for a moment when you swipe. In native there's also some auto-mipmap action. That means a low quality version of what you're loading is initially loaded and then the full quality is tiled in. That reduces the memory footprint to a full single image as well as 2 lower quality zoomed out siblings.

Objects have overhead and cutting them into pieces adds more and more overhead the more you do it. That's why you'll get a performance degredation more than anything. copyPixels works at memory speed. The more you do it the more you waste cycles.

If you're truely blitting then you're merely using memory and CPU, there would be no GPU texture upload lag because the memory isn't uploaded to the GPU except for specific situations. For instance, be weary of cacheAsBitmap on a mobile device which is memory greedy. When you start using the GPU and actually creating true textures to upload to the GPU you'll get that behavior, which is synronous (app halting) performance, but not with just loading bitmaps from a library and copyPixeling them into a BMD.

At this point, while GPUs are gaining more and more memory and without native code to thin you out, you may have to approach your gallery differently for a short time. See if you can approach it like the built in gallery. Bundle lower quality images to preload quickly for blitting. Load studders will disappear and there will only momentarily be a lag while the true image is loaded, ready for zoom/pan. Keep the lower quality photos in a vector so there's no need to load and your memory footprint will be tiny and performance will increase.

Ultimately, move to Stage3D. Blitting isn't the way anymore. Check out Starling if you haven't already.

[xTLS]

Sinuous: Thanks for the info. There seems to be a misunderstanding: to be clear, I'm not really blitting, as there is no animation involved. The code above that copypixels the gallery images into pieces is only run once. I didn't see any need to use Stage3D, as this is a business app, not a game, and contains no 3D content. Performance everywhere else has been excellent, even when scrolling 60+ smaller images or running realtime color filters on large graphics.

If the app is run in GPU mode, don't all on-screen graphics have to be uploaded to the GPU?

kglad: All right, thanks, I'll test that. Is that a confirmation that you can't profile an app that uses CameraRoll with the standard FB profiler?

[kglad]

i'm not familiar with fb so i can't answer that.

[sinious]

I'm not suggesting you use 3d, Starling is 2D but leverages the huge performance gains of Stage3D (GPU). They also have a open source framework where some components are being re-written to use Stage3D (via Starling) simply because it performs so well (FeathersUI).

The benefit of a profiler is telling you what is using that memory. It's nice to know you're using 98MB but knowing 45MB are improperly deallocated references of bitmapdatas left behind as kglad mentioned above is much more useful.

The drop-down for GPU/Direct/None is almost superfluous at this point. The compiler could and probably will handle this option for you in the future. Or the IDE may do it as it much easier. The setting is a way of making sure things are available if needed but it does not guarantee what is running is always on the option you select. For example a freshly loaded bitmap that is not on the display list has no business consuming valuable GPU resources just because the GPU setting is enabled. Blitting is primarily aimed at pure CPU+Memory. The objects on the display list will be handled behind the scenes by Flash, probably with the GPU.

If you're using the flash display list to animate objects then you'd be suprised what blitting could do to reduce memory usage and speed up the display, even to this day. But again, check out Starling linked above. That's the most bang for your buck.

[xTLS]

I appreciate the suggestion, but the Flash display list is more than adequate for translating and scaling images for this app, other than in the gallery, and in any case it's much too far along to transition the entire project over to Starling just for one image gallery.

The benefit of a profiler is telling you what is using that memory. It's nice to know you're using 98MB but knowing 45MB are improperly deallocated references of bitmapdatas left behind as kglad mentioned above is much more useful.

Yes, that's why I wanted to use the FB profiler. I'm looking into Monocle right now.

[sinious]

It could take a while to get in so in the meantime you can take a reading of the memory usage before and after you load your gallery to get a basic idea of the extra stress.

Starling is definitely a future recommendation. While Starling greatly simplifies getting into Stage3D, it's not a walk in the park. I recently submitted a bug fix request that fixed an issue starting/stopping Starling so it can actually just be used for smaller portions of a larger application. Just food for future thought.

[xTLS]

It was easy enough to get into the program, but I'm tired of spending an hour+ reinstalling Flash Builder versions, the Python script they claim is in "the package" to convert existing SWFs to Scoutable SWFs doesn't exist as far as I can tell, and a version I found online (after fixing the typos in it) gives me a certificate error or something like that; I need to double check it tomorrow.

Reading more about Starling I am very interested in its features for upcoming projects. Good stuff.

[sinious]

It's great but it's open source so sometimes an issue critical to your deadline won't be fixed rapidly, especially if it's extremely situational. That recent issue I submitted that was fixed wasn't entirely fixed. I submitted a new issue linked to a post with more information. Starling is still having a small issue starting, stopping and restarting again (something you'd do if you just want to use components).

https://github.com/PrimaryFeather/Starling-Framework/issues/208

Give it some time, they're really good about fixing critical bugs and Adobe works with them.

[kglad]

in addition to using monocle (now called scout), you can simply use System.totalMemory to test flash memory consumption on your development platform.  memory consumption will be the same on the deployment (iOS) platform as your development platform.

to pinpoint your problem, if you find one, scout or some other memory tracker can help.