Using StageVideo and Stage3D in AIR - BETA

[chris.campbell]

We're planning on introducing a new ActionScript feature that will allow hardware accelerated video to be used as a source texture in a Stage3D environment.  Currently, using video with Stage3D requires the use of the Video object, which is not accelerated, and manipulation of the bitmap representations of the video frames.  The planned feature, called VideoTexture, will allow direct access to a texture object that is sourced from a Netstream or Camera object.

The following sample code demonstrates the use of the VideoTexture object.

1) Create a VideoTexture object and attach a NetStream (or Camera) object to the VideoTexture object:

var ns:NetStream;
var context3D:Context3D;
var texture:VideoTexture;
texture = context3D.createVideoTexture();
texture.attachNetstream(ns);
ns.play("video.3gp");
texture.addEventListener(VideoTexture.RENDER_STATE, renderFrame);

2) A texture representation of the current video frame can be retrieved from the callback function for the VideoTexture.RENDER_STATES event.

function renderFrame(e:Event):void
{
    // Render on stage3D with VideoTexture
}

Please note that this will be an "extended beta" feature.  The initial implementation, available  in our AIR 15 beta, will be for Windows/AIR only.  We are committed to expanding this feature to AIR mobile and Mac in a following release.  We'll also consider an implementation in Flash Player if there is sufficient demand.

Getting your feedback will be critical to making sure we're providing a solution that you can use.  Please let us know what you think and keep an eye out for updated beta release notes in the next couple of weeks.

[Colin Holgate]

That might be nice, and I know at least one huge fan you're going to get! I'm going to post this link to a prerelease forum, expect to be inundated with praise.

Meanwhile, two things that would be useful, while you're working on these things, would be to have better support for StageVideo for playing local files (we had to give up in the end, and use FLVs and Video object), and, find a way for Stage3D to work in GPU mode, just like StageVideo and Stage Text does.

[Infin8]

It would be great if StageVideo could be progressed so the current bugs are eliminated. Especially for Air/Android. I have a working StageVideo AIR app on iOS which fails miserably on Android (black screen issues).

Perhaps this new Stage3D implementation might solve these issues???

[KubaBlade]

Great news Chris!!! If you put this into mobile as well I will start my new game, because it needs this feature

Keep the good job guys! Flash rocks no matter what "they" say!!!

[zeh]

That's great news, and I like the syntax. Thanks Chris.

However, I'd like to also ask what's the status on overlaying a Stage3D context and StageVideo. That alone would work wonders, without the need for any kind of texture support. I know that used to be supported a while ago - contexts could be "transparent" - but apparently it was dropped a year or so ago (it was under test I suppose). I can't find the original links for that unfortunately.

To share my perspective, I've recently had a project where I had to overlay both under several different circumstances and my solutions were two fold (one for each case):

1. Dump every frame into a texture in real time. This was needed because I was masking a texture. This was less than ideal because copying a bitmap on every frame takes a lot of CPU resources, so the video quality suffered (I had to downscale it).

2. Have StageVideo occupy part of the screen, and Stage3D occupy another part of the screen; they never overlapped. This also worked, but it meant never overlaying elements... the elements that needed to cover both had to be made using the old Flash display list. So it's a little bit of a crazy solution with lower performance than ideal.

So both were less than ideal but would have worked better if I could overlay stage3d/stagevideo.

[Colin Holgate]

I think that having hardware decoded video as a texture in Stage3D indirectly solves the problem of wanting transparent Stage3D over StageVideo. You only have to connect the net stream to your back most texture (or camera background if that exists in Stage3D), and you have what you wanted.

[zeh]

Yes, but "indirectly" being the keyword. It's better but it still sounds like more hops than necessary to me.

I guess I just never quite understood why the context overlaying feature was added only to be removed later.

[mikatalk]

This is great news! Updating the texture on each frame works for me but this seems so much nicer! Can't wait for it on mobile!

[Jason_Villmer]

This looks promising Chris.

---------------------------------------

Mobile + H.264 + Away3D

---------------------------------------

Currently, we cannot use H.264 video files as textures on 3D Objects (using Away3D, etc) for MOBILE apps.

I am very curious, and hopeful, whether this feature will resolve this. Both I (and Andy Dufresne) have been trying to get support for this for quite some time.

Best Regards,

Jason Villmer

Andy Dufresne

[Jonathan_at_Idle_Games]

I was hoping you'd find this thread

[itlancer]

It is important feature for combining GPU accelerated video and Stage3D interface/content. Waiting for release!

We need it on all of  desktop and mobile platforms.

[Neverbirth]

Superb. For me, this would cover the needs I've had in the past.

[aw4kening]

Cool! It is a necessary feature. With H264 we will have a smaller size, better quality, and higher performance 😃

[ozmachine]

There are so many nice possibilities with this feature.

[millingcutter]

So, what about FlashPlayer in this case?

[Jason_Villmer]

@ Chris

First, I want to thank you and Adobe for pursuing this feature. I have read what you've written but I was curious if you could clarify a few things.

1. Initial support will be for:

     Yes - Desktop AIR for Windows

     No - Desktop AIR for Mac

     No - Mobile AIR for Android

     No - Mobile AIR for iOS

     No - Flash Player

     ** I have a Macintosh computer and want to make mobile apps that utilize this feature.

2. Will this feature allow us to use an H.264 video file as a texture on a Away3D object? (in AIR Mobile)

I've been trying to get support for this feature for several years now.

Here's my thread at the Adobe Bug Base: Bug#3744843 - H.264 video + Away3D + AIR mobile = Does not work

Best Regards,

Jason

[MyOwnStartScreen]

Finally a dream comes true! Thanks to all people which are involved to make this happen! Number of AIR possibilities squared -> BAAAMMM

[chris.campbell]

Thanks for the feedback so far.  To clarify, our initial support is for Windows AIR only.  This will be an "extended" beta feature and once the v15 release goes out in September we'll start our work on the Mac and mobile (iOS and Android) AIR versions.  Based on your feedback, once the AIR side is done we'll take a look at what will be required to move this over to Flash Player.  Do you see strong use cases for this on the Player?

[millingcutter]

Yes, I see. It's good choice to overlay StageVideo by stage3d-based ui's to control media playback. Personally, I use Feathers. And now i can't do it this way.

[Alexandre Madurell]

+1

[clustaSean]

Hi Chris, I would like to integrate this into a projection mapping tool I'm building (running on a powerful windows desktop) and would be happy to provide feedback on its performance, stability etc. Any idea when this extended beta version will be available? Oh and thanks for all the hard work on the flash runtimes! 

[Infin8]

Will this new feature include the facility to control the movie and listen for events?

e.g. Play, pause, stop, rewind, seek and Event.READY, Event.COMPLETE?

If so that would make it an extremely useful addition.

[Colin Holgate]

StageVideo just uses netstream, which you can control. I don't think it will matter that the video is being used as a texture, you should still be able to control the netstream.

[Vyacheslav Alferov]

Hi! Please add this bugfix/feature not only to the AIR, but in web player too. I need it for my Starling+Feathers videoplayer that I can't use now. I make it for the web, tot for desktops.

[Jason_Villmer]

Hmm. Seeing how I'm using a Mac and primarily want to make mobile apps, it looks like I'm going to be waiting a bit until I can start working with this.

I want to repeat one question here as well - Will this feature allow us to use an H.264 video file as a texture on an Away3D object? (in AIR Mobile)

Thanks again for moving in this direction, I'm very hopeful!

Anyone interested in this can also take a look here: https://bugbase.adobe.com/index.cfm?event=bug&id=3744843

Thanks Chris & Adobe!

Jason