How do you use AIR's WebKit/htmlloader?

[chris.campbell]

Hi everyone,

A long standing issue in AIR has been the inclusion of an older version of WebKit.  The request to update this library has come up many times in the forum and is in the top 10 on the community driven Uplist feature page.  As with the recent and ongoing physics discussion, we're not committing to any changes purposed below at this time,  as we're purely in an investigation mode at this time.  We realize that this is an important feature and we need further clarification on what you're looking for.  Please read on for questions from our development team.

We are exploring updating WebKit, but due to our modification of the WebKit source, this will be difficult, and updating WebKit will change the HTML DOM, possibly breaking content authored for our existing DOM.

So we are also exploring leaving HTMLLoader alone, for now, but providing a reasonable alternative.

StageWebView was originally written as a replacement for HTMLLoader on mobile (because we could not use our WebKit on mobile, StageWebView took advantage of the browser provided by the platform).

But it was extended to the desktop as an unsuccessful solution to this problem.

On mobile, content was probably newly written (so it could be tested with StageWebView), and the browsers were similar enough to our WebKit that the DOM impact was minor.

But on the desktop, forcing older content written for our WebKit to run on Internet Explorer 8 (as an example) was a disaster due to the differences in the DOM’s. We resolved this by making HTMLLoader versus StageWebView on the desktop a choice.

Which gets to the questions (for desktop development only).

Which is more attractive, an embedded web browser or using the system browser? Is it valuable to provide both?

For instance, using the native browser can save on code size (perhaps 6 MB), but you must create and test portable HTML, and you face the risk that future updates to the system browser breaks your content.

How much interop do you need between AS and JS?

Hearsay suggests another problem with StageWebView is there is no interop between ActionScript and JavaScript, whereas HTMLLoader had a lot of support.

Would StageWebView be sufficient if it exposed an ExternalInterface, or sandbox bridge, level of functionality? For instance, if AS could register a list of functions which could be called from JS (and vice versa), would that be enough? Or is there something else that HTMLLoader does that is essential?

For instance, one of the ideas being explored is to provide an entirely new class (perhaps as an ANE) which links an unmodified version of the latest WebKit source as a static library. By using unmodified source, we can more readily update to newer versions of WebKit.  As well, if we leave the existing classes unchanged, we don’t risk breaking existing content. But if we use unmodified WebKit source, we may find some of HTMLLoader’s functionality impossible to match, which is why I’m interested in understanding the essential functionality, so we can decide if a sufficient, minimal (so it’s easier to support without customizing Webkit) interface for it.

Thanks,

Chris

[FredericCox]

Hi Chris,

I am currently working on a project which led me to upvote this issue. I am loading in a URL using HTMLLoader (the mx:HTML control) and after the HTML is loaded I can manipulate the DOM. So I can hide specific parts of the loaded webpage. This is working fine only I am facing an outdated version of webkit so new HTML5 functionality does not work well.

If you can expose the DOM via the ANE and allow functions to alter it this would be fine by me.

What I think is essential:

- locationChanging event (with support for #)

- locationChange event (with support for #)

- complete event (html page loaded)

- access to DOM to show/hide specific elements and/or rewrite/replace the source code of the webpage to see immediate effects (this we use for a WYSIWYG editor to preview changes), like domWindow.document.getElementById, getElementsByTagName, getElementsByClassName

and to be able to do things like:

foundElement = browserHTML.domWindow.document.getElementById("content");

  if(foundElement) foundElement.style.width = (this.width-36) + "px"; //take scrollbar into account

- location property for the current page (with support for #)

[Jeff__Ward]

My primary use case for a web view in AIR would be to open "accounts / settings / management" pages that are already available via the web to my users.  In this case, layering on top is fine, and ExternalInterface is very useful.  A recent version of webkit (to guarantee proper page rendering) is the biggest need.

So I'd be happy with an up-to-date and embedded (to guarantee cross-platform consistency) webkit providing StageWebView with an ExternalInterface bridge.

If I recall, HTMLLoader as a DisplayList object could be layered in the displaylist and rendered to Bitmap, which is cool, but I'm guessing is part of what made it complex.  But if HTMLLoader is staying as-is, then it could still be used for these use cases.

Aso if I recall, with StageWebView you can hack interop with location change events - ugly, but it provides basic communication capabilities.

[omuppi1]

StageWebView exposing an ExternalInterface would be perfect for a lot of things.

Exposing DOM elements would be even better (everything Frederic Cox mentioned above looks very interesting.)

[Muzak]

Pretty much what FredericCox said. It gets harder and harder loading html5 content with webkit being outdated and AIR apps that used to work fine when developed and first released keep breaking after a while when the (3rd party) loaded content is updated.

I was actually hoping to see webkit be replaced by something like chromium.

[GoldHolden]

What would be good would be to update the HTML loader but also to introduce a standalone library that could load modern web code and be manipulated within the runtime. External interface was nice originally because of the type cohesion between JavaScript and actionscript. So passing objects worked seamlessly etc. My understanding is that it should even be possible to cross compile something like chromium using flascc and actually include it directly in a project as an ABC library. The interface that's written in swig would probably determine the available capabilities. In theory though it would be a game changer to be able to create a browser object that uses latest sources and manipulate it like any other object in actionscript. This would also allow you to use HTML tools like edge along side actionscript in a given application. That is obviously a business value not to mention a way to combine technologies which was the original intention of Adobe's contribution of avm2 for tamarin.

[nemenv]

I would love to see the HTMLLoader updated but would be fine with a new HTML object - what is super important for me is the AS-JS integration. My app is tightly integrated with the HTMLLoader with code like the following :

// using jQuery to set element text
_htmlview.window.$("#mydiv").html("some text")

// getting an element's width
var nodeWidth:Number = parseFloat(_htmlview.window.document.getElementById("mydiv").style.width);

or registering its own handlers on an observer in the JS:

_htmlview.window.MyObserver.addListener("elementSelect", onElementSelect);

This allows me integrate with the DOM and draw UI above the HTMLLoader.

As for features I find myself missing:

• SVG
• CSS calc()
• element.classList API
• matchMedia
• HTML5 Audio/Video

[Harry248]

StageWebView not only needs some care on desktop. There are a lot of (critical) bugs (like #3572986) in the mobile version too, which should be fixed first before adding new features.

Despite that - absolutely must-haves for a new StageWebView / HTMLLoader:

- bullet-proof HTML5 Audio/Video support

- (easy) JS / AS Interface

- Touch/Gesture recognition (it's a shame that we are not even able to recognize touches on the Webview)

Thanks for listening Chris!

[Santanu Karar]

Hi Chris,

I have my problem/question about using WebKit for desktop AIR project. I don't know if there is any solution exists to my problem, but I'd like to share those thoughts with you regarding WebKit usage in AIR desktop.

As per How to publish your AIR application to Mac App Store (guide) - Pigsels Media , we're packaging desktop AIR application to submit to Apple App Store. The problem comes when we have WebKit usage (HTML) in the application. As per the article, one can not include WebKit classes while packaging for App Store, as Apple would reject the app.

1. Now, it’s time to make the magic: remove webkit.plugin from your bundle. This, of course, means, that you will not be able to use HTML rendering in your AIR application. The reason behind this is that Apple will refuse your app with WebKit plugin in it, because this plugin itself contains calls to undocumented Mac APIs, which is not tolerated by Apple.

Fortunately an AIR application will work just fine without WebKit if it has no use of HTML rendering.

In the kind of cases, WebKit seems not a solution at all. Our .pkg installed executable fail working with HTML component if we removes WebKit while packaging. I am also not sure what if we have StageWebView in place of WebKit component to AIR desktop, maybe we could cut down these complexities in packaging then

Thank you!

[周戈]

Hi Chris,

I'll straight up to the point:

1. StageWebView desperately needs two key features:

1.1. Add an autoSnapshot:Boolean property to the StageWebView Class, so that when setting autoSnapshot=true and the StageWebView is not currently interacted by user, StageWebView can automaticly draw its viewport to BitmapData, and automaticly add the Bitmap to DisplayList, in the meantime automaticly hide the true webpage view. This feature will make our jobs way more easier than before!

1.2 Add ExternalInterface support to StageWebView.

2. About the AIR's WebKit, well, I think it just needs to be updated to the latest version of WebKit, the current version is so out of date.

I'm very appreciated to all the great works of Adobe AIR Team, wishing it could go really far

DarkStone

2014-06-12

[Neverbirth]

Updating the embedded Webkit engine would be extremely useful. Also, StageWebView is severely lacking in a lot of areas... you can get an idea by looking at this commercial ANE:

iOS UIWebView and VideoPlayer Native Extension for AIR mobile | Darkredz Studio

It would be a shame to make the author work unneeded, but I think StageWebView should be improved and be on par with its native counterparts.

[cosmacol]

1. Support for osx retina display

Currently, I use HTMLLoader to implement hybrid Flash/HTML desktop application, and even configuring the descriptors files correctly, the HTML content will be rendered in compatibility mode (upscaled 2x) - it is very noticeable alongside with the Flash-based content renderered in native resolution.

2. Improve HTML5 compatibility

Many authoring applications (e.g. Articulate) are able to create HTML5 packages, but that content often cannot be rendered by the HTMLLoader, usually due to the lack of support for recent standards, such as WebFonts and SVG rendering. This is especially useful for Learning Management System solutions, or in general when external resources need to be rendered.

Thank you for asking,

Cosma

[mpguerin]

Hi Chris

We currently use the mx:HTML object in our desktop AIR application to load in a variety of web content. Some of this content we generate ourselves contains links which we capture using the locationChange handler and feed into the application to make various bits of the application work.

override protected function htmlLoader_locationChangeHandler(event:Event):void
  {
  if(htmlLoader.location && LinkUtils.doWeHaveAValidURLLink(htmlLoader.location))
  {
  event.preventDefault();
  LinkUtils.actionValidURLLink(htmlLoader.location);
  // set the location back to the one used before so the same bug link can be 
  // clicked twice in a row and this function still gets called.
  htmlLoader.loadString(htmlText);
  }
  else
  {
  super.htmlLoader_locationChangeHandler(event);
  }
  }

Key things we'd require for our application to continue to work are as follows:-

1) Access to a locationChange/locationChanged handler (We had to make the handler protected from private to enable our code to work as required)

2) Capacity to pass an HTML string containing the markup to display, rather than pointing the component to a URL.

3) Access to the internal htmlLoader  to call the loadString() method, or similar.

Desirable features for a new WebKit/HTML:-

1) SVG Graphics

2) HTML5 Audio/Video

3) Proper displaylist support.

4) Access to complete handler for page loading completion.

Getting support for HTML5, Video and better rendering of modern web pages is huge for my team and our application. I'd be willing to participate in any testing or beta programme that Adobe would offer here.

Thanks,

Marc

[Harbs.]

Pretty much every time I've used HTMLLoader, I've needed to do HTML manipulation using ActionScript. So js to as interop is pretty important to me.

I agree that it makes sense to at least consider chromium and possible CEF. The idea of combining AIR and CEF capabilities sounds like an interesting one to me, although I'm not sure off-hand how I'd use that.

Harbs

[Guest]

For what little I do with the HTMLLoader object, just being on the latest version of Webkit so I can have HTML5 video and audio would be enough for me. I also heavily use the events that HTMLLoader offers to make state changes to AS UI elements. On the concern of updating the DOM breaking existing content, correct me if Im wrong but it would only have the potential to break in an app if the app was updated/re-published. Hope this was helpful.

Edit: Also, just want to say thank you for considering updates and asking the community what we would like to see.

[practical_code1549]

I'm 100% in support of ditching embedded WebKit for the size savings, and adding 2-way communication for AS/JS in StageWebView.

Another option would be to just give us a compiler flag so that we can strip webkit manually when packaging.

Thanks for reaching out Chris! This is long overdue, but awesome that it's finally (maybe) happening.

[Neverbirth]

An advantage of the embedded webkit engine is that it allows content to be rendered as a DisplayObject. It could be achieved with StageWebview, but would require more resources.

I guess when compiling IPAs and APKs the embedded engine is already stripped, isn't it? if it isn't, it should.

[goldysilva]

Updating the embedded Webkit engine would be extremely useful.

I agree with FrederickCox. I am also working on a desktop app that needs to have the DOM exposed also a newer version webkit that loads modern sites.

ANE with the ability to update Webkit sounds good. I am not to keen on using the system browser for reasons mentioned by you.

What I think is essential (From FrederickCox)

- locationChanging event (with support for #)

- locationChange event (with support for #)

- complete event (html page loaded)

Also a closer interop between AS and JS. Example getting JS events like mouse click and knowing which DOM element was clicked, image, button, text etc.

Also locationChange event should also happen when a link on the page is clicked and not only when the 'location' property is set.

Also why even have a complex bridge/sandbox system in the new proposed html component? Why cant the developer add in her own security...

Many sites use 3rd party scripts from Google and other places (eg. jQuery from Google loads faster because most likely it was cashed on your machine before), not

being able to load sites that cross script disable some of their functionality.

I have read the other entries and I think most of us want this. Creating applications with AS capabilities and a "real browser" can prove to be very powerful.

Thanks for even considering this.

[Sascha Ahrend]

What is critical: HTML loader comes with a built-in XSLT-processor.

Please update it to also support EXSLT. This has been done by webkit browsers in the last years.

By this we would have a fully usable XSLT-processor in Adobe Air.

I would generally prefer an updated version of the HTML loader, even if this implies changing the DOM.

(There is still the option of using an older version of the SDK for business critical cases.)

As a general principle: Consistency over configuration.

I don't won't to rely on the user's system configuration to ensure my app is working.

Thank you!

[dlldlldlldll]

Glad to hear that Adobe finally going to update the webkit or proprocionar an effective means of embedding an updated browser in adobe air, the greatest need is that the interaction between AS and JS, and to integrate web content into applications without the need to use native client browser, most of my client use windows, with lamentable internet explorer, so the need to embed an updated browser in Adobe Air

One alternative that I thought to do was use the http://www.awesomium.com/download/ and create from it an ANE.

But I'd rather wait.

[practical_code1549]

I guess there's 2 main use cases, so the most important thing is that the solution can serve them both:

1. Mobile developers who need the smallest possible APK/IPA size, and can rely on StageWebView for any web needs.

2. Desktop/Mobile developers who need a modern embedded browser.

In both cases, we also need at least some basic communication tools between AS and the page itself:
1. Ability to access the dom.window element, to add event listeners or call global JS functions.

2. Improved event's as mentioned above (locationChanging, locationChanged, complete) with support for #

[flashdictionary]

Chris,

In our view, the embedded WebKit provides a competitive advantage to other HTML engines because of its exactness and consistency across the desktop platforms, and the power to operate with native code on the HTML Elements. In regards to architecting an update, the following are attractive to us:

* Ability to access and manipulate the DOM as ActionScript objects.

* Ability for ActionScript to call document.getElement()... and automate the click of a button, and enter text into a textfield.

* Interoperability and consistency across Windows and Mac OSX. Never have to worry about "browsers" and their versions.

* As a native display object, we can create multiple instances, and we can layer the display as we desire.

Things that we do not care about:

* An ExternalInterface to the JavaScript in StageWebView defeats the whole purpose of the WebKit which is in our minds the consistency and native manipulability.

Architecturally, we hope that Adobe does the following with the WebKit:

* Keep the HTMLLoader class as is. The class is already written, and works the way it was intended, which can still provide value to legacy applications that currently use it.

* If a desktop app has no references to HTMLLoader, and is compiled with captive runtime, then we want the captive runtime to exclude the old WebKit.dll (4,663 KB)

* Add a new class to the runtime (perhaps called WebKitDisplay or WebKitLoader) which contains today's version of WebKit. This new class would provide access to the DOM and the ability to getElements as native ActionScript objects.

* Regarding a new and separate "WebKitDisplay.dll", we want it to be included with the captive runtime only if there are references to it.

[pierBover]

What I'd love to see, is an easy way to load local files in StageWebView. There is content that is easier to simply produce in HTML instead of AS3, and being able to bundle that with the app (without any hacks) would be great.

Another feature would be to be able to communicate between JS and AS3.

[Maxijin2]

Hi Chris,

Thank you for taking care of community requirements.

I'm primary using AIR with starling + feathers to develop apps ! and here are requirments:

- Show googlemaps with StageWebView in starling app

- Play videos Youtube, etc with StageWebView in starling app

a deep integration with stagewebview where we can communicate from as3 to htmlcontent and vice versa is all we need. and it must be crossplatform.

Thanks

[Jeff__Ward]

I'm assuming that ExternalInterface is easy for Chris Campbell & team to implement, while deep AS <-> direct DOM manipulations are much more effort.  If that's not true, ignore this post.  😃

But I honestly don't get why people want deep actionscript integration to the DOM.  Maybe for convenience sake (though traversing XML structures is hardly ever convenient!), but if you can directly call JS via ExternalInterface, what more do you need?  I mean, interact with the DOM the way it's easy, using jQuery (or other js framework), and hook yourself up some JS->AS event bindings like so:

  // FYI - untested pseudocode, but I've used similar:

  static public function js(code:String):* {

    return ExternalInterface.call("eval", code);

  }

  static public function js_bind(selector:String, event:String, func:Function):void {

    ExternalInterface.addCallback(magic_name, func);

    ExternalInterface.call("eval", "$('"+selector+"').bind('"+event+"', fancy_magic_to_call_as_func);");

  }

  // Then just write/call javascript from your AS3:

  js("$('.some_class').css('display', 'none')");

  js_bind(".button", "click", function() { trace('HTML button clicked!') });

FYI, I'll fill in the magic and the unbind cleanup when I have some free time.