Legend

Question

Best Cross-Platform App Framework to use to support plugin.

Forum|Forum|14 years ago
February 27, 2011
6 replies
6142 views

It just occurred to me that a better way to have done DevAdjust (relative develop adjustments and presets), which suffers from lack of non-modal accessibility, would have been to create an application which presents the UI and talks to a background task in a plugin (via file IO), to apply the develop settings. In fact, many plugins could benefit from a distributed design, off-loading to real apps the things that they are best at e.g. non-modal UI, and interfacing to plugins to do the dirty work, e.g. writing the catalog.

Anyway, I was just wondering if anyone has any idea what would be the best platform for such an app these days. Flash/Flex/AIR?, Java?, dotNet?, Silverlight?, wxWidgets?, Qt?, Eclipse RCP?, ZooLib?, ...

I'm starting to favor wxWidgets with a wxLua binding for a plugin's app counterpart, here's an example app:

frame = wx.wxFrame(wx.NULL, wx.wxID_ANY, "wxLua Minimal Demo",
                   wx.wxDefaultPosition, wx.wxSize(450, 450),
                   wx.wxDEFAULT_FRAME_STYLE)

-- create a simple file menu
local fileMenu = wx.wxMenu()
fileMenu:Append(wx.wxID_EXIT, "E&xit", "Quit the program")
-- create a simple help menu
local helpMenu = wx.wxMenu()
helpMenu:Append(wx.wxID_ABOUT, "&About",
                "About the wxLua Minimal Application")

-- create a menu bar and append the file and help menus
local menuBar = wx.wxMenuBar()
menuBar:Append(fileMenu, "&File")
menuBar:Append(helpMenu, "&Help")
-- attach the menu bar into the frame
frame:SetMenuBar(menuBar)

-- create a simple status bar
frame:CreateStatusBar(1)
frame:SetStatusText("Welcome to wxLua.")

-- connect the selection event of the exit menu item to an
-- event handler that closes the window
frame:Connect(wx.wxID_EXIT, wx.wxEVT_COMMAND_MENU_SELECTED,
              function (event) frame:Close(true) end )
-- connect the selection event of the about menu item
frame:Connect(wx.wxID_ABOUT, wx.wxEVT_COMMAND_MENU_SELECTED,
        function (event)
            wx.wxMessageBox('This is the "About" dialog of the Minimal wxLua sample.',
                            "About wxLua",
                            wx.wxOK + wx.wxICON_INFORMATION,
                            frame)
        end )

-- finally, show the frame window
frame:Show(true)

-------------------------

PS - I downloaded wxLua and had this little app up and running in seconds. Its kinda bare-bones/simple&easy, but seems to work well, and there's gotta be some advantages in using the same language for the app as the plugin. On the other hand, Python has more code libraries readily available to support app dev. Anyway, it seems to make more sense to me these days to use a scripting language with cross-platform native UI support for desktop app dev than a traditional programming language like C++/C# or Java, and it seems to make more sense to use a true OS app rather than Flex or Silverlight which are more oriented toward graphics and internet deployment. Not sure what ZooLib would do, and Qt looks nice but contrary to what I read in one place, its expensive.

Any thoughts?

Thanks,

Rob

This topic has been closed for replies.

areohbeeAuthor

Legend

Status update...

The good news: I have a really nice messaging system built around xmlrpc. Both app and plugin have sendAndReceive functions, and both can listen for unsolicited messages from the other. So, its possible to have either end be master <-> slave, or any combination that suits the fancy of the sidekick-app/plugin designer. This code is unfinished but will eventually be available at the Elare Plugin Framework site.

The bad news: Although It is easy to extract the images from the previews once you know the preview id, I haven't found a way to get to the preview id corresponding to a photo without going through the lrcat file, which is locked when Lr is running. At the moment I dont have any idea how to scale this wall... I previously thought the uuid leading to the preview file (lrprev), which is the first part of its filename (and is embedded in the header of the file...), was that of the photo to which it corresponded, but sadly - I was wrong.

UPDATE: I've now got some ideas how to scale this wall, but I decided not to post them. PM me if you want to talk about it.

Rob

areohbeeAuthor

Legend

Update:

I have an app / plugin prototype pair up and limping that uses xml-rpc for communication.

What is working now:

- Plugin can send commands to app and receive reply.

Here's what I'm thinking about for the future:

- App can also send commands to plugin and receive reply.

Plugin solicits these commands via an XmlRpc client request, but conceptually, they are initiated by the app, so app will just have:

- send(Command)AndReceive(Reply)

- processing of incoming commands happens by way of registered rpc functions.

And plugin will have:

- send(Command)AndReceive(Reply)

- sendReply

- processing of incoming commands will happen when app responds to the plugins solicitation with a command instead of a "none at this time" response.

Also, preview image reverse engineering is more-or-less complete (it turned out to be quite easy - with info available on the net...)

So, first proof-of-concept / prototype app will be a thumbnail / image display app that includes folder navigation. Folders / files selected in Lightroom will be simultaneously selected in app, and vice versa.

I will be writing the app side in wxPython.

As far as reusability, I suspect some reusable stuff to grow from this, but its not my present objective to make a formal spec...

Rob

Anonymous

I tried HTTP based communication in the past, but I had to change direction because open HTTP connections block Lightroom from exiting. And I really need communication from the app to plugin.

Currently for my Lightroom + iPad remote controller plugin (lrpad.com), I am using file based approach. The plugin polls file and app writes it when it has something to be said. This allows Lightroom to cleanly exit - BUT my app does not know that (i never receive call to LrShutdownPlugin except when uninstalling the plugin).

There is a private remote communication API used by the tethering functionality, but it is not documented. At least on Windows, it uses named pipes. I managed to get it to start my app and message was received, but I haven't yet figured out how to write anything back to plugin. On the Mac side, I don't even know what it uses to communicate... I hope the interface gets published at some point.

And BTW, I'm writing the app in Lua. Using alien for platform specific features, such as reseting idle timeout and keyboard events.

ps. how do you actually obtain the image preview, any links?

areohbeeAuthor

Legend

Hi muhmu,

Your post was quite a lot for me to digest.

So if I've got this straight, you have an app running on the same machine as Lightroom, that talks to both the iPad and the lr-plugin, right?

And you could not exit Lightroom when using http to communicate app <--> plugin? Hmm, that's odd. I dont have such a problem, but then I think the default for http is to make a new connection for each exchange - were you somehow keeping the connection open?

Oh, I just read the default for HTPP/1.1 is a persistent connection - still I've had no trouble exiting Lightroom whilst doing xml-rpc. Or, am I not understanding the issue?

Yeah, the pipes would be best if anybody knows how to do it...

Thanks,

Rob

areohbeeAuthor

Legend

XmlRpc is a really nice way to have interprocess communication between a plugin and a Python app. This python server app implements the check for newer version compatibly for plugins that use the Elare Plugin Framework (tested - works):

-----------------------------------------------------------------------

from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler

# Restrict to a particular path.
class RequestHandler(SimpleXMLRPCRequestHandler):
rpc_paths = ('/RPC2',)

# Create server
server = SimpleXMLRPCServer(("localhost", 8000),
                            requestHandler=RequestHandler)
# Register an instance; all the methods of the instance are
# published as XML-RPC methods (in this case, just 'updateCheck').
class MyFuncs:
    def updateCheck(self, id):
        if id == "com.robcole.Scratch":
            return ("0.1", "file:///C:/") # return mandatory latest version number and optional download url.
        # else generate a "cannot marshal 'none'" fault for now...

server.register_instance(MyFuncs())

# Run the server's main loop
server.serve_forever()

-----------------------------

on client side (plugin) it is called via:

local xmlRpc = XmlRpc:new{ url = "http://localhost:8000/RPC2" } -- just do once during init.

local status, xml, values = xmlRpc:sendAndReceive( "updateCheck", { { type='string', value=_PLUGIN.id } }, 5 ) -- 5 sec tmo.

-- process status & values... (entire text of xml response returned just for debug)

In my opinion, that is pretty simple.

Rob

AlanUniqueName

Participating Frequently

Hi Rob,

Warned you it would be long winded! Just brainstorming right?

Firstly your PM indicates that Lua + LR SDK would have difficulty in serving up to an external sidekick app - thumbnails, previews and rendered images of the photos. If this is the case, then my interest is practically extinguished as this would not support the kind of advanced things I could envisage and be interested in. However, for the sake of continued off the wall brainstorming and throwing in my two pence, I’ll assume that this could be overcome.

I think the power of your suggestion is that it would enable almost full blown “mini applications” to sit alongside LR (A Sidekick) complete with their own rich UI and possibly ease leveraging other technologies and interfaces such as high end UI experience, other APIs, other systems, databases, alternate peripherals, high power compute hardware etc where inside a Lua Plugin design environment this might prove impossible, difficult or not sensible – I cannot comment on this latter aspect.

Understandably you have your own motivations which seem to be around the modal / non-modal issues for a plugin – I understand, but personally this does not interest me half as much as the “what if” around the Sidekick concept.

The point made that it would be possible to send generated Lua script down a communication path of some sort is very interesting, and given that, I could see it as the way to go, as defining a proper Sidekick API is probably never going to happen and could ultimately end in a mess. I take the other point about coupling though. A script based approach would in concept (detail TBD) also need to incorporate proper handshaking and error handling with the external app. I’m actually not worried as long as an efficient and robust language and technology agnostic communication path is opened. It is the “what if” concept of this that interests. I think ideally LR itself would make such a communication port available, but in the absence of that, then a plugin to provide it - ideally non file (disk) based comms for efficiency. Pipes were also mentioned.

Again I point out that I know nothing of Lua or the LR SDK… I’m simply interested at concept level.

As an aside it also broadens the developer community to those who want to do bigger and more off the wall things and can work with the tools and knowledge they have which may not be Lua – i.e. C++, Python, Visual Basic, C# .net whatever and just make a limited use of Lua and LR API where needed. The Lua aspect of what such tools could do might be just a small part of the functionality. A Sidekick given access to the rendered images and previews may also be able to do more advanced things such as some image analysis which may require a high performance language and possibly leverage hardware acceleration or advanced CPU instructions.

Again, I cannot really suggest what is sensible or say that I’ve thought anything through and this is just brainstorming – not trying to be too sensible or too damming at this stage.

So what kind of things might this Sidekick concept possibly give? Some examples, but I do not suggest that any of these things are needed or even practical … I’m just attempting to illustrate the diverse things such a concept might allow.

1) An Application which “integrates with” Lightroom.
I can envisage a full blown stylish and UI rich Flickr Manager application which leverages tech like WPF/Silverlight to primarily manage and interact with an account on Flickr. As a secondary aspect, this app could also integrate with LR to upload the images from LR to Flickr and download comments etc the other way – as previously said the LR publish is very seriously broken. It would also perform more advanced tag management and filtering etc. than LR currently offers and other management functions. I.e. Primarily A Flickr Manager application with LR integration.

2) Extending LR Functionality - A Light Table
From time to time in the Forum … people have requested Light Table functionality. Personally I have no need for this and can’t even articulate the requirements. It may not be high on the Dev team’s agenda… I’ve no idea. But this concept may allow a sidekick Light Table to be developed.

3) Advanced image proceeding.
It has been requested at least once in the forum that LR implement photo sharpness detection to provide efficiency on the first pass culling stage. I have no idea if such a thing is possible, practical or sensible. Presumably some sort of algorithm may exist. I suspect this kind of stuff is not high on the LR agenda. An external sidekick might be able to do this kind of thing… but it would need some serious crunch power I suspect.

4) Multi Image processing.
HDR Processing without Photoshop. Controversial I know. But potentially it would allow quality UI and fully integrated into LR HDR processing without intermediate files being thrown to and from the disk. Sidekicks of this type would desire serious crunching power through the language and technology.

5) Print ordering & Photo books.
It would allow bespoke apps for print shops to be created which integrate with LR. Similar apps get written using the Flickr Api.

6) Complex Camera Control and Capture.
It would allow highly bespoke apps to control advanced camera / multi-camera setups with tethered or PC controlled shooting to be created which integrate with LR and camera manufacturers APIs. No - I have no idea what type of setups! Just illustrating.

7) Integration with other databases.
It would allow the potential bridging and integration with alternate large or shared databases.

8) Integration with other peripherals.
Wacky idea I know… but it would allow someone to write an iPad app to act as an addition control surface - clearly on non iThingy things as well. This sounds wacky I know… but similar happens elsewhere. Yamaha have released four iPad apps to control my Motif synthesiser! I’m simply trying to illustrate flexibility – not be practical.

9) Etc – to be completed with more skill, imagination and a younger mind than mine!

I have no idea how useful any of the above is – that’s not the point, and have no idea how much could be done with the current environment… I just get the feeling it would allow more scope. You asked what it would be used for … the story goes that the guy / team who invented the laser said the same thing at the time. I just can’t say but it feels like it would broaden things and open doors.

What the Adobe LR take on this would be – who knows!

Alan.

areohbeeAuthor

Legend

Hi Alan,

Good answer... (& I dont shoot any ideas down at brainstorming phase).

I was recently trying to figure out what doors the sidekick app might open as well. Some were the same as you came up with, some different...

To summarize:

- Regular apps can access everything on the computer except the Lightroom catalog.

- Plugins have limited UI, limited access to the Lightroom catalog, and limited access to other computer resources.

- Neither one has full access to Lightroom catalog, nor preview images, yet.

So it seems that lack of access to image data is a big limitation. Another big limitation is catalog access, which it seems would have to go through the plugin.

Regarding #1 (previews). It is certainly possible to reverse engineer the previews, and be able to read existing previews and associate with a source photo. - There is an existing app that recovers photos from the previews that people have used who accidentally delete their sources and such. I dont know whether this would take a day to figure out, or a month..., but its doable. And, I dont know whether the limitations would be acceptable. e.g. how to assure the preview is up to date. In any case, this could be one use case for a general API - to support access to up-to-date previews.

Regarding #2 (catalog access). I dont know if there is any way to hack an unlock for the catalog, and if any of this were to be unpopular with Adobe, this would probably be one of the things. But, a plugin can gain exclusive write access to the catalog, so if it passed this privilege to its sidekick, which could circumvent the lock, then the sidekick would have complete access sql-wise to the catalog. I'm not a database-guy, so I dont know if this is possible... If not, then one use of a general API would be for catalog read/write.

Another wild idea for catalog access would be to make a copy before starting Lightroom, and updating it instead of the live catalog if necessary, then do a sync upon exiting Lightroom - just brainstorming, again - I'm not a database guy. Obviously, this would be risky, and subject to redoing for every new Lightroom version, but anything goes at the brainstorming phase, right?

Oh, I just figured out what y'all meant be sending generated Lua script to the plugin to execute, instead of a general API. I guess I always imagined a custom plugin for every app, that would generate whatever scripts might be needed to execute or whatever, from more raw params passed by the app (e.g. via xml-rpc), but I can see that an option to generate the scripts on the app side might be worthwhile too.

That's it for now,

Rob

DawMatt

Inspiring

Hi Rob,

Interesting question.

For your initial purpose this seems like a lot of effort to work around the modal only dialog limitation provided of the current SDK. Longer term we can hope a non-modal option will be provided but I wouldn't hold my breath. I suspect this is a UI design choice and providing this as an SDK function option would undermine it. Other plugins might benefit more from separating the code for the UI and underlying functionality but in general I'd hope people don't go to this much effort purely to work around a UI SDK limitiation.

re Cross platform UI framework. I've used straight wxWidgets via C++ a while back. It did what we needed it to and ran OK on Windows and Mac so it sounds like a reasonable approach. And if using wxLua can reduce the learning curve for sidekick development then so much the better. Picking up something new like Flex, Silverlight, etc would mean retooling as well as significant learning curves and could prove a barrier to adoption.

Personally I'd prefer not to use a file based communications mechanism for IPC but it should be possible. LightroomTether (the plugin from Mountain Storm that pre-dated Lightroom 3) used UNIX pipes to communicate, but only ran on Mac. I've used generated scripts when passing instructions from Lightroom to Photoshop Elements. It can work as long as you are careful about file locking and atomic reads/writes to ensure messages being passed don't accidentally get lost. Generating lua scripts is also possible but it would seem cleaner to pass messages between the two components rather than having one blindly execute scripts generated by the other. That approach requires the two components to be very tightly coupled - not ideal.

Matt

areohbeeAuthor

Legend

HI Matt,

DawMatt wrote:
For your initial purpose this seems like a lot of effort to work around the modal only dialog limitation provided of the current SDK.

Not much more work, really. I mean DevAdjust is 95% UI, & 5% guts (not even that much), and none of the UI is really dependent on Lightroom. I actually really like the LIghtroom UI capabilities in general: simple & easy to use, layout works well, ... But for an app like DevAdjust, the modal nature of it is no small deal. On the contrary, its a big deal. So big in fact that its a deal breaker for most people. In my mind, the biggest part of extra work would have come from having to re-release on two different platforms every bug fix.

DevAdjust, fully loaded, takes several seconds to come up, then blocks Lightroom while its up. One can skinny it down and then it only takes a few seconds, but imagine if you had to wait a few seconds every time you switched from the basic develop section to the tone curve or between any other two develop sections... - that would suck.

People want relative presets and adjustments ever-ready, so they can hit it without waiting and alternate between that and other tools... It would be a fair amount of work for me to port it from plugin to app at this point, but if I had it to do over again, I would certainly do it that way. And, I may still do that port - we'll see.

It would not surprise me either if Lr4 still does not have non-modal UI - I'm guessing it would have fairly far-reaching implications, and I'm sure Adobe considered it carefully way back when they decided against it. I hope I'm wrong and Adobe surprises me...

Once again, I'm not (yet) trying to invent anything reusable here, except for maybe a general mechanism for communication. And even with that, there is nothing to stop one app from sharing files with their plugin while another uses XmlRpc or whatever... But, I am on the edge of my seat to see what ideas Alan & others come up with ;-}

areohbeeAuthor

Legend

I must say after reviewing the add-ons for Lua, I'm not sure there is anything you can do with a Python app that you cant do with a Lua app:

http://lua-users.org/wiki/LibrariesAndBindings

I don't know about the relative quality of the various implementations though...

DFBurns

Inspiring

I'm not sure where this could go but at the least it's a fun mental exercise.

Right now, Lr can read and write via HTTP, FTP, and file. Other than responsiveness, I can't see where the complexity of embedding a local web-server or ftp server in an external app is justified though. One could design a file polling scheme with dynamic polling intervals in order to balance communication latency with the need to not put undue load on Lr.

I started out thinking that one could design a server-like command interface within an Lr plugin and then 'sidekick' (I like the term) apps would have to comply with that. That seems like a pain though to maintain and it's an arbitrarily artifical layer. It occurs to me though that if Lr can read a file then (security aside for now) why can't sidekick apps just generate Lua dynamically that a general purpose Lr plugin (aka sidekick server) reads and executes? So, every X seconds, rather than "fetch" a command and some arguments, it fetches a file. Can 'require' take a dynamic file name? Hmmm...

Also, I think with a general purpose communication mechanism devised, the language/platform for the sidekick app is moot. Use whatever you like as long as it can write a Lua file.

It'll be interesting to see where this goes.

DFBurns

Inspiring

Or use some combo of dofile, loadfile, or loadstring: http://www.lua.org/pil/8.html

AlanUniqueName

Participating Frequently

Hi Rob,

This concept sounds like a great idea and very worthy of exploring. I might be interested / motivated in developing some kind of App outside of the LR box which talks to a process inside LR - this concept opens up the possibility of creating new "Sidekick", as opposed to "Plugin" functionality in any language and not be constrained by being inside a Lua sand box of sorts - it might open up all sorts of powerful possibilities to anyone with imagination.

It may even be possible over time to build up a technology/language independent “SideKick” API to LR of sorts. However, it is possible that this kind of behaviour may be frowned on – especially the permanently running internal process bit.

I'm guessing you are right that using file IO between the SideKick and LR might be workable, but possibly inefficient. I was wondering if some sort of Web based communication might also be possible. Clearly I've not been motivated enough to learn Lua and the LR SDK so I effectively know nothing, but it seems to me that LR / Lua already communicates with the Flickr API and this effectively constitutes another form of communication out of the box There may be other forms of communication?

One motivation would be to fix the seriously broken LR Flickr publishing and perhaps go a step beyond. In terms of the language / tech for the Sidekick … the best tool for the job is the obvious answer… but that’s one of the inherent beauties of your suggestion. For me is would be C# and Silverlight … but I would only do something for personal / Windows use. I guess plugin writers will naturally gravitate towards cross platform tech.

I’ll keep in touch to see how you get on and PM you a question later about what might be possible.

Alan.

areohbeeAuthor

Legend

HI Alan,

Yeah - you kinda got me goin' on this one...

There are already plenty of plugin authors including background tasks with their plugins, so unless Adobe yanks that capability in Lr4, frowning wont be enough to stop it...

File IO would be inefficient but probably fine for a lot of apps, but its entirely possible to toss a tiny web server in with the app and use http. The only connections to the outside world that I know of are ftp, http, and files, actualy, one could also count the command-line, since its possible to communicate be calling a command and passing it a string, so if files didnt cut it, then I guess http would be next in line.

Anyway, I got a wxPython app up and running, complete with a packagable (self-contained) .exe file, and I think it would be easy to also create the .app package for Mac.

So this technology/language independent sidekick API - not sure what that would look like.

Personally, I was toying with just splitting functions between app and plugin, case-by-case, as opposed to a reusable API.

What would be some example functions in this API, and a use case...?

PS - I just noticed Python has an XmlRpcServer library, and my newer plugins have XmlRpcClient functions built in. This would be one way for them to communicate...

Talk to ya later,

Rob

AlanUniqueName

Participating Frequently

Hi Rob,

I'll try to put some rough thought together later tonight (French time) or tomorrow and might PM you as my response is likely to be long winded - unless you specifically want it public for debate.

Alan

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded