P: Include additional metadata in XMP (flags, collections, VC's, etc.)

[Hadmut Danisch]

Hi,

when letting Lightroom write all the picture settings to a xmp file, both the stacking and the collection settings are missing.

Basically, I'd expect to find just every work done about the picture in the xmp file (e.g. for use with other tools. If I put several pictures in collection, I'd like to use that information and the order of the pictures from other programs.)

Even worse, when making a virtual copy of a picture, it's settings do not appear anywhere in an xmp file.

regards

[Hadmut Danisch]

May I just ask what role "john beardsworth" is playing here?

Is he related to Adobe? Or is he a forum troll? What are his intents to continuosly deny other people's needs?

What's the point in offering LR users a forum for feature requests and improvement, if they have to battle around with third party people in pointless discussions?

(What- or whoever he is, he seems to have an extremly limited point of view and understanding of computer technology, and although I respect his opinion as apropriate for his workflow, it's absolutely irrelevant for my point of view both in my privat photograpic use and in my professional work. So I do consider his comments as insubstantial and jamming of that forum.)

[john beardsworth]

I'm a user of the software just like you and "a well-respected member of the Lightroom community" (from a recent thread here). My concise counter-arguments are because your longwinded posts ignore important aspects of Lightroom and demonstrate ignorance of how best to use it. Just because I pull apart your arguments is not a reason to throw insults.

[Lee Jay]

My entire catalog went corrupt for no apparent reason whatsoever the other day. LR wouldn't even open. I had had no crashes or problems of any sort.

I used a catalog backup from a few days before, but of course it didn't include anything I had done in LR since it was made. I sync'd and everything was back just how I had it exactly because I use xmp the way I do, and don't use any features not supported inside of xmp. If I hadn't done that, I would have lost several days of work.

[john beardsworth]

I wouldn't have lost any as I backup upon exit, and use all those features you can only dream of 😉

[Lee Jay]

If LR had a backup file management system (you know, keep the most recent three, one from a week ago, and one from a month ago or something like that), I might consider backing up on exit. As it stands, that would be a nightmare for me.

[Victoria Bampton LR Queen]

The limitations of xmp are as designed, however I do agree there is an argument for including all available data.

So bringing this back on track, it seems to me that the main argument you're making is not only about xmp holding all available data, but also for Lightroom to work as a file browser rather than a database. Would that be accurate, or have I missed something in amongst all this chatter?

[Victoria Bampton LR Queen]

I'll vouch for the fact that John is not a troll and has been a well-respected member of the community for a long time.

Though it may feel like the discussions have been pointless, as a result of John's questioning and counter-arguments you've raised a workflow situation that we don't come across very often - combining multiple users of the same catalog and a remote location making import/export catalog a less useful option - and we wouldn't have fully understood your reasons for wanting this feature from your initial post.

That kind of additional information is useful - if Adobe can't or won't include all data in xmp for one reason or another, understanding your reasons behind wanting that feature may provide an alternative solution.

[Lee Jay]

Victoria,

Long ago (about Beta 4 prior to 1.0), I suggested a working mode that was browser-like. It would work like this. You'd "browse" to a folder, and those images would be automatically imported into a temporary collection (possibly even in a temporary catalog). Autowrite would be turned on for those images automatically, and you'd do your thing. If you browsed away to another folder, the collection would be removed, but of course it could come back quickly if you browsed back because of the XMP data. Essentially, this is how RSE/RSP worked, and that's the tool I was using before Lightroom. I felt this approach would ease the introduction of the catalog concept and allow "quick looks" and "quick editing sessions" where those techniques would be appropriate.

[Victoria Bampton LR Queen]

Yeah, I could see how something along those lines could be useful in specific situations. How would you imagine it working if someone then wanted to add those photos to the full catalog? Or wanted to drag/drop between folders, treating it like a full-blown file browser?

[Lee Jay]

If someone wanted to do a transfer, the simplest thing would be to just do an import - no new UI required. I was trying to keep this simple.

If people want to do the whole folder-to-folder and other file management things, I'd require them to import and do it as we do now.

[Hadmut Danisch]

@Victoria: "File browser" is not really accurate, but it gives an idea of what I am expecting. It is a wrong view on the thing.

To be more precise in terms of programming, the storage engine storing the settings for each single picture should not be strictly bound to SQLite, but encapsulated and object oriented, i.e. separation of programming interface and implementation, thus allowing different implementations for the same thing.

Then, SQLite (as it is used right now) would be one implementation and xmp files another, both fulfulling the same task.

Your point of view („file browser”) is only user interface oriented. As my first and main profession, I am not a photographer, but a „Informatiker” (computer scientist + engineer + consultant), especially in IT security. That's why I do more intensively focus on keeping machines synchronized in case of loss/theft/damage. The idea is not to turn LR into a file system browser. It is not meant to change LR's appearance and usage at all.

It is meant to give LR just an optional second/alternative storage engine, which behaves diffently, e.g. allows much better synchronization between several machines of the same user or different users.

I usually synchronize my portable computers with my home server whenever I connect them to my home office. Since I use plenty of different programs, it would simply be absolutely unfeasible, if every single program expected my to manually perform import and export both on the server and each notebook to keep my machines in sync. Whoever believes that this manual export/import by mouse is a good way to achieve synchronisation has never ever had his machines in a good state. There must be a general way to synchronize several machines fast and automatically, no matter where you actually did some changes on your data (and it does not matter whether you were writing software, writing a book, or work on your pictures). And you can proove that it is not trivial to keep several machines in sync if you always produce export files. What if you have two appartments and two notebooks? Export and import every change for four times? How much time would you have to waste?

The second aspect is that I have a different view as a professional security consultant. I always highly recommend my customers to install software on their portable computers to a) keep them encrypted b) keep them automatically synchronized whenever they attach their notebook to their desk to minimize impact in case of theft or other sorts of loss.

The third aspect is, that because I used to have two appartments for a long time and to travel around a lot, I had to learn how to use several machines at the same time and having work done on one machine available on the others. This works with file system syncs and online storage. But not with having to start every single program to manually export/import things. Where should that end? Having 30 exports every months?

The fourth aspect is, that for technical and professional reasons I do not use Windows as my primary operating system, but Linux (that's why I started with Bibble 5). For all those photographic applications that don't run under Linux, I have a Windows in a virtual machine and another windows in a small machine at home to use over rdp. And my central storage servers are all Linux machines. I simply don't have that much time to waste to start a virtual Windows machine and a second Windows PC, to start LR on both machines, operate them with a mouse just to perform my daily sync. Both Windows entities use Linux as their underlying storage, either through virtualization or samba shares. So both storage devices must be able to sync without having a running LR instance.

The fifth aspect is, that for the very same reason every professional photo agency should store pictures on central servers with backups, and not on the local machine. Servers do not execute Software like LR. They just offer storage, dumb storage. So it must be possible to synchronize against any storage device, not just running LR software.

So, to repeat myself again, I do not want to turn LR into a different program, do not want to change it's appearance, or change it's usage. All I want to have is the storage engine replaced with the choice between the old one and a more robust and less colission-prone one that can sync against plain storage, and has a more precise state.

And, btw., Bibble 5 had pretty good reasons to offer both options, storing in local databases or storing in file system mode. User can choose between both (and even use them together). I've been using that for about a year, and it is working pretty well, simple and reliable. And yes, since Bibble5 has the file system mode where it does not use a central database but taking the files 'as they are with their sidecar', you can actually use it as a sort of file browser, if that's the term and point of view you prefer. You see almost no differences in usage between database and file system mode. But I can always sync a part of my image archive on my notebook for working outdoors, or take pictures onto the notebook and sync it into my central archive - without needing to run Bibble5 or even have it installed on the central machine. I can burn images with their sidecars onto CDROM or pack on USB pendrives and can work on them without the need to import them into a local database. It makes work so much easier.

[Victoria Bampton LR Queen]

I'm not knocking your request, by any means. I can certainly see uses for that kind of setup, and as I'm working between 7 different machines and 3 different operating system versions, I do understand the complications of keeping everything updated.

Allow me to understand your needs further...

Obviously one of LR's database's main strengths is the ability to view offline photos. How would you imagine that fitting with the file-system storage implementation? Would you imagine a database that could be synced with the xmp storage, which would still enable offline viewing, or would you just skip offline viewing altogether? Would that even be of importance?

Have you used Bridge at all? How do you imagine this being different from working with Bridge? What features do you imagine a Lightroom file-system option bringing to the party, that would be better than Bridge's existing features?

[Hadmut Danisch]

What is "Bridge" ?

I actually did not intend to view offline photos, but that's a good point. I'd actually see three different approaches how to deal with it, and I'd suggest to leave it to the user to choose between:

- No method to watch offline files
- Include a thumbnail in the xmp (which makes sense only if the xmp is still online, which I could not really imagine at the moment)
- Use a regular database as a underlying cache. If LR keeps its code really clean, that it is really simple to build an abstract storage engine consisting of two others, which writes always in both, and reads from the available if one is missing. Would make searching much faster than traversing the file system each time. Actually that's not too far from what LR does right now with file system synchronization against it's database.

So I'd prefer basically three modes of storage:

- Database only with manual writing of xmp (as it is just right now)
- xmp mode only
- mixed mode with xmp preference, but using a database as a cache and backup in the offline case (where offline and missing because of deleted are distinct things)

The more I think about it, the more I'd see the function of that mixed mode database in fast searching/caching than in offline reading. E.g. having 200,000 pictures in a large file system tree would really take time to find all pictures with a particular key word or maintain a collection.

I think having the choice between these three modes would be good.

[Hadmut Danisch]

What is "Bridge" ?

I actually did not intend to view offline photos, but that's a good point. I'd actually see three different approaches how to deal with it, and I'd suggest to leave it to the user to choose between:

- No method to watch offline files
- Include a thumbnail in the xmp (which makes sense only if the xmp is still online, which I could not really imagine at the moment)
- Use a regular database as a underlying cache. If LR keeps its code really clean, that it is really simple to build an abstract storage engine consisting of two others, which writes always in both, and reads from the available if one is missing. Would make searching much faster than traversing the file system each time. Actually that's not too far from what LR does right now with file system synchronization against it's database.

So I'd prefer basically three modes of storage:

- Database only with manual writing of xmp (as it is just right now)
- xmp mode only
- mixed mode with xmp preference, but using a database as a cache and backup in the offline case (where offline and missing because of deleted are distinct things)

The more I think about it, the more I'd see the function of that mixed mode database in fast searching/caching than in offline reading. E.g. having 200,000 pictures in a large file system tree would really take time to find all pictures with a particular key word or maintain a collection.

I think having the choice between these three modes would be good.

[Lee Jay]

Bridge is the file browser/organizer that comes with the Photoshop CS.

[Hadmut Danisch]

Photoshop CS? You mean that incredibly cheap software, almost for free?

[Victoria Bampton LR Queen]

Mmmmmmm, that mixed mode was the kind of scenario I was wondering about, and possibly more likely to make the cut. Not a complete departure from the current setup, but extending the xmp spec to contain all (or at least more) information and improving the metadata synchronization with those xmp files to give a more intelligent sync, or something along those lines.

[Hadmut Danisch]

But it should be possible to do that without polluting the regular databases. Maybe have a separate database used for that purpose only, which is subject to cleanup of expired entries.

[FeedbackCommunityMember]

Your logo.png is a raster file, it can't be scaled as a vector.

[FeedbackCommunityMember]

But isn't it supposed to scale the smart object layer?
So how am I supposed to do it the right way, if it's supported?

[FeedbackCommunityMember]

If a raster graphic is made into a smart object, and that smart object is scaled to 50%, then I put a prefix of "200%" in front of the smart object name, I would expect the generator feature to export a file that looks the same as the original raster graphic... but it doesn't.

[FeedbackCommunityMember]

In my case I've tried with a vector logo copy pasted to PS as a smart object without succeeding.

[FeedbackCommunityMember]

It should work fine if you use a vector file: AI, PDF, or EPS.

[FeedbackCommunityMember]

logo.png is the rasterized output, not the input. It's not that simple.

I suspect it's a bug in the support for asset generation, perhaps use retina-sized PSDs and scale down by 50% as an alternative workaround?

[FeedbackCommunityMember]

Then you'll need to provide a lot more detail about your document, or post a sample document we can see.