P: Automatic image tagging

[Hendrik M Halkow]

[Update: See this reply for a list of third-party plugins providing automatic image tagging: https://feedback.photoshop.com/photoshop_family/topics/automatic-image-tagging-with-google-cloud-vis... -- John Ellis]

Hi all,
who of you likes organising and tagging pictures in your image catalogue? I don't. I'd love to keep my focus on taking photos. When I read about the Google Cloud Vision API, I thought this cloud be the solution for my problem that I don't like organising, but love having things organised. Cloud Vision is a service that takes an image and returns a set of keywords, places, brancs, etc. it detected. Why not let Google do the work?
So I started learning Lua and Lightroom development to create such a Plugin by myself (hold on, it is nothing there yet).
When I started thinking about the actual plug-in, the question came up what would other Lightroom users expect it to look like? And what other features do you expect? So I forwarding these questions to you. What do you think of it? Do you think this plug-in is helpful to you? What are your expectations?

[Victoria Bampton LR Queen]

Have you tried the new image analysis search feature in Lightroom Web yet? It runs along similar ideas, except the keywords aren't actually visible.

[Hendrik M Halkow]

Thank you Victoria for sharing this. I didn't know about this. looks like Adobe is working on this. What I miss is that the keywords are suggested based on the content, which is what you mentioned, too.
And of course, the integration of this feature into Lightroom: I want all of my images to be indexed by this feature (besides, I also want all of my images to be available in Lightroom Web and Mobile).

@Adobe: What are your plans about this feature? Could you please add the automatic keyword suggestion to it?

[johnrellis]

"So I started learning Lua and Lightroom development to create such a Plugin by myself (hold on, it is nothing there yet)."

That sounds like an excellent idea! Please keep us apprised of your progress.

I think it's likely that over time, the Google service will be higher quality and more capable than the Adobe service -- Google has more technical resources and a bigger market to support their efforts, as well as access to orders of magnitude more training data for its machine-learning algorithms.  Just compare the quality of face recognition provided by Google and Facebook with that provided by LR (technology Adobe licensed from a third party).

The Lightroom SDK forum is a good place to ask questions about implementing plugins.  

[KRO-KODILU]

@Hendrik M Halkow, sounds amazing! Both thumbs up! I've been hoping for and looking forward for such a solution for years. That sounds like a really excellent idea! Please keep us posted of your progress. I've subscribed to this post. Have a nice day 🙂

[FeedbackCommunityMember]

Hello all!

I just came across this thread and thought I'd let you know I did come up with a plugin that does just that.
Lightroom CC users can grab it from https://creative.adobe.com/addons/products/16333
Sorry though, it's commercial...

[johnrellis]

I've just released the Any Vision plugin, which also uses Google Cloud Vision to automatically tag your photos with labels, landmarks, logos, face expressions, and dominant colors and extract embedded text (OCR).

In addition to Any Vision, there are three other products with significantly different feature sets and pricing: Cloud Tagger, Excire, and Lightroom Web search.

[Guest]

After much deliberation I chose Lightroom to be my primary photo management software over Google Photos and Apple Photos because among other things I am a photographer / videographer and love that it's keywords are ITCP level metadata (instead of some proprietary system).

But one thing I really fell in love with when testing google / apple photos was their auto keywording features.  Being able to have all "cars" "landscapes" "animals" etc automatically keyworded makes searching and adding my own keywords exponentially faster.

I found this cool plugin http://wordroom.org/ but it only keywords 1 photo at a time when you manually tell it to.

I know lightroom is in a bit of a different ballgame than google / apple photos but I'm guessing there are a lot more people like me who use Lightroom over those options but would love to see stay competitive in the auto-keywording area as well.

Thanks!

[FeedbackCommunityMember]

Keywording is a painful and cumbersome process especially in these days where many photographers have fat LR catalogs with hundred thousands of photos that are increasing every day with little or no time to pick, rate, keyword, and tag.

LR has automated face detection and geotagging features, and both are integrated nicely with keywording panel at the library module. However, many users may need to search for a specific photo based on some general concepts such as scene type (e.g. landscape, architecture), or more specific objects such as Eiffel tower.

At the moment, I'm aware of the experimental search feature which is available only at online version of LR but this is limited only for searching on synced collections without offering any option to keyword them.


I think it will be great if Adobe extends the artificial intelligence based search feature to desktop and mobile versions of LR  and adds the ability to suggest keywords and apply them into photos.

[johnrellis]

Here's an updated list of products for automatic image tagging within Lightroom. If you know of others, please post them here.

Cloud Tagger uses Google Cloud Vision, and it is intended as an aid to careful keywording of smaller numbers of photos rather than searching large catalogs. It currently has no recurring charges.

MyKeyworder is intended as an aid to careful keywording of smaller numbers of photos rather than searching large catalogs.  It has recurring charges.

Lightroom Keywords is intended for careful keywording of smaller numbers of photos rather than searching large catalogs. It has per-photo charges.

Wordroom is intended for careful keywording of individual stock photos. It's currently in beta and it's not clear what it will cost in the future.

Excire lets you search your entire catalog for specific content or for similar images. It doesn't use the cloud and it has a higher one-time upfront cost with no recurring charges.

LrTag is similar to Excire, but it has a monthly charge.

RAISE is an online photo-sharing service. Free?  Not clear if its LR plugin downloads tags back to the catalog.

Any Vision uses Google Cloud Vision to automatically tag your photos with labels, landmarks, logos, face expressions, and dominant colors and extract embedded text (OCR). It has recurring charges.

Lightroom Cloud (but not LR Classic) uses similar technology, but it doesn't search your desktop catalog, just photos you've synced with Lightroom Cloud, Web, and Mobile.

[Updated to reflect the release of Project Nimbus as Lightroom CC.]

[Updated to add Wordroom and LrTag.]

[Updated to add RAISE.]

[Fred_R1]

Is it possible to implement photo recognition to generate keywords?

I have seen this kind of functionality on an online photo community (I don't know if Adobe already provide it to its online one). I was thinking that this functionality will be a powerful tool for non professional photographer to improve their use of a software like Lightroom that provide tools to take advantage of keywords in photos.

I use Lightroom for long time, but in order to exploit my photo database I need an continuing improved set of keywords. As I develop my way to practice photography I'm more demanding about Lightroom features that are powerful like collections... But one limitation is brain and memory (speak for myself only) because I'm facing a problem : how can I extract photos that share the same features in order to work on them and see if I can make a coherent collection.

I'm not so wise to implement a complete dictionary of keywords, and giving coherently keywords to all my photos is a task that will take too long time and probably will be pledge by inefficiencies in the keywords definition.

So will it be possible to implement IA photo recognition and to generate set of keywords on each photo of a given database, like face recognition does. It will save precious time and give the opportunity to all users to discover common elements in their photographic practice. A analysis module could also provide automatic collection themes based on the most shared keywords or can provide interface that allows the user to select a specific theme and explore differently its production.

Exploiting past photos is a growing problem and this kind of feature will be helpful and powerful if connected to other LR existing functionalities. It will help me (and probably some others) to focus on ideas and photo selection/improvement.

Thanks 
Frederic

[smu95rp]

Currently, Lightroom can recognize faces and suggest People to label images. It would be really useful to be able to apply the same technology to objects or products. 

I take a lot of photographs of classic cars and it would be really helpful for Lightroom to be able to suggest specific makes/models. I'm sure that train, airplane and boat enthusiasts will have a similar use. Or marketers can use it to find appearances of their products in media.

Additionally, it can recognize specific cities by their buildings or landmarks. 

Also statues and artworks.

[johnrellis]

Google's Cloud Vision service does some of this. For example, it generally recognizes and labels the presence of "car", "antique", and "classic" or "vintage" car, and it correctly identified a Volkswagen Beetle, a Ford Model A, and a Mercedes Benz 500K. But more often it fails to identify the make or model.

It also usually identifies the presence of other categories -- trains, planes, etc.   And it often identifies buildings and landmarks by their map location.  

See this post for an overview of services that provide automatic tagging in LR: https://feedback.photoshop.com/photoshop_family/topics/automatic-image-tagging-with-google-cloud-vis...

[imanikongirl]

Can you please give us professionals that use Lightroom Classic Sensei too? I hate it when you give all the new tools to the shiny new penny and don't continue to do so with the older "classic" versions. This is alarmingly similar to my experience with purchasing Lightroom 6 only to discover it didn't have all the features of the CC version...so I did switch.

[johnrellis]

In a merged topic, *Victoria Bampton Champion wrote, "Sensei is all done in the cloud, so you'd have to get your photos up there. Since you have Classic though, you can sync photos to the cloud (it just sends small smart previews) so that Sensei searches them, and you can use the Sensei search on the mobile apps or web browser."

[johnrellis]

See this previous post for plugins that provide automatic tagging and search: https://feedback.photoshop.com/photoshop_family/topics/automatic-image-tagging-with-google-cloud-vis...

[imanikongirl]

THANK YOU for this!

[berliner_ffm]

I understand Sensei only works for online pictures. But why not enable search with sensei from Lightroom Classic for all synched photos? I dont want to change the application just for a search.

[FeedbackCommunityMember]

In my opinion, Adobe Sensei is one of the best new features and I'd love to use it on my  Lightroom Classic CC library.

Is there any technical limitations not to have it on this version? 

Thanks!

[johnrellis]

See this previous post in the topic: 
https://feedback.photoshop.com/photoshop_family/topics/automatic-image-tagging-with-google-cloud-vis...

[FeedbackCommunityMember]

If you wanne benfit from local image analysis and similarity search for lightroom, check www.excire.com

P.S.: Sorry, did not see, that John already mentioned it. Anyway, you really should test it 😉

[PhotoCitizen]

Adobe's artificial intelligence (Sensei) search works surprisingly well in their iOS Lightroom app. When will Sensei search be incorporated into the desktop Lightroom Classic?

[Victoria Bampton LR Queen]

It's unlikely to happen, because it's a cloud feature and they've said that Classic will be focused on desktop features rather than cloud ones. Of course you can sync photos to the cloud from Classic and use the web interface on your desktop to search for photos. Just go to https://lightroom.adobe.com

[Antoine HLMN]

I think this approach is really sad. Many photographers have a workflow that is difficulty compatible with the new Lightroom version: lack of fundamental features, need for constant connection with a high bandwidth and maybe the most important, the need of control on their images. Retaining features for their « cloud » apps from the desktop app for commercial reasons might be extremely frustrating for photographers.

[Antoine HLMN]

I’d love to see some automatic keywording tool based on Sensei (or any other AI) in LR classic. Like the author, I like to focus on taking pictures rather than organizing them. That’s why I almost never use keywords... Here’s where the software part can shine

[FeedbackCommunityMember]

You really should test the Excire Search Plugin (www.excire.com). It allows for automatic keywording and image similarity search based on AI. Furthermore it runs completely local on your machine, no cloud service is integrated or necessary. It will be continuously extended and improved. Rigt now, it supports more than 500 classes/scenes. New classes will come this year and user feedback and suggestiona are appreciated.

Sorry for double posting, but it seems that my above comment wasn't read.

Best regards,

Thomas (Excire Project Manager)