Organizer Import - Expecting too much?

[skimbell]

As an active musician, (at least before Covid came along) I take literally thousands of pics at our shows.  I've been doing this for the better part of the last 50 years.  I have well over 2 million pics and videos spanning 6 external HDs.

I've used PS Elements for many years but never used the Organizer, as I keep my files in meticulous order.  However, the prospect of using facial regognition was extremely alluring, so three days ago I started importing the files from my smallest drive, with 337478 files.  After three days it is at 88% and still running!

I'd like some input as to whether I'm simply expecting way too much from this program.  Can it handle groups of files this large?  Does anyone have firsthand knowledge of whether I'm wasting my time here?

Some details:  Windows 10Pro, i7 8700 CPU @ 3.7 GHz.  16 Gb RAM  PE 2019

It's not a Cray but it does pretty well. 

I don't know if Organizer is creating a single massive index file for all the imported pics, and perhaps this file has grown so large it's having issues.  I'm also not sure if it's trying to perform facial recognition as it imports.  It only permitted me to import the whole external drive...I wasn't able to start with a smaller subset of folders.

It appears that of the 16Gb of memory 8.1 Gb is reserved for cached processes, but it doesn't appear that its using it, and I don't know how to change that allocation.  Almost all the remaining 8 Gb is being used by Organizer.  I've shut down as many background processes as I can.

Some of the pics are of large concerts where there are upwards of 50,000 faces staring into the camera.  To make things worse, I use burst mode when shooting so I have a LOT of pics of these huge groups!

Since the initial import hasn't completed yet I have little clue how the facial recognition component works.  I remember using Picassa several years ago & it was easy to add folders, but the facial recognition was rather painful, although it was much better than nothing!

I'm also concerned about Organizer changing my file locations.  I don't want it to do this...I'm very happy with the file structure just as it is. 

So...suggestions?  Ideas?  Concepts?  Or don't even bother?

Thanks!

[Greg_S.]

I have well over 2 million pics and videos spanning 6 external HDs.

I've used PS Elements for many years but never used the Organizer, as I keep my files in meticulous order. However, the prospect of using facial regognition was extremely alluring, so three days ago I started importing the files from my smallest drive, with 337478 files. After three days it is at 88% and still running!

Adobe has never published a maximum size for media files in a single catalog.  I have been using the Organizer since it was first introduced (as Adobe Album).  My current catalog is approaching 200k files and I know of others who have two to three hundred thousand files.  The Organizer certainly handles them with ease.  (I'm not trying to sell you on upgrading, but I would say that face recognition performance improved in Elements 2020 over 2019.  And 2020 also face recognizes videos.)  Two million is an exponentially larger catalog, and I have never heard of one that approaches that number, so I think you would be in the vanguard on that one.

Facial recognition does take a long time, and what you are experiencing is nothing out of the ordinary.  However, you do not need the analysis to complete before you explore its features.  The analysis will take place in the background.

I don't know if Organizer is creating a single massive index file for all the imported pics, and perhaps this file has grown so large it's having issues. I'm also not sure if it's trying to perform facial recognition as it imports.

Yes, the Organizer creates a single database file, but it also creates many other files.  For example, it creates a .json file for each media file that it analyzes.  My catalog folder is about 17 GB.

It only permitted me to import the whole external drive...I wasn't able to start with a smaller subset of folders.

It sounds like you used the Import>In Bulk option.  If you use the Import>From Files and Folders option, you can import whatever set of files you want.  You can multi-select folders and then hit the Get Media button.

I'm also concerned about Organizer changing my file locations. I don't want it to do this...I'm very happy with the file structure just as it is.

The Organizer will do nothing to your file structure - unless you want it to.  In this regard, it is important that you make any changes to your file/folder structure within the Organizer.  This includes moving or renaming files and folders.  Otherwise the catalog will lose track of your files.

Some of the pics are of large concerts where there are upwards of 50,000 faces staring into the camera. To make things worse, I use burst mode when shooting so I have a LOT of pics of these huge groups!

Here is where I have to ask you what do you plan to do with your recognized faces?  The Organizer will probably recognize most of those 50,000 faces.  Since you shoot in bursts, it may create small or large stacks of each person in the crowd.  The idea for face recognition is to be able to add names to the faces.  Whose faces do you intend to name?  The artists on stage? Groupies in the front row?  Fans who come to every concert? 

My grandsons' school concerts and sports events are on a much smaller scale.  But the photos/videos I take have dozens of stacks of kids and parents that I have no interest in naming.  It is fairly easy for me to multi-select stacks of those people and delete large numbers with one command.  But I think it is going to take a lot of time for you to go through that process.  So, if you tell us a little more about your intentions, perhaps I can give you some strategies to speed up the process.

[skimbell]

Thanks so much for such an in-depth response!

Here is where I have to ask you what do you plan to do with your recognized faces?...Whose faces do you intend to name? The artists on stage? Groupies in the front row? Fans who come to every concert?...So, if you tell us a little more about your intentions, perhaps I can give you some strategies to speed up the process.

I think you probably get the gist of what I'm looking for.  There are probably a hundred fans who are FaceBook friends for whom I want to pull a pic for their birthday or something, and would like a grouping of all the pics I have of them.  Same with family members.  It is also extremely handy when I get a pic of someone whose name I can't remember!

in other cases I might want to identify pics that have one, two or more specific people as a group in the same pic.  Don't know if that can be done or not.

Unfortunately the vast majority of faces I don't need to keep as I have no clue who they are and will never need to know.  I recal when working with Google's Picassa I had literally hundreds of thousands of "Unknown" faces, and I'll have the same issue here.  What I'd LIKE to do is identify my core people I'd like to identify first.  For example, tell it to go through all the pics and find these people.  However, I assume that it will have to scan every pic first before I can take that step.

Burst mode on my camera was perhaps the best and worst thing that ever happened to me in terms of my amateur photography!

One other thing...as I scan new pics, or add others to the drive, does Organizer automatically look for new pics, or do I have to tell it to rescan a folder?  I don't necessarily want to keep it open all the time as with this many pics it could place a strain on my system's resources.  You mentioned moving pics around using Organizer, but what about new pics?

Thanks again for your response.  I'll no doubt have more questions as I explore further!

[Greg_S.]

Sorry for not catching your reply earlier.  I have responded to your other thread and will answer your other questions here.

in other cases I might want to identify pics that have one, two or more specific people as a group in the same pic. Don't know if that can be done or not.

That is one of the main purposes of facial recognition.  It is very easy to search for all photos containing particular people in them.  Simply click on the boxes next to the People tags and the images in the grid will be filtered to those people.

Unfortunately the vast majority of faces I don't need to keep as I have no clue who they are and will never need to know. I recal when working with Google's Picassa I had literally hundreds of thousands of "Unknown" faces, and I'll have the same issue here. What I'd LIKE to do is identify my core people I'd like to identify first. For example, tell it to go through all the pics and find these people. However, I assume that it will have to scan every pic first before I can take that step.

My suggestion is that you first scan through the stacks and identify people that you know and can recognize easily.  You will select their stacks and add names to their stacks.  Since you shoot in bursts, the same person will have several (if not dozens) of photos in a stack.  However, you will also find that there are multiple stacks of the same person.  You can select many stacks of the same person by Shift/Ctrl clicking on their stacks.  You then have the option to combine the stacks and add a name to the combined stacks.  (You can also drag and drop stacks on each other to combine them.  This is a good option if you want to review each of the faces in a stack before adding a name, since some of the faces may not be the same person.)

Once you have gone through an initial scan, you will then see a large number of unknown or unwanted faces.  Again, you can use Shift/Ctrl click to select multiple stacks and use the Don't Show Again command to remove them from the People grid.  What I sometimes do is add names to my wanted faces in a screen until only unwanted faces remain.  I then select all of the visible faces and remove them with Don't Show Again.  When going through this process, I make sure that the side panels are toggled off so I have more faces on screen to view and select.

as I scan new pics, or add others to the drive, does Organizer automatically look for new pics, or do I have to tell it to rescan a folder? 

The Organizer will automatically analyze new photos added to your catalog (if you have Face Recognition turned on in Preferences).