Highlighted

Backup organizer database, without media?

New Here ,
May 17, 2020

Copy link to clipboard

Copied

Hi,

I am running the Adobe Elements suite, including organizer (latest version, 2019 I think?) on my laptop. All my media files are located on a NAS, that I regularly backup. I have other compters/OS's working with the media files on the NAS as well so I do not want to make my backup solution for the entire library an Adobe one. I am however switching to a new laptop soon and would like to take my Elements database metadata with it to that laptop.

 

I have a few questions:

  1. Can I do a "database backup only" without backing up the actual media it refers to? (Just the metadata)
  2. When I do a restore from backup in Organizer, will it restore the files to it's original location? (That is to say: on the network drive it is not linked to.)
  3. Would it work if I installed Organizer on the new laptop and just copied the "ProgramData\...\Catalogs\" folder over to the new install? Making sure of course all paths on the new laptop point to the same location for the media.

 

I really wish Adobe would make it easier to move your database/files and not expect someone to use Organizer as a backup solution. It is not. I make backups of my NAS on an hourly basis and scheduled off-site backups once a month, can't do that with Organizer 😞

TOPICS
How to, Import and export, Organizer, Problem or error

Views

649

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

Backup organizer database, without media?

New Here ,
May 17, 2020

Copy link to clipboard

Copied

Hi,

I am running the Adobe Elements suite, including organizer (latest version, 2019 I think?) on my laptop. All my media files are located on a NAS, that I regularly backup. I have other compters/OS's working with the media files on the NAS as well so I do not want to make my backup solution for the entire library an Adobe one. I am however switching to a new laptop soon and would like to take my Elements database metadata with it to that laptop.

 

I have a few questions:

  1. Can I do a "database backup only" without backing up the actual media it refers to? (Just the metadata)
  2. When I do a restore from backup in Organizer, will it restore the files to it's original location? (That is to say: on the network drive it is not linked to.)
  3. Would it work if I installed Organizer on the new laptop and just copied the "ProgramData\...\Catalogs\" folder over to the new install? Making sure of course all paths on the new laptop point to the same location for the media.

 

I really wish Adobe would make it easier to move your database/files and not expect someone to use Organizer as a backup solution. It is not. I make backups of my NAS on an hourly basis and scheduled off-site backups once a month, can't do that with Organizer 😞

TOPICS
How to, Import and export, Organizer, Problem or error

Views

650

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
May 17, 2020 0
Adobe Employee ,
May 18, 2020

Copy link to clipboard

Copied

Hi Mecallie,

 

The third option you have mentioned will work. Please do it carefully, with all paths of the catalog as well as media maintained as before.

 

Thanks,

Somya

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
May 18, 2020 1
New Here ,
May 18, 2020

Copy link to clipboard

Copied

Great. In that case it's up to my own skills of getting all the paths right 🙂

I will try as soon as I get my new laptop!

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
May 18, 2020 0
Adobe Employee ,
May 18, 2020

Copy link to clipboard

Copied

Let me know if you face any issue while doing that. I would advise to maintain a third backup of the 'Catalogs' folder while doing this just to be extra careful.

 

Thanks!

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
May 18, 2020 0
New Here ,
May 29, 2020

Copy link to clipboard

Copied

may I ask a question about a related problem? I run Photoshop Elements and Premiere Elements and Organizer 2020 and have created a big catalog, which is stored in the folder "ProgramData\...\Catalogs\". I want to move that catalog to a folder, where it is part of my backup procedures. The media should not be removed from their current place. Does the mentioned solution number 3 will work - I copy the "ProgramData\...\Catalogs\" to the position I want it to have, using the windows explorer,  and then I open the catalog from that new place? 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
May 29, 2020 0
Adobe Community Professional ,
May 29, 2020

Copy link to clipboard

Copied

You can move or copy the whole catalog folder nearly anywhere from the Explorer/Finder. The only exception seems to be on a NAS with Macs (I don't know anything about Macs). You simply have to use the catalog manager to browse to the  new location once.

This is not clearly stated in Adobe's help docs, but a very frequent choice is to store the catalog and the library (the media files tree) on an external USB drive. This way, you can keep the catalog always updated and accessible from any computer with the correct PSE version.

To do so, you do a backup and restore to a master folder of your choice on the external USB drive. The result is that the catalog folder is moved together with the library and stored just under the new master folder. If you prefer to keep the catalog folder in the default location, the catalog manager lets you move it back. Any other move or copy (for backup for instance) has to be done from the explorer.

Having an incremental backup system including the catalog folder is generally much faster than a backup from the organizer, but if the original drive is missing, all the links in the catalog will be broken. So, the best combination for speed and safety is to use both a fast sync/backup utility including the catalog folder and to do periodic full organizer backups especially if you want to migrate to a new computer.

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
May 29, 2020 1
New Here ,
Jun 10, 2020

Copy link to clipboard

Copied

Thank you for your comprehensive explanation and sorry for the late response of my side. This is exactly the way I want to go: to store the media and the catalog at the same place, so the backup procedures keeps both in sync. 

Thanks gagain. 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 10, 2020 0
slast LATEST
New Here ,
Jun 13, 2020

Copy link to clipboard

Copied

I have been having some issues which seem to relate to this and have some very useful help from MichelBParis  - thank you! However I am still confused as it seems to conflict with what I read here, although it may well be just my lack of comprehension of terminology.

 

My problem is that my laptop's internal drive C is 256Gb. My image library is just under 100Gb and most of that is images I just want to archive with Organiser tags etc so I can locate them if necessary. I want to have both library and Organiser backed up so I could restore them in the event of a hard drive failure (as happened last year with a previous laptop). On that occasion I had all the images backed up else where but lost the catalog. So I understand from MichelBParis that this means "[my] goal is mainly to save space on a limited space SSD drive, [so I] must have another internal drive [my emphasis] with enough space." I inderstand this is because Organiser needs to have a physical drive identification to be able to find things. Consequently, although I can (and have) put my library on cloud storage (Onedrive), backup has to be local. I could do this but rather defeats the purpose of having bought Onedrive space and means having to keep yet another bit of hardware lying around.        

 

However, other posts and elsewhere it appears that network drives (NAS) CAN be used.  I have no experience of NAS but it appears from what I can find out to be like a private "cloud" storage. (I have seen some articles which seem to directly compare OneDrive and NAS e.g. https://nascompares.com/onedrive-vs-nas-drive/ , so I am still confused!

     

The steps I have taken so far, and my understanding of what they are doing is as follows, can someone please correct me if I am wrong so I better understand the limitations and benefits of the catalog system.

  • I moved my media files into a Onedrive folder on C.
  • which sync'd automatically to upload them to my Onedrive cloud space as a copy of the one left on my hard drive.
  • I made a new Organiser catalog by "get from files and folders". This appears to be "my catalog 1".    
  • Organiser tells me I should make a backup, which I want to do anyway especially to keep track of additions by incremental backup. Organiser tells me I must make a FULL backup before I can do an incremental one. I can choose the location of the backup but it has to be on a physical drive. It allows me to create and use a folder in c:/Onedrive.
  • I tell it to go ahead and make the backup. When I look at the contents of that folder I find it has files with names all starting with "B" (presumably "Backup"?) and sufficient numbers to allow up to 10 million files.
  • These are 3 types - .xml, .json and .jpg.  From looking into these it seems that the .jpgs are copies of my image files including one for each edited one saved as part of a data set, and the .json and .xml ones are database files with any tags, face recognition etc.  Images that are just being saved (never edited, no faces etc) contain just {} characters
  • These database files a very small part of the total size.  
  • The whole lot have now been uploaded to my Onedrive cloud storage and are visible via the web.
  • In the Windows Onedrive folder I marked the .jpg files ONLY as "files on demand" which as I understand it means that space occupied by the originals is free for Windows to use. My free space has now increased has by the size of the image library, and I can access them from the organiser and edit them as well. So far so good! 
  • In the Onedrive folder on the laptop, all the .jpgs now show status as  “available when online”.  The free space reflects that. It decreased after the backup was done, and is now back to only a little less, which would reflect the database files still being held locally.   
  • I can see all the files in Onedrive on a different device such as my Kindle.     

So I am now in the situation where I APPEAR to have saved the space I need without having another internal drive. I THINK I have my original library of images plus a complete backup of both catalog AND images in my online OneDrive space.

1) Is this correct? 

2) My question now is whether that is sufficient so that if I had a new laptop with an installed copy of Elements, would I be able to restore the catalog and library to it from the files on Onedrive? These include backup.tly and catalog.buc.

I have a follow up question but that is enough for now! 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 13, 2020 0
Community Beginner ,
May 29, 2020

Copy link to clipboard

Copied

Mecallie, I have the same setup you have (assuming you use Windows) - catalog on my C: drive and media on my NAS, both of which are backed up daily - C: drive via Acronis True Image and the NAS by its own backup.

 

The catalog MUST be on a non-removable drive in your computer. I have argued this with Adobe to no avail. The media can be anywhere.

 

You can move the catalog anywhere via the operating system (Windows File Explorer) as long as it is to a non-removable drive. The media, however, can only be moved via an Organizer backup and restore. When you go to restore the backup you will have the option to restore the media and catalog to a different location, and you can either retain the original media file structure or not.

 

So in answer to your questions:

1. You cannot do a catalog-only backup. I'm not sure what you mean by the 'metadata only'. Certain types of files (like jpg) have metadata as part of the media file itself so it automatically stays with it's file. 

2. As mentioned above, you have the ability to seperately select where you restore the catalog and media. If you simply accept the restore defaults it will place the catalog and media in their original locations. If the media were originally on the NAS, that is where it will be restored as long as the Media location is mapped on your new computer identically to where it was mapped on the original computer.

3. Yes, as Somya said. Actually the easiest way is to first, install Elements on your new computer, then copy or restore the ProgramData\...\Catalogs\ folder from your backup to the same location on your new computer. I assume the media location will not change on your NAS and that the new computer will be mapped to the NAS with the same drive ID. When you then start Organizer, if it doesn't already have the desired catalog open, you will need to open Files > Manage Catalogs and open the desired catalog.

 

By the way, I agree with you that Organizer is not a Backup solution. In fact, Adobe could stand a significant update to all of it's file handling. But then again, I'm not sure I would use it even if it was better. I like the backup solution I use as it is automatic so I don't have to take any action to make it happen.

 

Bob

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
May 29, 2020 0
Adobe Community Professional ,
May 29, 2020

Copy link to clipboard

Copied

The catalog MUST be on a non-removable drive in your computer. I have argued this with Adobe to no avail. The media can be anywhere.

Bob, 

Presently, my catalog (my main catalog shared between two computers) is a folder on an USB external drive. Something I have been routinely doing since more than ten years, and which is well documented in the forums, less so in Adobe's help docs.

1. You cannot do a catalog-only backup. I'm not sure what you mean by the 'metadata only'. Certain types of files (like jpg) have metadata as part of the media file itself so it automatically stays with it's file. 

I am sorry, but it is necessary to distinguish between data stored in a catalog and data stored in the metadata header of jpegs / tiffs / psds or in xmp sidecar files. What can be easily stored is the whole catalog folder. It's a simple copy.

The catalog folder keeps crucial data like stacks, version sets, albums which file metadata can't store. Conversely, there are specific exif fields which are not managed in the organizer.

3. Yes, as Somya said. Actually the easiest way is to first, install Elements on your new computer, then copy or restore the ProgramData\...\Catalogs\ folder from your backup to the same location on your new computer. I assume the media location will not change on your NAS and that the new computer will be mapped to the NAS with the same drive ID. When you then start Organizer, if it doesn't already have the desired catalog open, you will need to open Files > Manage Catalogs and open the desired catalog.

Just imagine that after a crash, you need to restore your files library to a new drive or NAS. You have a good external backup (I also have Acronis backups) and you have a copy of your catalog folder. You restore your files with the same folder structure as before. You open your catalog. What will happen? I don't have a NAS, but I can tell you that with conventional drives, even with the same folder structure, all your files will be disconnected and your catalog unusable without trying to reconnect all files. Good luck! The reason is that the location of files stored in the catalog is based on the drive identification (internal serial drive number) together with the folders path. What would work would be to use not a restored tree, but on a 'cloned' drive, which should keep the internal serial number. I don't know if you would have the same issue with a NAS, but the forum is full of horror stories...

 

Yes, the organizer backup is slow: about the same time as a simple copy on an external drive. With my slow computer and external USB3 drives, I need 3 to 5 hours to backup a 100 000 items catalog of 650 GB. It's a night job.

So, I also use Microsoft SyncToy after each session manually and I copy the whole catalog. Just a few minutes. If necessary I can assign the old internal drive serial number to the backup drive to avoid reconnection of all files. Acronis clones are mainly used to backup the system drive.

Note that the situation is very different with Lightroom. You have to manage the backup of the files independently, and the reconnection is not a problem.

 

 

 

 

 

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
May 29, 2020 0
New Here ,
Jun 07, 2020

Copy link to clipboard

Copied

I read your answer here and perhaps you can help with a related problem.  I just moved all my photo files (91 Gb) into a Onedrive folder. I want to have them as "files on demand" as the vast majority are really archives and I want  to free up hard disk space and be more secure against hardware problems. I am now showing 83 nGB free space on my hard drive. I had to start a new catalog and all the photos are now in it. I am now being prompted to make a backup catalog and I guess have to do a full backup first. It asks me where I want to put it, but then tells me I dont have enough disk space and delete some files. This implies the full backup is going to make a copy of everything including all the actual photo files back on the hard drive. Is this correct?    The folder at C:\Program Files\Adobe\Elements 2018 Organizer is less than 2 gb.  Can I not make the backup in the Onedrive folder which has 900gb available?      

Thanks in advance if you can clarify for me!   

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 07, 2020 0
Adobe Community Professional ,
Jun 07, 2020

Copy link to clipboard

Copied

The safety offered by Onedrive is due to the ability to keep your Onedrive local folder  'in sync' with the cloud. If your local computer becomes unavailable (broken, stolen...) you are supposed to be able to recover your media as before on a replacement computer or drive. That's good, automatic and fast.

 

Now, what about the catalog folder itself?

I do use Onedrive to manage a special photo library I want to access anywhere from other computers or smartphones. A small selection (about 2 GB compared with my current 700 GB library).

It's possible to create a catalog  for that library.

I have not tried to store such a catalog inside the Onedrive folder (performance? compatibility), but what works is to keep the dedicated Onedrive catalog either on default location or anywhere outside of the Onedrive folder. It's very fast to save a copy of the working catalog folder in the Onedrive folder as a backup after and editing session. That's something you should do to be able to recover your catalog in case of a crash.

 

Reminder:

- you can always move or copy a catalog folder outside of the default location from the Explorer. Nothing simpler to save it where you want, including in the Onedrive folder. However I have not tested using such a catalog as the working catalog.

- As you correctly understand, the typical organizer backup requires saving both the media files (the library) and the catalog at the same time. That's because the restore must be able to restore the links in the database so that they point to the new location.
- The problem with 'point to the new location' as mentioned before, is that the database defines the location by two pieces of data: the drive identification (which is always false if the original drive has been replace) and the full path in the folder tree.

 

So, if you lose your original drive storing your Onedrive folder, the links to the replacement drive will all be false. Only disconnected files. Let's be clear: trying the 'reconnect' function is not realistic, contrary to what is current in Lightroom. Your catalog folder restored from the cloud will be lost.

 

So, I think your solution is a valid one except for the risk of losing the original drive/computer. In that case, I would strongly advise to create a full organizer backup (catalog plus media) on an external USB drive (quite affordable in 500GB) at regular intervals. In the worst case, the new and changed files after the last backup will be recovered on disk anyway from the cloud and only missing in the catalog which may be repaired rather easily.

 

I have never tested really the above scenario for the recovery from the cloud to a new computer, all my above suggestions assume that the Onedrive recovery process works for you. 

 

I do use Onedrive to share my small library with two distant computers. You don't  mention if that will be a requisite for you in the future. No difficulty for sharing the media library at all. However you might want to also share your catalog. There the difficulty will be the problem with the drive identification in the database. The catalog folder will be good for the original computer, not for the 'synced' one via the cloud. Unless you 'clone' the drive or assign the same internal serial number via a Windows command? Probably not impossible.

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 07, 2020 1
New Here ,
Jun 10, 2020

Copy link to clipboard

Copied

Thank you for replying so comprehensively - sorry I didn't reply earlier.

 

If I understand correctly then, because of the drive identity issue, Elements backups have to be kept on a physical disk and not in the cloud in my Onedrive storage. This is because as far as the Elements catalog is concerned, the image files in the cloud storage are related to the Onedrive folder on that original physical disk.

 

The first backup done WITH the organiser has to be a full one, which also backs up the image file library, and that will put a copy of all my image files library back from Onedrive onto my hard drive - which rather defeats the purpose of putting them on Onedrive in the first place. It will also take an age because it has to download all 100Gb from the cloud.

 

But unless I make a full backup that way, I can't do an incremental backup of the catalog for changed files.

Your suggestion is to do full backup as described every now and then which would still mean the lengthy download cloud>USB drive each time. 

 

So while I can use Onedrive cloud storage for both the library and catalog originals, and free up the equivalent hard drive space,  I can't do a backup on in cloud storage alone to protect against loss of the hard disk. The backup has to be on an independent physical drive that could be accessed from a new computer that would identify that physical drive identity as being the same as the catalog expects. 

  

This seems a bit strange in this day and age but seems to be what I have to conclude?   

Steve

 

 

         

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 10, 2020 0
Adobe Community Professional ,
Jun 10, 2020

Copy link to clipboard

Copied

The organizer catalog can only manage local files, not the files on the Cloud. Onedrive has the ability to automatically sync a specific folder on your local drive (can be moved to another local drive) with the storage in the Cloud. It's a kind of backup solution and also a way to to share the library between several computers. But that automatically means a duplication and you can't spare the local storage, even if you can decide to move the dedicated folder from the default location on C: to another internal drive (I have moved it to another internal drive on my main computer). I don't imagine it's possible to move it on an external drive.

So, if your goal is mainly to save space on a limited space SSD drive, you must have another internal drive with enough space. Generally not a big problem with desktops. Organizer or other third-party backups can be made from that other local drive as usual if you want. The speed of your Internet connection is irrelevant, it's good so long as you are happy with the 'syncing' delay. Then the syncing with the Cloud provides the kind of  'incremental backup' you are after. 

The catalog folder itself has to work from either the default (hidden) location on C: or from any other 'custom' location excpet the Onedrive system folder. I don't think it's possible or manageable to have it there for performance reasons, but I have never tested it. What is possible is to simply copy that folder into the Onedrive folder after each session as a copy (a matter of a few minutes). This is a good level of safety, except if the drive holding the Onedrive folder is broken, stolen... Then, I repeat that it's a good idea to do regular organizer backups from time to time, generally as a night job. You never have enough safety.

 

 

 

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 10, 2020 0