Please explain Bridge's file/metadata management (incl. Collections, Smart Collections)

Hi, Stephen,

In fact, it would be preferable to answer my questions / comment on my remarks/allegations (which may be mistaken in part) in contra-order of my posts above (3-2-1, not 1-2-3) since indeed, some of my initial questions I could answer myself later on.

I hope that some Adobe official (or some user with deep technical knowledge of Bridge) will enter the discussion since I'm not a software engineer but a technical journalist (and a photo amateur), and my own tries with Bridge have not been really convincing while at the same time I have to admit that I left the try very early on: The installation files were very near 1 gb (compare that with your, implied, assertion that Bridge was just a file manager from which we shouldn't ask what a file manager isn't able to give), the installation was a monstruosity, and then I had a real psychological problem with the license and other "Adobe account" questions with regards to my data, so I tried to de-install Bridge before really trying it thoroughly, and that de-install went wrong (perhaps my fault?) to such a degree that I had to re-install Windows and all my other programs in order to make Windows and my system stable again.

That's why now, when I'm going to compare Bridge with other DAM tools, I'd like to get the necessary info the "easy" way, ie without having to install and then to quasi-reengineer Bridge in order to know how it works. Also, it is known that while Bridge is free and its (serious) competitors are not, Bridge seems very slow in parts, from what people in the web say, and I try to dissect why this may be the case and, hopefully, how that can be overcome, by smarter decisions re how to use Bridge:

As said above, probably it would been a good, initial, idea to only use Smart Collections, avoiding Collections, but from your SML examples above, for which I'm very grateful, it seems that Smart Collections are internally organized as Collections, too, ie in "endless" textfile lists which then will have to be searched/scanned one by one, the partial results then being combined (by addition or substraction) which, if true (no indexes?) will take some time of course for building up the results list from within really big file sets (which I don't have for real on a system where I could re-install Bridge again just for trying out seriously; also, XML use could indicate not multiple textfiles but one monster XML file so I should learn more about XML before making final assertions upon the data gathering, but it's known that XML is much slower than a well-designed db indeed, and it's obvious even now that the ubiquitous (third-party) web info "Bridge isn't/doesn't use a db but is a file manager for pics, etc." or similar is totally misleading since from the above, it's obvious that Bridges FUNCTIONS as if had a db as its back-end, it just uses (technically inferior and much slower) XML files instead of the db you'd expect from a DAM.

If this sounds like a rant: it isn't: I'm just trying to understand and to (finally, correctly) describe, but as an aside, LR is said to be very slow, too, and I myself once had bought, and then quickly shelved, LR 3 which was unbearable slow indeed (on my then quite slow-processored computer I admit) - the irony being that LR using a db (SQLite, as said). But then - and I hadn't been aware of this fact then -, LR speed problems also depend, and probably for its better part (but not entirely, it seems from what users say on the web), on the size you want the previews (it displays instead of the original files) to be in. (Unfortunately, technical info from Adobe itself, in the official descriptions / help files, for the user of both of their programs, is too sparse so that the user has to rely upon third-party info or Adobe info buried deep in these fora at best there, and which gives way to multiple bad workflow design decisions on the part of the user who often discovers by chance only how to organize their things better - I'm speaking of amateur LR users here of course, photo professionals using either program will try to be better informed, but this will cost them lots of effort then, because of the poor availability of technical info.)

From what I know up to now, it seems to me that Bridge was inferior to its competitors so that it would be a good idea to spend the $100-200 (often much more if server / multi-user) they cost, but then, I may be mistaken here again, so a list of Bridge's strong points (USPs) would be more than welcome. And again, what about thumbs of files on drives currently not being connected?

Also, I see a problem with the Collections. Let me briefly explain the difference between tagging (Smart Collections) and virtual folders (Collections).

For tagging (Smart Collections), the info "belongs to" is coded within the file itself or within some representation of the file, or, for speed reasons, it's replicated from the file to some representation of it, from which it's then fetched for the gathering. For example, you have 2 tags "Persons" and "Person x". If then you gather everything about Person x, you do a (live or stored) search "tag persons" AND "person x"; as you see from my example, we're into hierarchical tagging if a simple search "person x" will suffice but which internally is probably encoded as "px": tag "persons", then "person x" within that parent category/tag; this way, multi-thousand "person x" can easily be retrieved, without this needing much storage, and if you then just want to get the pics where "person x" is displayed together with "person y" or "person z" and in their common holiday in "locality z", you do a combined, "Boolean" search which internally will probably be coded "((px and py) or (px and pz)) and lz", simplified to "lz and px and (py or pz)" (the parentheses being here just for clarification since "or" has precedence over "and").

For virtual folders (Collections), though, you have a file-system-textfile "px" which perhaps then lists 5,000 full-paths for person x, you have another textfile "py" which may list 3,000 full-paths for person y, and so on, and if you only use those Collections for "parent categories", like persons, you'll have to maintain a textfile "p" with probably 500,000 full-path entries, for any persons: You see that we have, at the very least, a quite incredible waste of storage needs.

Then, when you try to combine these categories, as in our tagging example above (since you didn't do all these "Collection" assignments as tags, too, or does Bridge do this internally then, automatically?), first: is this even possible? And second, if it's possible, it's quite incredible internal processing: First, thousands of entries/full-paths of the file px must be stored, then thousands of entries of the file py must be compared, one by one, to any entry in the list px; ditto then for all the entries of the file pz (compare to px), and the program must build a sub-list which only retains the px-py joins and the px-pz joins, and from this intermediary-target list, any entry then must be compared with any entry in the file lz.

The program may maintain some sort of indexes, in order to speed this up, but in any case, this "file-based" virtual-folder listing would cause lots of unnecessary processing, and that's because for virtual folders, the "belongs to" info isn't stored with the pic-files, but all their paths, for just one single such "virtual folder" (Collection), are listed together, as redundant info and info which only with difficulty could be shortened (you could of course build additional accordance tables for translating shorter path codes into the full-paths).

From the above, it seems quite obvious that users should avoid Collections and just do hierarchical tagging instead, in order to get immediate results even for Boolean combinations of "Collection" criteria, and in order to hold their whole dataset (incl. all metadata internal to Bridge) as slim as possible, except for the fact that perhaps Bridge organizes it all quite differently from my description above? Then: How is the internal organization that Bridge has adopted?

From the above, it becomes obvious at least that in order to sensibly decide how they will best organize their "stuff" in and within Bridge, from a user-system interaction point of view, the users need to know about this internal data organization, so they can make knowledgeable decisions.

And: Did I overlook some aspect when saying virtual folders should be avoided but replaced by hierarchical tagging? Of course, there should be "virtual virtual folders", ie (adjustable) preset simili-virtual-folder structures (adjustable "defaults"), for display of the lists/trees, but which internally would be built up on-the-fly, in real time, by hierarchical tagging.

And it appears obvious to me - hence my question what I may overlook here - that a db is the technical solution of choice for all this, all the more so since not only the db's SQL power becomes available, but that then also any necessary index is built up and maintained/updated automatically by the db engine (which in LR is SQLite; embedded Postgres would probably have been a smarter choice there).

Could it be that Bridge's internal "list" management just derives from its origins, many years ago, and that a total code overhaul, needed by the implementation of a db, has then be postponed up to this day, or does the avoidance of a db in Bridge comes with real advantages I don't see, and which would overcome some of the drawbacks of the db missing? (It goes without saying, I think, that a good db design would come with export functionality, at the very least into the ubiquitous .csv format, so that (meta)data exportability reasons would not apply here.)