512px Image Previews/Thumbnails from 10GB+ PSB Files

[Chris Sw]

Is it possible to quickly (milliseconds) extract a 512px image thumbnail/preview from large images (10GB+ PSBs) on MacOS?

USE CASE

I need to quickly get an image preview/thumbnail (~512px) of a batch of large multi-GB images on import into FileMaker to accompany metadata that is accessed with the aid of ExifTool.

SOURCE IMAGES

The original images could be as large as 10GB+, 8-bit to 32-bit, RGB or CMYK, layered or flattened, with or without Alpha/Spot Changes, with or without background transparency, and are saved from Photoshop as either TIFs, PSDs, PSBs, PNGs, JPEGS, or HEICs with Thumbnails and Maximize Compatibility enabled.

EXPERIMENTATION:

ExifTool can generate 160px thumbnails from the embedded -PhotoshopThumbnails tag, but that’s smaller than I’d like:

exiftool -b -PhotoshopThumbnail in_path > out_path

If the ExifTool output below captures everything, the only thumbnail tag in the metadata is -PhotoshopThumbnail, which again is too small:

exiftool -a -b -W %d%f_%t%-c.%s -preview:all DIR

QuickLook has limited success getting 512px images with this code:

qlmanage -t source_path_to_file -s 512 -o out_path

Unfortunately, QuickLook has problems with larger PSBs (starts having challenges at 2.12GB, and >4.3GB+ will fail). It also doesn't always play well with TIFFs (particularly CMYK with Alpha and/or Spot Channels), or if they are over 2GB. The same is true of Sips, which is even more finicky about first converting to RGB color space, etc., and starts losing the ability to retrieve other metadata as the files get into the 4.3GB+ range.

ImageMagick is too slow when working with multi-GIG PSBs and it’s not feasible to generate from Photoshop.

Adobe Bridge seems to be able to get large previews quickly of 10GB+ files when set to process files this big. How does it do that/where does that data come from? Is there a way to replate that outside of Bridge? JavaScript? 

MacOS Finder Preview also is unable to generate previews of PSBs that get into the ~4.5GB range.

Any other ideas that might work in combination with FileMaker to quickly access/produce previews/thumbnails on 10GB+ PSBs in the 512px range on import? AppleScript? JavaScript? Python? Have been playing with the PIL library a bit.

* Found an old post related to this here, but the PSB QuickLook plugin referenced there wasn't helpful for my images.

[Stephen_A_Marsh]

You have obviously tried a lot of things.

If the native thumb isn't the right size, then there isn't much that you can do for *extraction* as the size is what it is.

The best thing you can do is use automation such as Image Processor or Image Processor Pro to open such large files and then save smaller copies.

As you know, this will not take milliseconds, even with a script.

Perhaps somebody else has a creative idea... If the Bridge preview files could be accessed that may be your best bet.

EDIT: If you look at your Bridge cache prefs, you will see a path to the folder, then there are subfolders for 256px or 1024px and then further subfolders that contain the preview images –

/Users/username/Library/Caches/Adobe/Bridge/Cache/v36/1024/DesktopEAFAFFC3/my template.psd.png etc. Some files are PNG and others JPEG, I don't know why the format isn't consistent.

[Chris Sw]

Thanks @Stephen_A_Marsh, if only I could figure out how to generate those without having to use Bridge that would be brilliant. When you open a Photoshop file with Shift+Option, it allows you to read the composite data instead. I wonder if there is a way to access that without going into Adobe? That opens in Photoshop nearly instantaneously. Not sure if this data is stored for quick access in the file or generated on open. From there would need to discard Alpha Channels, convert to 8-bit if not already, convert to RGB/sRGB if not already, discard all alpha channels if not already, then resize. Will try to connect with Adobe in the new year to see if they have any suggestions...

[Stephen_A_Marsh]

@Chris Sw 

Of all the methods that you have tried, Bridge cache thumbnail images seemed the fastest for your purposes.

The composite data is a good idea, but I have no idea of how to access that through an action or script. Perhaps a keyboard macro program could simulate opening with shift+opt as part of a wider automation, however, it will still take time to resize, remove or change unwanted data, save. Not milliseconds. :]

EDIT: What about "Open as smart object"...

[Conrad C]

I recently had to generate hundreds of thumbnails, fitting each proportionally within a square of specific pixel dimensions, for a similar task but not for FileMaker. After playing around with several tools, the one that ended up working the best was GraphicConverter (shareware), using its File > Drag and Drop Converter command although it has other conversion/batch options. I only had to define a Batch Function preset and output folder, threw the images into the drop zone, and it was done in seconds by using all CPU cores.

However, the files I thumbnailed were not as large as the ones you have mentioned. And I don’t know if it handles PSB. However, GraphicConverter is scriptable.

(I tried several Adobe tools first, but for various reasons including the specific file formats I had to thumbnail, no Adobe tool could actually do it properly.)

[Chris Sw]

Thanks for the thought Conrad. It doesn't appear to like files this large using (GraphicConverter 12 on MacOS). Opening an 8GB PSB just gives me a white result. If I composite the PSB and try it, then it will open it. GC isn't really a viable solution though since we are running from multiple locations and users.

[r-bin]

This test script extracts the thumbnail from test.psb and writes it to test.jpg. Check on your large files.

var file = new File("F:\\((\\test.psb");
var jpg  = new File("F:\\((\\test.jpg");

main(file, jpg);

function main(file, jpg)
    {
    try { 
        file.open("r");
        file.encoding = "BINARY";

        var offset = 0;

        var hrd_len = 26;

        var buff = file.read(hrd_len);
        offset += hrd_len;

        var color_mode_len = get_integer(file.read(4));

        offset += 4 + color_mode_len;

        file.seek(offset, 0);

        var image_resource_len = get_integer(file.read(4));
        offset += 4 + image_resource_len;

        var buff = file.read(image_resource_len);

        file.close();

        var x = buff.indexOf("8BIM\x04\x0C\x00\x00");

        if (x < 0) { alert("Thumbnail not found"); return; }

        var len = get_integer(buff.substr(x+8, 4));

        buff = buff.substr(x+12);
        
        var format = get_integer(buff.substr(0, 4));
        var w      = get_integer(buff.substr(4, 4));
        var h      = get_integer(buff.substr(8, 4));

        buff = buff.substr(28, len-28);

        jpg.open("w");
        jpg.encoding = "BINARY";
        jpg.write(buff);
        jpg.close();    

        alert("Thumbnail " + w + "x" + h);
        }
    catch (e) { alert(e); throw(e); } 
    }

function get_integer(s)
    {
    try { 
        var len = s.length;            

        var mult = 1;
        var ret = 0;                 

        for (var i = len-1; i >=0; i--)
            {
            ret += s.charCodeAt(i) * mult;                 
            mult *= 256;
            }

        return ret;
        }
    catch (e) { alert(e); throw(e); } 
    }

[Chris Sw]

@r-bin thank you. That produced a 160px jpg, same as the ExifTool command. I was hoping there was a way to get a larger (~ 512px image).

[Stephen_A_Marsh]

@r-bin thank you. That produced a 160px jpg, same as the ExifTool command. I was hoping there was a way to get a larger (~ 512px image).

By @Chris Sw

You can only extract what is there to begin with. You are now talking of generating a new pixel size image. I presume that scaling up a smaller image isn't acceptable.

[Chris Sw]

Correct; that wouldn't be acceptable.

[r-bin]

Composite data, if it was created when saving with “maximum compatibility”, is located in the very last section of the file, i.e. at the end. When you try to position the pointer of a file whose size is more than 2 (or 4 gigabytes, I don’t know exactly), the file error “I/O error” occurs in the script.

[Chris Sw]

Ah, thank you, I was not aware. I have files set to always save with transparency and Max Compatibility on. Could it be something Adobe would possibly consider changing how the data is stored in future generations of the PSB format? Would it be larger thumbnails or composite data at the beginning of the file?

[Stephen_A_Marsh]

@Chris Sw 

I have created a couple of scripts for you, I think the second one may be a bit faster than the first, but you will need to test it on your large files and report back.

I'm hoping that these will be faster than using Batch Actions, Image Processor or Image Processor Pro etc.

At the moment you are asked to select one or more files. This could be easily changed to select a top-level/root folder and then process all files of a given file type at this root level. Another option would be to process all files in all nested subfolders.

A new directory "512px" will be created on the desktop for the thumbnails.

The first script resizes to fit 512px on the longest edge, maintaining the original aspect ratio. The second script creates a 512px 1:1 square aspect ratio. Either of these could be changed.

Script version 1.0A:

/*
Create 512px Thumbs.jsx
v1.0A 26th December 2023, Stephen Marsh
https://community.adobe.com/t5/photoshop-ecosystem-discussions/512px-image-previews-thumbnails-from-10gb-psb-files/td-p/14313667
*/

var selectFile = File.openDialog("Select the file/s:", Multiselect = true);

var saveFolder = new Folder("~/Desktop/512px")
if (!saveFolder.exists) {
    saveFolder.create();
}

var timeDiff = {
    setStartTime: function() {
        d = new Date();
        time = d.getTime();
    },
    getDiff: function() {
        d = new Date();
        t = d.getTime() - time;
        time = d.getTime();
        return t;
    }
};

timeDiff.setStartTime();

var counter = 0;

for (var i = 0; i < selectFile.length; i++) {

    openAsSmartObject(false, true, new File(selectFile[i]), true);

    activeDocument.flatten();
    activeDocument.convertProfile("sRGB IEC61966-2.1", Intent.RELATIVECOLORIMETRIC, true, false);
    activeDocument.bitsPerChannel = BitsPerChannelType.EIGHT;

    if (activeDocument.height > activeDocument.width) {
        activeDocument.resizeImage(null, UnitValue(512, "px"), null, ResampleMethod.BILINEAR);
    } else {
        activeDocument.resizeImage(UnitValue(512, "px"), null, null, ResampleMethod.BILINEAR);
    }

    saveJPEG(new File(saveFolder + "/" + activeDocument.name.replace(/\-\d+$/, '')), true, true);

    activeDocument.close(SaveOptions.DONOTSAVECHANGES);

    counter++;

}

alert("512px Thumbnail Generation Completed!" + "\r" + "Files selected: " + selectFile.length + "\r" + "Files processed: " + counter + "\r" + "(" + timeDiff.getDiff() / 1000 + " seconds)");
saveFolder.execute();


///// FUNCTIONS /////

function openAsSmartObject(dontRecord, forceNotify, null2, smartObject) {
    function s2t(s) {
        return app.stringIDToTypeID(s);
    }
    var descriptor = new ActionDescriptor();
    descriptor.putBoolean(s2t("dontRecord"), dontRecord);
    descriptor.putBoolean(s2t("forceNotify"), forceNotify);
    descriptor.putPath(s2t("null"), null2);
    descriptor.putBoolean(s2t("smartObject"), smartObject);
    executeAction(s2t("open"), descriptor, DialogModes.NO);
}

function saveJPEG(saveLocation, copy, lowerCase) {
	function s2t(s) {
        return app.stringIDToTypeID(s);
    }
	var descriptor = new ActionDescriptor();
	var descriptor2 = new ActionDescriptor();
	descriptor2.putInteger( s2t( "extendedQuality" ), 6 ); // Compression value
	descriptor2.putEnumerated( s2t( "matteColor" ), s2t( "matteColor" ), s2t( "none" ));
	descriptor.putObject( s2t( "as" ), s2t( "JPEG" ), descriptor2 );
	descriptor.putPath( s2t( "in" ), saveLocation );
	descriptor.putBoolean( s2t( "copy" ), copy );
	descriptor.putBoolean( s2t( "lowerCase" ), lowerCase );
	executeAction( s2t( "save" ), descriptor, DialogModes.NO );
}

Script version 1.0B:

/*
Create 512px Thumbs.jsx
v1.0B 26th December 2023, Stephen Marsh
https://community.adobe.com/t5/photoshop-ecosystem-discussions/512px-image-previews-thumbnails-from-10gb-psb-files/td-p/14313667
*/

var selectFile = File.openDialog("Select the file/s:", Multiselect = true);

var saveFolder = new Folder("~/Desktop/512px")
if (!saveFolder.exists) {
    saveFolder.create();
}

var timeDiff = {
    setStartTime: function() {
        d = new Date();
        time = d.getTime();
    },
    getDiff: function() {
        d = new Date();
        t = d.getTime() - time;
        time = d.getTime();
        return t;
    }
};

timeDiff.setStartTime();

var counter = 0;

for (var i = 0; i < selectFile.length; i++) {

    app.documents.add(512, 512, 72, "_Temp", NewDocumentMode.RGB);
    placeFile(new File(selectFile[i]), false, 0, 0);
    var docName = activeDocument.activeLayer.name.replace(/\.[^\.]+$/, '');
    activeDocument.convertProfile("sRGB IEC61966-2.1", Intent.RELATIVECOLORIMETRIC, true, false);
    saveJPEG(new File(saveFolder + "/" + docName + ".jpg"), true, true);
    activeDocument.close(SaveOptions.DONOTSAVECHANGES);
    counter++;

}

alert("512px Thumbnail Generation Completed!" + "\r" + "Files selected: " + selectFile.length + "\r" + "Files processed: " + counter + "\r" + "(" + timeDiff.getDiff() / 1000 + " seconds)");
saveFolder.execute();


///// FUNCTIONS /////

function placeFile(null2, linked, horizontal, vertical) {
    var s2t = function (s) {
        return app.stringIDToTypeID(s);
    };
    var AD = new ActionDescriptor();
    AD.putInteger(s2t("ID"), 1);
    AD.putPath(s2t("null"), null2);
    AD.putBoolean(s2t("linked"), linked); // false for embedded
    AD.putEnumerated(s2t("freeTransformCenterState"), s2t("quadCenterState"), s2t("QCSAverage"));
    AD.putUnitDouble(s2t("horizontal"), s2t("pixelsUnit"), horizontal);
    AD.putUnitDouble(s2t("vertical"), s2t("pixelsUnit"), vertical);
    AD.putObject(s2t("offset"), s2t("offset"), AD);
    executeAction(s2t("placeEvent"), AD, DialogModes.NO);
}

function saveJPEG(saveLocation, copy, lowerCase) {
	function s2t(s) {
        return app.stringIDToTypeID(s);
    }
	var descriptor = new ActionDescriptor();
	var descriptor2 = new ActionDescriptor();
	descriptor2.putInteger( s2t( "extendedQuality" ), 6 ); // Compression value
	descriptor2.putEnumerated( s2t( "matteColor" ), s2t( "matteColor" ), s2t( "none" ));
	descriptor.putObject( s2t( "as" ), s2t( "JPEG" ), descriptor2 );
	descriptor.putPath( s2t( "in" ), saveLocation );
	descriptor.putBoolean( s2t( "copy" ), copy );
	descriptor.putBoolean( s2t( "lowerCase" ), lowerCase );
	executeAction( s2t( "save" ), descriptor, DialogModes.NO );
}

https://prepression.blogspot.com/2017/11/downloading-and-installing-adobe-scripts.html

[Chris Sw]

Thank you @Stephen_A_Marsh! I'm vacationing at the moment without my computer but will try when I return.

[Chris Sw]

@Stephen_A_Marsh, this took 72 seconds to complete on an 8GB file. It's opening/reading the entire file, as opposed to just the composite data as would be the case with Shift+Option on Open.

[Stephen_A_Marsh]

I was hoping that either opening as a SO (1st script) or placing as a SO (2nd script) would offer some performance benefit. I used place embedded, perhaps place linked would be faster... So you could try changing the function call parameter from false to true.

From:

placeFile(new File(selectFile[i]), false, 0, 0);

To:

placeFile(new File(selectFile[i]), true, 0, 0);

As I previously wrote, I am not aware of a scripting method to open the composite version and more advanced scripting people have not offered this as a solution. Have you looked into macro software to simulate keyboard presses in an automation?

Still sounds like using the Bridge preview cache images is the fasted method for you.

[Chris Sw]

@Stephen_A_Marsh, that made it take longer.

Here are a few benchmarks using my Test File (8GB layered PSB w/ lots of Alphas):

• Bridge Workflow/Export........ 5 sec
• ImageMagick.................................. 30 sec
• Stephen Script 1b (true)........... 54.3 sec
• Stephen Script 1a.......................... 55.7 sec
• Stephen Script 1b (false).......... 66 sec
• PS Image Processor..................... 1 min 33 sec

Opening the Test File in Photoshop reveals the most time spent is on reading all of the data:

• Open (Regular) - 48 sec
• Open (Composite / Holding down Shift+Option on Mac on Open) - 2 sec

This would explain why your scripts are taking longer than 48 seconds. Therefore, the trick, if possible, is to find a way to ony read the composite data and skip reading all the layers, etc. I think this may be what Bridge is doing to get their previews so quickly?

Surprisingly, Photoshop's Image Processor doesn't appear to be as efficient as Bridge's when converting images... The PS Image Processor opens the entire file, taking 1:33 to complete a conversion on the test file vs a ~5 second export from Bridge. Seems the Photoshop Image Processor could be improved by working with the composite data when available instead (assuming that's what Bridge is doing)?

ImageMagick reads the composite with [0] at the end of the filename. Is there anything comparible for your JavaScript code? Without that, I get the following error on the Test File: "unable to decompress image `/.../test.psb' @ error/psd.c/ReadPSDChannel/1515".

The following ImageMagick code took ~30 seconds on the Test File:

magick 'pathToTestImage[0]' -resize 512x512^ -gravity center -extent 512x512 -density 72x72 pathOut.png

The ImageMagick code appears to be the fastest that I would be able to use so far, but it's still too slow. While it would be ideal to get it in less than a second, if I could get it created in less than 5 seconds (around what Bridge does), that could be workable. I just don't know how to get the Bridge results without going into Bridge. If it could be accomplished with Adobe's Scripting API for Bridge, that may work, but I don't have any experience with that.

Ideally, I'd be able to come up with some script that could be run with a Terminal command, AppleScript, JavaScript, Python/PIL library, etc. that could quickly access the composite data on a big file like this without having to open Bridge to do it.