Known Participant

Question

Need precise description of tile data received after advanceState()

Forum|Forum|16 years ago
June 20, 2009
3 replies
3697 views

I am working on my first filter. It isn't running yet, so the code I use to get a tile of data from Photoshop (modified slightly from the Dissolve sample) is just a guess:

            SetInRect(inRect);
        SetOutRect(inRect);
        if (filterRecord->haveMask) SetMaskRect(inRect);
        filterRecord->outLoPlane = filterRecord->inLoPlane = 0;
        filterRecord->outHiPlane = filterRecord->inHiPlane = filterRecord->planes - 1;

int result = filterRecord->advanceState();

where inRect is the tile rectangle.

As far as I can tell, the tile data arrives in filterRecord->outData. How is the data arranged for the various image modes? I've highlighted my major questions by using bold below.

My best guess is that the data is interleaved:

pixel1 data, pixel2 data, ...

Within each set of pixel data, we have:

color channels, then ...?

So here is where the confusion begins. My best guess for RGB is:

R, G, B, alpha, inverted alpha, mask, inverted mask

Is this correct (given the LoPlane and HiPlane I used)? If not, what is the correct data received and order?

I would be happy with:

R, G, B, alpha, mask

Is there any way to get just this data? This is not critical.

For the other modes, I'm guessing RGB would be substituted by G (for Grayscale), CMYK, HSL, HSB and LAB, with the channels in the order listed. Correct?

For 8-bit modes, the values range from 0..255. How is the "H" in HSL and HSB handled since it normally goes from 0..359?

In 16-bit modes, values range from 0..32768, at least for RGB. What are the ranges for the other modes?

For 32-bit mode, I'm guessing that the values range from 0.0..1.0. Correct?

Are the alpha and mask layers always 8-bit, regardless of the color data size?

Finally, I'm assuming that the alpha layer controls transparency but the mask layer tells me how strongly to apply my filter effect. A mask value of 0 would leave the pixel unchanged. Is this right?

Do I have to handle masking on my own or will Photoshop automatically handle masking?

Answers to the bolded questions should allow me to process the tile contents correctly. Feel free to link to a resource that answers the question. I've gone through most of the SDK documentation and examples and have not found that to be a good way to resolve these questions.

Thanks for your help!

SDK

This topic has been closed for replies.

T

TonyFreixasAuthor

Known Participant

Well, it's been about two months since I posted my questions. I now have a working filter plugin, although it will require some testing to ensure it works with most color modes and depths. At least it works pretty well with 8-bit RGB with or without an alpha channel. The preview mode supports zooming from 10% to 1600%, scrolling and color picking. The performance is pretty good, even with a full size preview window on my 1920x1200 monitor.

Anyway, I can now try to answer some of the questions I posted. Let's start with the easy ones:

For 8-bit modes, the values range from 0..255. How is the "H" in HSL and HSB handled since it normally goes from 0..359?

Okay...duh. HSL and HSB are not native Photoshop color formats (see Image / Mode on the Photoshop meu for available formats). HSL and HSB color spaces are defined, but they have nothing to do with the data obtained through advanceState().

I believe that when Photoshop uses HSB and HSL, it uses at least 16-bit depths. I would guess that an attempt to convert from 8-bit RGB to 8-bit HSL would generate an error, but I don't know.

In 16-bit modes, values range from 0..32768, at least for RGB. What are the ranges for the other modes?

For the image modes available through the Photoshop menu, the ranges are 0..255 for 8-bit, 0..32768 for 16-bit and 0.0..1.0 for "32"-bit depths. I put "32" in quotes because floating point numbers use the data type "nativeFloat", which on my machine is 64 bits. If you decide to get clever somewhere and try to use sizeof(somedata) * 8 to figure out the depth, you'll get the expected answer for uint8 and uint16, but not necessarily for nativeFloat.

For 32-bit mode, I'm guessing that the values range from 0.0..1.0. Correct?

Yes.

Are the alpha and mask layers always 8-bit, regardless of the color data size?

No. It's important to understand that the word "mask" is used in two ways: to indicate a selection and to indicate a mask layer. If you see the word "mask" without the following "layer", it's probably a selection.

Selections are always 8-bit and the selection data is accessed through the masData field in the FilterRecord.

The alpha channel and the mask layer have the same depth as the other channels in the image.

Finally, I'm assuming that the alpha layer controls transparency but the mask layer tells me how strongly to apply my filter effect. A mask value of 0 would leave the pixel unchanged. Is this right?

The alpha layer controls transparency; that's correct. The "mask" (selection, not the "mask layer") controls how strongly to apply the filter effect.

Do I have to handle masking on my own or will Photoshop automatically handle masking?

Yes. You can set up the advanceState() operation to work either way by setting the FilterRecord's autoMask field (default is true, I believe).

More answers later...

Message was edited by: TonyFreixas Correction: I said: For the image modes available through the Photoshop menu, the ranges are 0..255 for 8-bit, 0..32768 for 16-bit and 0.0..1.0 for "32"-bit depths. I put "32" in quotes because floating point numbers use the data type "nativeFloat", which on my machine is 64 bits. If you decide to get clever somewhere and try to use sizeof(somedata) * 8 to figure out the depth, you'll get the expected answer for uint8 and uint16, but not necessarily for nativeFloat. Apologies, but it seems that 32-bit really is 32-bit. My confusion came from seeing nativeFloat as the only floating point typedef. I'm sure there's some good reason the Photoshop people didn't define a float32 type. Corrections are welcome.

Chris Cox

Legend

32 bit per channel is floating point data: there is no defined range. Negative infinity to positive infinity.

And we try not to let NaNs through, but they occur in some file formats.

Photoshop doesn't need to define float - the language already defines it...

NativeFloat is a holdover from the days when a double could be 64, 80 or 96 bits depending on configuration.

Selections are 8 bits, always.

Layer data matches the depth of the document, and that includes opacity and alpha channels.

T

TonyFreixasAuthor

Known Participant

Chris Cox wrote:
32 bit per channel is floating point data: there is no defined range. Negative infinity to positive infinity.
And we try not to let NaNs through, but they occur in some file formats

Thanks. This is not clear—I'll have to play with some files to see what happens. In the test cases I've been using, the values have been in the range of 0..1, but so far, all my test cases came by converting an image originally in 8-bit.

What would be the best way to get 32-bit image with data outside the range of 0..1? A RAW image? Create an HDR?

Photoshop doesn't need to define float - the language already defines it...
NativeFloat is a holdover from the days when a double could be 64, 80 or 96 bits depending on configuration.
Selections are 8 bits, always.
Layer data matches the depth of the document, and that includes opacity and alpha channels.

This matches my understanding exactly. I've been too busy lately optimizing my filter's display to have had time to update this thread. Thanks for your contributions, particulary the clarification about 32-bit,

T

TonyFreixasAuthor

Known Participant

Well, this is a surpirse...no one knows how the data is structured? Maybe all the people who know are on vacation—it is summer, after all! :-)

I got my plugin running, but without a user interface. The code is broken right now as I'm working on adding the GUI. When I get the code working again, I can create some specially constructed images and try the various color modes and depths, with and without alpha, transparency and selections. I'll post my results back here.

When the plugin ran, the image was in 8-bit RGB mode with an alpha channel, no mask layer and no selection. Setting the low and high planes as shown below gave me the bytes as R G B A R G B A R G B A ...

I believe I should be setting wantsAbsolute in the FilterRecord to prevent getting the color channels re-ordered on me. That's about all I know right now.

Chris Cox

Legend

No, I'm just wondering how you skipped the part of the SDK documenting the data - and where you specify how you want the data structured as part of the filterRecord and your advance state call.

T

TonyFreixasAuthor

Known Participant

Hi, Chris!

I'll assume you meant the question literally and not as some sort of put-down. I'll deal with it in two parts:

First, go to your Adobe Photoshop CS4 SDK\documentation\html and do a grep for pgwinplugins.html. There are links to this file from pages.html and programmersguidemain.html (and three self-references). Now grep for programmersguidemain.html. There is a link to this only from pages.html. Finally, grep for pages.html. There are no links to this page—therefore, unless you go scrounging around in the raw HTML documentation, you will miss out on some valuable information.

I hope this explains one way someone may "skip part of the SDK documentation".

For the second part of my answer, I read the information provided in the SDK detailing how the data is delivered. This information (at least, what I found) contains ambiguities. These ambiguities may not jump out at someone who already knows the answer.

If you are willing to point to the location(s) of the documentation you are referring to, I can point out the ambiguities—unless, that is, the documentation is buried somewhere hard to find and is not something I was able to find. In that case, I can at least answer the "how I skipped the part of the SDK" question.

T

TonyFreixasAuthor

Known Participant

I figured I might as well start answering some of my own questions:

If not, what is the correct data received and order?
Are the alpha and mask layers always 8-bit, regardless of the color data size?
Do I have to handle masking on my own or will Photoshop automatically handle masking?

I found this relevant piece of documentation:

Boolean FilterRecord::autoMask

Enables or disables auto-masking.

By default, Photoshop automatically masks any changes to the area actually selected. If isFloating=FALSE, and haveMask=TRUE, the plug-in can turn off this feature by setting this field to FALSE. It can then perform its own masking.

If the plug-in has set the PiPL bit writesOutsideSelection, this will always be FALSE and the plug-in must supply its own mask, if needed.

void* FilterRecord::maskData

A pointer to the requested mask data.

The data is in the form of an array of bytes, one byte per pixel of the selected area. The bytes range from (0...255), where 0=no mask (selected) and 255=masked (not selected). Use maskRowBytes to iterate over the scan lines of the mask.

If the plug-in has set the PiPL bit writesOutsideSelection, this will always be FALSE and the plug-in must supply its own mask, if needed

In answer to #1, it looks like mask data is received separate from channel data. Or it may be that there are two meanings for the word "mask". One may apply to the masking performed by a selection and the other to the mask layer. This is a new question: Is the word "mask" used with two meanings?

In answer to #2, the selection "mask" is always 8 bits.

In answer to #3, it looks like Photoshop will handle masking automatically and by default in most cases.

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded