File Format - baffling bug

[qu1j0t3]

Hi,

A user of my Paint Shop Pro file format plugin (now 6.5 years old) recently reported a bug and provided some test .pspimage files to trigger it.

The issue is quite simple, on reading these files, the red channel is solid white, but all other channels are read correctly.

The baffling part is: I've confirmed that the correct channel data is being passed to Photoshop. In addition, the following parameters are set:

imageMode=3 imageSize=(630,627) depth=8 planes=4 res=72dpi

           theRect=(2,0,632,627) loPlane=0 hiPlane=0

           colBytes=1 rowBytes=630 planeBytes=395010 data=0xaa2f000

           theRect=(2,0,632,627) loPlane=1 hiPlane=1

           colBytes=1 rowBytes=630 planeBytes=395010 data=0xaa2f000

           theRect=(2,0,632,627) loPlane=2 hiPlane=2

           colBytes=1 rowBytes=630 planeBytes=395010 data=0xaa2f000

           theRect=(2,0,632,627) loPlane=3 hiPlane=3

           colBytes=1 rowBytes=630 planeBytes=395010 data=0xaa2f000

Is anyone aware of anything that might trigger Photoshop to ignore the red (first) channel passed in?

Note that this only happens with certain files, and always to those files, and never to other files. One might justifiably argue that the issue must be input data dependent. However, as I say, I have confirmed that the correct channel data is being passed to Photoshop in pb->data.

Tested on PS 7.0 and PS CS2 on PowerPC. The exact code tested is revision 44 of http://telegraphics.com.au/svn/pspformat/trunk/

--Toby

[Chris Cox]

As no other file format is seeing this problem, I'd have to guess it's a bug in your code.

You may be overwriting the data, or using an incorrect channel mapping.

[qu1j0t3]

Chris,

I'm a little disappointed, though not surprised, that you've never seen this in other plugins. Nor have I. I agree that the most logical cause is a bug in my code. It would make me ecstatic to confirm that.

I am not overwriting the buffer in my code. You see from read.c that my debugging dump of channel data (which matches expected data) is done immediately before returning from read_start. The buffer is allocated using malloc(). Is that potentially an issue? If I am passing buffers back to read_start and read_continue, should they be a particular flavour of allocation?

I've already checked the planeMap, and it's always the identity mapping.

Until she sent me the files a week ago, nobody had reported this issue in six years. I can't find anything different about the files either, which adds to the mystery...  (I've now also reproduced this in Win32 using Photoshop 5.0.)

--Toby

[Chris Cox]

Ok, I'm baffled.

No, the allocation source doesn't matter. (as long as you don't deallocate until after the data has been copied in AdvaceState or after a Continue call).

If it were an API bug, I'd expect to see this in a lot of files and different formats.  But we haven't seen it anywhere else.

Something to try:  don't write the fourth channel (I'll bet it was opaque, aka white).

[qu1j0t3]

Hi Chris,

Thanks again for responding.

No, the 4th channel is a transparency mask, and it is read in correctly. I concur that an allocation issue appears unlikely. The buffer is allocated once in read_start, used for all channels, and free'd in read_finish. (Obviously all other channels are read in correctly.)

What I'll do next as a sanity check is dump the raw channel data to files at the point where read_start or read_continue end. I'm pretty sure this will confirm that the content is correct.

--Toby

[qu1j0t3]

The file dump confirms that the image data is being parsed correctly. Yet somehow the result in Photoshop is a clear Red channel.

Since I am stumped, I am offering a $50 PayPal bounty and project commit rights to anyone who identifies the bug in my code responsible for this. Refer to the attached file for working and non-working input files, and the raw dumps created from the non-working file.

The source code (GPL) is at http://www.telegraphics.com.au/svn/pspformat/trunk

(also note that this project is registered on Ohloh: http://www.ohloh.net/p/10163 )

The code also requires http://www.telegraphics.com.au/svn/common/trunk to build (check out as sibling directory named 'common').

Tested makefiles and build environments are:

- OS X CLI (make+gcc, not Xcode), targeting Photoshop CS2,3,4 on OS X;

- Visual Studio C++ Express Edition, targeting all versions of PS on Windows;

- MPW, targeting all versions of PS on MacOS Classic and PS 7 & CS on OS X.