Debugging a plugin

[_Viktor_Linder_]

I am having trouble with a plugin.
When I access it from the menu I get the message box "Could not complete the Export command because there is not enough memory (RAM)". The message box appears before my PluginMain is called.
It seems to be related to the linking somehow; when I do not link against a specific library my plugin works. Weirdly, it also works when I am linking against the library and not referencing any stl containers. The dll is loaded (DllMain() is called) but PluginMain() is never called.

dumpbin /exports gives me this output:

Section contains the following exports for photoshopexporter.8be

00000000 characteristics
463B2018 time date stamp Fri May 04 13:59:20 2007
0.00 version
1 ordinal base
1 number of functions
1 number of names

ordinal hint RVA name

1 0 00001163 PluginMain

Summary

52000 .data
2000 .idata
F7000 .rdata
2B000 .reloc
2000 .rsrc
449000 .text

When looking at the dumpbin output of the "Outbound" sample I notice that the PluginMain() has an RVA of 1000. There is also a comment in the "Outbound" source saying "...most development systems require [PluginMain] be the first routine in the source." Is there a way to control the RVA of an exported function, and does it matter to Photoshop?

I have checked the DLL with depends.exe without finding any unresolved dependencies.

Is there a way to get more information from Photoshop as to why the plugin fails? Like a "Photoshop Console" or similar?

[qu1j0t3]

What development system are you using?

Can you show your source/Makefile?

Did you start by compiling one of the sample plugins?

[_Viktor_Linder_]

I am using vc2005.

I guess I could post a code snippet, but as it is the code isn't really doing anything; it never reaches PluginMain().

I have compiled the "Outbound" plugin, and it works perfectly. I have also compared my project settings to "Outbound" and not found any notable differences (of course, _something_ must be different though :).

[qu1j0t3]

Have you checked your PiPL for correctness?

[_Viktor_Linder_]

Yes, at least I think it is correct. I compile it with the same commandline as is used in the "Outbound" example:

#define plugInName "My Test Exporter"
#define plugInCopyrightYear "2007"
#define plugInDescription \
"Test export plug-in module for Adobe Photoshop¨."

#define ResourceID 16000

#include "PIGeneral.h"

//-------------------------------------------------------------------------------
// PiPL resource
//-------------------------------------------------------------------------------

resource 'PiPL' (ResourceID, plugInName " PiPL", purgeable)
{
{
Kind { Export },
Name { plugInName "..." },
Version { (latestExportVersion << 16) | latestExportSubVersion },

CodeWin32X86 { "PluginMain" },

SupportedModes
{
noBitmap, doesSupportGrayScale,
doesSupportIndexedColor, doesSupportRGBColor,
doesSupportCMYKColor, doesSupportHSLColor,
doesSupportHSBColor, doesSupportMultichannel,
doesSupportDuotone, doesSupportLABColor
},

EnableInfo
{
"in (PSHOP_ImageMode, GrayScaleMode,"
"IndexedColorMode, RGBMode,"
"CMYKMode, HSLMode,"
"HSBMode, MultichannelMode,"
"DuotoneMode, LabMode)"
}

}
};

[qu1j0t3]

Yes, that looks kosher. Re-reading your original message, hmm, I see it was unlikely to be the PiPL anyway.

I don't use VS to build my Win32 plugins, so I can't be sure, but I know that CodeWarrior used separate DLL versions of some libraries. Are you setting everything up correctly in VS for a DLL build product?