GPU Plugin Error Help - Cannot call WorldIsWritable()

[Nathan Lovell_52]

Hello,

I've built the SDK_Invert_ProcAmp project and another simple plugin to work with GPU, and they've worked as far as initialising the ParamSetup and creating the UI. However, I get an error in each plugin that says: 'After Effects error: internal verification failure, sorry! {Cannot call WorldIsWritable() before checkout is rendered}'. I tried searching through dependencies and headers to see if I could find any method/function with this name, and also searched online, but was unable to find anything. Any ideas on what could be causing this? Happy to provide code or other resources

Regards,

Nate

 

[Mylenium]

Perhaps it's simply being picky with driver access? I have no knowledge of these SDK specifics, but on a more basic, generic level, checkout errors in AE typically mean that there may be a permissions issue somewhere and/ or combinations of layer switches cause weird behaviors. also the writable worlds stuff used to come up a lot a decade ago when developers got their feet moist in 3D rendering plug-ins, so I would further suspect that there may be some specific piece of code at play. perhaps you may also need to build some virtual dummy comp/ layer structure somewhere to ensure correct handling of some internal functions. Again, just shooting my brain farts into the blue here, but perhaps it can give you a clue where to look...

Mylenium

[shachar carmi]

do other compiled sample projects work correctly?

[Nathan Lovell_52]

All the normal projects work fine, however CUDA/GPU integrations have all gotten me to this error after fixing the file structure, linking, etc. I first used the included SDK_Invert_ProcAmp project and got this error, then I moved on to a simple grain plugin that just randomises pixel values, integrated the GPU stuff the same way as the included example project, and it also throws this error. 

I am using:
AE SDK 17.1.2

CUDA 10.2

Boost 1.73

AE CC 2018

Visual Studio 2019

I have also run tests to see how far it processes specifically. Adding breakpoints to each of the EffectMain switch cases reveals it runs the following order: GlobalSetup(), ParamSetup(), PreRender(), then it gets held up on that. Also, if it helps it loads the UI of the plugin, but leaves the affected layer as a black solid.

[shachar carmi]

ah. GPU is not my strong suite. let's see if any of the guys (and gals) here have an insight.

@Deleted User  perhaps?

[Nathan Lovell_52]

Yes or @Bruce Bullis I know has a great depth of scripting/extension/plugin knowledge! I wonder if it is some kind of memory management error?

[Guest]

Unfortunately I haven't encountered that specific error myself.

Since it also seems to occur with the provided samples, it might sense to check if the same situation with the compiled plugins also occurs on other computers - if yes, it is most likely a source code or compiler issue, if not, it hints at a problem with the local GPU driver and/or host application.

Also, seeing that you are testing with AE CC 2018, it might sense to check with the latest AE CC 2020, maybe there have been some bugfixes in the meantime.

[Nathan Lovell_52]

I've done lots of testing and experimenting, hopefully my findings will help. So I have 2 computers, a macbook air running Catalina, AE CC 2020, XCode 11.3, and Boost 1.73.0. And I also have a W7 Desktop running AE CC 2018, VS Code 2019, Boost 1.73.0, and CUDA 10.2.  I followed the PDF guide for compiling GPU plugins for AE on the Mac, and it WORKED! Although I don't have a CUDA enabled GPU on that device, it shows up with the proper GPU acclerated symbol and renders properly:

Now onto the Windows computer, it compiles fine and seems to like where all the dependencies and libraries are linked from, but when actually running the plugin in AE, I do NOT get the GPU Accelerated symbol, my GPU usage stays idle, and I get the aforementioned error, which prevents me from doing anything with the plugin or viewing it's effect.

With that being said is it safe to say this is a CUDA problem? One other thing is that in my .cu file the nvcc section is grayed out, does that also mean it's not recognising my CUDA device?

I have tested the .aex plugin file itself on a few other computers remotely, and it throws the error before starting up AE that you should reinstall the plugin. Hopefully this is something relatively simple like a versioning/compatability issue, as at this point it seems like linking GPU libraries is quite difficult! 

Lastly I did try to dig deep and search for anything in the guide that would be of help, and came across a patch note from version 16.0 saying: 'See “GPU Effects Changes” for details.' However there is only 1 instance of GPU Effects Changes in the entire document, so that was a dead end.

[Guest]

This could be an incompatibility with the CUDA runtime libraries, as far as I remember, the are not always downward compatible, so you should make sure you have the correct Cuda SDK and runtime version installed (latest version is not necessarily the one you need), and then compare what you are linking to with what is used at runtime (check for example using "Dependencies" freeware tool).

[Mylenium]

That would be my guess, too. I think CUDA 10 and up totally deprecated backward compatibility for anything lower than v7 and if I remember correctly, even in v9 you already had to run an extra install for legacy CUDA components.

Mylenium

[Nathan Lovell_52]

Awesome thank you Toby and Mylenium for the great advice. Couple more questions just to clarify:

1) What does the CUDA SDK have to be compatible with that you are talking about? Do certain GPUs require certain versions or what compatability am I checking/comparing? (I have a GTX 1080 running driver version 27.21.14.5655)

2) Which dependencies freeware are you referring to? (Quite a few freewares with that name)

Again much appreciated, feels like I am just about close enough to go off and figure it out once I understand what versioning/compatability is required!

[Guest]

You can check at compile time against what CUDA libs you are linking. These are usually either in the Visual Studio directories page of the solution/project properties or directly in the "Additional libraries" on the linker page. This determines which version of the CUDA runtime is required on the end-user's system.

You should check which runtime is required by a binary/dll by using tools like "Dependencies" (https://github.com/lucasg/Dependencies). It can show the full path of the CUDA DLLs it sees for that binary.

[Nathan Lovell_52]

May have made some more progress!

In my linker section, it just has cudart.lib and OpenCL.lib, so I don't believe that is the issue.

However, inside of Properties -> Configuration Properties -> VC++ Directories, I noticed it lists an older version of CUDA I had as well. Could this be the issue causing versioning conflicts? If so how do I change the $(CommonExecutablePath) macro to remove the older version? Hopefully this is the direction you were trying to lead me in, and if not get steer me back on track!

One other small note is that while skimming through all of the debug information upon build time, I noticed some instances of CUDA 9.0 being referenced, but a clean build resolved that.

Which files should I be dragging into Dependencies? I thought it would take the CUDA .lib files.

[Guest]

If it just says cudart.lib and OpenCL.lib in the linker section, it means it will try to locate these files using all the paths in the VC++ Directories setting, using the first one it finds. Having different CUDA versions (9.0 and 10.2) in the path for the binaries might be problematic, but you should primarily find out where these lib files are located on your disk (and if multiple instances are found, find which one gets used by the linker based on your folder priority setting).

As for your second question: lib files are used in the compilation/linking stage, not in the actual binary. Open Dependencies and drag your plugin binary (.aex) into it, then it will show the CUDA/OpenCL DLL (not LIB!) libraries it references and - again based on path/priorities - which one gets used currently on your system.

[Nathan Lovell_52]

Dependencies is showing me it's using CUDA v10.2. Another thing I tried was changing cudart.lib and OpenCL.lib to reference the full CUDA v10.2 path, but to no avail. 

Additionally, I restarted my computer and searched my harddrive for all possible instances of other cudart.lib files and found there to only be 2, both v10.2 in these locations: 

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\lib\x64
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\lib\Win32

Could this problem still be related to the CUDA 9.0 that is lingering around, or might it be something else entirely? I'd imagine in VS that a non-existant folder shouldn't cause this large of an issue. Are there other ideas you have for debugging and pinpointing this issue?

[Nathan Lovell_52]

@Deleted User @Mylenium So I used an old NVIDIA GPU Box Blur Plugin example as a new reference and essentially recreated the same way it includes and uses CUDA in my plugin. The great news is that this way of doing it led to it compiling successfully and at least for 8bpc, is working great and running some convolutions on my GPU now!

The only downside to this is that it only appears to be working on my computer (Win7, CC 2018). When I test it on a W10 machine on both CC 2019 and CC 2020, as soon as I apply the effect I get an Entry Point Error. 

To try and solve this, I did some searching around the forums, but only found this sort of issue for XCode compiling. I have tried testing with my Debug and Release versions, as well as trying to compile using different AE SDK versions. I also saved the previous version of this plugin before I started implementing the GPU stuff, and this works just fine on these computers, so it seems to point in the direction of possibly the CUDA or other stuff causing the issues?

Could this be an OS/AE/Visual Studio versioning error? Why might it be working only on my computer (so far) and giving this error on others/newer ones? Help greatly appreciated!

-Nate

[shachar carmi]

the entry point issue sounds like a dependency issue. most porbably, you've compiled the plug-in using /MTd, which makes it dependent on debug dlls which aren't present on a non-dev machine.

try compilining using /MT and then run it on the other machine.

you can copy "dependency walker" onto the win10 machine and check your non-working plug-in there. you'll see if it's indeed a dependency issue.

[Nathan Lovell_52]

@shachar carmi That is most likely the issue, now that you mention that!

I earlier had to change my Release Runtime Library to Multi-threaded Debug DLL. (Had to meaning to make it work a while back)

When I change it to the proper release version (/MD) I get a ton of _ITERATOR_DEBUG_LEVEL errors. 

After some googling and attempting to solve this, I tried adding _ITERATOR_DEBUG_LEVEL = 0 to my Preprocessor Definitions, but it still seems to have issues. I'm probably missing something simple here, but how do I get past this iterator issue?

[Guest]

Don't just add the preprocessor definition without changing anything, that will only lead to more problems.

Your iterator problems hint at that you are linking to libs compiled with incompatible settings. All linked libs need to have been compiled with the same settings as your main program/plugin. The most important relevant settings are architecture (32bit vs. 64bit), thread model (/MT vs. /MD) and debug vs. release mode (/M? vs. /M?d). Either you get the exact settings of each lib, or you compile all the libs you want to use yourself with the right settings as well.

[Nathan Lovell_52]

What's the best way to find these libraries that were compiled incorrectly? And are we talking like CUDA libraries that I'm including or other dependencies? I'm having trouble finding any debug/release mode settings for some of my libraries.

[Guest]

Check what .lib files you are including in the solution settings.
Then find out where these libraries come from and in the documentation for these libraries find out:
- for what compiler and compiler version they are made
- if they are 32bit or 64bit
- if they are debug or release
- if they use multithreaded mode or not (/MT or /MD)
- static or dynamic linking
- calling convention
Then adjust your own project settings accordingly.
Dynamic link libraries are usually safe if the interface is pure ANSI-C, then the linking is usually consistent across different compilers. In all other cases, you should not mix libs compiled with different setting or use libs with different settings than your main project or you will likely run into problems, often very hard to track down. This is not specific to CUDA or After Effects plugins, this is just how compilers work.

[shachar carmi]

i'd like to add on top of toby's answer, that if your previous compile settings were /MD (i.e. not debug), you can copy the dependent on dlls to the same folder as your plug-in, and on runtime the plug-in will look them up there.

it's perfectly fine for a plug-in to be dependent on distributable dlls, and it's up to your installer to deploy the components where they need to go. (but having the dlls in the same folder as the plug-in should be fine)

[Guest]

Shachar, that is only true for standalone applications, not for plugins, on Windows. For plugins that require external DLLs, these will be looked for in the directory of the host application (i.e. AE), *not* in the directory of the plugin. This was an intentional change by Microsoft some years ago.

[shachar carmi]

meh. just ignore the rumblings of the old man that i am...

[Nathan Lovell_52]

Thank you for the detailed information! I would like to ask a few clarifications/questions since compilers and libraries are getting into a new area of knowledge for me.

Should the only difference between my Debug and Release 'Preprocessor Definitions' be _DEBUG ?

Are these 'Additional Dependencies' the libraries I need to be ensuring are the right versions/compiled files? I have the same ones for both Debug and Release, which may be an issue?

Lastly, I also have a few .lib files within the plugin directory itself, could these also be libs causing this issue?

Thanks!