Debugging Plugin jumps into Disassembly in XCode (AWS_CUI_RevertAlert)

[mjordi]

Hi all,

I'm using the CS5 SDK and I'm trying to write a plugin for Illustrator. I've noticed some weird behavior when debugging, though. For example, when I call sAIArt->GetArtFirstChild(art, &child), XCode jumps into disassembly, specifically the function AWS_CUI_RevertAlert. Why is this?

It seems the code runs through fine. All pointers are not null pointers, I don't get errors in Illustrator or in the debug output. My code also seems to do what it should.

This is still annoying for debugging. When does XCode jump into this function and why? It doesn't happen for all SDK calls, but for many (e.g. GetArtType(), but not GetArtSibbling())

Hope anyone knows how to fix this.

Thanks,

Jordi

[Guest]

This has been happening since the switch to XCode from CodeWarrior several years ago. Our theory is that it has something to do with certain API calls spawning separate threads which send the debugger off into lala land.

We've never been able to fix it. We either add another breakpoint after the known API calls that cause this behavior, or, if we forget, typing "finish" into the console will most often send you back to the right place.

Any other suggestions?

[inspector-g]

Yes, typing "finish" at the GDB prompt in the debugger will sometimes get you out of that assembly madness.

I recently found a solution that appears to work every time. If you step through a few instructions you'll see the runtime stack update in Xcode

with the proper information (until then it appears to be one frame "behind").

It seems to happen on the "mov" instruction. Now, I don't know assembly very well, but I'm assuming that means the program's internal memory (index) pointer will move to a new location for what I can only assume is a function's starting address.

Anyway, if you just hit "step over" 3 or 4 times, then "step out" you'll be back in your code.

I wrote a little GDB macro to make this easier. Add it to the ".gdbinit" file in your home folder and you'll have it every time GDB launches.

# definition to get out of lame AI SDK assembly code
define aifinish
       stepi
       stepi
       stepi
       stepi
       finish
       p "DON'T FORGET TO CLICK THE FIRST LINE IN THE XCODE RUNTIME STACK BECAUSE IT HAS THIS STUPID REDRAW ISSUE WITH SOME MANUAL GDB COMMANDS"
end

As the last line says, Xcode sometimes doesn't update the debugger UI info when manual GDB commands are run. Just click in it and you'll see the update immediately.

This little thing has made my life much easier. Enjoy.