Working on a plugin for creating custom mask types, like identifying certain types of objects in an image, and would like information on how best to interface with Lightroom's masking system

[View this post in your web browser. It contains formatting and images that don't appear in email.]

You’ll find that the LR plugin SDK won’t provide most of the core functionality you’re looking for. See my post about how the pixel representations of masks are stored:

https://community.adobe.com/questions-675/mask-information-in-lrc-where-is-it-stored-does-the-sdk-allow-me-to-read-and-write-to-it-1630362?sort=oldestFirst#comments

how best to access the base image to read from after cropping, or rotating, or heal/remove has been applied. 

The only way to access the pixels of the base image are by reading them directly from the photo file (especially difficult for proprietary raw formats) or by using LrExportSession to export the photo to a temporary file on disk in a non-lossy format (e.g. TIFF or PNG) and then using some external utility to decode the file into an easier pixel format that the plugin can read with Lua. The method photo:getDevelopSettings() returns parameters describing cropping and rotating, whose interpretation is undocumented but has long been reverse engineered. (Search this forum for posts from me years ago describing their interpretation -- the horrible “new” forum search is useless.)  

For getting access to the pixels after Remove and masking has been applied, you’ll need to use the export method to access the pixels.