Hi Michael...
If you're getting a DitaOpenToolkit menu, then you probably don't have the FM8 DITA installed (if it was installed, the OT plugin is added to the bottom of the DITA menu as "Generate Output"). If you're testing DITA-FMx (the likely reason that the default DITA plugin would be uninstalled), it comes with its own Generate Output command by default. Don't use the Adobe OT plugin with DITA-FMx (it might actually work, but it will confuse things further).
The Adobe OT plugin was created with OT 1.3.x in mind, and works reasonably well with that version of the OT. It does work with later versions, but you need to make sure that the environment variables that are defined by the startcmd.bat file are declared globally. The OT 1.3.x required this, but the OT 1.4.x provides a batch file with declares the variables within a shell. FM8 DITA requires these variables to be global.
To confuse things further .. DITA-FMx provides a setting in the ditafmx.ini file that lets you just point to the startcmd.bat batch file (actually a slightly modified version) so it's easy to use the OT 1.4.x.
Best of luck! :)
...scott