To my knowledge there isn't a place in RoboHelp that allows you to set that color.
To my knowledge, the CHM viewer simply uses the colors defined in the Microsoft Windows operating system. For me, on my Windows 7 Pro setup, if I right-click the desktop and choose to Personalize, then click Window Color followed by Advanced appearance settings. The Windows Item that seems to control this is "Selected Items".
LOL, now to try and sort what mine was before experimenting!