do you know how to enable mouse scroll wheel to scroll the scrollbar in TOC? It doesn't work under default circumstances.
Scrolling wheel will scrol the whole window including TOC. It doesn't matter whether the TOC is separate or in overlay. If you really want this feature, please log a feature request. Here is the link to the form.
I'm not sure why it isn't working for you. I tested this out in responsive and non-responsive designs with my mouse, and it works fine.