This is the same question asked here for Theme_1 Standard: the answer was
For that layout:
from William van Weelden
I don't see the same properties in either of these places:
Does anyone know what the right properties to change to make the TOC font smaller for the Charcoal_Grey Layout? I'm afraid that if I start to mess around with it I will do something irreparable. On that note, if I want to copy the entire project so that I do have a test environment, is there a SAVE AS type functionality, I only see RENAME (apologies for the two-fer-one question).
For Charcoal Grey, you will want to use the SCSS to generate the CSS instead. (See; RoboHelp screen profiles and layouts)
The selector that you're looking for is: %toc-desktop-theme3
Add a font size to this element and it will scale as you provide. (Use em measures for best results. The rest of the content also scales using em.)