Hi Angela:
FrameMaker is tough to teach yourself—it's not a hard program to use once you understand it, but it doesn't work like any other application so it's not intuitive. I have a bunch of free videos you can watch online—listed here. They don't replace attending a training class, but they may address some of your questions.
Back to this question—thank you for sharing your screenshots. Without being able to examine your file, I find your reference page baffling. But here's a general idea: all of the lines should look essentially the same in terms of how the building blocks are set up. Those two lines at the top look completely different.
If the orange underlined line has the style name that points to the heading that isn't working—which I'm not convinced it is—try changing it to read:
<$paranum> <$paratext>\t<$volnum>.<$chapnum>–<$pagenum>¶
(\t is a tab, and you can just press the Tab key on keyboard. I can't tell if you need one or two.)
Then, update your book.
If it doesn't change at all, it wasn't the right line on the reference page—but one of them is. If possibly changing the wrong line makes you nervous, copy the TOC.fm file to a different location before you start. If this messes it up, you can just restore the copy.
~Barb