That is by design and desirable. The TOC is simply a way of bringing a reader's attention to key topics. It does not have to include all the topics in the build. Let me give an example.
I have a project with around 12,000 topics. Only a very small number of those would be in TOC, the rest being accessed from links in the topics.
All topics in Project Manager will be in the build by default. If you have some reason for leaving them in the project but do not want them in the build, such as different versions, then you need to apply a Conditional Build Tag and then use a Build Expression to exclude those topics when you generate the output.
You need to create different layouts for the different versions, each with the required build expression for that output.
See www.grainge.org for RoboHelp and Authoring tips