Willi's suggestion to use the more flexible cross-references feature is a good one, and bypassing the more limited TOC feature can give you a great deal more control over a TOC when the book doesn't follow a simple schema.
Pretty much all solutions using the TOC feature lie in the formats applied to each TOC element. One is to not 'pull' the chapter or section numbers from the text, but apply paragraph numbering to the TOC entries. That is, ignore the Chapter 1, Chapter 2, etc. and just pull the chapter name paragraph, and apply autonumbering to it. That's a bit risky in that you are disconnecting the actual content from the TOC, but it's worked for me on a number of books.
The other approach is to pull the Chapter Number paragraph and the Chapter Name paragraph, and stack them horizontally using spacing and by setting one to zero leading. I can't remember which way worked best, but if the TOC numbering paragraph has, say, 16pt line spacing, set the TOC name paragraph to 0pt, so it will remain on the same line.
Another borderline hack but it's relatively simple to apply to an otherwise straightforward TOC setup — and again, everything depends on the TOC paragraph styles applied, from each level of entry to the page numbering to the leader style in between. All those features are there; make good use of them.
—