After a breakout room session has been started, folks left in the main room can't hear each other

Our meeting room is set up in standard view. When I start a breakout room session, 5 of us hosts were left in the main room. 2 of us could hear each other, the other 3 could only hear themselves - not us - almost like we were in our own 2 breakout rooms within the main room. 

After dragging the other 3 folks into a breakout room and then placing them back in the main room, all of our audio worked again. This happens rather often since we moved over to v11. Any ideas what could be causing this?