When I click on retake quiz, I do go to first question but feedback is showing. How do I make sure the feedback does not show?

I really need more details to be able to help. What do you mean by 'feedback'? Did you set the attempts on quiz level to multiple attempts, how many? Did you keep both Retake and Review button on the score slide? I tried to explain the design of question/score slides in some articles, maybe they could help?

Question Question Slides in Captivate - Captivate blog

Question Question Slides - Part 2 - Captivate blog