Never remove any embedded objects from a quiz, Knowledge check or score slide. That Skip button is an embedded object.
With the default setup it will never appear. You may have edited the setup of the slide or in the Quiz Preferences. If you do NOT want the Skip button to appear on all Quiz and Knowledge Check slides, use Quiz, Preferences, Setup and uncheck the option 'Skip'. It will never appear on any slide in that case.
If you want to have it disappear only on some individual slides, uncheck the button in the Quiz Properties of those slides.
Beware: if you also have regular Quiz slides in the same project, you may need the second scenario, because the Quiz, Preferences, Setup is valid for both. Similarities and differences between both are described in one of my blog posts: