Why is it that "Fit to Frame" only works sometimes? Is there a pattern or logic...?

I am currently importing a lot of graphics on the reference pages of a document. And I need to make these graphics fit to a specific frame, but the feature "Fit to Frame" is greyed out. This has happened to me many times in different document but now I would like to get to the bottom of this mystery. WHY does it only work sometimes? Are there specific circumstances I must adhere to?