Objects got distorted when transforming

I got a problem that is very annoying. I placed an image as a smart object to Photoshop. After that, I use some selection tool to make a selection and then create a mask. However, when I transform the layer, it popped up the notification: "A Warp transform has been applied to the Smart object. But warp transforms are not allowed for linked layer masks associated with Smart Objects. Please unlink the layer mask and try again".

I am sure that I have not used any transform on this layer before. I just placed and created a mask on it. I tried forcely transform it but the object got distorted and shrink into weird size. And they can't go back to normal properties after some transform.

I tried to reset the selection tool or preferences setting but it keep happening. I really don't know what is the problem. Does anyone got this problem?