How to squeeze video in irregular shape ?

If you are just trying to mask using something like a heart shape then all you have to do is draw a mask on the layer and then scale the layer. You can also use a track matte. This is really basic stuff. 

If you are trying for a distortion then Andrew's suggestions are right on the money. You didn't really give us much to go on. A little clearer explanation or an example would be very useful. 

If you are new I suggest you start with Adobe's learning resources. Learn AE

I'd suggest either the Bezier Warp effect, or the Liquify tool.

But you may also find Mesh Warp or Reshape useful.

https://helpx.adobe.com/au/after-effects/using/distort-effects.html