I installed huge pack of plugins for AE but I don't need them in Media Encoder as it's loading for too long with plugins
If your plug-ins are installed in the Mediacore folder then you can't prevent AME from loading them. That's a shared location that all Adobe video apps (and now Audition) use. After the plug-in cache is create the first time, loading should be much faster on subsquent launches.
What plug-ins did you buy?
The Mediacore folder is a common location for shared plug-ins between After Effects and Premiere. So if those plug-ins show up in both of those apps, then that's why AME loads them. Also, AME, by default, loads Premiere Pro sequences natively (as opposed to using Dynamic Link), meaning you don't need PR installed. So if you don't have Premiere installed, but you're loading a sequence with Boris FX plug-ins, AME will need to access them.