PDF Embed API - hide sidepanel

We are currently using embedMode as "FULL_WINDOW" with ui configs showPageControls,showZoomControl,etc set as false, still sidepanel is present. As we already have created custom toolbar, we want to hide the one coming from Adobe services 

Out of all embedMode, "SIZED_CONTAINER" seems to have least ui configs and it has ui most similar to our requirement but pdf pages within this mode are not scrollable

As a workaround we used clip-path css property to visually hide it from user, but with tab navigation from keyboard along with links in buttons from sidepanel are also accessible. Is there a way to hide sidepanel?