is baking viable when...

OK so the baking looks to have worked but - baking does not actually raise or lower the surface of the low poly. It simulates details on the surface when rendering so when looking at the surface straight on or at angles the light paths are changed to give the appearance of surface height changes in accordance with those those baked details. However if a surface is viewed at a very low angle or even at 0 degrees i.e. looking straight along the surface, to the object or background behind it, then that simulation falls down as there is no actual surface displacement so no real height changes in that surface.

In short a normal map merely simulates detail on a surface it doesn't actually raise that surface. To do that requires both a height map and tessalation of the surface - which of course adds polygons.

Example below

A plane cube:

A normal map used to add surface details

Top and side viewed at low angle that detail almost disappears as it is not really there:

Surface tessalated and surface displacement driven with height map. Now those low angles have real surface detail so are visible at low angles, but that is at the expense of adding polygons:

Dave