There has obviously been a change to the way "lib" is declared and scoped in the html5 output. I have been using my own javascript files added to the html page to add, control and remove instances from the library at runtime. After updating Animate CC to 2017.5 build 16.5.1.104, I was getting the same error of lib not defined. After clearing, my cache, republishing a new html5 page and generally pulling my hair out - as my code has been working just fine before the update, I started to look at where 'lib' was declared in the published html5 page.
While I'm not sure where it was declared previously (I couldn't find that right off in my old files), I found it is now declared several times and no longer available at the root level of the DOM, but is scoped to the composition. In the HTML page, it is only declared locally to functions init() and handleComplete().
However, if you are using frame scripts within Animate to instance a library object by its class name, it all works fine - as it's in the composition scope. So, 'lib' must be scoped to the composition somewhere. It just appears as it's not longer available to the html page itself.
To get it to work with my external scripts, I changed 'lib' to be scoped to the page level (added it to the var list above the init function) and ensured it was only assigned once in the init() function. This might not be the best way, but all seems to be working fine for me now.