URL Status Checker crashes website, host blocks IP
I used a script called Gryperlink to add URLs to all our product SKUs in our catalog. The script finds the SKUs, appends the SKU to the end of a uniformed URL to generate the product's specific URL.
We have very big catalogs with 1500-2500 products in each. The script took a while when testing on 100 products, so I set to run on a catalog overnight. Came back the next morning, and the catalog looked great, but no one inside the building could access our website. Our host blocked our IP because of the many requests InDesign was making to the website. I turned off the "Automatically Update URL status" and that seemed to prevent the issue. This leaves me with some questions:
Has anyone else experienced this with a catalog with lots of links?
How does InDesign check the status of the link? Does it actually load the page, ping it, or just crawl the page like a search engine indexer? I'm guessing it does this from the host computer, and not from a Adobe server?
And finally, a product suggestion: This feature should be turned off by default. It slows down InDesign and on large documents with lots of links can create a DoS attack on the site.