Skip to main content
S
speakerboom
Participant
October 2, 2006
Answered

Using CFHTTP to extract from a website requiring JavaScript-enabled browsers

  • October 2, 2006
  • 1 reply
  • 517 views
How do you use CFHTTP to extract content from another website when that website is built to ensure that a userAgent has JavaScript enabled? I'm using the userAgent attribute, and I've tried several different values for it, but each attempt only returns a "Sorry, you must have a JavaScript-enabled browser to view our site" type of message.

How can I make my CFHTTP request appear as a browser *with* JavaScript-enabled?

I'm using CFMX7 and the URL I'm trying to scrape is:
http://www.flacareers.com/flacareers/renderpage.asp?pageid=5036
    This topic has been closed for replies.
    Correct answer speakerboom
    Well, it appears that when I use Googlebot as the userAgent...

    Googlebot/2.1 (+ http://www.google.com/bot.html)

    ...it works just fine! I was trying only *browser* userAgent strings before. I guess the *bot* userAgents have more class and get into more clubs :)

    1 reply

    S
    speakerboomAuthorCorrect answer
    Participant
    October 3, 2006
    Well, it appears that when I use Googlebot as the userAgent...

    Googlebot/2.1 (+ http://www.google.com/bot.html)

    ...it works just fine! I was trying only *browser* userAgent strings before. I guess the *bot* userAgents have more class and get into more clubs :)
    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices