CFHTTP I/O Exception: null

[SimonGSW]

Greetings,

I am trying to get PDF content using CFHTTP from the following link: https://policy.ucop.edu/doc/3220479/BFB-BUS-38.

It works in browser, but CFHTTP returns following:

ErrorDetail	I/O Exception: null
Filecontent	Connection Failure
Header	[empty string]
Mimetype	Unable to determine MIME type of file.
Responseheader	test - struct [empty]
Statuscode	Connection Failure. Status code unavailable.
Text	YES

My CF code is:

<cfhttp method="get" url="https://policy.ucop.edu/doc/3220479/BFB-BUS-38" timeout="50" result="test"></cfhttp>

<cfdump var="#test#" label="test">

CF Server is CF11 developer edition, version 11,0,15,311399.

Looking at Response headers in browser, I noticed that Transfer-Encoding: chunked and Content-Length is missing(Server Apache). Is there problem with CFHTTP and Transfer-Encoding: chunked?

Any ideas on how to make it work?

Regards,

Simon

[WolfShade]

I'm not sure, but I think that if you're going to have the web server connect to a secure (SSL/TLS) website, you have to manually insert the certificate for CF to transmit via Apache.  Per the CFHTTP docs:

Manually import a certificate

1. Go to a page on the SSL server in question.
2. Double-click the lock icon.
3. Click the Details tab.
4. Click Copy To File.
5. Select the base64 option and save the file.
6. Copy the CER file into C:\ColdFusion9\runtime\jre\lib\security (or whichever JRE ColdFusion is using).

7. Run the following command in the same directory (keytool.exe is located in C:\ColdFusion9\runtime\jre\bin):

   keytool -import -keystore cacerts -alias giveUniqueName -file filename.cer

HTH,

^ _ ^

[SimonGSW]

Thanks for your reply.

I think that is not the case.

1. Certificate for the site is signed by authority that JSSE should definitely recognize InCommon RSA

2. Error Detail should be different, like authentication issue(not NULL)

Regards,

Simon

[Dave Watts]

Nevertheless, that is by far the most likely cause of your error. Generally, when you get a connection failure attempting to connect via TLS, that is the cause. The Java keystore typically does not contain the certificate chain for your site's certificate. It doesn't contain nearly as many root and intermediate certificates as a browser. And there is no error that CF can really see - it is literally unable to connect to the site. It doesn't receive an authentication error, TLS connection doesn't require authentication. If I was betting on the cause of your error, I'd put every dollar I have on this.

Dave Watts, Fig Leaf Software

[WolfShade]

I just went into FireFox options > Advanced > Certificates, and I do not see InCommon listed.  Are you absolutely sure that JSSE recognizes InCommon?  Their partner is Comodo, which just changed their name to something else.

V/r,

^ _ ^

[SimonGSW]

Thanks,

No, I am not sure but I just tried to get just HTML content  from the same site. URL - https://policy.ucop.edu/

I think, it should be similar issue in case of  Certificate problem. No problem getting proper response for that URL

Accept-Ranges bytes Connection close Content-Type text/html; charset=utf-8 Date Mon, 03 Dec 2018 23:01:01 GMT Explanation OK Http_Version HTTP/1.1 Server Apache Set-Cookie NSC_qpmjdz=ffffffff8338570745525d5f4f58455e445a4a423660;Version=1;Max-Age=120;path=/;secure;httponly Status_Code 200 Transfer-Encoding chunked Vary Accept-Encoding X-Frame-Options DENY
Statuscode	200 OK
Text	YES

Interesting that Transfer-Encoding is also chunked, but no problem.

Regards,

Simon

[WolfShade]

It may be because you are trying to get a binary, vs the "string" that the CFHTTP (underlying code) normally presents.  It may work via HTTP, but not HTTPS unless you have the cert stored in CFs keystore.

Just guessing.

V/r,

^ _ ^

[Dave Watts]

If you're able to successfully retrieve content from one HTTPS URL but not from another one with the same hostname, you're right, it's not a certificate problem. Unfortunately, at that point, I don't know what kind of problem you might have. If possible, try to monitor the HTTPS requests and responses using a tool like Fiddler and see what's different about them.

Dave Watts, Fig Leaf Software

[Charlie Arehart]

Simon, I don't see that anyone has asked what JVM version you have CF using. It may be Java 7 (as CF11 originally came out on that). You can see the JVM version CF is using in the CF Admin "setttings summary" page.

And it may need that you simply need to update to Java 8. (And as long as you have applied CF11 update 3, which added support for Java 8.)

Sometimes (despite the lack of clarity in the errors you see), issues of CFHTTP failures calling out to https URLs are down to differences in the DESTINATION's support for various SSL or TLS versions and encryption algorithms, and some are NOT supported by Java 7 or earlier.

(And this is also why you find it "works" in a real browser but not in CF. Like Dave said about certs, what browsers support is different than what is supported by the java-based httpclient class in CF, which is simulating a browser for us.)

Are you aware of how to update CF to point to a new JVM? There are various resources out there. And then as for things that can go wrong when updating the JVM and how to recover/avoid those problems, I have a blog post on it, listed at the top of my https://www.carehart.org/top.cfm page.

Let us know if this is it or not.

[SimonGSW]

Thanks Charlie,

CF Admin reports Java Version 1.8.0_25 and I have latest CF11 update installed. 

It would be interesting if anybody can check that issue with CFFTP in CF 2018 or CF 2016 using URL that I supplied https://policy.ucop.edu/doc/3220479/BFB-BUS-38https://policy.ucop.edu/doc/3220479/BFB-BUS-38http://

That just a link to a PDF document in UC California.

Regards,

Simon

[Charlie Arehart]

That's an old Java 8. A later one might work: the latest is 191.

Ans I'm afraid I'm seeing this on my phone so challenging to test it while out. Perhaps someone else will if I don't get to try that for you. 

/Charlie

Sent via the Samsung Galaxy S7 active, an AT&T 4G LTE smartphone

[BKBK · Accepted solution]

SimonGSW  wrote

I am trying to get PDF content using CFHTTP from the following link: https://policy.ucop.edu/doc/3220479/BFB-BUS-38.

It works in browser, but CFHTTP returns following:

ErrorDetail	I/O Exception: null
Filecontent	Connection Failure
Header	[empty string]
Mimetype	Unable to determine MIME type of file.
Responseheader	test - struct [empty]
Statuscode	Connection Failure. Status code unavailable.
Text	YES

My CF code is:

<cfhttp method="get" url="https://policy.ucop.edu/doc/3220479/BFB-BUS-38" timeout="50" result="test"></cfhttp>

<cfdump var="#test#" label="test">

CF Server is CF11 developer edition, version 11,0,15,311399.

Looking at Response headers in browser, I noticed that Transfer-Encoding: chunked and Content-Length is missing(Server Apache). Is there problem with CFHTTP and Transfer-Encoding: chunked?

Any ideas on how to make it work?

Hi SimonGSW​, I have an idea on how to make it work. The site may be expecting you to pass it some parameters. To know which parameters to pass, open any URL in the browser and press F12 on your keyboard to open the Developer Tools screen.

Then, with the Developer Tools still visible, open https://policy.ucop.edu/doc/3220479/BFB-BUS-38. You should see the PDF file as expected.

Now, click on the Network tab in Developer Tools. Copy all the request headers and use them as HTTP parameters in your cfhttp call. In your cfhttp tag, change the method from Get to Post. Also add the attributes, Port and Getasbinary. The result should be something like this:

<cfhttp method="post" url="https://policy.ucop.edu/doc/3220479/BFB-BUS-38" timeout="50" port="443" getasbinary="yes">

    <cfhttpparam type="header" name="Accept" value="text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" />

    <cfhttpparam type="header" name="Accept-Encoding" value="gzip, deflate, br" />

    <cfhttpparam type="header" name="Accept-Language" value="en-US,en;q=0.5" />

    <cfhttpparam type="header" name="Connection" value="keep-alive" />

    <cfhttpparam type="header" name="Host" value="policy.ucop.edu" />

    <cfhttpparam type="header" name="Upgrade-Insecure-Requests" value="1" />

    <cfhttpparam type="header" name="Cookie" value="PHPSESSID=40kdc682uolbvouq7ufv7t81s3; NSC_qpmjdz=ffffffff8338570445525d5f4f58455e445a4a423660" />

</cfhttp>

<cfcontent type="application/pdf" variable="#cfhttp.filecontent#">

[SimonGSW]

Thanks BKBK,

Your comment helped me to get a resolution. The issue is in passing cookies in CFHTTP request and also setting getasbinary="auto"(that should be default, but without that option, I got empty PDF file)

So,

<cfhttp method="get" url="https://policy.ucop.edu/doc/3220479/BFB-BUS-38"  getasbinary="auto">

   <cfhttpparam type="header" name="Cookie" value="PHPSESSID=40kdc682uolbvouq7ufv7t81s3; NSC_qpmjdz=ffffffff8338570445525d5f4f58455e445a4a423660" />

</cfhttp>

works without problem, but more work needs to be done in order to get those cookies(I can't expect static values to work forever).

I used another CFHTTP with option redirect='no' to get those cookies and include those in the final CFHTTP. That has done the trick.

It is actually even more complex if request has more than one redirect. Just try link httpp://policy.ucop/edu/doc/3220479.  That one has additional redirect from http to https and requires additional level of cfhttp to get the final result.

My task was actually collecting documents(multiple formats) from links on University policy site.

Regards,

Simon

[BKBK]

Glad to see it is working. I experimented by setting the cookie values to 0, and I still got the PDF.

<cfhttp method="post" url="https://policy.ucop.edu/doc/3220479/BFB-BUS-38" port="443" getasbinary="yes" >

    <cfhttpparam type="header" name="Cookie" value="PHPSESSID=0;NSC_qpmjdz=0" />

    <cfhttpparam type="header" name="Connection" value="keep-alive" />

</cfhttp>

<cfcontent type="application/pdf" variable="#cfhttp.filecontent#">

[SimonGSW]

BKBK,

That is correct in this example, but it could be different cookie names if you have to get documents from multiple URL's.  For my task, I had about 30 different URL's  and needed to build solution to use not just once.

Regards,

Simon

[BKBK]

How to deal with different or changing cookies for 30 URLs is beyond your original question. Ask the ucop.edu webmaster for help.

For now, we have solved the CFHTTP I/O Exception problem.