Skip to main content
T
tbarroqueiro
Inspiring
March 2, 2018
Question

ColdFusion 2016 S3 issue

  • March 2, 2018
  • 5 replies
  • 3045 views

We recently upgraded our CF version from CF 10 to CF 2016.  We ran into a few issues but sorted them out, mostly.  We're now having an issue where our calls to S3 are not working.  The server just spins until it finally times out.  We don't see anything useful in the CF logs and have no real way of troubleshooting the problem.

When we revert the server back to CF10 the call works fine.

Here's the code:

<CFDIRECTORY  directory="#application.S3URL#/medium/#lcase(variables.manufacturer)#" name="myBucket" />

where application.S3URL is like so: s3://#application.accessKeyid#:#application.awsSecretKey#@#application.S3Bucket#

Has anyone encountered this or have suggestions of possible settings mismatch?

    This topic has been closed for replies.

    5 replies

    Homestar9
    Homestar9
    Inspiring
    May 23, 2018

    I have a possible theory as to what may be wrong with the cfdirectory "list" action and why it could be timing out.  The bucket I've been working with has about a hundred sub object/folders in it, with thousands of files each.  I bet that the "list" is trying to get every file in the bucket (not just the root) and it is causing the timeout.

    I stumbled upon the following project on GitHub: https://github.com/jcberquist/aws-cfml which seems to have a more modern implementation of S3 and when requesting the files in a bucket it limits the results to 1000.  I was able to use this project to successfully query a list of files from the bucket in question.  Hopefully this library will also help OP with their problem as well.

    BKBK
    Community Expert
    BKBKCommunity Expert
    Community Expert
    May 24, 2018

    Homestar9​, thanks for sharing the link to the AWS project on GitHub. Handy.

    Homestar9
    Homestar9
    Inspiring
    May 23, 2018

    I'm curious if OP ever arrived at a solution for working with Amazon S3 directly in ColdFusion 2016?  I ran into a similar problem and any time I run <cfdirectory action="list"...> on a bucket or bucket sub-folder the request waits for a very long time before eventually timing out.

    I experimented with Joe Danziger's S3 wrapper on RIAforce (and Github: GitHub - joedanz/cf-amazon-s3: ColdFusion Amazon S3 REST Wrapper but it doesn't work with the modern version of Amazon's API.  I searched Github for another project or possibly a more recent branch of this project without luck.

    Has anyone been able to list the contents of a bucket "directory/object" using ColdFusion 2016?

    T
    tbarroqueiroAuthor
    Inspiring
    March 7, 2018

    The results for a functioning folder are the same (except CF11 seems to be escaping the spaces with +).  A non functioning folder just times out on both systems.

    A few notes/reminders. 

    Nothing has been touched on S3 in a few years.  So no changes there.

    Using the Cloudberry UI for S3, I double and triple checked that the ACLs were correct, and that I could see the folders.

    I even created a copy of a non functional folder and it also times out.

    I use the exact same API credentials for the Cloudberry UI as I am using in my CF modules.
    Basically, one day everything worked fine, the next it's flakey.  Neither S3 nor CF had any changes made in regards to the code.

    CF11

    s3Dir1 = s3://[Masked]:[Masked]@dev.capitol.fusionapps.com/medium/2 thousand degrees

    directoryExists(s3Dir1): NO

    s3Dir2 = s3://[Masked]:[Masked]@dev.capitol.fusionapps.com/medium/2 thousand degrees

    directoryExists(s3Dir2): NO

    <CFDIRECTORY directory="s3://[Masked]:[Masked]@dev.capitol.fusionapps.com/medium/2 thousand degrees" name="myBucket"> =

    Fetched 183 Records. First 20:

    medium/2+thousand+degrees/bolinas_ceiling_lg_ambr.jpg

    medium/2+thousand+degrees/bolinas_ceiling_lg_w_mf.jpg

    medium/2+thousand+degrees/bolinas_ceiling_sm_a_mf.jpg

    medium/2+thousand+degrees/bolinas_ceiling_sm_white.jpg

    medium/2+thousand+degrees/bolinas_large_a.jpg

    medium/2+thousand+degrees/bolinas_large_white.jpg

    medium/2+thousand+degrees/bolinas_small_amber.jpg

    medium/2+thousand+degrees/bolinas_small_white.jpg

    medium/2+thousand+degrees/cabo_grande_bz.jpg

    medium/2+thousand+degrees/cabo_grande_sn.jpg

    medium/2+thousand+degrees/chambers_flushmount_dsrtclay.jpg

    medium/2+thousand+degrees/chambers_flushmount_white.jpg

    medium/2+thousand+degrees/delancey_brown.jpg

    medium/2+thousand+degrees/delancey_dclay.jpg

    medium/2+thousand+degrees/delancey_red.jpg

    medium/2+thousand+degrees/delancey_white.jpg

    medium/2+thousand+degrees/dillon_brown.jpg

    medium/2+thousand+degrees/dillon_white.jpg

    medium/2+thousand+degrees/echo_pendant_amber.jpg

    medium/2+thousand+degrees/echo_pendant_aqua.jpg

    CF2016

    s3Dir1 = s3://[Masked]:[Masked]@dev.capitol.fusionapps.com/medium/2 thousand degrees

    directoryExists(s3Dir1): NO

    s3Dir2 = s3://[Masked]:[Masked]@dev.capitol.fusionapps.com/medium/2 thousand degrees

    directoryExists(s3Dir2): NO

    <CFDIRECTORY directory="s3://[Masked]:[Masked]@dev.capitol.fusionapps.com/medium/2 thousand degrees" name="myBucket"> =

    Fetched 183 Records. First 20:

    medium/2 thousand degrees/bolinas_ceiling_lg_ambr.jpg

    medium/2 thousand degrees/bolinas_ceiling_lg_w_mf.jpg

    medium/2 thousand degrees/bolinas_ceiling_sm_a_mf.jpg

    medium/2 thousand degrees/bolinas_ceiling_sm_white.jpg

    medium/2 thousand degrees/bolinas_large_a.jpg

    medium/2 thousand degrees/bolinas_large_white.jpg

    medium/2 thousand degrees/bolinas_small_amber.jpg

    medium/2 thousand degrees/bolinas_small_white.jpg

    medium/2 thousand degrees/cabo_grande_bz.jpg

    medium/2 thousand degrees/cabo_grande_sn.jpg

    medium/2 thousand degrees/chambers_flushmount_dsrtclay.jpg

    medium/2 thousand degrees/chambers_flushmount_white.jpg

    medium/2 thousand degrees/delancey_brown.jpg

    medium/2 thousand degrees/delancey_dclay.jpg

    medium/2 thousand degrees/delancey_red.jpg

    medium/2 thousand degrees/delancey_white.jpg

    medium/2 thousand degrees/dillon_brown.jpg

    medium/2 thousand degrees/dillon_white.jpg

    medium/2 thousand degrees/echo_pendant_amber.jpg

    medium/2 thousand degrees/echo_pendant_aqua.jpg

    T
    tbarroqueiroAuthor
    Inspiring
    March 5, 2018

    It seems like this may not be CF version specific.  We reverted the production servers back to CF10 over the weekend and are still seeing the same behavior.  We also have another server with CF2016 that runs a process that involves S3, and that works fine.

    for reference:

    When trying to run the following command, some directories return just fine, others do not.  As far as I know, nothing has changed on S3 in years.

    <CFDIRECTORY directory="#application.S3URL#/medium/#lcase(variables.manufacturer)#" name="myBucket" />

    As a point of reference, the following command works just fine.  Which basically uses the exact same paramaters/paths to S3.

    <CFSET StoreSetACL("#application.S3URL#/medium/#lCase(variables.Manufacturer)#/#lcase(qry_images.Filename)#.jpg","#variables.perms#") />

    BKBK
    Community Expert
    BKBKCommunity Expert
    Community Expert
    March 7, 2018

    Does the following directory exist on your new CF2016 installation: #application.S3URL#/medium/#lcase(variables.manufacturer)#. The following tests will give the answer:

    <cfset s3Dir1 = application.S3URL & "/medium/" & lCase(variables.Manufacturer) >

    <cfset s3Dir2 ="s3://" & application.accessKeyid & ":" & application.awsSecretKey & "@" & application.S3Bucket & "/medium/" & lCase(variables.Manufacturer)>

    <cfoutput>

    directoryExists(s3Dir1): #directoryExists(s3Dir1)# <br>

    directoryExists(s3Dir2): #directoryExists(s3Dir2)#

    </cfoutput>

    T
    tbarroqueiroAuthor
    Inspiring
    March 7, 2018

    Doesn't matter what directory I use, they ALL say NO to directoryExists. But some directories fetch a cfdirectory and some don't.

    Here's one that works. As you can see, it still says NO to directoryExists, but fetches fine.

    s3Dir1 = s3://[MASKED]:[MASKED]@dev.capitol.fusionapps.com/medium/2 thousand degrees

    directoryExists(s3Dir1): NO

    s3Dir2 = s3://[MASKED]:[MASKED]@dev.capitol.fusionapps.com/medium/2 thousand degrees

    directoryExists(s3Dir2): NO

    WolfShade
    WolfShade
    Legend
    March 2, 2018

    When the server was upgraded to CF2016, did it keep the IP address?

    I'm trying to think of reasons why S3 wouldn't respond (which appears to be the case.)  Or, have you been watching the networking (F12) to see what _is_ coming back from S3?

    V/r,

    ^ _ ^

    T
    tbarroqueiroAuthor
    Inspiring
    March 2, 2018

    No, the IP stayed the same.

    We have 3 environments that are exactly the same, and the behavior is the same across all 3.

    ColdFusion 2016 S3 issue

    created by WolfShade<https://forums.adobe.com/people/WolfShade> in ColdFusion - View the full discussion<https://forums.adobe.com/message/10217165#10217165>

    WolfShade
    WolfShade
    Legend
    March 2, 2018

    Try it in CF2016 with FireFox and open Developer Tools (F12) and click the NETWORK tab.  Each call from the browser to the server will populate the screen; click any of the calls, and on the right side will be tabs for analyzing the responses from the server.  Somewhere in there you might find an error message.  Or you might find that nothing is coming back from any call.  But, either way, you'll have a troubleshooting start point.

    V/r,

    ^ _ ^

    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices