Skip to main content
M
meett9325076
Inspiring
June 1, 2017
Answered

AWS S3: Download media file get corrupted

  • June 1, 2017
  • 3 replies
  • 7170 views

Hi All,

I am trying to download media file (image or video) from AWS S3 using nodejsaws-sdk module. The file is downloaded but I couldn't open as it looks corrupted. Also noted that file size is not same as source file. Refer below code:

var AWS = require('aws-sdk');
var options = {
accessKeyId: "XXX",
secretAccessKey: "XXX"
};
var s3 = new AWS.S3(options);
var bucketName = 'test-bucket';
var keyName = 'test.jpg';
var params = {Bucket: bucketName, Key: keyName};
var fsModule = "fs";
var fs = require(fsModule);

// This requires when running this code in panel
AWS.util.stream = require('stream');
var writeStream = fs.createWriteStream('D:/test.jpg');
var readStream = s3.getObject(params).createReadStream();
readStream.pipe(writeStream);

I have added aws-sdk node module in my extension folder.

Note:
(1) I can download and open txt file without any issue
(2) If I run this code outside panel context (run in local nodejs server) it works for all type of files including text and media!!


Premiere Pro version: 9.0.0
Extension Type: Panel


Thanks & Regards,
Meet Tank

This topic has been closed for replies.
Correct answer sberic

meett9325076​ It looks like you're not waiting for the download to complete before attempting to pipe. Check out the documentation. The getObject() function takes a callback as its second parameter, a common approach for asynchronous functions. That callback has the form:

function(err, data) { ... }

Where the data parameter has a Body property, which itself is a ReadableStream (amongst other useful types).

Given that, your code should probably look something more like:

var writeStream = fs.createWriteStream('D:/test.jpg');  
var awsRequest = s3.getObject(params, function(err, data){
    if (err)
    {
        console.log(err, err.stack);
    }
    else
    {
        data.Body.pipe(writeStream);
    }
});

Note that this also provides you with access to the AWS Request handle that's generated when you call getObject().

I hope this helps.

3 replies

sberic
sbericCorrect answer
Legend
June 1, 2017

meett9325076​ It looks like you're not waiting for the download to complete before attempting to pipe. Check out the documentation. The getObject() function takes a callback as its second parameter, a common approach for asynchronous functions. That callback has the form:

function(err, data) { ... }

Where the data parameter has a Body property, which itself is a ReadableStream (amongst other useful types).

Given that, your code should probably look something more like:

var writeStream = fs.createWriteStream('D:/test.jpg');  
var awsRequest = s3.getObject(params, function(err, data){
    if (err)
    {
        console.log(err, err.stack);
    }
    else
    {
        data.Body.pipe(writeStream);
    }
});

Note that this also provides you with access to the AWS Request handle that's generated when you call getObject().

I hope this helps.

E
e_d_
Inspiring
June 1, 2017

Does adding

writeStream.end();

as last line  (23) make a difference?

Bruce Bullis
Bruce Bullis
Legend
June 1, 2017

Not directly applicable, but I'm told several partners have used this guidance to good effect, in their panels.

http://www.hacksparrow.com/using-node-js-to-download-files.html

Terms & ConditionsAccessibility statement
Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices