Javascript Solution for Getting Data from multipart/mixed Poll Status (/ops/id/{id}) Response

[dhildreth]

I'm working with Retool, attempting to automate a PDF generation workflow for my internal team.  This is not a traditional javascript application with HTML.  I need to 1.) use the Retool utility for downloading the file and 2.) persist the data to a DB (likely a Blob field type).

I have downloaded the Postman collection and environment and modified it to work with my account successfully.  I can also successfully call the Document Generation POST endpoint (/ops/:create), get the location ID, and finally call the Poll Status GET endpoint (/ops/id/R1UeCerQ14fM1CMsagyOaMlSFXZ8thxi) to get the final multipart/mixed response.  It looks like this:

--Boundary_380665_2133110495_1627937751536
Content-Type: application/json
Content-Disposition: form-data; name="contentAnalyzerResponse"

{"cpf:inputs":{"documentIn":{"dc:format":"application/vnd.openxmlformats-officedocument.wordprocessingml.document","cpf:location":"InputFile0"},"params":{"cpf:inline":{"outputFormat":"pdf","jsonDataForMerge":{"customerName":"Kane Miller","customerVisits":100,"itemsBought":[{"name":"Sprays","quantity":50,"amount":100},{"name":"Chemicals","quantity":100,"amount":200}],"totalAmount":300,"previousBalance":50,"lastThreeBillings":[100,200,300],"photograph":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP88h8AAu0B9XNPCQQAAAAASUVORK5CYII="}}}},"cpf:engine":{"repo:assetId":"urn:aaid:cpf:Service-52d5db6097ed436ebb96f13a4c7bf8fb"},"cpf:status":{"completed":true,"type":"","status":200},"cpf:outputs":{"documentOut":{"cpf:location":"multipartLabel","dc:format":"application/pdf"}}}
--Boundary_380665_2133110495_1627937751536
Content-Type: application/octet-stream
Content-Disposition: form-data; name="multipartLabel"

%PDF-1.6
%����
44 0 obj
<</Linearized 1/L 42582/O 46/E 37060/N 1/T 42277/H [ 458 154]>>
endobj
{{!-- SNIPPED --!}}
endstream
endobj
startxref
116
%%EOF

--Boundary_350829_355281493_1627941421328--

I've reviewed the Adobe PDF Service API Documentation and I understand the response is broken up into two parts:  the JSON with data about the output and location name ("multipartLabel") and the octet-stream containing the PDF data.

What the documentation does not do is go one step further to explain how to isolate the PDF data and do something useful with it.  In Postman, you can click "Save Response -> As File" and get the PDF successfully. 

My question is, now that I have the response, how can I do something useful with it such as saving the PDF data to a database or PDF file?

I was able to parse the PDF data using plain javascript, but I'm unable to do anything useful with it.  Not to mention, it has no knowledge of multiple documents in the multipart/mixed response, so it will break if there is anything more than one document (if ever).

function getPdfData(data) {
  let contentType = data.metadata.headers['content-type'][0];
  let contentTypeParts = contentType.split(';');
  let boundary = contentTypeParts[1].split('=')[1];
  let parts = data.data.message.split(boundary).filter(part => part !== '--' && part !== '');
  let parsedParts = parts[1].split('\n').filter(part => part !== '');
  let shifted = parsedParts.splice(4, parsedParts.length - 2);
  return shifted.join();
}

var pdfData = getPdfData(pollStatusResponse);

The entire process has me questioning my sensibilities and I'm thinking there has to be a simpler way.  Maybe there's a URL parameter or header that allows me to specify the response type?  Something like /ops/id/R1UeCerQ14fM1CMsagyOaMlSFXZ8thxi?contentType=application/pdf.  

The next step for me would be to get the data, and then use one of the following Retool commands to download the PDF:

utils.downloadFile(pdfData, 'response', 'pdf'); // downloads a file called response.pdf
utils.downloadFile({base64Binary:pdfData}, 'response', 'pdf'); // Special function for dealing with base64 binaries.

Finally, I would use a MySQL command to store the data into the Database.

How would I reliably do this!?

Similar to: https://community.adobe.com/t5/document-services-apis/downloading-pdf-extract-output/m-p/12222372

Retool Community Post: https://community.retool.com/t/download-a-file-from-response-with-content-type-multipart-mixed/5776

Retool Utilities Documentation: https://docs.retool.com/docs/scripting-retool#utilities

Adobe PDF Service API Documentation: https://documentcloud.adobe.com/document-services/index.html#how-to-get-started-

[dhildreth]

Somewhat related, I just had an awesome experience with a competing PDF API service (Anvil PDF Filler).  Their API response looks like:

{
  "base64Binary": "JVBERi...",
  "fileName": "shv5ADziQHdFRlL51fmN",
  "fileType": "pdf"
}

So, in one line of JS code in Retool, I got exactly what I was trying to accomplish with the Adobe PDF API:

utils.downloadFile({base64Binary:fillPDF.data.base64Binary}, fillPDF.data.fileName, fillPDF.data.fileType);

 
Cringing to think I spent hours with the Adobe solution when I was able to get the other solution to work within 15 minutes.

I repeat -- there must be a better way or something I am missing!

[fpMonster]

With the multipart/mixed response, you have you either write a parser or use a multipart parser library to break up the "contentAnalyzerResponse" json part and the "multipartLabel" binary part. Then save each part as Blob file.

[Raymond Camden]

Have you considered using the Node.js SDK? It makes this much easier to use.

[Test Screen Name]

What you may be missing is that this is MIME, an  industry standard for multipart data. You just need to write a MIME parser, or (preferably) use an existing MIME parser on the data.