How to replace default file system (ASFileSys) via plug_in for files being opened in Acrobat?

[wedwdwdewdwd]

Problem: to open PDF files encrypted as a whole using 3rd party methods, such encrypted files have custom file extension (like foo.pdf.newext).

Solutions tried:

1. wrote a new Acrobat plug_in (based on Acrobat SDK's sample DMSIntegration), implemented custom ASFileSys to encrypt/decrypt underlying file on disk. To make Acrobat use my ASFileSys I replaced default file open dialog (AVAppOpenDialog()) with my own by replacing its HFT entry and provided my own ASFileSys instead of ASGetDefaultFileSys() as out param of replaced call. It works for files opened via file open dialog. But it does not work for files opened directly from Explorer or Comman prompt as no file open dialog is involved. Any help here on how to replace ASFileSys files opened from Explorer?
2. wrote a new Acrobat plug_in (based on Acrobat SDK's sample DMSIntegration), implemented custom ASFileSys to encrypt/decrypt underlying file on disk. To make Acrobat use my ASFileSys I registred for File open notifications (AVDocWillOpenFromFile and AVDocDidOpen), but I can't replace ASFileSys here and Acrobat will fail to recognize file format using its default file sys (file is encrypted as whole) and will show error message after calling AVDocWillOpenFromFile. I can initiate file open programatically with my own ASFileSys on detecting this error but by then Acrobat's alert message is already shown to user. Need to replace Alert message's HFT entry to not show it in such cases. This is not ideal. Any help here on how to replace ASFileSys via AVDocWillOpenFromFile()?
3. Also tried intercepting AVDocOpenFromASFileWithParams() by replacing its HFT entry but looks like it is not called by Acrobat to open files.
4. Any other solution?

[Test Screen Name]

What you want to do is outside the design aims.

1) Maybe you can associate your file type to a helper app that communicates with Acrobat

2) It's too late. The file is already open as an ASFile on a specific ASFileSys

[wedwdwdewdwd]

Thanks for your quick response. Could you please elaborate your suggestion for (1)? Do you mean that helper app should decrypt the file and handover to Acrobat? Intention is to keep file encrypted on disk at all times. This is the reason we are exploring ASFileSys to perform decryption in-place.

[Test Screen Name · Accepted solution]

Perhaps the Helper app could pass a request to a plug-in, and have the plug-in do the open on a custom ASFileSys. I do not say this is easy, but it does not sound impossible. What you are doing seems very forced and against the way Acrobat is intended to work, but you may be able to get something even if the user experience is not ideal.