Adobe AIR EncryptedLocalStore Problems

[boushley]

Encrypted local store doesn't function once an application is installed on my machine. I've been having problems on this computer with encryptedlocalstore, so I setup a basic test program.

Things work fine when run from adl, but do not function when run normally. If I run adl on the installed application from inside the .app/Contents/Resources folder the application works fine, but when run normally it fails.

I've tried deleting my entire ~/Library/Application Support/Adobe/AIR/ELS folder but to no avail. I uninstalled and reinstalled Adobe Air 2.5.1 and still ELS fails. The folder for the Application is created within the ELS folder, but none of the calls to ELS function, and throw an error with the message "general internal error". There is no stack trace attached to the error messages.

I'm running Mac OSX 10.6.6  I'm using Flex sdk 4.5.0.19786, if that might have an affect.

Any ideas?

[chris.campbell]

Hi,

I tried this out (using the sample code you posted over on stackoverflow) and I'm seeing the following behavior on 10.6.6.

1. Compile and launch code from Flash Builder (ADL).  App starts and loads ELS properly.  Quit the app.

2. Install and launch the release version of the same application.

Result: I'm presented with a keychain dialog that says "testApp wants to use your confidential information stored in "Adobe.APS" in your keychain.  I have Always Allow, Deny, and Allow as my choices.  If I select Deny, I get the "general internal error" message.  If I select Allow then everything works properly.

If I delete the ELS folder, then reverse those steps so that the release app is started first, it works fine, but then ADL will prompt me for keychain access.

I took a look at the docs, and I did see the following:

When debugging an application in the AIR Debug Launcher (ADL), the application uses a different encrypted local store than the one used in the installed version of the application.

Have you ever been prompted with this keychain dialog?  If you delete the ELS folder, then start the release build of your app, does it still fail?  Finally, I've seen some comments about failures occuring on case sensitive file systems.  Is your drive formated with case sensitivity enabled?

Thanks,

Chris

[boushley]

I have seen this dialog requesting permission to access the keychain,

I understood what it was asking and why, so I selected Always Allow.

However I still receive this error. I have deleted my ELS folder and

still see the problem. I have deleted the ELS folder, uninstalled and

reinstalled Air and stil see the problem.

My filesystem is case sensitive (Mac OSX), can you link me to some of

these posts where people have had problems with this? Are there any

solutions or workarounds for case sensitivity issues?

Aaron

[chris.campbell]

Here's a post about the case sensitive issue, but I'm concerned this might not apply.  Take a look though and tell me what you think.

http://blogs.adobe.com/koestler/2009/07/unreadable_encrypted_local_sto.html

Out of curiosity, could you try creating a new user account, then running the application through ADL and then it's normal stand alone state?  Does it still fail?

Thanks,

Chris

[boushley]

The solution for the case sensitivity was deleting the ELS folder,

which I've done several times. I setup a new user account on the same

machine, and the new user account had the same problems. I should

also mention that I'm getting an error that says "EncryptedLocalStore

database access error" sometimes. It seems like the first time I run

the application after deleting ELS I get this, and then it goes back

to the general internal error problem.

Is there a possibility that something in my keychain is messed up

causing these issues?

Aaron

[Guest]

We've actually had a problem identical to this many many times.

It only seems to happen for us on Mac OSX 10.6 w/ Firewall enabled. It doesn't seem to happen when the firewall isn't enabled.

It also has nothing to do with case sensitivity - we have never changed our application ID.

It seems to happen after an upgrade occasionally - but not consistently. Once it happens, here are the steps we've taken to resolve it.

(P.S. would love to see Adobe weigh in on this one - it happened on our CEO's machine just last night!)

<it happens enough we had to wiki it!>

Fixing Mac EncryptedLocalStore database access error

This error is caused when the application is updated and the  cached data stored in the /var/db/DeferredSignature file is out of date  with the newly updated signature of the application.  It seems that  Adobe AIR actually iterates through this file looking for a matching  signature.  For some reason it stops (likely a bug) and no longer  continues generating the appropriate Mac keychain to store the ELS key  information.

To work around this problem you need to perform the following operations:

• Cleanup the application data by deleting the data in 
  
  • ~/Library/Application SupportAdobe/AIR/ELS/XYZApplication
    
  • ~/Library/Preferences/XYZApplication

• Delete the Adobe AIR keychain using the Keychain Access application on the Mac.
  • In the Keychains window you should see a Keychan called PrivateEncryptedDatak - right click on it and select Delete.
  • If the Keychain does not get removed from the list exit and restart Keychain Access.
  • If the PrivateEncryptedDatak keychain is still visible select  the Keychain Access menu and the Keychain First Aid  menu item.  In the  popup enter an administrator username/password, select Repair and hit  start.  The keychain should now be gone.

• Okay, now the fun part.  You need to start a terminal up - this will open up a shell window    
  • If you are not logged in as an administrator you need to  change the user this terminal is logged in as via the 'su' command by  typing 'su - username' where username is the user you want to login as.
  • sudo rm /var/db/DetachedSignatures

That's it - you should now have everything cleaned up and can try  re-installing the application (if the new version is already  installed you probably can simply launch it now).

[boushley]

Glad to have that great information on how to fix that issue, but even following these steps my ELS is still non-functional.

Does anyone have any other ideas on how I might get this working?  From what I've read, the only other ideas people have is the completely reinstall the operating system.  While this is an option, I'ld prefer to track down the actual bug.

[Guest · Accepted solution]

Oops! Looks like I missed a step in the post: after the deletions you need to reboot the system before attempting to use your applications.

Adboe folks: I would love to do something less destructive/exterme. How??? What are you storing in that DetachedSignatures file that I could remove? Losing ELS causes my app to lose its ability to open its encrypted SQLite DB which in turn forces us to re-fetch potentially gigs of data from the server.

And reinstall you OS is never an option. I'd sooner re-write my app in another language and move away from AIR than deal with that as an option!

[boushley]

You're awesome!!!  Apparently reboot is a very important step.  So to be clear to anyone else who tracks this down, I resolved the issue by following the list of steps aboe (remove ELS, and Application Support for your programs), cleanup the keychain, deleted the /var/db/detachedSignatures and then reboot.  I had multiple users, and cleaned up both of them, but I forgot about that and rebooted between them, so I went through a few reboots, but this totally worked.

Thanks!

[chris.campbell]

Thanks guys for tracking this solution down!  I've forwarded this thread along to our engineers that worked on ELS and hopefully they can provide additional feedback.

Chris

[Guest]

Hey Chris,

We could really use an update on this one. It seems to happen to at least one of my Mac users every 2nd or 3rd time I release a client. It really messes us up because we store the password for our sqllite db in the ELS - once ELS has been deleted the whole database needs to be deleted, and this triggers a complete re-sync with the server. Because of the nature of our app, the resync can be Gigs of data, take a very long time, and end up costing us a decent amount of money in bandwidth.

I wouldn't be so concerned if the ramifications weren't so bad, and frankly the remedy is more than slightly questionable.

[chris.campbell]

Hi guys,

Sorry I haven't updated you on this one.  I had one of our security experts take a look and this was new to him.  I've added this as an internal bug (#2820203) and I'll ask around a bit more to see if anyone else might have an idea what could be going on.

Thanks,

Chris

[Guest]

Hey Chris,

I just saw something similar to this happen with yet another customer - this time on Windows XP. While deleting his ELS fixed the issue, this is really not a very acceptable solution for me. If we can't count on ELS being stable for storage, it becomes pretty useless. I store my sql lite db passwords in there to protect them and when the ELS gets killed I have to re-download potentially gigs of data to the local system - both time consuming and expen$ive.

Should we be using something different other than ELS?

[chris.campbell]

Hi Chris,

If at all possible, we'd love to get the steps to reproduce the problem on Windows.  I've updated our bug to point to this post but it would be nice to have a separate bug on Windows (or at least some platform specific steps in the original bug.)  If we have evidence that our implementation has defects that make it less reliable, we will certainly fix them as a high priority.

As for guidance on using ELS, please see Oliver's recent blog post on this topic.

EncryptedLocalStore and Storage Reliability

Thanks,

Chris

[Guest]

That Windows failure happened under the exact same circumstances as the Mac ones did - users upgraded to our latest build via the installer update mechanism. Nothing special that I'm aware of.

Also, I read the blog post. While I agree with the principles, the simple fact of the matter is that ELS is really appearing to be fragile. Much less so than the Encrypted SQLite.

[chris.campbell]

I hear you Chris.  I'll update the bug report, specifically calling out that this also occured on Windows.

Chris

[dashart21]

Same problem on Windows, Mac OS and Linux.

Steps to reproduce:

1. Delete the PrivateEncryptedDatakey for the Adobe AIR application in the keychain on Mac OS, on Windows maybe different

or

1. Manipulate /Users/USER/Library/Application Support/Adobe/AIR/ELS/APPID/PrivateEncryptedDatak

Added a bug report to the JIRA

http://bugs.adobe.com/jira/browse/ASL-202

[chris.campbell]

Thanks for opening the bug on Jira.  I've updated our internal bug (#2820203) with your notes and the jira link.

Chris

[Guest]

For months we never seemed to see this problem. Now that 2.7 is out, we are seeing it quite frequently again.

Could it possibly have something to do with building our kits using an older SDK and running with the newer version of the runtime?

When I first saw this problem frequently was when we were building our application using the AIR 2.5 SDK, but 2.6 was out and about. Once we switched to using the 2.6 SDK, problems seemed to go away. We have not yet gotten around to switching over to the AIR 2.7 SDK yet, but now we are suddenly seeing the problem in ~30% of our MAC installations again. Way to high a number to be acceptable.

This NEEDS to be addressed.

Also, later today I will post new, less destructive work-around steps.

[Guest]

Less destructive recovery instructions:

Open a terminal window. 

You then want to bring up the SQLite admin tool with this command (running as root using sudo):

sudo /usr/bin/sqlite3 /var/db/DetachedSignatures

Then inside the command line admin you want to run the following commands (substituting your application id for xyzapplication):

delete from global where id in (select distinct(global) from code where lower(identifier) like '%xyzapplication%' );

delete from code where lower(identifier) like '%xyzapplication%';

.quit

That's it. No reboot required. Now you can re-launch your application and it will work just fine - even be able to access your old ELS without hassles.

Obviously this needs to be fixed. It is 100% unreasonable for a real average user to be doing these things and it is happening often enough right now that I'm afraid to ship my product because I'll be swamped with support calls from our numerous MAC users.

I'm a little puzzled as to how my team has been able to figure out a workaround for this and Adobe hasn't in all the time that people have been complaining about this. Why hasn't it been made a priority given the extreme pain involved in recovering from this?

[lharp]

Hello all,

this issue is really annoying and affects also our Goalscape Desktop customers quite often.

Workarounds described above work fine but it's really a nonsense to ask every affected customer to remove/reset or hack some application data and instruct him to do all required workaround steps to make the app work again. Not speaking the fact that those steps are not easy to follow for any non-tech users.

However, there is some related issue filed in Adobe Bug Tracking system.

So please vote for it to get some more attention on this: https://bugbase.adobe.com/index.cfm?event=bug&id=2955326

AIR 3.0 seems to be near to release and this could get finally fixed there!

Tomas

[chris.campbell]

Thank you Tomas, I'd also like to encourage everyone affected to please vote and post their comments.  We take this feedback very seriously.  Additionally, I've been working to get this addressed internally and I'll continue to do so.

Since Jira is no longer used for AIR and Flash Player, please see the following bug over at bugbase.adobe.com

2955326 - Adobe AIR - bad handling of ELS errors

Thanks,

Chris

[lharp]

Hi Chris,

thank you for responding on this one!

It's really good to hear you're already working on that to get it addressed.

Finally we can again hope this will be resolved soon since it's really important for us as it compromises our efforts big time!

Could you please tell me some time estimate for a fix? Or does it first need to get enough votes?

Do you think it could get into the next major release of AIR?

Thanks,

Tomas

[chris.campbell]

Hi Tomas,

Unfortunately, I don't have an ETA on fixes for ELS.  If you haven't already, please check the AIR 3 beta release, though I don't hold much hope that it will be automagically fixed.  As I mentioned, I'm working internally on this but the work required isn't as straight forward as I originally hoped.

I really believe that votes and comments illustrating the impact that this will have on your application and business will be a valuable approach.

Thanks,

Chris

[Guest]

Folks, I think we *finally* have a simple, reliably reproducible test case!

Step 1 - Turn on your MacOSX firewall

Step 2 - Install and run your application and have it store a simple key in ELS

Step 3 - In Finder, mount an SMB network share (like a fileserver)

Step 4 - Execute code in your application that does an air.File.browseForOpen()

Step 5 - You should now see a popup from OSX asking to grant permission to your App to bypass the firewall. Choose ALLOW and then enter your username/password to finish the approval process

Step 6 - Pick a file

Step 7 - shutdown

Step 8 - bump the version number in application.xml and repackage your application with the same certificate

Step 9 - Upgrade your application

Step 10 - Launch the upgraded application and try to read your simple key from ELS - you will get a message about ELS being corrupted.

I'm going to attach a simple sample app that demonstrates this to the bug if I can figure out how to do it. But it seems to work to corrupt ELS every time I try to do it.

-- EDIT --

Since the bug was apparently closed for not reason I can understand, I've opened a new one with the sample app that reproduces it as https://bugbase.adobe.com/index.cfm?event=bug&id=3001156