Code review: Slow to start

Report · Nov 08, 2018

I have an Adobe CEP project for Illustrator CC 2018-2019 that I've finished, more or less. I plan on distributing it through the art department of our company only. However, one problem that my beta tester (read: guinea pig) has noted is that it is slow to start. That is, it takes a significantly long period of time between the time that the panel appears and the blank gray area is filled with the HTML content.

Normally, I'd say that this behavior is indicative of too many libraries being loaded up in the header. However, since this is a completely local project (no CDN references) and there are only a few small CSS files loading in the header, I honestly don't know why it takes a good 30+ seconds for the panel to completely load.

Can anyone please help me to identify where the problem lies or is this just normal for CEP panels? You can clone my repo of the project from Github. Also, I have the packaged ZXP of it, if there's a way I can post it here for testing. (Although I guess you wouldn't be able to see the source code in it.)

Thank you.

Report · Nov 12, 2018

I think your project not so heavy.

If you using timestamping, an extension communicates with a timestamp server to validates timestamping when startup it.

Probably, In your cases, validation processing over the network takes time for some reasons.

Report · Nov 12, 2018

Hi, Ten, thanks for replying.

The project does not use any timestamping of which I am aware. Even if it did, shouldn't the HTML "page" be rendered in the panel before any scripting is done? All of the scripts are at the end of the <body> element for exactly this reason. There shouldn't be any processing over the networking before the scripts are called, right?

Report · Nov 12, 2018

Have the client time the load time with the debug flag set to true.

If it loads quickly then the problem is hashing the files.

There is what to do about that but see if that is the problem

Report · Nov 12, 2018

If you mean to time it using the Chrome DevTools, I would not be able to do so, since those tools are not available until after the panel is fully loaded. (Unless you know of a way for me to do so.) If, however, you mean to time the loading of the extension using a stopwatch or similar tool, then I can tell you right now that it takes a long time to start up even on my dev machine.

That said, I'm not entirely sure what you mean by the "debug flag." Do you just mean the existence of the ".debug" file in the root folder of the extension? Or the "defaults write" command on macOS? Or both? Actually, I realize now that I have packaged that ".debug" file in the .ZXP file; Is that okay and won't cause slowdowns on the production machines it will be running on? Or do I need to remove it before packaging the extension? FYI, none of the production machines will have that "defaults write" command run on them.

Something else that I just realized which you wouldn't immediately see from my Github repo: This project has a `node_modules/` folder. Along with the usual default subfolders and files in it, I've added a few packages necessary for my project. Recursively, it has a total of almost 13,000 files. Could this be at least part of the reason for the slowdown? Maybe CEP scans/indexes all files in the extension's tree before rendering the HTML for the panel? Please forgive my newbie ignorance.

Report · Nov 12, 2018

See Setting Cep Debug Mode and Log Level the Easy Way | Creative-Scripts.com

If the flag is set to 1 then it should go a lot quicker, the node modules will really slow things down.

You can try use https://rollupjs.org/guide/en to reduce the amount of files you are using and then the startup time can be greatly reduced.

Report · Nov 12, 2018

Just to clarify a bit what's happening.

Adobe wants to provide some security protection for both the users and developers. They do this hashing all the files. For example if we have a file x.js whose contents is "abc" some calculation will be applied to the contents to produce some code unique to that file like 123 that process can be quite quick but is the contents is somewhat longer and if there are 13,000 files to process then it's going to really long time. If in any of those file any change is made then the hash won't match the files and we know the extension has been fiddled with, so it won't load.

Developer need to be able to work with non signed extensions otherwise the development process would take ages.

So one can set a debug flag that will disable the hash checks and allow for fiddled unsigned extensions to run. This will allow the extensions to start up much quicker.

If the extension is internal, i.e. for your own org then you can disable the check, if it's external then you have to minimize the code.

There are different ways of installing node modules with huge differences in the amount of files but even the reduced one will contain loads of code which is not needed so for the distribution version of the extension you can roll up the code.

The rollup will remove any files that are not need and combine the needed ones into 1 file that only contains the functions required for the code to work.

In short it could be you can reduce those 13000 files into a handful of files and that's going to make the hash time insignificant.

Report · Nov 13, 2018

Thank you, Trevorׅ, that really does help me to understand what's going on and why it is so slow on startup. And I've long since already set the debug mode flag, but that doesn't appear to help in the speed department.

I've only briefly looked at rollup.js so far, but it seems to be just an alternative to Webpack, unless I'm mistaken. I am already using Webpack in this project, but only for the `dom_html` side for obvious reasons. Regardless, the issue appears to be the simple fact that the giant `node_modules/` directory exists in the project folder. Thus, when I am using ZXPSignCmd to package it up, it packages up that `node_modules/` folder along with everything else. (I don't think there's any way I can exclude it directly in that command.) And, indeed, when I manually copy over just those libraries I need from `node_modules/` into a subfolder of `dom_html/` or `dom_app/` (relinking them in the code, of course), then deleting the `node_modules/` folder entirely, then it does start up rather rapidly.

So, I got to thinking that perhaps all I need is a deployment script that 1) copies over everything except the `node_modules/` folder to a separate location, along with my `.p12` and `ZXPSignCmd` files, then 2) runs them to package it up into a .zxp file. Since I have a PHP background and am already familiar with it, I think I'll use Deployer. That way, I can run just one command and have it do what I need. Theoretically. I'll report back here with my success/failure in doing so.

That said, if rollup.js is a better method and can actually lead to excluding the `node_modules/` folder without the need for me to manually copy over the libraries, please let me know. Although, I'd probably need a link to a good rollup.js tutorial for this purpose (extension-making), as I always have a hard time learning new software from just the software's own docs.

Report · Nov 16, 2018

Sorry for the delay. Things have been busy where I work.

So, my method of using Deployer seems to have worked. It’s kind of a batch-processing tool and I’ve made a recipe for it so that it will ultimately package up a .zxp file without the node_modules/ folder. The only caveat to this method is that I update the "local" copies of libraries whenever they are updated through npm update or yarn upgrade.

As I said before, I’m totally willing to learn about Rollup, but I’ll need to investigate it further and find some good tutorials. That will take some time, which I sadly do not currently have. I have also heard that both npm and yarn will eventually have some tools (forgot what they are called) which will prune that node_modules/ folder to only the packages that are used. So I’ll try and keep an eye out for that, too.

For now, I’ll just go ahead and mark your answer as the correct one, as it seems to have put me on the right path, at the very least. Thank you so much for your help, Trevorׅ!

Report · Apr 29, 2020

I had the same problem with node_modules. 5000+ files. I've used the tool zeit/ncc, you ll find it on npm. This can compile one node app in one file. If you are lucky it will work. For socket.io I had to add some files. I compiled all pug files to html cause pug uses uglyfyJs. I worked a whole day (lost a whole day) on a compile tool to make this stuff workable. I ended with a working extension that uses several npm libraries with only 62 files. Adobe, solve this if you want us to use extensions in our companies!

Report · Aug 13, 2020

moved to the exhcange forum

Code review: Slow to start

1 Correct answer