Highlighted

AE performance and render-preview SIGNIFICANTLY slowing down when linking text to .CSV/TSV data.

Community Beginner ,
Sep 02, 2020

Copy link to clipboard

Copied

Hey guys. I am having a real issue with something in AE and would really appreciate anyone's expertise on this.  Solving this problem would be huge for me, as it would significantly increase my work flow. So any help would be greatly appreciated!
 
In my projects, I work with multiple compositions displaying lots of football stats. For years now I have been manually typing all the stats myself in AE which can be very time consuming. I then stumbled across a tutorial on how to link all these stats to an external CSV. Finding this looked like an absolute gamechanger for me, and was going to save me so much time inputting data
 
However, I am now experiencing a big problem with it. When I link up the data, and display even a small number of say 20 different bits of text, the performance of AE when doing this is incredibly slow, spefically the renderp-preview. My old comps before linking previewed in real time or faster, meaning that editing the project is very quick, convenient and easy to sync my voice. However, the exact same composition with the text boxes now being linked to a .CSV/TSV runs incredibly slow and makes the project unworkable. The render preview takes minutes to even preview a few seconds. I have tried this with both CSV and TSV to the same result.
 
I have also linked them in both ways via source text; one in the project using dataValue and one by dragging the csv directly into the comp and whipping to the correct piece of data. It all links perfectly, but the render-preview is painfully slow even when nothing is happening on screen. Through vigourous research, I am thinking that AE is constantly executing this link for every frame and looking for a change in 'dynamic data', therefore slowing the render-preview right down. However my data is static and don't need it to do this.
 
I am not experienced with expressions or coding in AE, however over the past few days of research I have tried
posterizeTime(0) within my source text and that hasn't helped. Also I tried using a javascript comment '// cacheCompareSamplesPerSecond 0' to no avail. I have tried both render engines in project settings. I have a fairly powerful MBPro with 64GB of RAM.
 
I cannot explain how much help this would be to my workflow if I can solve this problem. So any advice or experience on this would be greatly appreciated!
 
I have attached an unlisted youtube video explaining the issue. it is 5 minutes long but I hope you don't mind watching and trying to help me. (It's much easier to explain via voice and screen rather than text!) https://www.youtube.com/watch?v=-UGgDY6JZgI
 
 
 
Enjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.

Topics

Expressions, How to, Import and export, Performance

Views

101

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

AE performance and render-preview SIGNIFICANTLY slowing down when linking text to .CSV/TSV data.

Community Beginner ,
Sep 02, 2020

Copy link to clipboard

Copied

Hey guys. I am having a real issue with something in AE and would really appreciate anyone's expertise on this.  Solving this problem would be huge for me, as it would significantly increase my work flow. So any help would be greatly appreciated!
 
In my projects, I work with multiple compositions displaying lots of football stats. For years now I have been manually typing all the stats myself in AE which can be very time consuming. I then stumbled across a tutorial on how to link all these stats to an external CSV. Finding this looked like an absolute gamechanger for me, and was going to save me so much time inputting data
 
However, I am now experiencing a big problem with it. When I link up the data, and display even a small number of say 20 different bits of text, the performance of AE when doing this is incredibly slow, spefically the renderp-preview. My old comps before linking previewed in real time or faster, meaning that editing the project is very quick, convenient and easy to sync my voice. However, the exact same composition with the text boxes now being linked to a .CSV/TSV runs incredibly slow and makes the project unworkable. The render preview takes minutes to even preview a few seconds. I have tried this with both CSV and TSV to the same result.
 
I have also linked them in both ways via source text; one in the project using dataValue and one by dragging the csv directly into the comp and whipping to the correct piece of data. It all links perfectly, but the render-preview is painfully slow even when nothing is happening on screen. Through vigourous research, I am thinking that AE is constantly executing this link for every frame and looking for a change in 'dynamic data', therefore slowing the render-preview right down. However my data is static and don't need it to do this.
 
I am not experienced with expressions or coding in AE, however over the past few days of research I have tried
posterizeTime(0) within my source text and that hasn't helped. Also I tried using a javascript comment '// cacheCompareSamplesPerSecond 0' to no avail. I have tried both render engines in project settings. I have a fairly powerful MBPro with 64GB of RAM.
 
I cannot explain how much help this would be to my workflow if I can solve this problem. So any advice or experience on this would be greatly appreciated!
 
I have attached an unlisted youtube video explaining the issue. it is 5 minutes long but I hope you don't mind watching and trying to help me. (It's much easier to explain via voice and screen rather than text!) https://www.youtube.com/watch?v=-UGgDY6JZgI
 
 
 
Enjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.

Topics

Expressions, How to, Import and export, Performance

Views

102

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Community Beginner ,
Sep 02, 2020

Copy link to clipboard

Copied

To anyone who has read this thread, just to let you know I have been in touch with someone and may have a work around. This is no longer urgent but feel free to provide your thoughts if you do have experience with this.

 

I will post this possible solution tomorrow, thanks!

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Adobe Community Professional ,
Sep 02, 2020

Copy link to clipboard

Copied

Hi Brett, did you ensure posterizeTime(0) is the first line in the Expressions it's been used?

There are two other options -

1) Nest or PreCompose this comp and use Time Remapping on the new Comp such that AE only has to calculate for the frames that are required to load the data. So, if the comp loads all data by the 2-second mark, apply Time Remapping on the nested comp and set a second keyframe at the 2-second mark; ensuring you delete the last keyframe created by the Time Remapping routine.

 

2) Bake the Expressions once you're happy with they work. There's a script that automates the process - https://aescripts.com/easy-bake/

HTH

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Community Beginner ,
Sep 04, 2020

Copy link to clipboard

Copied

Hey Roland, thank you very much for your response.

 

Yes I did try posterizeTime(0) and didn't notice any improvement. I must point out I am a rookie when it comes to coding and expressions though. I did add the expression in the first line, but there's still a chance I didn't implement this technique correctly.

 

However your solution to precompose the text layer sounds very similar to what someone else advised, and that has worked a treat. The method was to make the text layer display each bit of information from the CSV every frame. You can then 'Freeze Frame' this composition, and use time-remapping to call on the exact frame required to display the correct cell.

 

It was worked fantastically and now the render preview is as quick as ever! The key is to stop AE from executing the expression every frame in case of 'dynamic data'. This is what was slowing everything down. My data is static so the freeze frame and time remap has worked perfectly.

 

Many thanks for your help!

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Adobe Community Professional ,
Sep 04, 2020

Copy link to clipboard

Copied

Thanks for reporting back Brett.

 

You should be able to set up your 'final comp' such that it's Time Remap keyframes read the time stamp of Markers you place in your 'working comp' – this is to automate the Time Remap routine.

 

Lastly, I agree with your observation that AE is not intelligently rendering the datadrivern comp even when posterizeTime(0) invoked. Please file a Bug Report and include your AEP and TSV/CSV files in your Bug Report. It always helps when Engineering has a source file to dig into the issue.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Adobe Community Professional ,
Sep 05, 2020

Copy link to clipboard

Copied

It looks like your animation with the linked CSV file renders fairly quickly for the first few seconds, then starts slowing down. This actually kind of makes sense because the expression is going to reach back to the CSV file every time it goes to a new frame. I'm not sure how to stop that. Maybe a valueAtTime() call would help. You might get lucky and have Dan Ebberts take a peek at this thread.

 

If this was my project my comps would not be any longer than the time it takes to add a new set of data and move it onto the screen. I would then render the comps and do the editing in Premiere Pro. If I needed to extend the screen I would just make a cut and freeze frame or adjust the speed in Premiere so that the audio matches up. My average project is 30 minutes or more, but my average AE comp is less than 7 seconds. Any sequences that I create in AE are no longer than a sentence or phrase or 8 bars of music. Your video shows only a small portion of the comp revealed and that small section is more than 30 seconds. It looks like that comp is probably about six or eight minutes long. I never make comps that long because it is just way too cumbersome to do any editing.

 

If you must keep everything in one comp I would pre-compose that section that is using the CSV file, enable Time Remapping, and set a keyframe as soon as the animation stops and delete the last TR keyframe. See if that speeds up the previews and rendering. 

 

Edit: I just made a test comp with 40 text layers and a CSV file. Adding valueAtTime(inPoint) does not change anything. Setting the in and out point of the layer to 1 frame does not change anything. My initial preview was 15 fps for the first 10 seconds of my 30-second comp then it crept up to realtime (30 fps) and it didn't seem to make any difference. I didn't notice any slowdown at all even whey I tripled the number of text layers.

 

I have noticed significant slowdowns in AE if you have a bunch of shape layers. Are you using the CSV file to do anything to shape layers? That might be the source of your problem.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...