Skip to main content
jordans57050364
jordans57050364
Participant
September 2, 2020
Question

AE performance and render-preview SIGNIFICANTLY slowing down when linking text to .CSV/TSV data.

  • September 2, 2020
  • 3 replies
  • 1767 views
Hey guys. I am having a real issue with something in AE and would really appreciate anyone's expertise on this.  Solving this problem would be huge for me, as it would significantly increase my work flow. So any help would be greatly appreciated!
 
In my projects, I work with multiple compositions displaying lots of football stats. For years now I have been manually typing all the stats myself in AE which can be very time consuming. I then stumbled across a tutorial on how to link all these stats to an external CSV. Finding this looked like an absolute gamechanger for me, and was going to save me so much time inputting data
 
However, I am now experiencing a big problem with it. When I link up the data, and display even a small number of say 20 different bits of text, the performance of AE when doing this is incredibly slow, spefically the renderp-preview. My old comps before linking previewed in real time or faster, meaning that editing the project is very quick, convenient and easy to sync my voice. However, the exact same composition with the text boxes now being linked to a .CSV/TSV runs incredibly slow and makes the project unworkable. The render preview takes minutes to even preview a few seconds. I have tried this with both CSV and TSV to the same result.
 
I have also linked them in both ways via source text; one in the project using dataValue and one by dragging the csv directly into the comp and whipping to the correct piece of data. It all links perfectly, but the render-preview is painfully slow even when nothing is happening on screen. Through vigourous research, I am thinking that AE is constantly executing this link for every frame and looking for a change in 'dynamic data', therefore slowing the render-preview right down. However my data is static and don't need it to do this.
 
I am not experienced with expressions or coding in AE, however over the past few days of research I have tried
posterizeTime(0) within my source text and that hasn't helped. Also I tried using a javascript comment '// cacheCompareSamplesPerSecond 0' to no avail. I have tried both render engines in project settings. I have a fairly powerful MBPro with 64GB of RAM.
 
I cannot explain how much help this would be to my workflow if I can solve this problem. So any advice or experience on this would be greatly appreciated!
 
I have attached an unlisted youtube video explaining the issue. it is 5 minutes long but I hope you don't mind watching and trying to help me. (It's much easier to explain via voice and screen rather than text!) https://www.youtube.com/watch?v=-UGgDY6JZgI
 
 
 

    3 replies

    R
    Community Expert
    Rick GerardCommunity Expert
    Community Expert
    September 5, 2020

    It looks like your animation with the linked CSV file renders fairly quickly for the first few seconds, then starts slowing down. This actually kind of makes sense because the expression is going to reach back to the CSV file every time it goes to a new frame. I'm not sure how to stop that. Maybe a valueAtTime() call would help. You might get lucky and have Dan Ebberts take a peek at this thread.

     

    If this was my project my comps would not be any longer than the time it takes to add a new set of data and move it onto the screen. I would then render the comps and do the editing in Premiere Pro. If I needed to extend the screen I would just make a cut and freeze frame or adjust the speed in Premiere so that the audio matches up. My average project is 30 minutes or more, but my average AE comp is less than 7 seconds. Any sequences that I create in AE are no longer than a sentence or phrase or 8 bars of music. Your video shows only a small portion of the comp revealed and that small section is more than 30 seconds. It looks like that comp is probably about six or eight minutes long. I never make comps that long because it is just way too cumbersome to do any editing.

     

    If you must keep everything in one comp I would pre-compose that section that is using the CSV file, enable Time Remapping, and set a keyframe as soon as the animation stops and delete the last TR keyframe. See if that speeds up the previews and rendering. 

     

    Edit: I just made a test comp with 40 text layers and a CSV file. Adding valueAtTime(inPoint) does not change anything. Setting the in and out point of the layer to 1 frame does not change anything. My initial preview was 15 fps for the first 10 seconds of my 30-second comp then it crept up to realtime (30 fps) and it didn't seem to make any difference. I didn't notice any slowdown at all even whey I tripled the number of text layers.

     

    I have noticed significant slowdowns in AE if you have a bunch of shape layers. Are you using the CSV file to do anything to shape layers? That might be the source of your problem.

    Roland Kahlenberg
    Roland Kahlenberg
    Legend
    September 3, 2020

    Hi Brett, did you ensure posterizeTime(0) is the first line in the Expressions it's been used?

    There are two other options -

    1) Nest or PreCompose this comp and use Time Remapping on the new Comp such that AE only has to calculate for the frames that are required to load the data. So, if the comp loads all data by the 2-second mark, apply Time Remapping on the nested comp and set a second keyframe at the 2-second mark; ensuring you delete the last keyframe created by the Time Remapping routine.

     

    2) Bake the Expressions once you're happy with they work. There's a script that automates the process - https://aescripts.com/easy-bake/

    HTH

    Very Advanced After Effects Training | Adaptive & Responsive Toolkits | Intelligent Design Assets (IDAs) | MoGraph Design System DEV
    jordans57050364
    jordans57050364Author
    Participant
    September 4, 2020

    Hey Roland, thank you very much for your response.

     

    Yes I did try posterizeTime(0) and didn't notice any improvement. I must point out I am a rookie when it comes to coding and expressions though. I did add the expression in the first line, but there's still a chance I didn't implement this technique correctly.

     

    However your solution to precompose the text layer sounds very similar to what someone else advised, and that has worked a treat. The method was to make the text layer display each bit of information from the CSV every frame. You can then 'Freeze Frame' this composition, and use time-remapping to call on the exact frame required to display the correct cell.

     

    It was worked fantastically and now the render preview is as quick as ever! The key is to stop AE from executing the expression every frame in case of 'dynamic data'. This is what was slowing everything down. My data is static so the freeze frame and time remap has worked perfectly.

     

    Many thanks for your help!

      Roland Kahlenberg
      Roland Kahlenberg
      Legend
      September 5, 2020

      Thanks for reporting back Brett.

       

      You should be able to set up your 'final comp' such that it's Time Remap keyframes read the time stamp of Markers you place in your 'working comp' – this is to automate the Time Remap routine.

       

      Lastly, I agree with your observation that AE is not intelligently rendering the datadrivern comp even when posterizeTime(0) invoked. Please file a Bug Report and include your AEP and TSV/CSV files in your Bug Report. It always helps when Engineering has a source file to dig into the issue.

      Very Advanced After Effects Training | Adaptive & Responsive Toolkits | Intelligent Design Assets (IDAs) | MoGraph Design System DEV
      jordans57050364
      jordans57050364Author
      Participant
      September 3, 2020

      To anyone who has read this thread, just to let you know I have been in touch with someone and may have a work around. This is no longer urgent but feel free to provide your thoughts if you do have experience with this.

       

      I will post this possible solution tomorrow, thanks!

      M
      mediacontig21600295
      Participant
      April 21, 2022

      Hi, no more follow-up on this subject?

      I'm experiencing the same issue. very slow performance on csv driven comps.

      The weird thing is it's slowing down all the time. First comp would render in 2 minutes then next in 3, 14, 24, 38...

      So it feels like some cache or memory is filling up.

      I'm also not sure if this is only a mac-issue, my colleages on windows machines don't complain...

      If there's any more info on this subject, I would very much apreciate it.

       

      Thanks

       

      Peter

      Terms & ConditionsAccessibility statement
      Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices