3D Camera Tracker: unable to acquire rendered frame

[bsined]

I can't stabilize any 360 footage using the 3D Camera Tracker from VR Comp Editor. 95% of the time, I run the analysis to create the camera for stabilization, it fails with the incredibly non-descriptive and undocumented error message "unable to acquire rendered frame." The error appears for videos of any length and any resolution. It also appears at various progress stages of the process and even within the same file (it can be at 5%, or at 55%, or at 98% completion).

At this point, I've exhausted any possible solution I could find:

• Reduced the Comp width down to 260 pixels and aspect ratio to 1:1: it speeds up processing but doesn't guarantee success even with a 6-second video
• Converted the video to sequences of JPEGs and PNGs: the success rate is slightly higher (the process is likely to complete than not) but this slows down the process exponentially (a 20-second video takes 5+ minutes to get converted into images then takes longer to analyze and re-encoded)
• Played with every combination of settings when creating the 3D Edit
• Disabled the antivirus (Windows Defender)
• Turned off all cloud syncing and took the computer offline
• Moved the source files to an M.2 SSD from RAID0 (2x7200rpm WD Black drives)
• Moved disk cache to the M.2 SSD
• Increased paging file size to 16GB
• Tried changing from CUDA to OpenCL and back

The source camera is Nikon KeyMission 360. The videos are stitched in-camera and exported as equirectangular 2:1. Tested video resolution was 3840×1920 and 3840×2160.

Computer specs: Dell XPS 8930, i7-8700, 32GB RAM @ 2667MHz, GeForce GTX 1070 8GB, AE 16.0.0 build 235 with 429 plugins.

Every single tutorial talks about how easy this feature is and has no tips for troubleshooting. I'm hoping an actual Adobe (or Mettle) developer chimes in on this.

[Rameez_Khan]

Hi bsined,

Sorry to hear about your issue and thanks for a detailed post.

Great job trying to troubleshoot the issue so far.

Let us try to reproduce this issue in-house. We'll get back to you for additional questions.

Thanks,

Rameez

[bsined]

RK, do you guys want my source files for testing?

[Sumeet Kumar Choubey]

Hi bsined,

Yes, a download link for the source files would be helpful in reproducing the issue and diagnosing it properly.

Regards

Sumeet

[Shawn Zeilenga]

I'm hoping you guys find the solution.  I am having the exact same problem.  Reduced my footage down to 10 second increments and it stills gives me the same error that bsined had. 

>GoPro Fusion footage exported as .mp4 file from premiere.

>Imported into After Effects

>Apply AE 3D Camera Tracker

>Unable to acquire rendered frames error

• Cache cleaned

• Laptop workstation - Dell Precision 7710

• Processor - Intel i7-6920HQ / 2.9GHz / 4 Core

• Mercury GPU Acceleration (CUDA) enabled

Thanks

Shawn

[Shawn Zeilenga]

I applied a 2nd 3D Camera Tracker and it worked. I just turned off the first one that kept failing.

[bsined]

Thanks! Applying multiple 3D Camera Trackers and disabling previous ones seems to work. However, I've had success only with the 3rd or 4th iteration and not on the first try (I get the error and restart analysis a couple of times). There's definitely some pesky bug hidden somewhere.

[Shawn Zeilenga]

Yeah, it worked one time for me, not perfectly I might add, and then I had

trouble duplicating it. Definitely something Adobe needs to figure out and

inform us on how to move forward.

I agree with your previous statement, I've watched a video on premium beat

on the process and it works fine. I might reach out and see what their

hardware was when creating the video. But my Dell workstation should be

more than capable.

Thanks

Shawn

[bsined]

Why did forum staff mark this as "Answered"? This method doesn't provide consistent results. It's a possible workaround but the software still doesn't work as intended and we have no fool-proof ways to fix it.

[zeno_more]

I first encountered this problem somewhere last week, it occurred only a couple of times (after several successful trackings) after that I was able to do a steady tracking and thought it was an anomaly - However starting 2019 I can't get the tracking analyser to surpass 30% just as stated by Krabaz - anything I try is cut off by this ridiculous 'unable to acquire rendered frame'  could it have been the last update? 

RameezKhan Any update on this?

iMac 2017

Radeon Pro 580 8192 MB + eGPU Radeon RX 580

3,8 GHz Intel Core i5 32gb RAM

using the canvas360 plugin in conjunction with element 3d and several others  - however the problem also persists when trying a complete blanc tracking without the use of these plugins... 

[zeno_more]

Ok. Some good news.
So I'm currently trying tracking with AE cc2018 and its working instantly as of the first try (it might fail in upcoming tests as it worked flawlessly with cc2019 at first as well)  bsined​ any luck on your end? 

[Balcony Labs]

I can confirm. I'm getting the error while tracking 360 footage in AE 2019. Downgraded to AE 2018, and it works again.  Any updates?

[Shawn Zeilenga]

So I changed the format of the file and that almost worked. 

It got hung up on "solving camera".

I took the mp4 file I was using, imported it into Media Encoder and rendered it as a jpeg sequence.

Then I imported that file into After Effects. Then the usual, >Add 3D Edit >Properties >Analyze Background

Thanks
Shawn

[bsined]

I've tried export as an image sequence. I think it worked more consistently but the time drain of exporting the images, analyzing them, and then re-encoding into a video is too much.

I searched again for mentions of the original SkyBox by Mettle and it looks like the bug pre-dates 360 video production. The earliest mention on this forum is from October 2011 and relates to Warp Stabilizer, no answer found. Another guy had the same issue with 3D Tracker in CS6 on Mac in August 2012 but he found it to be related to another software. When Mettle introduced camera tracking in SkyBox Studio 3 years ago, some folks commented about it but never got a response beyond "Some folks have had problems with this technique most have not."

I guess, I can start killing every user-launched process until the camera tracker works but I would rather have Adobe tell me what causes the error message in the first place.

[Shawn Zeilenga]

Hey bsined,

So I was messing around with the 3D Camera Tracker function today and I had some success, but still had an issue.

Under the "Advanced" tab, select the option for "Detailed Analysis".  It pushed the analyzing passed step 1, which is better than before. However, it still failed after "stabilizing" a few times.

Hopefully someone from adobe can offer some advice, since this is critical for a lot of projects moving forward.

I did get success after the 3rd time through.

As you can see the motion tracking points on the bottom image. It was a stable 360 camera rather than a moving one.  And the footage was only 47 seconds long.

Thanks

Shawn

[Shawn Zeilenga]

The only down side is that the tracking points follow the camera, rather than sticking to the environment.

Thanks

Shawn

[krabaz]

Same problem here. I have tried all the tricks from this forum.

1000 frames in 360° 6k/4k/1080 29,97fps as mp4, tiff, jpg and png sequence etc.

It didn't help at all. => "unable to acquire rendered frame."

Also the 24th Tracker start did not bring any improvement. it always breaks off. once earlier, once later.

The basic problem: we work here on client projects.

It can't be due at the PC.  (i9-9900, 32gb-ram 3200mhz, 7000 cuda cores, m2 system and m2 data etc.)

i despair of it. o.O

[Rick Gerard]

I have not tried the camera tracker with 360º footage but I have serious doubts about getting a reliable track. Here's why. AE's cameras have no distortion. AE does not have a spherical camera. The Camera Tracker is going to look at parallax changes in the fixed geometry it finds in a scene and try and figure out where the camera is and how it is moving. If you are camera tracking footage show with wide angle lenses that have a fair amount of barrel distortion you never get a very good track. I almost always try and remove any lens distortion if I need to do accurate camera tracking.

I think you are going to have to look for other options for most 360º footage.

[bsined]

It's literally an advertised and documented feature of After Effects: Stabilize and track 360/VR footage with VR Comp Editor | . We aren't discussing how to improve the quality of the stabilization but the fact that it doesn't work as advertised in AE 16.0.

[zeno_more]

Rick Gerard

It works. By using plugins, as for example canvas 360 - it tracks perfectly as it uses a clever work around.
You can even track 360 footage while using the immersive video dialogue box, set the camera tracker to a view of 90 degrees (or whichever zoom you have your camera set to).   Please research the topic before dismissing it - plenty of tutorials to go around!

Currently have an incredibly steady track with 360 drone footage, added 3D objects, Plexus and Trapcode Form and it stays in place. However this only works with cc2018 somehow the tracking doesn't work within 2019... 

[bsined]

zeno_more, krabaz, ShiverMeTimbers, crookedfoot: I installed CC2018 (15.1.2) and the tracker finishes the analysis without the annoying error. I'm having trouble with camera solving but that's progress.

RameezKhan , Sumeet Kumar Choubey : What's the word from the development team? This is clearly a bug that was introduced in CC2019.

[zeno_more]

So far tracking and solving have gone 10/10 on my end.

Do you use a third party plugin, or the native 'immersive video' / Mettle dialogue box ?
Try setting the camera tracker to 'specify angle of zoom' - adjust to the same angle as your camera zoom (its written in pixels and then in angle degrees within ( ) brackets)

[bsined]

VR Comp Editor is the only plugin applied to the composition. Did you uninstall CC2019? I'm running 15.1.2 and 16 side-by-side.

[zeno_more]

no I haven't uninstalled 2019, currently not running them side by side but when testing I was, everytime tracking and solving was fixed with no hassle. As for this specific project I really need the 3D tracking of 360 footage so im going all out in 2018,  #timetravel so no need to keep 2019 running.

[Rameez_Khan]

Hi bsined,

Sorry for my delayed response!

Let me check this internally and update the thread.

Thanks,

Rameez