3D Camera Tracker: unable to acquire rendered frame

Explorer ,
Oct 20, 2018

Copy link to clipboard

Copied

I can't stabilize any 360 footage using the 3D Camera Tracker from VR Comp Editor. 95% of the time, I run the analysis to create the camera for stabilization, it fails with the incredibly non-descriptive and undocumented error message "unable to acquire rendered frame." The error appears for videos of any length and any resolution. It also appears at various progress stages of the process and even within the same file (it can be at 5%, or at 55%, or at 98% completion).

At this point, I've exhausted any possible solution I could find:

  • Reduced the Comp width down to 260 pixels and aspect ratio to 1:1: it speeds up processing but doesn't guarantee success even with a 6-second video
  • Converted the video to sequences of JPEGs and PNGs: the success rate is slightly higher (the process is likely to complete than not) but this slows down the process exponentially (a 20-second video takes 5+ minutes to get converted into images then takes longer to analyze and re-encoded)
  • Played with every combination of settings when creating the 3D Edit
  • Disabled the antivirus (Windows Defender)
  • Turned off all cloud syncing and took the computer offline
  • Moved the source files to an M.2 SSD from RAID0 (2x7200rpm WD Black drives)
  • Moved disk cache to the M.2 SSD
  • Increased paging file size to 16GB
  • Tried changing from CUDA to OpenCL and back

The source camera is Nikon KeyMission 360. The videos are stitched in-camera and exported as equirectangular 2:1. Tested video resolution was 3840×1920 and 3840×2160.

Computer specs: Dell XPS 8930, i7-8700, 32GB RAM @ 2667MHz, GeForce GTX 1070 8GB, AE 16.0.0 build 235 with 429 plugins.

Every single tutorial talks about how easy this feature is and has no tips for troubleshooting. I'm hoping an actual Adobe (or Mettle) developer chimes in on this.

Annotation.png

Views

11.2K

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

3D Camera Tracker: unable to acquire rendered frame

Explorer ,
Oct 20, 2018

Copy link to clipboard

Copied

I can't stabilize any 360 footage using the 3D Camera Tracker from VR Comp Editor. 95% of the time, I run the analysis to create the camera for stabilization, it fails with the incredibly non-descriptive and undocumented error message "unable to acquire rendered frame." The error appears for videos of any length and any resolution. It also appears at various progress stages of the process and even within the same file (it can be at 5%, or at 55%, or at 98% completion).

At this point, I've exhausted any possible solution I could find:

  • Reduced the Comp width down to 260 pixels and aspect ratio to 1:1: it speeds up processing but doesn't guarantee success even with a 6-second video
  • Converted the video to sequences of JPEGs and PNGs: the success rate is slightly higher (the process is likely to complete than not) but this slows down the process exponentially (a 20-second video takes 5+ minutes to get converted into images then takes longer to analyze and re-encoded)
  • Played with every combination of settings when creating the 3D Edit
  • Disabled the antivirus (Windows Defender)
  • Turned off all cloud syncing and took the computer offline
  • Moved the source files to an M.2 SSD from RAID0 (2x7200rpm WD Black drives)
  • Moved disk cache to the M.2 SSD
  • Increased paging file size to 16GB
  • Tried changing from CUDA to OpenCL and back

The source camera is Nikon KeyMission 360. The videos are stitched in-camera and exported as equirectangular 2:1. Tested video resolution was 3840×1920 and 3840×2160.

Computer specs: Dell XPS 8930, i7-8700, 32GB RAM @ 2667MHz, GeForce GTX 1070 8GB, AE 16.0.0 build 235 with 429 plugins.

Every single tutorial talks about how easy this feature is and has no tips for troubleshooting. I'm hoping an actual Adobe (or Mettle) developer chimes in on this.

Annotation.png

Views

11.2K

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Oct 20, 2018 1
Adobe Employee ,
Oct 31, 2018

Copy link to clipboard

Copied

Hi bsined,

Sorry to hear about your issue and thanks for a detailed post.

Great job trying to troubleshoot the issue so far.

Let us try to reproduce this issue in-house. We'll get back to you for additional questions.

Thanks,

Rameez

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 31, 2018 0
Explorer ,
Oct 31, 2018

Copy link to clipboard

Copied

RK, do you guys want my source files for testing?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Oct 31, 2018 0
Adobe Employee ,
Nov 01, 2018

Copy link to clipboard

Copied

Hi bsined,

Yes, a download link for the source files would be helpful in reproducing the issue and diagnosing it properly.

Regards

Sumeet

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Nov 01, 2018 0
Explorer ,
Nov 01, 2018

Copy link to clipboard

Copied

I'm hoping you guys find the solution.  I am having the exact same problem.  Reduced my footage down to 10 second increments and it stills gives me the same error that bsined had. 

>GoPro Fusion footage exported as .mp4 file from premiere.

>Imported into After Effects

>Apply AE 3D Camera Tracker

>Unable to acquire rendered frames error

• Cache cleaned

• Laptop workstation - Dell Precision 7710

• Processor - Intel i7-6920HQ / 2.9GHz / 4 Core

• Mercury GPU Acceleration (CUDA) enabled

Thanks

Shawn

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Nov 01, 2018 0
Explorer ,
Nov 01, 2018

Copy link to clipboard

Copied

I applied a 2nd 3D Camera Tracker and it worked. I just turned off the first one that kept failing.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Nov 01, 2018 1
Explorer ,
Nov 02, 2018

Copy link to clipboard

Copied

Thanks! Applying multiple 3D Camera Trackers and disabling previous ones seems to work. However, I've had success only with the 3rd or 4th iteration and not on the first try (I get the error and restart analysis a couple of times). There's definitely some pesky bug hidden somewhere.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Nov 02, 2018 0
Explorer ,
Nov 02, 2018

Copy link to clipboard

Copied

Yeah, it worked one time for me, not perfectly I might add, and then I had

trouble duplicating it. Definitely something Adobe needs to figure out and

inform us on how to move forward.

I agree with your previous statement, I've watched a video on premium beat

on the process and it works fine. I might reach out and see what their

hardware was when creating the video. But my Dell workstation should be

more than capable.

Thanks

Shawn

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Nov 02, 2018 0
Explorer ,
Dec 16, 2018

Copy link to clipboard

Copied

Why did forum staff mark this as "Answered"? This method doesn't provide consistent results. It's a possible workaround but the software still doesn't work as intended and we have no fool-proof ways to fix it.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Dec 16, 2018 0
Community Beginner ,
Jan 02, 2019

Copy link to clipboard

Copied

I first encountered this problem somewhere last week, it occurred only a couple of times (after several successful trackings) after that I was able to do a steady tracking and thought it was an anomaly - However starting 2019 I can't get the tracking analyser to surpass 30% just as stated by Krabaz - anything I try is cut off by this ridiculous 'unable to acquire rendered frame'  could it have been the last update? 

RameezKhan Any update on this?

iMac 2017

Radeon Pro 580 8192 MB + eGPU Radeon RX 580

3,8 GHz Intel Core i5 32gb RAM

using the canvas360 plugin in conjunction with element 3d and several others  - however the problem also persists when trying a complete blanc tracking without the use of these plugins... 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 02, 2019 1
Community Beginner ,
Jan 03, 2019

Copy link to clipboard

Copied

Ok. Some good news.
So I'm currently trying tracking with AE cc2018 and its working instantly as of the first try (it might fail in upcoming tests as it worked flawlessly with cc2019 at first as well)  bsined​ any luck on your end? 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 03, 2019 1
Community Beginner ,
Jan 07, 2019

Copy link to clipboard

Copied

I can confirm. I'm getting the error while tracking 360 footage in AE 2019. Downgraded to AE 2018, and it works again.  Any updates?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 07, 2019 2
Explorer ,
Nov 02, 2018

Copy link to clipboard

Copied

So I changed the format of the file and that almost worked. 

It got hung up on "solving camera".

I took the mp4 file I was using, imported it into Media Encoder and rendered it as a jpeg sequence.

Then I imported that file into After Effects. Then the usual, >Add 3D Edit >Properties >Analyze Background

Thanks
Shawn

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Nov 02, 2018 0
Explorer ,
Nov 02, 2018

Copy link to clipboard

Copied

I've tried export as an image sequence. I think it worked more consistently but the time drain of exporting the images, analyzing them, and then re-encoding into a video is too much.

I searched again for mentions of the original SkyBox by Mettle and it looks like the bug pre-dates 360 video production. The earliest mention on this forum is from October 2011 and relates to Warp Stabilizer, no answer found. Another guy had the same issue with 3D Tracker in CS6 on Mac in August 2012 but he found it to be related to another software. When Mettle introduced camera tracking in SkyBox Studio 3 years ago, some folks commented about it but never got a response beyond "Some folks have had problems with this technique most have not."

I guess, I can start killing every user-launched process until the camera tracker works but I would rather have Adobe tell me what causes the error message in the first place.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Nov 02, 2018 0
Explorer ,
Nov 12, 2018

Copy link to clipboard

Copied

Hey bsined,

So I was messing around with the 3D Camera Tracker function today and I had some success, but still had an issue.

Under the "Advanced" tab, select the option for "Detailed Analysis".  It pushed the analyzing passed step 1, which is better than before. However, it still failed after "stabilizing" a few times.

Hopefully someone from adobe can offer some advice, since this is critical for a lot of projects moving forward.

I did get success after the 3rd time through.

As you can see the motion tracking points on the bottom image. It was a stable 360 camera rather than a moving one.  And the footage was only 47 seconds long.

Thanks

Shawn

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Nov 12, 2018 0
Explorer ,
Nov 12, 2018

Copy link to clipboard

Copied

The only down side is that the tracking points follow the camera, rather than sticking to the environment.

Thanks

Shawn

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Nov 12, 2018 0
New Here ,
Dec 29, 2018

Copy link to clipboard

Copied

Same problem here. I have tried all the tricks from this forum.

1000 frames in 360° 6k/4k/1080 29,97fps as mp4, tiff, jpg and png sequence etc.

It didn't help at all. => "unable to acquire rendered frame."

Also the 24th Tracker start did not bring any improvement. it always breaks off. once earlier, once later.

The basic problem: we work here on client projects.

It can't be due at the PC.  (i9-9900, 32gb-ram 3200mhz, 7000 cuda cores, m2 system and m2 data etc.)

i despair of it. o.O

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Dec 29, 2018 0
Adobe Community Professional ,
Jan 07, 2019

Copy link to clipboard

Copied

I have not tried the camera tracker with 360º footage but I have serious doubts about getting a reliable track. Here's why. AE's cameras have no distortion. AE does not have a spherical camera. The Camera Tracker is going to look at parallax changes in the fixed geometry it finds in a scene and try and figure out where the camera is and how it is moving. If you are camera tracking footage show with wide angle lenses that have a fair amount of barrel distortion you never get a very good track. I almost always try and remove any lens distortion if I need to do accurate camera tracking.

I think you are going to have to look for other options for most 360º footage.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 07, 2019 0
Explorer ,
Jan 08, 2019

Copy link to clipboard

Copied

It's literally an advertised and documented feature of After Effects: Stabilize and track 360/VR footage with VR Comp Editor | . We aren't discussing how to improve the quality of the stabilization but the fact that it doesn't work as advertised in AE 16.0.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 08, 2019 0
Community Beginner ,
Jan 08, 2019

Copy link to clipboard

Copied

Rick Gerard

It works. By using plugins, as for example canvas 360 - it tracks perfectly as it uses a clever work around.
You can even track 360 footage while using the immersive video dialogue box, set the camera tracker to a view of 90 degrees (or whichever zoom you have your camera set to).   Please research the topic before dismissing it - plenty of tutorials to go around!

Currently have an incredibly steady track with 360 drone footage, added 3D objects, Plexus and Trapcode Form and it stays in place. However this only works with cc2018 somehow the tracking doesn't work within 2019... 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 08, 2019 1
Explorer ,
Jan 08, 2019

Copy link to clipboard

Copied

zeno_more, krabaz, ShiverMeTimbers, crookedfoot: I installed CC2018 (15.1.2) and the tracker finishes the analysis without the annoying error. I'm having trouble with camera solving but that's progress.

RameezKhan , Sumeet Kumar Choubey : What's the word from the development team? This is clearly a bug that was introduced in CC2019.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 08, 2019 0
Community Beginner ,
Jan 08, 2019

Copy link to clipboard

Copied

So far tracking and solving have gone 10/10 on my end.

Do you use a third party plugin, or the native 'immersive video' / Mettle dialogue box ?
Try setting the camera tracker to 'specify angle of zoom' - adjust to the same angle as your camera zoom (its written in pixels and then in angle degrees within ( ) brackets)

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 08, 2019 0
Explorer ,
Jan 08, 2019

Copy link to clipboard

Copied

VR Comp Editor is the only plugin applied to the composition. Did you uninstall CC2019? I'm running 15.1.2 and 16 side-by-side.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 08, 2019 0
Community Beginner ,
Jan 08, 2019

Copy link to clipboard

Copied

no I haven't uninstalled 2019, currently not running them side by side but when testing I was, everytime tracking and solving was fixed with no hassle. As for this specific project I really need the 3D tracking of 360 footage so im going all out in 2018,  #timetravel so no need to keep 2019 running.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 08, 2019 0
Adobe Employee ,
Jan 08, 2019

Copy link to clipboard

Copied

Hi bsined,

Sorry for my delayed response!

Let me check this internally and update the thread.

Thanks,

Rameez

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 08, 2019 0
Community Beginner ,
Mar 13, 2019

Copy link to clipboard

Copied

RameezKhan​ - have you made any headway here? It looks like the problem has persisted for more than 6 months, and I'm sad to say I'm having exactly this issue trying to stabilize 360 video in After Effects.

My specs:

iMac (Retina 5K, 27-inch, Late 2015)

3.2 GHz Intel Core i5

40 GB 1867 MHz DDR3

AMD Radeon R9 M380 2048 MB

After Effects CC 2019 version 16.0.1 (build 48)

I've tried all the "hacks" above and nothing is really working. It's odd because it was working a few weeks ago on a different shot. I've yet to try a second machine, but that's my next troubleshooting idea at this point.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Mar 13, 2019 1
Adobe Community Professional ,
Mar 13, 2019

Copy link to clipboard

Copied

Some shots just won't track. If you can't get four or five seconds of a shot to track it's either the shot or the format, or your AE is not up to date.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Mar 13, 2019 0
Explorer ,
Mar 13, 2019

Copy link to clipboard

Copied

Rick, we've discussed this with you a few months ago. You're wrong here. There's a bug in AE that presents itself with either any 360 footage or none at all. Nobody can figure out what the conditions are. The only thing that we've established is that the previous version of AE doesn't seem to have it so the advice of having the software updates is useless.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Mar 13, 2019 3
Community Beginner ,
Jan 08, 2019

Copy link to clipboard

Copied

I spoke too soon. My first track in 2018 went smoothly, but since then, I can get analysis to finish but solve hangs indefinitely.

Camera tracking for 360 footage is broken for me at this point.

zeno_more​: Could you explain your method a bit more or point me in the right direction?

Thanks.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 08, 2019 1
Explorer ,
Jan 08, 2019

Copy link to clipboard

Copied

This is incredibly frustrating. People in this thread so far have different hardware (Windows and Macintosh) and different file formats and footage quality, which means the bug is a lot more common than it should be.

I tried downgrading all the way to CC 2017 and installing the standalone SkyBox version of the VR Comp Editor (one of the YouTube tutorials has the "super-secret" download link on Adobe servers, which saves time dealing with e-mail support). In both CC 2018 and CC 2017, I could not stabilize the file, which I had miraculously stabilized in CC 2019 (happened only once). The earlier AE/SkyBox versions fail at the Solving Camera stage regardless of the resolution I use. They either outright fail with the appropriate message or the entire application hangs up after 10 minutes of analysis on a 20-second video. A clean install of CC 2019 still brings the "unable to acquire rendered frame" error.

RameezKhan, can you please ask someone from the VR tools team to join the discussion?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 08, 2019 0
Adobe Community Professional ,
Jan 08, 2019

Copy link to clipboard

Copied

I can't really offer anything other than saying how important it is to start looking at the footage when you are having tracking problems. If the footage contains enough fixed geometry to track and you are getting failures then look at the detail level, contrast, check for reflections and see what can be done to mitigate those problems. If you are still having problems and the footage compressed then think about rendering a visually lossless copy using a standard production format. No matter what kind of tracking you are doing it's important to remember that not all shots can be tracked, camera tracked or stabilized. If you are planning on tracking you owe it to yourself to run a few tests before you get into production and then carefully plan and execute your shots.

As others have said, there also seem to be a lot of system and version inconsistencies. I actually kind of expect that because VR footage is far from standardized and a very small segment of the post-production world. I always keep at least 3 versions of my compositing software on my production machines so I have some easy options to try if things are not working as expected.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 08, 2019 1
Explorer ,
Jan 08, 2019

Copy link to clipboard

Copied

Rick Gerard, thank you for the great idea! You mentioned the completely irrelevant limitations of the camera tracker for 2D shots (geometry, composition, etc.), which gave me the idea of how we all can standardize our troubleshooting. The official Adobe tutorial for this tool has sample footage. In the tutorial, it stabilizes nicely as expected. When I tried to run it on my machine, the analysis failed.

If anyone wants to contribute to troubleshooting and testing this issue, head over here: Stabilize and track 360/VR footage with VR Comp Editor and download the sample files in the "What You'll Need" section right above the video. Follow the tutorial and post your results here.

P.S. Rick, "stabilization" for 360 video means that the image "sphere" is rotated to maintain the horizon at a constant level. This can be done manually with GoPro VR Horizon plugin and many-many keyframes but VR Composition Editor is supposed to make that a lot easier.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jan 08, 2019 0
Community Beginner ,
Apr 26, 2019

Copy link to clipboard

Copied

bsined​ At this point, I actually am now taking your approach with support in order to cut out 50 unrelated questions about my own footage being the issue. I downloaded the footage from https://helpx.adobe.com/after-effects/how-to/stabilize-track-360-footage.html and am using it with support.

I had a long support call last night with a tech who remoted in and verified the issue using these files. Based on the responses from him, I get the impression that this isn't on Adobe's radar at all.

I am supposed to receive a call from a senior tech today. I will promote this thread to every support person I can.

My support case number is: ADB-6830840-P0B4 if any of you want to refer support to it for your own needs.

I'm so frustrated with the situation that I'm starting to look at other 3D Camera Trackers for 360 footage, but I really don't want to shell out any more money at the moment for a basic feature that I'm already paying for.

Will update this thread as I go.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Apr 26, 2019 2