How to Convert a Live Action Video into Anime

[JBJB1029]

How do I use The Cartoon Effect in Adobe After Effects to Convert a Live-Action Video into an Anime Video (OR is there a Better Way to accomplish this task, and if there is, How do I go about doing it)?

[Mylenium]

You're asking too simple a question, no offense. This extends just as much to how you shoot your footage in the first place, what specific look you are after and what additional techniques you plan on using. As you would imagine, it's going to be much easier to convert a well-lit cosplay costume with bright colors in front of a green screen than doing the same for a dark everyday attire in a shadowy alley. Rinse repeat for other things that influence how you even acquire the footage like on-set markers for tracking, additional effects lighting, scene blocking or the camera's capabilities. Conversely, in post it matters how quickly and easily you are able to extract the characters and then how much time you want to spend to do further masking for color corrections, separating individual areas for color ing/ texturing, creating more shapes and masks for additional contours and speed lines and so on. In short: This is much more complex if it's supposed to look good and basically one of those things where advising on specific steps and techniques would require to know what look you have in mind vs. what footage you have/ are able to shoot. Outside that of course you can endlessly watch YouTube tutorials and then piece the techniques together to apply to your project, but there's really no way to explain this in 5 minutes in a few short sentences. These things take practice and experience.

Mylenium

[JBJB1029]

So (for example), how do I make these Live Action Footages:

Into Anime like this?

@Mylenium?

[Mrtn Ritter]

Rotoscoping is the answer.

See this video from "A scanner darkly", where this techinque was used to create an anime style look from actual (greenscreen) footage:

https://www.youtube.com/watch?v=eke5VnpNcNk

Keep in mind, that Hamtaro was drawn in the first place. No effect nor KI from the current state will be able to recreate such a thing from footage, especially if it wasn't filmed for this purpose like the example of Sinister 2. Outlines will be shaky, foreground and background merge into each other and a lot of splill.

For highest quality like in Hamtaro, you need to redraw your footage.

*Martin

[JBJB1029]

If Rotoscoping is the answer, then There has to be a way to do all the Frames in one go, than one frame at a time @Mrtn Ritter

[Mrtn Ritter]

What Mylenium said...

There is currently no way to automate this process with a good result. You'll need a KI, which can identify at least foreground and background, as well as object edges. You need to seperate foreground and background, because on cartoons/animes, background usual has less details in order to tell the viewer where to look to.

Just make a screenshot, go to photoshop and apply one of the art-effects (there is also a cartoon style, i think) and you'll see in an instant, where the issue lies.

*Martin

[Clara29916118cpzb]

I disagree. The question is quite simple. If I have filmed a video at home, and I want it animated, how do I animate that video. The question was simple, you just decided to give someone a filming lesson instead. No offense.

[Mylenium]

This stuff is simply extremely laborious and if you refer to the "A Scanner Darkly" example Martin referred to you can see armies of artists doing extra roto despite the movie being specifically shot around the cartoon look. Same for pretty much any other VFX work - even if they were created with lots of green screen, markers and other techniques in mind, there's still armies of roto artists, matchmovers and so on sitting there for hours on end every day chewing through frame by frame. and those people have much more sophisticated tools than AE available... So for what it's worth, unless you are willing to put in the work, you cannot expect certain things to happen. If it's too much to do yourself, then find collaborators or hire other professionals, but otherwise you may simply need to get to grips with the fact that your cool Anime style YouTube video may take weeks or months even instead of just a few days.

Mylenium

[ShiveringCactus]

Here's a video that popped up on my YouTube front page a while back showing how one guy takes live action footage and converts it to animation.  He uses EBSynth to do the most of the frame-by-frame work.  

You could also look at using Character Animator to create a puppet driven by your webcam.  This isn't a full video-in-one-goal option, but gives you a lot of control.

[JBJB1029]

@Mrtn Ritter(&) @Mylenium, (and @ShiveringCactus), I suggested an Effect for Boris FX to create called "BCC Anime Look": A filter that would be designed to emulate the look of rotoscoped Anime (Japanese: アニメ, IPA: [aɲime]). To give a Live-action Footage [an/that] Anime Look. As you can see in these videos (Below):

• https://youtu.be/023oOTQXipY 
• https://youtu.be/oUi829DFv4I 
• https://youtu.be/KRE4QZAXYu4

And guess what? Jonathan from Boris FX said that it's pretty cool and that he'll pass it along. However, if they end up making it, it would likely be months away or more. But I gave them some additional information that may help speed up its creation.

[Mrtn Ritter]

That is cool, but all those videos show pretty impressive the state of this technique. None of those KIs are getting close to the look and feel of actual drawn anime. They all do look more of less like the carton filter of photoshop.

*Martin

[JBJB1029]

Is that a Good thing or a Bad thing @Mrtn Ritter?

[Mrtn Ritter]

Haha, well this is pretty philosophical, isn't it?

If you are aiming for a hand drawn anime, it's pretty bad that todays technology won't be able to do that. But if you widen your scope and be open for new styles to emerge, its pretty impressive what those AIs and filters can do.

I think newer technologies shouldn't try to imitate older ones. This will always fail. They have to find there own place. Something to play with: https://app.wombo.art

*Martin

[JBJB1029]

So, um ... Your Point is @Mrtn Ritter?

[Mrtn Ritter]

My point is: if you want hand drawn anime, you have to hand draw it. There is currently no way to recreate this style without doing the hard work.

You can get close to the style with a lower quality or another look and feel, though, and this might be already enough for the purpose or even just what you are aiming for.

*Martin

[JBJB1029]

What about using a Sketch Effect @Mrtn Ritter?

[Mrtn Ritter]

Just do it, if you like the result.

*Martin

[Joost van der Hoeven]

Out of the box:

You can install https://snapcamera.snapchat.com/ on your computer and use the anime lens. Then install a tool like splitcam and apply the lens to an existing video.
https://www.youtube.com/watch?v=0zoZ-Ap4FHc

[JBJB1029]

Out of the box:

You can install https://snapcamera.snapchat.com/ on your computer and use the anime lens. Then install a tool like splitcam and apply the lens to an existing video.
https://www.youtube.com/watch?v=0zoZ-Ap4FHc

By @Joost van der Hoeven

And does this video at https://www.youtube.com/watch?v=0zoZ-Ap4FHc explain how to Convert a Live Action Video into Anime?