Feature Focus: Body Tracker

Live-perform character animation using your body. Powered by Adobe Sensei, Body Tracker automatically detects human body movement using a web cam and applies it to your character in real time to create animation. For example, you can track your arms, torso, and legs automatically.

Get started now with an example puppet

1. Download the Hopscotch_Body Tracking puppet.
2. Open Character Animator (Beta), choose File > Import, and then select the downloaded .puppet file.
3. Click the Add to New Scene button at the bottom of the Project panel to create a scene for the puppet.
4. In the Camera & Microphone panel, make sure the Camera Input button () and Body Tracker Input button () are both turned on. If off (gray), click it to enable that input.
5. Now try moving your body, waving your arms, and if you’re far enough from the camera to see your lower body, raising your feet. Body tracking is that simple!

Additional example puppets are available for you to try out.

Control body tracking

Puppets that respond to body tracking use the Body behavior. When a puppet track is selected in the Timeline panel, the Properties panel shows the Body behavior (among others) that is applied to the puppet. It is this behavior that allow you to customize how the Body Tracker works. For example:

• Set When Tracking Stops to how you want the tracked parts of your puppet’s body to move when your body can’t be tracked for some reason, such as if your legs or arms move out of view from the camera. Hold in Place keeps the tracked handles where they last were before tracking was lost. The Return to Rest and Snap to Rest options, instead, will move the handles to the default state of the artwork, with the former option allowing you to control how fast it moves back with the Return Speed value, whereas the latter option moves back immediately.
• Select the specific Tracked Handles on your puppet that you want to be controlled by your body movement. By default, just the wrists are tracked.

When you want to capture your body tracking performance to later export to a movie file or use in an After Effects or Premiere Pro project, you’ll want to record the performance (Help documentation on recording in Character Animator).

Set up your own puppet for body tracking

The Body behavior lets you control your puppet’s arms, legs, and torso by moving your body in front of your web cam. When you use the Body behavior, be sure to also have the Face behavior applied to control the position of the character and to track your facial expressions. You can also use the Limb IK behavior to prevent limb stretching, inconsistent arm bending, and foot sliding.

Setup

After importing your artwork into Character Animator, do the following steps to rig the puppet so that the Body Tracker can control it:

1. Open the puppet in Rig mode.
2. In the Properties panel’s Behaviors section, click “+”, and then add the Body behavior.
3. Create tagged handles for the different parts of the body’s artwork so that the Body behavior can control their movements:

• Arms: Move the origin handle to the shoulder area from where the arm should rotate; but do not tag it as “Shoulder” directly on the arm. You’ll want to tag your shoulders and hips one level higher in your puppet hierarchy – commonly the Body folder or the view folder (e.g., Frontal) – to keep your limbs attached during body tracking.
  Now, add handles at the locations of the elbow and wrist and apply the Left/Right Elbow and Left/Right Wrist tags to the handles.
• Legs: Like the arms, move the origin handle of the leg to the hip area, but do not tag it as “Hip” directly on the leg. Add tagged handles for Left Knee, Left Heel, and Left Toe to the left leg and foot, and the Right Knee, Right Heel, and Right Toe tags to the handles on the right leg and foot.
• Shoulders & Hips: The Shoulder and Hip handles should be on the body, not on the limbs. Select the parent group containing your limbs and add handles in the same shoulder and hip positions as the arm and leg origin handles. You’ll see green dots from the limbs where you can place them, or you can copy/paste the handles directly from the limbs to position them exactly. It should work fine either way. Tag these handles as Right Shoulder, Left Shoulder, Right Hip, and Left Hip.

Note: Body and Walk behaviors on the same puppet might cause issues because both want to control the same handles. If you do have both behaviors applied, consider turning off the Body behavior when animating walk cycles, and turning off Walk when doing general body animation. The Body behavior should work fine in side and 3/4 views.

Controls

This behavior has the following parameters:

• When Tracking Stops controls how the puppet moves when the camera cannot track your body, such as if you moved out of view. Hold in place keeps the untracked body parts at the same place that they were last tracked. Return to rest moves the untracked body parts gradually to their original locations; use with the Return Speed parameter. Snap to rest moves them quickly to their original locations.
• Return Speed controls how quickly untracked body parts return to their resting locations. This setting is available when the When Tracking Stops parameter is set to Return to rest.
• Tracked Handles controls which of the tagged handles should be controlled by the Body Tracker. There are separate controls for the neck, the joints on the left and right arms, and the joints on the left and right legs.

Troubleshooting

If you’re getting unexpected tracking results, try calibrating the Body Tracker, as follows:

• Click the Set Rest Pose button in the Camera & Microphone panel; when Body Tracker Input is on, Set Rest Pose will show a five-second countdown so that you can move back from the camera to get your entire body into view
• Click the Refresh Scene button () in the Scene panel, when Body Tracker Input is on
• Turn on Camera Input, when Body Tracker Input is also on
• Turn on Body Tracker Input 

The overall positioning of the character comes from face tracking. As such, if you notice issues such as floating feet, choppy movement, and/or not enough horizontal movement, consider checking the following:

• lighting on your face
• starting head position when you click Set Rest Pose 
• Face behavior’s Head Position Strength setting

As with face tracking, the darker lighting conditions in the room can reduce the quality of the tracking and introduce jittery movement. Try to have adequate lighting when tracking your body.

If the arms are not moving, make sure both of your shoulders are being tracked in the camera.

Known issues and limitations

Body Tracker is still in development.

In this first public Beta version (v4.3.0.38), please note the following:

• Foot position might slide or jump around unexpectedly.
• The Walk and Body behaviors do not cooperate well since they are both trying to operate on the same handles simultaneously.
• When your body moves outside of the camera’s view, the bone structure overlay stays visible in the Camera & Microphone panel

What we want to know

We want to hear about your experience with Body Tracking:

• What are your overall impressions?
• Are you able to control your puppet’s movement the way you want?
• Have you found interesting ways to use the Body Tracker?
• Which Body behavior parameters and handles did you find useful or not useful?
• How can we improve the Body Tracker?

Also, we’d love to see what you create with Body Tracker. Share your animations on social media with the #CharacterAnimator hashtag.

Thank you! We’re looking forward to your feedback.

(Use this Beta forum thread to discuss the Body Tracker and share your feedback with the Character Animator team and other Beta users. If you encounter a bug, let us know by posting a reply here or choosing “Report a bug” from the “Provide feedback” icon in the top-right corner of the app.)