Eye gaze and Face input source as webcam or X/Y positions

Report · Feb 07, 2023

Eye gaze and Face can only use the webcam as inputs. It would be good to allow another mode where X/Y properties control which direction the face is facing (controls parallax) and which way the eyes are looking.

In my case I want to hook up to an external MIDI source so can use HTC Vive VR equipment to control eye gaze and parallax. I could possibly use Transform behaviors with a lot of care, but its tricky as you need to get the transform for both eyes correct (calibration is hard). It also means the puppet can be used one way or the other - not both.

Report · Feb 07, 2023

In Version 2.0, would tagging those handles with the new Transform tag, then exposing the Position X/Y parameters as controls that are mapped to MIDI help?

Report · Feb 07, 2023

I need to check. The v2 eye gaze and Face do not support new input types other than before. The trickiness was getting both eyes moving well together. For example, if the two eyes have different eye pupil ranges, the results are disproportional. It is hard to get that going right.

Report · Feb 07, 2023

The new transform tag sounds like it just allows transforms to be added to handles (before you added a Transform behavior, which had to be to a group (non-leaf level in hierarchy)). The new capability is a bit cleaner, but does not make it any easier to do eyes or face.

But I don't know how to use it yet. I put a "Transform" tag on an eye and the puppet in the scene window went back to 100% scale (the Transform behavior in the properties seemed to have no effect) and it changed the origin of the image. I probably don't understand how to use it yet, but nothing indicates it allows me to control the Face or Eye Gaze behaviors.

Also, if you add a "Transform" behavior to the pupil, it turns off the Eye Gaze capability. So you cannot use normal eye gaze on the puppet once you add a transform behavior.

So no reason to think it is fixed (but also with only one other vote not expecting this to get fixed any time soon either!)