How to put animation over video in HTML5 canvas

[yigit14]

Hello, i have a project similar jibjab, basically people will upload their face photo and their face will replace in the video. Because of mobile platforms popularity i want do this project mobile friendly.

-I created HTML5 canvas on Adobe Animate CC 2017

-Added video from components

-Opened Actionscript 3 document, added video as flv embed, put head png and track manually the original head.

-Copy paste the head png with layer to HTML5 canvas but couldnt success.

(I have done this with Actionscript 3 helloyigit.com/deneme)

My question is:

How can i do this project efficiently with HTML5 and show user as a video? Also head animation and video must sync %100

[ClayUUID · Accepted solution]

You can't put canvas elements on top of video, because the video actually exists in a div layer floating above the canvas. The only way around this, theoretically, is to pipe the video from an offscreen video element into a Bitmap object.

javascript - Adding video to an HTML 5 Canvas using Easel.js - Stack Overflow

As for syncing, should be possible by monitoring the video's time code.

Syncing Content With HTML5 Video – Smashing Magazine

[ClayUUID]

So I figured I'd verify my own advice, and came up with this simple function. Seems to work fine on desktop Windows IE, Firefox, and Chrome.

// accepts target clip to create bitmap in, file path to video
// returns reference to bitmap object, with access to video element via "video" property
function makeBitmapVideo(clip, path) {
    var vid = document.createElement("video");
    vid.src = path;
    var bmp = clip.addChild(new createjs.Bitmap(vid));
    bmp.video = vid;
    return bmp;
}

An example of how to use it:

var myClip = makeBitmapVideo(this, "placeholder.mp4");
myClip.video.play();
myClip.rotation = 45;

Note that apparently this will not work on an iPad: Putting Video on Canvas

That's Apple for ya. Kill off the Flash plugin, then half-a** support the thing that's supposed to replace it.

[yigit14]

Thank you very much ClayUUID for advice, all these codes i have to add on actions layer 1 frame or need to do something different? Can you please help me about steps that i have to follow?

[avid_Ray5CC8]

The video loads so  $this._element.attach($('#dom_overlay_container'));  in JS

After insert z-ndex в <canvas> and <div id='dom_overlay_container'>

And remove backgound color from <canvas>

[sasori3900861]

Thank you, this is an easy way out that works.

[Guest]

This worked great and was the last key to doing what I wanted once I figured out what it was doing. Thanks!

The z-index: style is the key. You don't even need to use JS to attach the canvas to a div if you're using CSS to make them overlap. I did this using Flask, so the {{width}} and {{height}} are jinja that's being replaced by variables from python code.

    <div id="div_outer" class="outerImageContainer" style="width:{{width}}px;height:{{height}}px;">
    <div id="div_inner" class="innerImageContainer">
        <canvas id="can" class="coveringCanvas" style="z-index: 999;" width="{{width}}px" height="{{height}}px" ></canvas>
        <video id="vid" onplaying="startDrawing()" onsuspend="stopDrawing()"
               src="/static/myvideo.mp4" controls="true" type="video/mp4" class="coveredImage"></video>
    </div>
    </div>

Then the CSS that makes canvas and video overlap the same area.

.outerImageContainer
{ margin: 20px;}
.innerImageContainer
{ width: 100%; height: 100%; position: relative; }
.coveredImage
{ width: 100%; height: 100%; position: absolute; top: 0px; left: 0px;}
.coveringCanvas
{ width: 100%; height: 100%; position: absolute; top: 0px; left: 0px; pointer-events: none;}

Finally I have this to draw the animations in canvas

function draw() {
    var time = vid.currentTime;
    if(time !== lastTime) {
        var c=document.getElementById("can");
        var ctx=c.getContext("2d");
        //no reason to redraw image but you can. it will overlap video controls:
          //ctx.drawImage(vid,0,0,{{width}}, {{height}}, 0,0,{{width}}, {{height}});
         //draw stuff here on top of the video each frame
        lastTime = time;
    }
    requestAnimationFrame(draw);
}
function startDrawing() {
    draw();
}