Help with AE camera matrix >> Opengl Matrix?

[Chris_Roe]

I've been toying with the SDK for a few weeks now, I'm finally getting somewhere, but I've gotten to a point that I can't figure out. I'm trying to use the After Effects camera data to move around in my opengl scene. Here is the code I have

First I get the Camera Matrix.

-------------

//Variables...
    A_Matrix4            camMatrix;
   
    GLfloat                glMatrix[4][4];

    GLfloat                glMatInverted[16];

    GLfloat                glMatrix2[16];

...
    A_FpLong planeDist;

     ERR(suites.PFInterfaceSuite1()->AEGP_ConvertEffectToCompTime(in_data->effect_ref,
                                                                     in_data->current_time,
                                                                     in_data->time_scale,
                                                                     &comp_timeT));
       
     ERR(suites.PFInterfaceSuite1()->AEGP_GetEffectCamera(in_data->effect_ref,
                                                             &comp_timeT,
                                                             &camera_layerH));

    ERR(suites.CameraSuite2()->AEGP_GetDefaultCameraDistanceToImagePlane (compH, &planeDist));
   
    ERR(suites.PFInterfaceSuite1()->AEGP_GetEffectCameraMatrix( in_data->effect_ref,
                                                                &comp_timeT,
                                                                &matrix,
                                                                &planeDist,
                                                               (A_short*)in_data->width,
                                                               (A_short*)in_data->height));

.. Here's where it's most likely going wrong.. ..

            // AE Matrix
            /*
            
             0   1  2  3
             4   5  6  7
             8   9 10 11
             12 13 14 15
            
             OGL Matrix
            
             0    4    8    12
             1    5    9    13
             2    6    10    14
             3    7    11    15
             */

// First I convert to the opengl matrix format

            for (x = 0; x < 4; x++)
            {
                glMatrix[0] = matrix.mat[0];
                glMatrix[1] = matrix.mat[1];
                glMatrix[2] = matrix.mat[2];
                glMatrix[3] = matrix.mat[3];   
            }

//  Then I convert the matrix into a 1-d array of floats

                   
            int count=0;
            for (x = 0; x < 4; x++) // rows
            {
                glMatrix2[count]     = glMatrix[0];
                glMatrix2[count++] = glMatrix[1];
                glMatrix2[count++] = glMatrix[2];
                glMatrix2[count++] = glMatrix[3];   
               
                count++;  
            }

-------------

// I then invert the matrix using a function I borrowed from Mesa3d, it is at the bottom of this post for reference.

gluInvertMatrix ( glMatrix2, glMatInverted );

-------------

-------------

And then down in my drawing routine, when setting up my GL scene, I use the following

-------------

            //set the matrix modes
            glMatrixMode( GL_PROJECTION );
            glLoadIdentity();
           
            gluPerspective( 45.0, (GLdouble)widthL / heightL, 0.1, 100.0 );
           
            // Set up the frame-buffer object just like a window.
            glViewport( 0, 0, widthL, heightL );
            glClearColor( 0.0f, 0.0f, 0.0f, 0.0f );
            glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT );
           
            //spin using slider [TODO] as slider control
            glMatrixMode( GL_MODELVIEW );
            glLoadIdentity();
           
            glLoadMatrixf( glMatInverted ); // <---- I am loading in the matrix that I created from the ae camera here.

-------------

When I don't include the LoadMatrix call, everything is visible, but when I do, I see nothing, so my assumption is I am way off in my calculations somewhere, I'm hoping someone familiar with this can spot my obvious error.

I am debugging this and logging my matrix before and after conversion, and it is definately working, I don't *think* i'm losing data anywhere... Any help is greatly appreciated.

-Chris

-------------

bool gluInvertMatrix(GLfloat m[16], GLfloat invOut[16])
{
    GLfloat inv[16], det;
    int i;
   
    inv[0] =   m[5]*m[10]*m[15] - m[5]*m[11]*m[14] - m[9]*m[6]*m[15]
    + m[9]*m[7]*m[14] + m[13]*m[6]*m[11] - m[13]*m[7]*m[10];
    inv[4] =  -m[4]*m[10]*m[15] + m[4]*m[11]*m[14] + m[8]*m[6]*m[15]
    - m[8]*m[7]*m[14] - m[12]*m[6]*m[11] + m[12]*m[7]*m[10];
    inv[8] =   m[4]*m[9]*m[15] - m[4]*m[11]*m[13] - m[8]*m[5]*m[15]
    + m[8]*m[7]*m[13] + m[12]*m[5]*m[11] - m[12]*m[7]*m[9];
    inv[12] = -m[4]*m[9]*m[14] + m[4]*m[10]*m[13] + m[8]*m[5]*m[14]
    - m[8]*m[6]*m[13] - m[12]*m[5]*m[10] + m[12]*m[6]*m[9];
    inv[1] =  -m[1]*m[10]*m[15] + m[1]*m[11]*m[14] + m[9]*m[2]*m[15]
    - m[9]*m[3]*m[14] - m[13]*m[2]*m[11] + m[13]*m[3]*m[10];
    inv[5] =   m[0]*m[10]*m[15] - m[0]*m[11]*m[14] - m[8]*m[2]*m[15]
    + m[8]*m[3]*m[14] + m[12]*m[2]*m[11] - m[12]*m[3]*m[10];
    inv[9] =  -m[0]*m[9]*m[15] + m[0]*m[11]*m[13] + m[8]*m[1]*m[15]
    - m[8]*m[3]*m[13] - m[12]*m[1]*m[11] + m[12]*m[3]*m[9];
    inv[13] =  m[0]*m[9]*m[14] - m[0]*m[10]*m[13] - m[8]*m[1]*m[14]
    + m[8]*m[2]*m[13] + m[12]*m[1]*m[10] - m[12]*m[2]*m[9];
    inv[2] =   m[1]*m[6]*m[15] - m[1]*m[7]*m[14] - m[5]*m[2]*m[15]
    + m[5]*m[3]*m[14] + m[13]*m[2]*m[7] - m[13]*m[3]*m[6];
    inv[6] =  -m[0]*m[6]*m[15] + m[0]*m[7]*m[14] + m[4]*m[2]*m[15]
    - m[4]*m[3]*m[14] - m[12]*m[2]*m[7] + m[12]*m[3]*m[6];
    inv[10] =  m[0]*m[5]*m[15] - m[0]*m[7]*m[13] - m[4]*m[1]*m[15]
    + m[4]*m[3]*m[13] + m[12]*m[1]*m[7] - m[12]*m[3]*m[5];
    inv[14] = -m[0]*m[5]*m[14] + m[0]*m[6]*m[13] + m[4]*m[1]*m[14]
    - m[4]*m[2]*m[13] - m[12]*m[1]*m[6] + m[12]*m[2]*m[5];
    inv[3] =  -m[1]*m[6]*m[11] + m[1]*m[7]*m[10] + m[5]*m[2]*m[11]
    - m[5]*m[3]*m[10] - m[9]*m[2]*m[7] + m[9]*m[3]*m[6];
    inv[7] =   m[0]*m[6]*m[11] - m[0]*m[7]*m[10] - m[4]*m[2]*m[11]
    + m[4]*m[3]*m[10] + m[8]*m[2]*m[7] - m[8]*m[3]*m[6];
    inv[11] = -m[0]*m[5]*m[11] + m[0]*m[7]*m[9] + m[4]*m[1]*m[11]
    - m[4]*m[3]*m[9] - m[8]*m[1]*m[7] + m[8]*m[3]*m[5];
    inv[15] =  m[0]*m[5]*m[10] - m[0]*m[6]*m[9] - m[4]*m[1]*m[10]
    + m[4]*m[2]*m[9] + m[8]*m[1]*m[6] - m[8]*m[2]*m[5];
   
    det = m[0]*inv[0] + m[1]*inv[4] + m[2]*inv[8] + m[3]*inv[12];
    if (det == 0)
        return false;
   
    det = 1.0 / det;
   
    for (i = 0; i < 16; i++)
        invOut = inv * det;
   
    return true;
}

[shachar carmi]

hi chris.

i'm no expert on openGL, but i think i found a problem in your code.

         int count=0;
            for (x = 0; x < 4; x++) // rows
            {
                glMatrix2[count]     = glMatrix[0];//  <----- this is not right. it should be [count++]
                glMatrix2[count++] = glMatrix[1];
                glMatrix2[count++] = glMatrix[2];
                glMatrix2[count++] = glMatrix[3];   
               
                count++;   // <------ and this is not needed.
            }

currently the first value in this loop get's overwritten by the second value. (unless you wish to change the syntax to [++count] instead)

if that doesn't solve it, we'll keep looking.

🙂

[Chris_Roe]

Ha! That was indeed wrong, but alas, still nothing on the screen...

Also - Turns out that the file proj file wasn't actually being saved, everything else was . . . so I was "debugging" an old version last nite. ugg...

Would it be possible for me to give you my  source file, or I can give the whole project to see if I'm doing things in the wrong order? I've put break points and it *was* not working when using the plane_width and plane_height in the GetEffectCameramatrix - None of the functions in the SDK return a A_short - of the width, so I'm typecasting the composition A_longs that I am getting,  it *seems* to work, but honestly I'm not sure that I'm even getting appropriate matrix information at this point.

ex:

                                    ERR(suites.PFInterfaceSuite1()->AEGP_GetEffectCameraMatrix( in_data->effect_ref,
                                                                                               &comp_timeT,
                                                                                               &matrix,
                                                                                               &focal_lengthL,
                                                                                               (A_short*)&compwL,
                                                                                               (A_short*)&comphL));

I guess I need to study up on Opengl's model view and see if I'm missing something..

Frustrating...

-chris

[gutsblow]

Chris,

Good to see you here in Plug-ins instead of scripts! I am not a biggie in SDK but GetEffectCameraMatrix never works the way I wanted. Instead I am using AEGP_GetLayerToWorldXformFromView. You may not need to inverse this matrix, which I am not so sure. Check the Sample project 'Resizer'. It has the code to obtain the Camera and light(s) information from a given comp using that function.

Good Luck!

[Chris_Roe]

Thanks for the suggestion, I got it working with GetLayerToWorldXformFromView, works just as I thought it would now!

And yea, I'm not limiting myself to scripts... I love programming stuff way too much to stick to one thing, I'm sure you're the same way .

By the way, have you had any success getting the openFrameworks stuff integrated yet? I saw a few of your posts of on the oF forums.

Thanks again guys, great stuff.

-Chris

[gutsblow]

Hello Chris,

Great. I never really got into writing scripts because they are not as

visually exciting as plugins! I dont have much prior programming

experience, so I get stuck up on things easily. But Shachar here in the

forums always solved my problems! I was actually in the forums today to post

a new question and then I saw your post. So here it goes to you and Shachar!

First of all,

Did you successfully get an OpenGL scene working in AE? Can you explain me

how did you get the Window-less offline rendering in Opengl. To be brief,

how can I get gl primitives drawn on my screen without a window?

Secondly, yeah OpenFrameworks. I have been pestering those guys about the

integration too. It has a very easy to use syntax and amazing drawing tools.

But, the problem comes again to one situation, the commands from it's code

work only like an application, and right there is no way to render stuff

without opening a window. You might have a better understanding at this than

me, since you are working in OpenGL.

Thanks again!

[Chris_Roe]

Check out the sample project GLAtor - it grabs the screen, writes it to a texture, then applies the texture to a quad in the scene. It's actually quite easy to figure out from there, just pay attention to your scale and positions, this will be the biggest headhache...

When you get the Matrix ( exactly as you do in the Resizer project ), you have to conver the matrix to colums - you can see how I do this in my first post, easy stuff, then you have to create a 1d array of 16 floats isntead of a 4x4 matrix, also in my first post, then to use that array, you glLoadMatrixf ( THE_MATRIX_YOU_CONVERTED_TO_FLOATS_HERE ); - then if you're still in the GLAtor project, move the camera to 0,0,0 and you *should* see your scene.

Good luck!

[gutsblow]

I,m on OSX 10.6 and GLator doesn't compile. May be I should check on a

Windows Machine. Also does GLator sample work in CS3 for you?

Thank you!

[Chris_Roe]

I'm on 10.6 as well and don't have any issues with it compiling - I haven't tried to run it on CS3, I only have CS4 now to develop it. What kind of errors are you getting?

[gutsblow]

Precompile Carbon.h error

Stdarg.h: No such file or directory

Float.h: No such file or directory

Xmmintrin.h: No such file or directory

After googling, I found that GCC 4.2 doesn't read these files. So I changed

the Precompiler to GCC 4.0 but still no luck.

Thank you.

[Chris_Roe]

I actually got this error the other day on a project out of no where. I have the iphone SDK which includes the base sdk and the iphone sdk... so I thought maybe it was that, but couldn't find any information to say that was the case..

The solution was a fresh install of xCode, don't know why it worked, but it did.

[gutsblow]

Great. That did the trick. I guess I figured out why too. By default Xcode

in 10.6 doesn't install Mac OS X 10.4 support and GLator uses 10.4 API. When

re-installing I checked 10.4 support and then changed compiler from GCC 4.2

to 4.0 and worked like a charm.