OpenCV and AfterEffects

Question

Hello,I am looking for some guidance for integrating OpenCV with the AE SDK.Notes:I've read through most of the (excellent) documentation and successfully experimented with all of the sample projects.I've successfully built AE SDK linked with OpenCV4.1 dll's and can import and instantiate the basic OpenCV building blocks (eg. cv::Mat).I assume some of the native AE functionality uses OpenCV since I can see some opencv dll's in the AE binary folder. Also, plug-ins like 3D Camera Tracker likely use feature detection and tracking from OpenCV.Where I'm stuck is copying input data to a cv::Mat object so that I can leverage the OpenCV library for computer vision.After trying for a few days I'm quite stuck with memory allocation errors etc. and it is quite clear from the AE SDK docs that allocating your own memory without using the AE SDK macros is a recipe for disaster (fair enough).I'm wondering if there is any hope for trying to accomplish the following (copied from the Render() function of the AE SDK Skeleton example):static PF_Err Render (PF_InData *in_data,PF_OutData *out_data,PF_ParamDef *params[],PF_LayerDef *output ){PF_Err err = PF_Err_NONE;AEGP_SuiteHandler suites(in_data->pica_basicP);// get input frame pixels from in_dataPF_EffectWorld *inputP = &params[0]->u.ld;// initialize 4-channel 8-bit Matcv::Mat mat = cv::Mat((int)output->height, (int)output->width, CV_8UC4);// copy pixels to opencv mat err = copyInput(&mat, inputP);// do some computer vision stufferr = doAwesomeComputerVision(&mat);// copy opencv mat to outputerr = copyToOutput(&mat, output); return err;}Any guidance is greatly appreciated.Thank you!

Anonymous · Answer

You are almost done then, you just have to implement the copyInput() and copyToOutput() functions in your examples .

No, but seriously, this shouldn't be too much work.

You are using the old Render() framework, in future you should switch to SmartRender to support other bit depths and get better optimisation/performance, but this will do for now for testing. When using Render(), you will get the pixel data as 8bpc uncompressed values in an interleaved way with the channel order being BGRA. There will also be some unneeded data (for you) at the end of each row, so be sure to take the rowbytes value in account which will give you the amount of actual bytes per row, so you know how much to skip over.

Apart from that, it is pretty simple and you can use a function like this to get a pixel:

inline PF_Pixel* getPixel(PF_EffectWorld* inputP, const A_long x, const A_long y)

{ return (PF_Pixel*)((char*)inputP->data + (y * inputP->rowbytes) + x * sizeof(PF_Pixel)); };

The simply copy the pixel values channel by channel to your matrix.

Setting a pixel in the output follows the same principle, read from your matrix, copy it to the output buffer.

Warning: this will be very slow if you do it in a linear pixel-by-pixel fashion, so again, just use this for testing. For actual implementation, you should then use a multithreaded pixel copying process, either by using your own thread functions or via the provided AE iterate suites. These will allow you to process a bunch of lines per thread. Check the examples in the SDK, e.g. SDK Noise for that (or the forum here).

Notes:

Anmelden

Login über soziales Netzwerk

Willkommen

Login über soziales Netzwerk

Scanne Datei nach Viren

Diese Datei kann nicht heruntergeladen werden