Questions about After Effect extensions development

[Thomas Fugier]

Good morning.

I am currently thinking about a new project that I may lead in my company. This project involves a potential homemade AE extension. But before digging myself too much into actual development, I would like to ask if you think the followings problems are actually doable :

• Reading pixels values from a video file and then having some algorithm that would place automatically sound files from predefined paths into the timeline. (For instance, blue pixel at X/Y position means having soundclip1.mp3 at specific time)

• Reading a specific text file with json in it (Or just different text lines with a specific convention that we would create). The file would contains informations like "soundclip1 at time 3 seconds" and then, same as with the first idea, the extension would automatically load the correct sound file and then put it accordingly in the timeline.
• Reading metadatas from a video file (And the metadatas would in fact contains the same text as with the text file idea)

Sorry for asking you such strange thing, have a nice day !

[Mylenium]

Metadata is a biiitch and I'd stay away from it unless you have full control over it yourself. You never know how other programs, media converters, editing suites and whatnot modify it or screw it up. Even just an inexperienced user using a wrong export preset somewhere could prevent the metadata you may need from being generated/ embedded. Unless you have a database system where you could verify this info like an Avid News Server or a similarly managed storage system, this could end up being a nightmare and you still have to sift through all the video files manually. Encoding file info to pixels sounds good on paper, but you'd have to have a system for encoding the pixels ready. It needs to be 100% foolproof and future-safe, so the coloring of the pixels itself would likely have to depend on some algorithm calculating it. If you get my drift: It may sound cool now to use red for sound A and blue for sound B, but what happens a few years down the line when you may have a huge audio and video library with many different clips? Also you'd likely have to use pixel triplets or something like that to encode a specific hash value because a single pixel might not be exact enough and/ or be affected by color shifts and encoding artifacts in the video. That's also something to consider. With all that being said, JSON is probably the easiest to handle. Everyone has some form of text editor somewhere or you could even create a web site/ intranet page for generating the files, which would also have the benefit that you can control the text files and enforce the syntax. Maybe I'm overcomplicating things, but maybe my thoughts are useful to consider your strategy.

Mylenium

[Mathias Moehl]

Technically, the answer to all questions should be yes.

- Reading pixel values: not possible directly via scripting, but you could create an expression on some dummy property that reads a pixel value (sampleImage expression).

- reading text files see

https://extendscript.docsforadobe.dev/file-system-access/index.html

-parsing JSON see

https://aescripts.com/learn/After-Effects-Scripting-JSON-Tutorial/

- xmp metadata see

https://extendscript.docsforadobe.dev/scripting-xmp/index.html

By the way, I am currently developing an extension called Automation Blocks, which allows to write these kinds of extensions by using blocks instead of programming code. It is still in private beta testing phase, but if you want to try to implement your project with this, I can give you access to the beta. My experience is that you can impement much quicker using this system. If you are interested, please send me an email to info@mamoworld.com. Here is an example, how these block programs look like: