New Text Scripting Hooks starting After Effects Beta 24.0x14

[Sébastien Périer]

Hello everyone,

We’re very happy to announce that starting in today’s build (24.0x14), we have added 42 new scripting hooks for text and font manipulation. Our goal with these new hooks is to match in scripting what is available in the UI.

In addition to extending the TextDocument object, we’ve also added two brand-new objects:

• Fonts Object: The Fonts object provides information about the current font environment on your device. It will allow you to retrieve all the available fonts and iterate them.
• Font Object: The Font object provides information about a specific font and the font technology used, helping disambiguate when multiple fonts sharing the same Postscript name are installed on the system.

You can see the full list right here, along with documentation for the new hooks and objects: https://ae-scripting.docsforadobe.dev/introduction/changelog.html. Remember, these scripting hooks are currently in Beta and are subject to change.

We’ve also attached a demo script panel to this post that exercises all of the new hooks. You will need to rename the file's extension from .jsx.txt to .jsx to run it inside AE.

We're excited to see what you'll be able to do with these new scripting capabilities!

[ariel_n]

Hi Sebastian, thanks a lot for these!, really happy to see RTL in there amont all the rest!

Any plans for adding TextDocument styles and convert from/to paragrah/point text?
There are already many mantion of these in the forums and would be a huge help to avoid a lot of crazy non-reliable workarounds 😉

[Douglas_Waterfall]

> and convert from/to paragrah/point text?

Is this really a common enough workflow?

In what circumstances would you use scripting to edit and convert Text Layers in this way?

Douglas Waterfall

After Effects Engineering

[ariel_n]

Hi Douglas!,

We do that mostly as a workaround for getting the "line height" of the text.
By "line height", I mean, when creating a paragraph text, After Effects places the text's first line baseline some distance from the top of the bounding box,

Computing the same "line height" for point text is simply not possible, and we could never figure out a way to do that reliably.  There is the AVLayer.sourceRectAtTime, but that's not the same, fonts can get crazy sometimes and the pixel perfect bounding box is not the same as the "line height". A similar issue was for computing the line left and right bearings for point text, but I don't exatly remember all the details, it's been a while.

What we used for years now is creating a box text, and trying to copy the contexts of the point text and then querying the baselineLocs, but it does not always work and when dealing with rich text or multiple languages, it's simlply a nightmare.

[Douglas_Waterfall]

Ah.

Would you not rather have explicit support for what you are trying to get, rather than the difficult workaround you've been trying to get to that data?

Is it that you want access to baselineLocs with point text?

[ariel_n]

That's true, getting more information about the font metrics would be a amazing, but the scope would also be far bigger too.

We do have aceess to baselineLocs for a point text though, it's just that the first line is always located at 0.0, and that makes sense for a point text ofc!.

I guess the reasoning behind convert from / to point and parafraph text is also that the functionality is already present in AE, and it is simply not exposed to ExtendScript or being able to do the same with only scripting.

[Douglas_Waterfall]

Still trying to narrow this down. Font metrics are notoriously complicated and obscure and the terms are not well agreed upon.

So you are using the distance from the top of the Box to compute your line height of the first line?

How do you compute the line height of the second line? Do consider baseline shift such as superscript or subscript? Different fonts on the same line? Different point sizes on the same line?

Now that you have your line height, what can you do with it?

There are various requests floating around to enable returning composed state of the text, what metrics would you be looking for? How would this data change when considering Text Animation?

[ariel_n]

"So you are using the distance from the top of the Box to compute your line height of the first line?"

Yes, that's exactly how we compute the line height.

"How do you compute the line height of the second line? Do consider baseline shift such as superscript or subscript? Different fonts on the same line? Different point sizes on the same line?"

We do, we basically go though each character in the line, and try to get the style of it using expressions. we also turn off baseline shifts and other features when possible (not all features are available in both engines). It doesn't always work though, and it gets, really, really slow and complicated.
We also exclude text animators, we compute their transformations separately.

BaselineLocs is also someehat not very usable in some cases, for ex., when using RTL or the bbox is smaller than the text, it all becomes undefined or even when using diffferent justifications. It also includes baseline shift if I remeber correctly and things like bold and italic can make it return different values. It is somewhat the drawing baseline location rather than the text baseline location.

"Now that you have your line height, what can you do with it?"

We mainly export this information and use it in combination with other softwares, but other use cases could include text highlighting, strikethrough, underlining, and alignment with other text layers and shapes.

We have also use the bboxes to add backgrounds shapes.

"There are various requests floating around to enable returning composed state of the text, what metrics would you be looking for? How would this data change when considering Text Animation?"

We never understood how the line height is computed, we extract font metrics in a post process using FreeType, but none of the valuues really matches to what's happening in AE.
Other values we've used are glyph bearings, x and y advances, and individual glyph's bounding boxes.

Not strictly related to font metrics, but bidirectionality information could be useful as well.

We also use harfbuzz and fribidi to get a lot of these metrics.

[Douglas_Waterfall]

Thank you. You are effectively interested in exact data of the composed state of each glyph.

That is very complicated, as you well know.

The composer can decide what it wants in such a way that the renderer data does not necessarily match up 1:1 with the model. Faux italic/bold for example. Or ligatures (of any kind). Styling too.

BTW, you mentioned Expressions - these changes as described by Sebastien only apply to Application level scripting.

[ariel_n]

Thanks a lot Douglas for taking the time and exploring all this.

We absolutly understand that exposing precise matrics is a very complicated topic, but the ability to convert a point to a paragraph text could help a great deal, and that's something AE does naturally and effectively already.
I though exposing that functionality might be a lot easier and it is indeed useful enough to be a feature on its own in AE.

Regarding the expressions, we use them in combination with ExtendScript to get information about the text styles, a kind of weird and very slow workaound and has some other problems, but it get's us closer until we get TextDocument styles support.

It goes something like this:

var copy = layer.duplicate();
for (var i = 0; i < doc.text.length; i++) {
    copy.sourceText.expression = "getStyleAt(" + i + ")";
    // now, we can query the style of the first character.
    var style_at_char = getStyleFromTextDoc(copy.sourceText.value);
    // do something crazy with the style here...
}
copy.remove();

[Paul Tuersley]

I'm very happy to see these new features!

Following on from what ariel_n said, I also do stuff like this for creating masks to create per-line boxes around subtitles. I've tried all kinds of convoluted workarounds over the years. I've tried to get a scripted box/point text conversion working, possibly for similar reasons.

I measure a one and two line height on a test layer and use that info to make some approx calculations. Paragraph text is more work, where I have to keep adding one character after another until the bounds change to figure out where the line wraps are, so I can then also test the width of each of those lines then try to calculate some reasonably accurate bounds.

On the subject of RTL, looks like I have a workaround of adding a line return at the start to force a bounding box to correctly report the width of a single line of RTL text.

I'm currently developing a workaround for measuring the actual character count when ligatures are used,

compared to the reported string length, by reading a newly created range animator's 'End' character index.

And that itself is because I use my workaround method of applying 'style tags' to certain characters by using Stroke (Bold), Skew (Italic) and Color text animators, since we don't have per character access to text styles....and the animator ranges are incorrect if you go by the string count when ligatures are being swapped in.

It wouldn't surprise me if other people jump through hoops like this so they can separate the text styles into different text layers, so need accurate positional data to reconstruct them. But this again is a convoluted workaround to the lack of per character style control.

So I'll add my vote for control over per character text styles (getting per char styles and setting style in a range) and whatever other info about the state of the text positioning we can realistically get.

[jeroenr13836525]

Very exciting! This opens up quite a lot of possibilities!
I noticed the docs saying elements apply only to the first character of a text layer for now quite often. Is there anything you could share around that for now notation already?
But I'm jumping ahead a bit too much now - for now, really excited to start playing with this.

[JohnColombo]

Hi all!

A small update: in response to your feedback, another new read-only attribute has been added in build 24.0x019: TextDocument.composerEngine. This attribute allows you to determine which composer engine was used to create a Text layer, which may prove useful when opening older projects and interacting with those Text layers via scripting. More details are available in the linked documentation.

[Paul Tuersley]

I've been playing with this new functionality the last few hours. Really impressed by how thorough it seems to be. There's lots of new stuff in TextDocument too which will take a long time to digest. I have no idea what half of it even does yet from just the basic descriptions on docsforadobe.dev. FontObject.writingScripts is interesting. Seems like it could be another way to help a user narrow down relevant fonts. How many fonts do support Klingon I wonder!

I've got a basic font family dropdown going, that then populates a style dropdown onChange. I guess if our own dropdowns have the full font family list we can query the allFonts entry for styles by the selection.index, otherwise we can loop through allFonts to find the family to then query the styles, which seems fast enough. And likewise use getFontsByFamilyNameAndStyleName to retrieve the postScriptName from the pulldown selection.text to apply to a textDoc.font, and getFontsByPostScriptName to turn textDoc.font back into the UI style family and style naming.

Is there any possibility of tapping into a Character panel font favorites flag (or method for getting/setting it) for font family? I always figured I'd need a favorites manager of my own, but requiring a user to create another set of favorites if they already have them in the Character panel isn't ideal. If the favorites flag could be read/write then even better as we could offer a manager that could affect the Character panel font list too. You could even do things like let users create multiple different favorite profiles for different jobs.

[Douglas_Waterfall]

Hi Paul:

I implemented scripting access to both favorites and the mru for my own testing, but it remains internal at this point. Let me think about exposing that.

Your strategy for building your font dropdown is...flawed. The reason of course is that PostScript names or Family/Style names are not guaranteed to be unique. So all this translation is going to fail in mysterious ways.

You might instead simply keep around a reference to the Font Objects which you get from app.fonts.allFonts in our dropdown. Then when you pick you'll know which font it was without any translation.

The new api at TextDocument.fontObject returns an actual Font Object, rather than simply the PostScript name which is returned from TextDocument.font. You might give that a try to see if that works better for you.

One thing I suspect you have not yet realized is what happens when fonts are added or removed from the font environment. A FontObject does not have a strong enough reference to keep a font from disappearing and consequently any Font Object can become invalid at any time...which I have found annoying as well while doing automated testing around font environment changes.

To fully do what you want you'll need even more properties exposed.

Douglas Waterfall

After Effects Engineering

[Paul Tuersley]

Thanks for all that info. Been playing with this some more. I can see how sticking with textDocument.fontObject over textDocument.font (which pretty much becomes obsolete?) keeps things more certain. Having duplicate fonts feels something of an edge case though, as does the specific case where a user uninstalls a font while AE is running.

What I've struggled with is resolving textDocument.fontObject back to the entry/index in allFonts. For example if I'm reading a text layer's font, I need to be able to autoselect it in my font menu. Currently I'm looping through allFonts looking for an initial familyName match, then comparing each allFonts[x][y] object to the textDocument.fontObject converted using JSON.stringify.

If a font is not installed but found in a project, a new isSubstitute:true entry is added at end of allFonts and to missingOrSubstitutedFonts. Seems pretty straightforward.

If a font was installed but has been removed while session is running, I'm still seeing the original entry in allFonts, plus a new one at the end which is what is assigned to TextDocument.fontObject. Strangely I'm getting 2 identical instances in missingOrSubstitutedFonts, both of which match the original entry in allFonts. It seems if I made a reference to the original but now missing allFonts[x][y] fontObject that reference becomes invalid, in a way I can only detect by querying an attribute in a try/catch.

A newly added font during a session will automatically insert a new fontObject into allFonts at the expected position. I didn't actually know that was a thing you could do in AE these days.

So I'm still trying to wrap my head around exactly the best way to keep track of some list of fonts. Keeping a list of allFonts indices is always a bad idea if new fonts can be added at any time. An array of references to objects in allFonts has potential for those references to become invalid. I suspect JSON would be the way to go for storing a list in prefs at least. And then if allFonts.length ever changes I probably need some routine for re-evaluating the state of the font list. Just going with postScriptName and an ignorance is bliss attitude seems quite tempting still.

Is the docsforadobe description for TextDocument.fontObject correct?

"The text layer’s Font object specified by its PostScript name".

Not sure how it relates to postScriptName as it's a FontObject surely?

Hope you can expose favorites! What is mru? Something cool and interesting no doubt.

[Douglas_Waterfall]

Thanks for all that info. Been playing with this some more. I can see how sticking with textDocument.fontObject over textDocument.font (which pretty much becomes obsolete?) keeps things more certain. Having duplicate fonts feels something of an edge case though, as does the specific case where a user uninstalls a font while AE is running.

You are not the first to declare edge cases uninteresting...but that leads to disaster as it is actually quite common and any architecture built upon simplistic assumptions will break absolutely once reality sets in. That may sound a bit overstated, but the whole point of my rework of the FontServer was exactly that - viewed across all our customers it happened all the time.

A super easy example of duplicates is this - you open a project, we have to create a substitute font, you then install a proposed matching font...if it happens to be an exact match then boom - there you have a duplicate.

There were some very hard problems to work though here as issues around duplicates were ignored or papered over for many years and in particular, just think of what it means to be dynamically adding (or removing) fonts while the App is running.

Before Adobe Fonts came along, this was probably more rare, and "good practice" was probably to stick with the fonts you had and only change them when the App was off. Now with Adobe Fonts, and other places in the web to find fonts, it is super easy to change the running font environment at will.

What I've struggled with is resolving textDocument.fontObject back to the entry/index in allFonts. For example if I'm reading a text layer's font, I need to be able to autoselect it in my font menu. Currently I'm looping through allFonts looking for an initial familyName match, then comparing each allFonts[x][y] object to the textDocument.fontObject converted using JSON.stringify.

If a font is not installed but found in a project, a new isSubstitute:true entry is added at end of allFonts and to missingOrSubstitutedFonts. Seems pretty straightforward.

The allFonts list is a copy of my internal state, which is subject to change at any time. If you change the sort order in the Char Panel with "Show English Font Names" you're going to get both different order but also different Family/Style names as well.

I have an ID attached to each internal Font object, so I can compare them quite easily. That is not yet exposed to you.

The closes you could do at this point would be to use the recently revealed hasSameDict to compare two Font objects - that would be an exact equal comparison...except in the case of Font objects which are from Virtual Fonts in which case you would also have to check the design vectors.

If a font was installed but has been removed while session is running, I'm still seeing the original entry in allFonts, plus a new one at the end which is what is assigned to TextDocument.fontObject. Strangely I'm getting 2 identical instances in missingOrSubstitutedFonts, both of which match the original entry in allFonts. It seems if I made a reference to the original but now missing allFonts[x][y] fontObject that reference becomes invalid, in a way I can only detect by querying an attribute in a try/catch.

Not sure what you mean by "still see" in allFonts, but the newly created substitute would be expected behavior.

There remain some vulnerable transitions while removing and installing fonts while it the App is running and you are seeing some of those workaround, albeit not perfect ones.

I would rather not get into deep conversations about what those are as they are expected to change over the next couple of releases as changes in other code are put in place which will allow me to simplify things and make my code more correct and predictable.

Your last note is correct though - the Scripting Text object is just a proxy to my internal text object, and once all the references/usages of it are released internally, I delete it. This can leave your Scripting Text object becoming effectively invalid - it will throw an exception on any access of its properties/methods.

I do not particularly like that behavior, but the alternative to allow the Scripting system to have a hard reference on my internal text object was considered undesireable.

Let me say that I am interested in following along with your explorations and discoveries as it represents an attempt to establish a new workflow with the Scripting Font objects - did we provide enough support? for which workflows? where does it fall down? do we care enough about those cases? Dunno yet.

A newly added font during a session will automatically insert a new fontObject into allFonts at the expected position. I didn't actually know that was a thing you could do in AE these days.

Why does the surprise you? Remember, you are getting a copy of internal data. The internal data had been resorted after the insert. That is what you are seeing.

So I'm still trying to wrap my head around exactly the best way to keep track of some list of fonts. Keeping a list of allFonts indices is always a bad idea if new fonts can be added at any time. An array of references to objects in allFonts has potential for those references to become invalid. I suspect JSON would be the way to go for storing a list in prefs at least. And then if allFonts.length ever changes I probably need some routine for re-evaluating the state of the font list. Just going with postScriptName and an ignorance is bliss attitude seems quite tempting still.

Yea, do not use indexes. The native code I inherited was using that to map the UI dropdown tables to the internal font list which created terrible race conditions (and crashes!) trying to keep the UI in sync after the font environment and list changed.

Until you get the same ID that I have internally, your task is made more difficult.

However, I think there is still a lot you could do, as long as you do not mind spending a few cycles.

Use this in your comparison/key:

• PostScript name
• Family name
• Style name
• hasDesignVectors / + designVectors
• technology
• writingScript[0]
• isSubstitute

That will get you a pretty long way.

Is the docsforadobe description for TextDocument.fontObject correct?

"The text layer’s Font object specified by its PostScript name".

Not sure how it relates to postScriptName as it's a FontObject surely?

That does seem rather awkwardly written, now that you point that out. Let me go clean that up.

Probably a text carry over from "font".

Hope you can expose favorites! What is mru? Something cool and interesting no doubt.

mru => Most Recently Used.

It is a the top of the dropdown, it lists by Family Name (but not the style) the last # picks.

Douglas Waterfall

After Effects Engineering

[Paul Tuersley]

If a font was installed but has been removed while session is running, I'm still seeing the original entry in allFonts, plus a new one at the end which is what is assigned to TextDocument.fontObject. Strangely I'm getting 2 identical instances in missingOrSubstitutedFonts, both of which match the original entry in allFonts. It seems if I made a reference to the original but now missing allFonts[x][y] fontObject that reference becomes invalid, in a way I can only detect by querying an attribute in a try/catch.

Not sure what you mean by "still see" in allFonts, but the newly created substitute would be expected behavior.

I meant that the original but now missing entry is still in the same place in allFonts array order, but then another substitute version has been added to the end of the allFonts list. When I examine with stringify, the newly added one has an altered family name (and other stuff) compared to the existing allFonts entry. The original one exactly matches the info for that font in the missingOrSubstitutedFonts list (i.e. it's the same object), while the newly added one is a different object and is the one now returned from TextDocument.fontObject.

While it's easy to check if a font isSubstitute, to check if any allFonts entry is newly missing you'd have to do a comparison to all the entries in missingOrSubstitutedFonts.

Not saying there's anything wrong with this. It's just stuff I need to be aware of to keep on top of the relationship between what was one instance a font has now become two instances of that same font but in different missing states.

A newly added font during a session will automatically insert a new fontObject into allFonts at the expected position. I didn't actually know that was a thing you could do in AE these days.

Why does the surprise you? Remember, you are getting a copy of internal data. The internal data had been resorted after the insert. That is what you are seeing.

It didn't surprise me. It's exactly what I expected. I was just listing the ways I was aware of that the allFonts list could change, from my tests so far. Good to know allFonts can change even without the length changing with something like "Show English Font Names".

Thanks again. I'll keep going with my testing and posting my thoughts when I get the chance.

[Douglas_Waterfall]

Something for you, @Paul Tuersley 

https://community.adobe.com/t5/after-effects-beta-discussions/more-scripting-hooks/m-p/14083457#M308...

[Paul Tuersley]

@Douglas_Waterfall Thanks for the heads up. I've been implementing font lookups using fontID. I have one array of stored IDs in the order of my family / style pulldowns so I can lookup the ID from the dropdown indices, then another by sorted fontIDs where I can lookup the indices of my family / style dropdown entries to set them to match a selected text layer. Maybe overkill. I'll have to see how expensive it is in comparison to keep trawling through 1000+ array entries looking for a fontID match.

I haven't seen app.fonts.allFonts[x][y] returning a different order when I switch 'Show Font Names In English'. But I do see generally that fonts don't all return in the order of the font IDs, with some at the end being from lower value fontIDs. But knowing if anything in allFonts[x] can ever change the order beyond things added to the end (and how to trigger it for testing) would affect whether any UI refresh needs to be total or just an addendum.

I checked to see if the fontIDs would be contiguous but that doesn't seem to be the case. While my last fontID was 1063, when I then removed a font the new missing version of it appeared as fontID 1068. That's fine. Just an observation and something for me to be aware of with my array of sorted fontIDs.

I'm thinking if app.fonts.allFonts.length ever changes then that is one definite cue to refresh my dropdowns. And that I could expect new fontIDs in that case but they would only ever be higher in number than my previously highest recorded fontID? Perhaps a user could install an additional style into an existing font family and that wouldn't affect the overall allFonts.length but would add a new higher ID style inside that existing allFonts[x] entry?

I tried adding a font through Creative Cloud but it didn't seem to show up in AE until I restarted. Don't know if that's expected or I did something wrong? Removing a font through CC did work as expected though, triggering a new [missing entry] in allFonts.

Is there any likelihood favourites will be exposed with read/write access? I find the standard Adobe font panel not especially nice to use. Hard to see scrollbar (too dark), limited vertical height, no way to add/remove favourites in bulk, or search and favourite at same time. Being able to create a better favourites editor with switchable profiles seems like something people would appreciate, more so if it synced with the Character panel.

[Douglas_Waterfall]

I haven't seen app.fonts.allFonts[x][y] returning a different order when I switch 'Show Font Names In English'.

This all depdends on the fonts you have on your system. With Show Font Names In English we ask for the ASCII apis (familyName, styleName) but with it reversed we ask for the Native versions (nativeFamilyname, nativeStyleName). If all your fonts return the same value then the sort order would not change. You can write a simple script to iterate all your fonts and see which ones return different answers for the native vs ASCII apis.

Calling app.fonts.fontServerRevision is the only reliable way to know that the font environment has changed in any way. Looking at the length of app.fonts.allFonts is not reliable.

The fontID is contiguous when they were handed out, but that order is not the same as the sort order. Removals of font instances will leave gaps behind the most recent one handed out. These might as simple as a dynamically created variable font instance which was created and then released. Treat them as a kind of key - anything else is going to cause you grief.

I tried adding a font through Creative Cloud but it didn't seem to show up in AE until I restarted.

No, it should be there pretty quickly. The fontServerRevision will change. When we open up a project with missing fonts from AdobeFonts they get activated automatically and this very much works. Not sure what you are seeing when you say it requires a restart to see.

Favorites is on my list, but I was only able to get fontID out for you on this drop - it was going to be too involved to do it at the same time. The next drop is going to be something more widely usable and more work for us to document so that has pushed back the Favorites reveal a month or two.

Thank you for trying this out and reporting back.

Douglas Waterfall

After Effects Engineering

[Davide Boscolo]

Hi there,
Is there a way to know if a character is selected so we can run a script on that selected portion of a text layer?

 

[Douglas_Waterfall]

Nope, not at this time.

Good idea, though.

Douglas Waterfall

After Effects Engineering

[Davide Boscolo]

Thanks for replying!
If you can add it, it would be wonderful. Have a think about it.

Cheers