• "Filtering layers" isn't recorded in an "action"?

Not so sure this would work anyway. The layer name is what is recorded in the action, not layer type or any other attribute.

If names differ, the action just stops. The only relative layer selection allowed is counting up or down from the active layer.

That you can do it manually is no guarantee it will work in an action. You can think; the action doesn't.

I wonder if there is a way to change the AM code below, perhaps the “undefined” bit?

var idlayersFiltered = stringIDToTypeID( "layersFiltered" );
executeAction( idlayersFiltered, undefined, DialogModes.NO );

Just like an action, the key bit is not recorded...

If your Smart Object layers are always in the same count and same naming convention or Smart Object layers are ordered one above another in the Layers panel then it can be done, otherwise if you want to use action in any document just to select all smart object layers regardless of their count, position, naming convention it cannot be done (recorded as action step) using filtering options in the Layers panel.

You can write a script to select Smart Object layers, and call that from an action.