action script with condition if the layer is a smart object

Perhaps I'm overthinking this...

A brute force approach would simply cycle through every layer and convert it to a SO if the layer kind supported this feature. If you record the Topaz filter into an action, then the script can run the filter on each SO layer by calling the action from within the script:

var doc = app.activeDocument;
var layerVisble

doc.activeLayer = doc.layers[0];

for (var i = 0; i < doc.layers.length; i++) {
	doc.activeLayer = doc.layers[i];
	layerVisible = doc.activeLayer.visible;
	doc.activeLayer.visible = true;
	main();
	doc.activeLayer.visible = layerVisible;
}

function main() {
	if (app.activeDocument.activeLayer.kind == "LayerKind.NORMAL") {
		app.runMenuItem(stringIDToTypeID('newPlacedLayer'));
		// Change the action set and action as required...
		app.doAction("My Action Name", "My Action Set Name.atn");
	}
}

Change the name of the referenced action and action set as required.

The downside of this simplistic approach is that any flattened Background image layers would be converted to a regular layer. A more complex conditional could check for such files and work around this issue if it was a problem.

As I wrote earlier:

What may be hard to automate is targeting/selecting the layer to filter. If you have varying layer structure, is there anything that is constant, such as layer name or relative position of the layer in the stack (such as the bottom layer).

It is all about identifying and targeting the correct layer, although it sometimes seems that scripts can work magic, they can't.

Once the normal raster target layer is selected, it is easy enough to convert it to a SO layer:

rasterToSO();

function rasterToSO() {
	if (app.activeDocument.activeLayer.kind == "LayerKind.NORMAL") {
		app.runMenuItem(stringIDToTypeID('newPlacedLayer'));
	}
}

I don't think we need to over-complicate the script for conditionally testing if the layer is already a SO, as multiple applications of converting to a SO layer have no effect if the layer is already a SO.

I'm a little confused, a true Background "image" layer by definition can't be a smart object... However, you could have another layer type (raster, shape, adjustment etc) named "Background", however, this would not be a true Background "image" layer.

if (app.activeDocument.activeLayer.kind == "LayerKind.SMARTOBJECT") {
    alert("This is a Smart Object layer");
} else {
    alert("This is not a Smart Object layer");
}

Simply replace the alert lines with whatever you wish to do...