Translucent square appears with captions in Adobe Media Encoder, but not Premiere Pro

I exported a project yesterday, and when watching the exported file, discovered that a little translucent grey square appears whenever text appears.  It doesn't matter where the captions are located; the square always appears in the same area.

The strange thing is that the square is only visible if I export with Adobe Media Encoder (25.3).  It is not visible if I export in Premiere Pro.  And it's not visible in Premiere Pro's preview window either.

I'm pretty sure it's an empty text object with a translucent grey background.  Although Premiere Pro doesn't show it in the preview window or its exports, when I switch to the Text tool and mouse over that area, the cursor changes, so I can select and delete it.  However, I would have to delete it for *all* of my text captions, which is a pain.

So my questions are:

1) Why would Adobe Media Encoder's exports differ from that of Premiere Pro?  I thought the whole point of Adobe Media Encoder was to provide an alternative rendering interface to Premiere Pro, but that the renderings would be identical.  Does this mean I cannot trust Adobe Media Encoder to always export exactly what Premiere Pro would export?

2) Have other people reported this problem, and is a fix confirmed for a future version of Adobe Media Encoder?

3) In the meantime, assuming it's an empty text object, how do I locate and delete it for *all* the captions in my project?  I'd hate to have to pixel-hunt for every one.  Is there a way to at least make them all visible, so that I can quickly see (in Premiere Pro itself) which ones I still need to delete?

Thanks!