How do I have Action Wizard ignore non-PDFs in selected folder?

I'm trying to run OCR, scan optimization, and file size reduction on documents in several folders using Pro XI. When I select a folder, it automatically includes spreadsheets and word documents that I do not wish to convert into PDFs. I have tried selecting them anyways then removing them, but then I need to remove each document individually. The other option I've found is to add files individually. Both of those work arounds are time consuming and seem counter-intuitive on batch operations.