delete silence durations too long

I don't know if this is an issue or that I'm doing something wrong, but I'm editing a podcast and I want to strip away the long pauses.

Using the delete silence in diagnostics tool gives me really weird selections of "silence". Most clips selected are durations around 30 seconds.. Using the Find levels aswell as entering manually a db guideline will give me same results..

Anyone can tell me what am I doing wrong?