Strange static / hissing sound in audio after I export

I am running a podcast with only two vocal tracks. When i mixdown and export to WAV or MP3 format, the audio has a loud hissing sound on many of the silent parts of the audio. The hissing is always the exact same every time. Am i doing something wrong when exporting? Currently I had to go through the entire podcast and silence those parts which took a long time .