It depends upon whether you record in Multitrack or Waveform view. If you record in Waveform view and don't save the file, then it remains as a temp file, and will disappear completely when you close Audition. If you record in Multitrack, then it's direct to file, and whatever you record will remain - you had to name it before you recorded it, that that stays - which is a good thing as far as most people are concerned! If you try to import a file at the wrong sample rate, then Audition will automatically create a rate-converted version of it as a standalone file, but this won't be saved unless you take specific steps to save it, and you will get a warning/reminder about this when you close the session. If you comp vocals in Audition, it will (unfortunately) save absolutely every take until you delete them, but that's quite a specific use case that I guess you're unlikely to run into if you're using it with Premiere.
Premiere is different from this POV though, as you get into the whole business of rendering, which is different from the way Audition works in this regard, and will inevitably store more junk.
As far as backup storage is concerned, the only automatic backups in Audition are its .sesx files, which generally aren't that large. When you save a project though, you will be asked whether you wish to store copies of the audio in the same location as the .sesx file - and it's often a good idea to do that. Not always though, and you do get the choice. In general, Audition is a lot more transparent about what's backed up than Premiere is, I think. OTOH that may be because I'm rather more familiar with Audition!