Hi Tommes — there's no per-object "exclude from downsampling" switch in InDesign; the Compression settings in the Export Adobe PDF dialog apply to the whole document.
But there's a clean workaround that does exactly what you want: InDesign only downsamples images whose effective resolution is above the "for images above [X] ppi" threshold. So check your special image's effective PPI in the Links panel, then in Export Adobe PDF > Compression set the threshold above that value — e.g. "Bicubic Downsampling to 300 ppi for images above 450 ppi". Your special image then keeps its full resolution while everything else gets downsampled.
One caveat: the JPEG compression quality setting is still document-wide, so the image will still be JPEG-compressed (just not downsampled). If you need it truly untouched, you'd have to switch the Compression to ZIP (lossless) for all images — at the cost of file size.
Reference: https://helpx.adobe.com/indesign/desktop/save-export-and-publish/save-and-export/adobe-pdf-export-options.html (Compression section)