The primary determinant of PDF file size is the degree to which you maintain resolution and compression quality in raster images in the PDF file when performing export. You have already downsampled the imagery to the point at which they will look pretty cruddy on many if not most modern computer monitors as well as mobile devices (typical high quality phones and tablets are upwards of 300 dpi, not 72). The only thing you can do further in the so-called “optimization” process is to change the compression to something lower in quality such as JPEG or JPEG low quality.
One other possibility is that you have exceptionally-complex vector diagrams (many thousands of vectors, for example). Converting those to raster images could significantly cut the size and also the quality of your PDF file.
Note that unless a font is tagged as non-embeddable, InDesign subset embeds any and all fonts used in the InDesign document. In general, this contributes relatively little to the size of the PDF file (unless you are producing font glyph charts)!
Bottom line is that you can't legislate the size of a PDF file. The size of a PDF file ultimately comes down to how much content you have in the original document and to what degree you are willing to degrade the quality of that content, especially raster images, in the resultant PDF file. And once you have degraded (ruined) the quality of the PDF, there really isn't much you can do!
Sorry!
- Dov