Converting colors increases file size five times

Thank you JR! Advanced Optimization did the trick. Although I can't use it for flattening, because it doesn't allow setting the required blending color space, so I used the Flattener Preview tool for flattening and Advanced Optimization for downsampling.

I moved the flattening step before color conversion, used JPEG compression on color images and managed to get the file down to 501 MB.

Sure. I uploaded it here: https://acrobat.adobe.com/id/urn:aaid:sc:EU:5997c6e8-4f47-4ab9-9e74-0c7bae2fb86d

As the name suggests, this version is right before the color conversion step in the process that I described.

Thank you for the quick response JR Boulay,

I tried using ZIP compression already on the large file in attempts to reduce its size (both using Adobe exporting tools and 7ZIP) and the funny thing is - the file got a tiny bit larger, not smaller. It's probably because I already used ZIP compression in the flattening step of my workflow.

In case it helps I'll elaborate on my workflow. Many settings here have a reason that was learned the hard way through experimentation:

• Export from Scribus (PDF 1.4 compatibility. Fonts - Subset all)
  1. Subsetting fonts is necessary, otherwise Acrobat just doesn't render Oxygen italic and bold. I have tried subsetting only Oxygen Italic and Bold which I seldom use throughout the document and embedding regular fonts without subsetting them, but that didn't have a noticeable effect on the file size.
     
• Open it in Adobe Acrobat
• Set black text to overprint fix:
1. Open Preflight - Acrobat Pro DC 2015 Profiles
2. Fixes (wrench icon)
3. Color spaces, spot colors, inks - Set black text to overprint - Fix 
4. Saving the file at this point produces a file of 374 MB.
• Convert Colors (CGATS21 profile, Relative Colorimetric, Check all Convert Options, Ok)
  1. Saving the file at this point produces a file of 1.99 GB.
• Flatten:
1. Select Print Production - Flattener Preview
2. Page-Level Transparency Blending Color Space - CGATS21-2-CRPC1 for all pages.
   
3. Raster Vector balance set to 100% vector.
4. Line art and text resolution: 600 PPI
   
5. Gradient and mesh resolution: 300 PPI
6. Convert All Text to Outlines - not checked
7. Preserve Overprint - checked
8. Compression settings - ZIP for all
9. Apply to PDF - All pages in document - Apply
   
10. Saving the file at this point produces a file of 1.76 GB.
• Downsample
  1. Here I run a custom Preflight fix under Acrobat Pro DC 2015 Profiles to downsample all color images above 300dpi to 300dpi and then do ZIP compression. 300dpi is the minimum AND maximum required by the printing house.
     
  2. Saving the file at this point produces a file of 1.62 GB.
• Downgrade to X-3 (2002) using the custom Standards profile
1. Open Print Production - Preflight - PDF Standards - Toolbox (profiles) - "Convert to PDF/X-3 (CGATS21-2-CRPC1)" profile - hit "Analyze and fix".
2. I made this profile myself by taking the SWOP X-3 profile and changing its color profile to CGATS21 as Acrobat didn't have one.
3. X-3 standard is required by the printing house I will have to use.
4. Saving the file at this point also produces a file of 1.62 GB.

So neither compressing, embedding fonts nor downsampling every image above 300DPI solves the problem. There are no more quality compromises I can make. There has to be a technical issue here with the color conversion, as other publishers manage to produce 400 page color books and fit within 1GB constraint.