That is correct and the issue comes from mixing workflows.
In a Diffuse/Specular/Glossiness workflow, metal areas are black in the Diffuse map and any metal reflectance colour comes from the Specular map.
In a Basecolor/Metal/Roughness workflow, metal area reflectance colours are in the Basecolor map and metal areas are white in the Metalness map.
Work with either Diffuse/Specular/Gloss or Basecolor/Metal/Roughness. If you mix Diffuse with Metalness you lose the metal reflectance colour.
This guide explains the two workflows further :
https://substance3d.adobe.com/tutorials/courses/the-pbr-guide-part-2
Dave