Create a color space for a one-of-kind printer?

I am working with a large sporting goods manufacturer that is trying to automate a complex process. One of the challenges we are facing would require us to define a unique Color Space specific to a non-standard printer with a very limited color range. The goal is to map unprintable colors to a table of colors within this printer's gamut. The printer is one-of-a-kind, and no predefined color space or gamut exists.

Since it is clear that printer profiles can be created because they are a part of most CC apps, we would like to create our own. However, we cannot find any public documentation on this process.