AIAPI SDK AIMaskFlattener - FlattenArt, Am I doing it right?

[iLLMonkey]

Hi,

I'm trying to flatten selected art. The FlattenDocumentToLayer() works fine so the Suite seem to be linked correctly. But can't get the FlattenArt() to process. Am I doing something obviously wrong here? When I run this, nothing happens.

Doc Reference:

AIErr(* FlattenArt )(AIArtSet artSet, AIFlatteningOptions *options, ai::int16 paintOrder, AIArtHandle prep)

	AIFlatteningOptions myFlatteningOptions;
	myFlatteningOptions.balance = ai::uint16{ 100 };
	myFlatteningOptions.flags = ( kAIFlattenOutlineText | kAIFlattenOutlineStrokes );
	myFlatteningOptions.rasterResolution = AIReal{ 300 };
	myFlatteningOptions.meshResolution = AIReal{ 150 };
	myFlatteningOptions.progress.start = AIReal{ 0.0 };
	myFlatteningOptions.progress.end = AIReal{ 1.0 };

	AIArtSet artToProcess = NULL;
	sAIArtSet->NewArtSet(&artToProcess);
	sAIArtSet->SelectedArtSet(artToProcess);

	AIArtHandle prepArt = 0;

	sAIMaskFlattener->FlattenArt(artToProcess, &myFlatteningOptions, kPlaceDefault, prepArt);

Thanks!

[A. Patterson · Accepted solution]

I've never used that function, but my guess is that you can't specify nullptr for the prepArt parameter. Usually it wants some combination like 'kPlaceInsideOnTop' and a layer's group handle or 'kPlaceAbove' and a piece of art. It's possible it doesn't need it -- not super clear from the function description. I've never used `kPlaceDefault` so maybe that works fine with a null prepArt.

You should capture the error value -- it can include helpful information. I won't look like it, but the value can often be translated into a useful message.

[iLLMonkey]

Thanks, After reading about it, according to the manual and included SDK examples, kPlaceAboveAll should work with a NULL. It throws an error anyway. This is how I determined an error was thrown:

AIErr error = sAIMaskFlattener->FlattenArt(artToProcess, &myFlatteningOptions, kPlaceAboveAll, NULL);
if (error) {
	sAIUser->MessageAlert(ai::UnicodeString("error"));
}
else {
	sAIUser->MessageAlert(ai::UnicodeString("not error"));
}

I don't know how to read the contents of the error. This is my first attempt at C++/SDK, I've come this "far" just by scouting sample-code (with AstroGrep) and comparing it to the AISDK API reference manual. I've used CountArtSet to make sure the ArtSet it populated. I've also tried not to specify any settings in myFlatteningOption and just leave declared but at Default.

[iLLMonkey]

Got it to work! You're right. It wouldn't take NULL for an answer (heh) Once I used an actual art-object thingy inside the handler it worked even though the manual said kPlaceInsideOnTop wouldn't need the prepArt.

[A. Patterson]

Sorry, been a busy weekend.

kPlaceInsideOnTop requires a prepArt because otherwise you'd have to ask "Inside what?" 😁 kPlaceAboveAll should work with a null prepArt though.

As for reading the error code, not every function returns a useful value. If you see a number that seems crazy big, that is probably one with information though. You take the number and  you can convert it into hex (Windows 10+ calculator can do this easily). You should end up with an 8-digit hext, or four pairs of 2-digit hex values. For each pair, look them up on an ASCII table (just Google it). They should spell out a 4-letter error code. I suspect you'd have gotten 'PARM' in this case, which is the usual error when there's something wrong with what you're feeding an SDK function.

[iLLMonkey]

Thanks!, That's some really useful information. And you're right again, the kPlaceAboveAll and kPlaceBelowAll are the onces that should work with a NULL prepArt. Thanks again, the code is working and I've learned alot.