Understanding Tolerance Calculation in Adobe Illustrator's Magic Wand Tool

[Ghulam Mohyuddin]

I understand that to select objects based on their fill color using Adobe Illustrator's Magic Wand tool, I can choose the Fill Color option and set a Tolerance value between 0 and 255  for RGB or 0 and 100  for CMYK. Lower tolerance values select objects very similar to the clicked object, while higher tolerance values select objects with a broader range of the selected property.

However, I would like to know the exact process Illustrator uses to calculate and apply tolerance for selecting objects based on fill color in both RGB and CMYK modes. For example, if I have an object with RGB values of (60, 90, 120) and set the tolerance to 20, what range of colors will be selected for each channel? Additionally, could you explain the process for a CMYK object as well? Thank you.

[Monika Gause]

I assume the fastest procedure would be that you find out what works for your artwork by experiment.

[Kurt Gold]

As for CMYK process colours the rule is what one may call a "common responsibility".

That is, to be considered and selected as objects with the "same" colour, none of the four inks (relative to the reference colour) are allowed to exceed or fall below the percentaged tolerance setting in the Magic Wand palette. Otherwise the Magic Wand won't select them.

Probably that's valid for RGB colours as well, but the calculation is obviously a bit more complicated.

[Ghulam Mohyuddin]

I understand your point, but I would like to know the precise method by which the Magic Wand tolerance is calculated.

[Doug A Roberts]

See if this thread helps:

https://www.photoshopgurus.com/forum/threads/tolerance.52555/post-1533711906

[Ghulam Mohyuddin]

"I have reviewed your reference material related to raster images. I am now seeking clarification on how the tolerance setting operates within Adobe Illustrator. Could you provide a detailed explanation of its function in Adobe Illustrator?"

[Doug A Roberts]

I don't see why it would operate differently.

Edit: It does appear different, strangely.

If I create a 255/0/0 to 0/0/0 gradient in Photoshop and select with a tolerance of 10, I get a selection from 255 to 245.

If I create a 255/0/0 to 0/0/0 object blend in Illustrator and select with a tolerance of 10, I get a selection from 255 to 230:

[Ghulam Mohyuddin]

"I believe the information regarding how tolerance is calculated in Adobe Illustrator should be accessible to users, as it is crucial for effectively using the tool. Specifically, I am interested in understanding how tolerance is applied in the Magic Wand tool. For instance, if we set a tolerance level of 20 in Adobe Illustrator for a CMYK color with values C=70, M=47, Y=55, K=60, does the tolerance apply as a ±20 range for each channel (e.g., C=50-90, M=27-67, Y=35-75, K=40-80)? Or is it based on the overall color difference between the sample color and the object you want to select? Your clarification on this would be greatly appreciated."

[Monika Gause]

---

@Ghulam Mohyuddin  schrieb:

"I have reviewed your reference material related to raster images. I am now seeking clarification on how the tolerance setting operates within Adobe Illustrator. Could you provide a detailed explanation of its function in Adobe Illustrator?"

---

In this forum 95% of participants are users just like you. But even if they were engineers it's not very likely that they would give you that information. 

[Ghulam Mohyuddin]

"I believe the information regarding how tolerance is calculated in Adobe Illustrator should be accessible to users, as it is crucial for effectively using the tool. Specifically, I am interested in understanding how tolerance is applied in the Magic Wand tool. For instance, if we set a tolerance level of 20 in Adobe Illustrator for a CMYK color with values C=70, M=47, Y=55, K=60, does the tolerance apply as a ±20 range for each channel (e.g., C=50-90, M=27-67, Y=35-75, K=40-80)? Or is it based on the overall color difference between the sample color and the object you want to select? Your clarification on this would be greatly appreciated."

[Kurt Gold]

For instance, if we set a tolerance level of 20 in Adobe Illustrator for a CMYK color with values C=70, M=47, Y=55, K=60, does the tolerance apply as a ±20 range for each channel (e.g., C=50-90, M=27-67, Y=35-75, K=40-80)?

--------------------------------------------------------------------------------------------------------------------------------

Yes, exactly. That's the basic rule about CMYK colours I was trying to explain in my rather generalised statement above.

It gets a bit different when there are 0 % inks. Basically, the 0 % inks do not count, unless you change at leat one of the non-zero % inks. Then the zero % inks kind of may get some bonus points.

A long time ago I created a comparative table that should explain the issue in detail. Will have to look at one of my archives and post it in case I can locate it.