Known Participant

Question

Understanding Tolerance Calculation in Adobe Illustrator's Magic Wand Tool

Forum|Forum|1 year ago
July 17, 2024
5 replies
5624 views

I understand that to select objects based on their fill color using Adobe Illustrator's Magic Wand tool, I can choose the Fill Color option and set a Tolerance value between 0 and 255 for RGB or 0 and 100 for CMYK. Lower tolerance values select objects very similar to the clicked object, while higher tolerance values select objects with a broader range of the selected property.

However, I would like to know the exact process Illustrator uses to calculate and apply tolerance for selecting objects based on fill color in both RGB and CMYK modes. For example, if I have an object with RGB values of (60, 90, 120) and set the tolerance to 20, what range of colors will be selected for each channel? Additionally, could you explain the process for a CMYK object as well? Thank you.

This topic has been closed for replies.

Ghulam MohyuddinAuthor

Known Participant

I have previously asked several questions about how tolerance functions in Adobe Illustrator's Magic Wand Tool, but I have not received a satisfactory explanation. Could someone please provide a detailed explanation of how tolerance works? For instance, does a tolerance setting of 20 mean that it will select colors within a range of +20 and -20 shades from the specified color? Or does tolerance work differently? I am seeking a precise explanation of the behavior of tolerance in Adobe Illustrator. Thank you.

Monika Gause

Adobe Expert

You will not get other information than in your other threads:

https://community.adobe.com/t5/illustrator-discussions/how-tolerance-works-in-magic-wand-tool-in-adobe-illustrator/m-p/14810909#M417614

and

https://community.adobe.com/t5/illustrator-discussions/understanding-tolerance-calculation-in-adobe-illustrator-s-magic-wand-tool/m-p/14751566#M413688

So maybe you finally post an exact case you need a solution for.

Ghulam MohyuddinAuthor

Known Participant

Despite applying a tolerance of ±20 for each color channel, the expected selection did not occur. For example, consider a base object with a specific color and a set tolerance range of ±20. Another object with colors within this range should logically be selected, but it remains unselected. This discrepancy suggests that additional factors or constraints are influencing the selection process.Look at this example

Anshul_Saini

Community Manager

Hi Ghulam Mohyuddin,

Thanks for reaching out. The Tolerance setting in the Magic Wand tool in Adobe Illustrator can indeed be a bit tricky. The tolerance setting controls the range of color variation that the Magic Wand tool will select. A tolerance of ±20 means that the tool will select colors that fall within a range of 20 units above or below the base color's RGB values.

If you’re finding that the tool isn’t selecting as expected despite setting the tolerance correctly, consider the following:

Ensure that the objects' color mode (RGB or CMYK) is consistent. If the color mode is inconsistent, the Magic Wand tool’s tolerance setting might not work as expected.
The Magic Wand tool doesn’t always account for opacity or specific effects applied to objects. Check if any transparency or effects might be affecting the selection.
Sometimes, color profiles or slight variations in color values can influence selection. Double-check that the colors are within the expected range.
Ensure that the objects are entirely within the selection bounds. The tool might not select them if they are partially outside or overlapping with other objects.

If you still encounter issues, please provide more details or a specific example, and I’d be happy to help further!

Best,

Anshul Saini

Ghulam MohyuddinAuthor

Known Participant

"Hello Anshul_Saini,I have a base object with the following color values: C=70%, M=5%, Y=100%, K=10%. I also have another object with color values: C=90%, M=5%, Y=80%, K=10%. I set the tolerance to 20, and according to the logic you provided, the object should have been selected, but it was not. Could you please help me understand why this is happening?"

Ghulam MohyuddinAuthor

Known Participant

I am seeking a detailed understanding of how the tolerance settings work with the Magic Wand tool in Adobe Illustrator for both CMYK and RGB color modes.

CMYK Tolerance Calculation:

Reference Color: CMYK (70, 0, 100, 0)
Tolerance: 20%
Could you please specify the range of color values for each channel (Cyan, Magenta, Yellow, Black) that would be selected with this tolerance setting?
RGB Tolerance Calculation:
- Reference Color: RGB (255, 81, 47)
- Tolerance: 30
- Could you provide the range of color values for each channel (Red, Green, Blue) that would be selected with this tolerance setting?
  Tolerance Application:
  - Does the Magic Wand tool apply tolerance settings to each color channel individually, or does it use a combined average of all channels to determine color similarity?

Your insights on these queries would greatly assist in understanding how the Magic Wand tool processes color selections with varying tolerance levels.

Monika Gause

Adobe Expert

Please continue in your other thread.

Kurt Gold

Adobe Expert

As for CMYK process colours the rule is what one may call a "common responsibility".

That is, to be considered and selected as objects with the "same" colour, none of the four inks (relative to the reference colour) are allowed to exceed or fall below the percentaged tolerance setting in the Magic Wand palette. Otherwise the Magic Wand won't select them.

Probably that's valid for RGB colours as well, but the calculation is obviously a bit more complicated.

Ghulam MohyuddinAuthor

Known Participant

I understand your point, but I would like to know the precise method by which the Magic Wand tolerance is calculated.

Ghulam MohyuddinAuthor

Known Participant

For instance, if we set a tolerance level of 20 in Adobe Illustrator for a CMYK color with values C=70, M=47, Y=55, K=60, does the tolerance apply as a ±20 range for each channel (e.g., C=50-90, M=27-67, Y=35-75, K=40-80)?

--------------------------------------------------------------------------------------------------------------------------------

Yes, exactly. That's the basic rule about CMYK colours I was trying to explain in my rather generalised statement above.

It gets a bit different when there are 0 % inks. Basically, the 0 % inks do not count, unless you change at least one of the non-zero % inks. Then the zero % inks kind of may get some bonus points.

A long time ago I created a comparative table that should explain the issue in detail. Will have to look at one of my archives and post it in case I can locate it.

Despite applying a tolerance of ±20 for each color channel, the expected selection did not occur. For example, consider a base object with a specific color and a set tolerance range of ±20. Another object with colors within this range should logically be selected, but it remains unselected. This discrepancy suggests that additional factors or constraints are influencing the selection process.Look at this example

Monika Gause

Adobe Expert

I assume the fastest procedure would be that you find out what works for your artwork by experiment.

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded