Welcome Dialog

Welcome to the Community!

We have a brand new look! Take a tour with us and explore the latest updates on Adobe Support Community.


JavaScript Subtraction Value is different

Engaged ,
Aug 25, 2021 Aug 25, 2021

Copy link to clipboard

Copied

Hi,

 

Can anyone explain why subtraction value is different?
Expected output : 12
Result output: 11.9999999999998

baseline
Result: 732
textColumn.endBaseline
Result: 720
baseline - textColumn.endBaseline
Result: 11.9999999999998

SumitChaurasia_0-1629878919271.png

 

 

-Sumit
TOPICS
Scripting, SDK, Server developers

Views

124

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines

correct answers 1 Correct answer

Adobe Community Professional , Aug 25, 2021 Aug 25, 2021
A rounding error. This is typical of all computer languages. In scripting you can make allowances so that you check not whether something has a precise value, but is within a range.

Likes

Translate

Translate
Adobe Community Professional ,
Aug 25, 2021 Aug 25, 2021

Copy link to clipboard

Copied

A rounding error. This is typical of all computer languages. In scripting you can make allowances so that you check not whether something has a precise value, but is within a range.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Engaged ,
Aug 25, 2021 Aug 25, 2021

Copy link to clipboard

Copied

Thank you Peter for your valueable.

-Sumit

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Aug 25, 2021 Aug 25, 2021

Copy link to clipboard

Copied

Due to this be careful when you use the greater than and less than operator in your code, you might be in for some surprises if you are not mindful of it and it's nasty to debug

-Manan

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Engaged ,
Aug 25, 2021 Aug 25, 2021

Copy link to clipboard

Copied

Thank you Manan,

 

I used below snippet for the time being and asked to expert like you, how to solve this problem.

 

 

var difference = Math.round((baseline - textColumn.endBaseline) * 100) / 100;

 

 

-Sumit

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Aug 25, 2021 Aug 25, 2021

Copy link to clipboard

Copied

Hi @SumitKumar 

 

In addition to what my colleagues wrote and in order to prevent any confusion, it's worth noting that there is no rounding error with integers as such. That is, the subtraction 732-720 would obvously lead to 12 in any programming language:

 

var baseline = 732;
var endBaseline = 720;
var diff = baseline - endBaseline;
alert( diff ); // 12 (of course.)

 

Here the issue comes from the fact that either your baseline variable does not actually contain 732, or textColumn.endBaseline does not actually contain 720, despite what the JS console claims. So, the “rounding error” is that particular case arises from the fact that floating-point numbers are mistakenly shown as integers.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Engaged ,
Aug 25, 2021 Aug 25, 2021

Copy link to clipboard

Copied

Hi @Marc Autret,

 

Thank you for the response. Below you can see data browser and available space in page.

 

SumitChaurasia_0-1629947966980.png

Under the last line exactly 12pt space is available.

 

SumitChaurasia_2-1629948324575.png

Here you can see data browser shows:
baseline: 732
endBaseline: 720
diff: 11.9999999999998

 

-Sumit

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Aug 26, 2021 Aug 26, 2021

Copy link to clipboard

Copied

Hi again @SumitKumar ,

 

Yes, I can see what is displayed in InDesign and what the “data browser shows.” My point is, the shown values may be rounded without your knowledge. In your capture above, baseline is not strictly 732 and/or endBaseline is not strictly 720.

 

To convince you, here is another screenshot of a real example (no fake, I swear!) with additional tests of strict equality:

 

WrongIntegers.jpg

As you can see, the JS console claims that 252 is strictly different from 252 😉

In fact, my baseline variable is not strictly equal to 252—despite what is prompted.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Engaged ,
Aug 26, 2021 Aug 26, 2021

Copy link to clipboard

Copied

LATEST

Thank you @Marc Autret ,

 

I believe on you and on your post so please do not swear. I learn lot of things from your website and also I am your fan.

https://www.indiscripts.com/

 

And the question is how to resolve it? Below code is okay to resolve this issue?

var difference = Math.round((baseline - textColumn.endBaseline) * 100) / 100;

 

Regards,

-Sumit

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines