addition problem - is actionscript/flash able to calculate correctly????

5 replies
522 views
Translate

Hi,
I am adding two float numbers that return an incorrect result. Here is what I am doing...

var simpleMath:Number = Number(parseFloat("110.50")) - Number(parseFloat("107.10"));
alert("Simple Math=" + String(simpleMath));

The result I get is... Simple Math=3.40000000000001

How is this possible?
I have tried...
var simpleMath:Number = Number(110.50) - Number(107.10);

var simpleMath:Number = 110.50 - 07.10;

var numba:Number = 110.50;
var numbb:Number = 107.10;
var simpleMath:Number = numba - numbb;

They all give the same result...

When I try this:
Final_Payment.text = Number(Total_Payment_Due.text) - ((Number(Payment_Count.text) - 1) * Number(Payment_Amount.text));
Where Total_Payment_Due.text = 110.50 and Payment_Count.text = 36 and Payment_Amount.text = 3.06
I get this result: 3.39999999999999...

Can someone please tell me what I am going wrong. Or is there problem with actionscript/flash...

I am using Cold Fusion 7.0.1 and actionscript inside a Flash Form savecontent variable.

This is driving me crazy...

Also, is there a round function for floats? I know about the Math.round() but it only works on integers.

Any help would be extreemly appreciated!
Thanks in advance,
Ed

ActionScript

This topic is closed to new replies. Start a new post to keep the conversation going.

N

Newsgroup_UserCorrect answer

Inspiring

"ed_skinner" <webforumsuser@macromedia.com> wrote in message news:e3dcfr$irm$1@forums.macromedia.com...
>

> Hi,
> I am adding two float numbers that return an incorrect result. Here is
> what I am doing...
>
> var simpleMath:Number = Number(parseFloat("110.50")) -
> Number(parseFloat("107.10"));
> alert("Simple Math=" + String(simpleMath));
>
> The result I get is... Simple Math=3.40000000000001
>
> How is this possible?

It's the same old problem we have always had with floating point stored in computer memory.. not just Flash. For dealing with
dollars and cents you can make it easy on yourself by multiplying both sides by 100 and rounding to eliminate decimals, then in
final calculation divide by 100.
You can also multiply/divide by 1000 if you need higher precision in the pennies.
tralfaz

N

Newsgroup_User

Inspiring

It doesn't matter how advanced a computer it is .. 0.1 (for example) can not
be expressed exactly in binary (which computers use) .. no matter what you
do, it is always only an approximation (a bit like 1/3 =
0.3333333333333333.... in decimal). Its just the mathematics of it.

Fixed point arithmetic (where you implicitly multiply by some number.. like
multiply by 100 for currency amounts to give you cents) is often done. For
instance, the x and y coordinates in Flash are internally fixed point
numbers (multiples of 1/20 = 0.05)
--
Jeckyl

Translate

E

ed_skinnerAuthor

Participant

Hey Thanks guys. I used the technique of *100, calculate, round and then /100 to get it to work right. One might think that in all the advancement in computers that floating point numbers would be handled correctly by now!
Oh well, past another problem with the help of fellow programmers.
Folks... and may the force be with you!

Translate

kglad

Community Expert

whoa, there's jedi in the jeckyl. who knew?

Translate

N

Newsgroup_User

Inspiring

That is why most commercial programs store currency as whole numbers of
cents, rather than as floating point numbers (which by their very nature
cannot be exact)

You need to change your program to take the nature of numbers in computers
into account to avoid errors. The bug is really in how you are doing your
calculations, not in Flash, and the solution also lies with you.
--
Jeckyl

Translate

N

Newsgroup_UserCorrect answer

Inspiring

"ed_skinner" <webforumsuser@macromedia.com> wrote in message news:e3dcfr$irm$1@forums.macromedia.com...
>

> Hi,
> I am adding two float numbers that return an incorrect result. Here is
> what I am doing...
>
> var simpleMath:Number = Number(parseFloat("110.50")) -
> Number(parseFloat("107.10"));
> alert("Simple Math=" + String(simpleMath));
>
> The result I get is... Simple Math=3.40000000000001
>
> How is this possible?

It's the same old problem we have always had with floating point stored in computer memory.. not just Flash. For dealing with
dollars and cents you can make it easy on yourself by multiplying both sides by 100 and rounding to eliminate decimals, then in
final calculation divide by 100.
You can also multiply/divide by 1000 if you need higher precision in the pennies.
tralfaz

Translate

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.