Skip to main content
Inspiring
October 4, 2010
Question

rounding issue?

  • October 4, 2010
  • 6 replies
  • 4133 views

maybe it's me, but are:

#decimalformat(70.625)#
#decimalformat(70.725)#

supposed to give two different rounding results?

    This topic has been closed for replies.

    6 replies

    BKBK
    Community Expert
    Community Expert
    October 6, 2010

    This is a worrisome bug. You can use the 'bad' values in my table above to reproduce similar bugs in the function round(). I'll use the values from the sixth row to illustrate the boogy bugginess:

    <cfoutput>
    round(100*16.525)= #round(100*16.525)#<br>
    round(100*17.525)= #round(100*17.525)#<br>
    round(100*18.525)= #round(100*18.525)#<br>
    round(100*19.525)= #round(100*19.525)#<br>
    </cfoutput>

    Inspiring
    October 6, 2010

    so what is to be done then? does adobe monitor these forums?

    Community Expert
    October 6, 2010

    Yes, they do, but you should submit bugs here:

    https://www.adobe.com/cfusion/mmform/index.cfm?name=wishform

    Dave Watts, CTO, Fig Leaf Software

    http://www.figleaf.com/

    http://training.figleaf.com/


    Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on

    GSA Schedule, and provides the highest caliber vendor-authorized

    instruction at our training centers, online, or onsite.

    Dave Watts, Eidolon LLC
    BKBK
    Community Expert
    Community Expert
    October 5, 2010

    I evaluated decimalFormat() for all 1000 decimals of the form

    n.m25

    where m is any of the digits 0, 1, 2, ..., 9 and n is any of the numbers 1, 2, 3, ..., 100. I am on CF 9.0.1.

    In every case, Coldfusion rounded the result to 2 decimal places as expected. In 923 cases, decimalFormat rounded the number up, ending in the digit 3, as expected. However, in the remaining 77 cases, Coldfusion rounded down, ending in the digit 2.

    You will find the 77 "bad" cases below. I'll illustrate using the sixth row as example. When you evaluate decimalFormat(x) successively for x=1.525, 2.525, ..., 99.525, 100.525, you will find that the result is rounded up, except 16.52, 17.52, 18.52 and 19.52.

    Expression, for n = 1,2,3,...100

    _________________________

    Coldfusion rounded down decimalFormat(x) for n = ...

    _____________________________________________________

    x = n + 0.0251, 16, 17, 18, 19, 20
    x = n + 0.125
    x = n + 0.2254, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80,  81
    x = n + 0.3258, 9
    x = n + 0.4252, 32, 33, 34, 35, 36, 37, 38, 39, 40
    x = n + 0.52516, 17, 18, 19
    x = n + 0.625
    x = n + 0.7254, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80,  81
    x = n + 0.8258, 9
    x = n + 0.92532, 33, 34, 35, 36, 37, 38, 39, 40

    You can immediately see that there is a method to the madness. There is a mysterious cycle of fives. Thus, the "bad" integers in row 1 are similar to those in row 6(=5+1), the bad integers in row 3 are similar to those in row 8(=5+3), and so on. There were no bad results for row 2 and for row 7(=5+2). As they say, there is order in chaos.

    Participating Frequently
    October 5, 2010

    Did you check precisionEvaluate? Would it help here?

    Inspiring
    October 6, 2010

    Did you check precisionEvaluate? Would it help here?

    Dunno.  In what way are you suggesting to use it?

    --

    Adam

    Inspiring
    October 5, 2010

    What's more interesting is this code:

    <cfset lOriginals = "70.625,70.725">

    Original:<br />
    <cfoutput>
    decimalFormat(70.625): #decimalFormat(70.625)#<br />
    decimalFormat(70.725): #decimalFormat(70.725)#<br />
    <hr />
    Looped:<br />
    <cfloop index="n" from="70.625" to="70.725" step="0.1">
        decimalFormat(#n#): #decimalFormat(n)#<cfif listFind(lOriginals, n)>*</cfif><br />
    </cfloop>
    <hr />
    Longer Looped:<br />
    <cfloop index="n" from="70.025" to="71.025" step="0.1">
        decimalFormat(#n#): #decimalFormat(n)#<cfif listFind(lOriginals, n)>*</cfif><br />
    </cfloop>
    <hr />
    List Loop:<br />
    <cfset lNumbers="70.025,70.125,70.225,70.325,70.425,70.525,70.625,70.725,70.825,70.925,71.025">
    <cfloop index="n" list="#lNumbers#">
        decimalFormat(#n#): #decimalFormat(n)#<cfif listFind(lOriginals, n)>*</cfif><br />
    </cfloop>
    </cfoutput>

    Output:

    Original:
    decimalFormat(70.625): 70.63
    decimalFormat(70.725): 70.72


    Looped:

    decimalFormat(70.625): 70.63*

    decimalFormat(70.725): 70.72*


    Longer Looped:

    decimalFormat(70.025): 70.03

    decimalFormat(70.125): 70.13

    decimalFormat(70.225): 70.22

    decimalFormat(70.325): 70.32

    decimalFormat(70.425): 70.42

    decimalFormat(70.525): 70.52

    decimalFormat(70.625): 70.62*

    decimalFormat(70.725): 70.72*

    decimalFormat(70.825): 70.82

    decimalFormat(70.925): 70.92

    decimalFormat(71.025): 71.02


    List Loop:
    decimalFormat(70.025): 70.03
    decimalFormat(70.125): 70.13
    decimalFormat(70.225): 70.22
    decimalFormat(70.325): 70.33
    decimalFormat(70.425): 70.43
    decimalFormat(70.525): 70.53
    decimalFormat(70.625): 70.63*
    decimalFormat(70.725): 70.72*
    decimalFormat(70.825): 70.83
    decimalFormat(70.925): 70.93
    decimalFormat(71.025): 71.03

    I did the list version of the loop because I was concerned the differences in the long loop was due to a creeping floating point inaccuracy.

    Initially I was going to say it's something to do with the rounding scheme Java uses (http://download-llnw.oracle.com/javase/6/docs/api/java/math/RoundingMode.html


    But looking @ that output... I think there's a bug here as I cannot see what's going on.

    It's interesting, also, to see the differences in results when using dollarFormat() instead.

    Adam Lehman, you're monitoring these forums... any insight?

    --

    Adam

    Inspiring
    October 5, 2010

    yes, I can't see it as anything else but a bug, and with those results, there isn't even any discernable pattern to it.

    Inspiring
    October 4, 2010

    sorry, that's not what i meant.

    i would think it should be 70.63 and 70.73, but it's not, it comes back with 70.63 and 70.72

    i've duplicated on both CF8 and CF9

    ilssac
    Inspiring
    October 4, 2010

    DecimalFormat

    Converts a number to a decimal-formatted string.


    A number as a string formatted with two decimal places and a thousands separator.

    So yes, I would expect those two function calls should output "70.63" and "70.73" respectively.  NOTE the "two decimal places" part of the description