Number Multiplication AS3

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Number Multiplication AS3

Σπύρος Αγγελόπουλος
Hi Everyone,

Can anyone verify the results of calculation below ?

 

var n1:Number=18.9;

var n2:Number=100.0;              

var n3:Number=n1*n2;  //1889.9999999999998

 

 

is it bug ?

 

thanks

 

 

spiros

 

 

 

 

Reply | Threaded
Open this post in threaded view
|

Re: Number Multiplication AS3

Erik J. Thomas
The Number data type, when including a decimal point is a binary floating point number and binary floating point math can result in rounding errors like yours.

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#680 <https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#680>

Adobe says this about Number:

"The Number data type uses the 64-bit double-precision format as specified by the IEEE Standard for Binary Floating-Point Arithmetic (IEEE-754) which means rounding errors can occur... The Number data type uses 52 bits to store the significand, with the result that numbers that require more than 52 bits to represent precisely, such as the fraction 1/3, are only approximations. If your application requires absolute precision with decimal numbers, you need to use software that implements decimal floating-point arithmetic as opposed to binary floating-point arithmetic."

Erik

On Feb 2, 2018, at 7:22 AM, Σπύρος Αγγελόπουλος <[hidden email]> wrote:

Hi Everyone,

Can anyone verify the results of calculation below ?



var n1:Number=18.9;

var n2:Number=100.0;              

var n3:Number=n1*n2;  //1889.9999999999998





is it bug ?



thanks





spiros










Reply | Threaded
Open this post in threaded view
|

Re: Number Multiplication AS3

Erik J. Thomas
You can sort of fix this issue by rounding, but it just hides the issue and should not be used for financial applications:

var setPrecision:Function = function(number:Number, precision:int) {
 precision = Math.pow(10, precision);
 return Math.round(number * precision)/precision;
}

var number:Number = 10.98813311;
trace(setPrecision(number,1)); //Result is 10.9
trace(setPrecision(number,2)); //Result is 10.98
trace(setPrecision(number,3)); //Result is 10.988 and so on
https://stackoverflow.com/questions/632802/how-to-deal-with-number-precision-in-actionscript <https://stackoverflow.com/questions/632802/how-to-deal-with-number-precision-in-actionscript>

Erik

On Feb 2, 2018, at 8:39 AM, Erik J. Thomas <[hidden email]> wrote:

The Number data type, when including a decimal point is a binary floating point number and binary floating point math can result in rounding errors like yours.

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#680 <https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#680>

Adobe says this about Number:

"The Number data type uses the 64-bit double-precision format as specified by the IEEE Standard for Binary Floating-Point Arithmetic (IEEE-754) which means rounding errors can occur... The Number data type uses 52 bits to store the significand, with the result that numbers that require more than 52 bits to represent precisely, such as the fraction 1/3, are only approximations. If your application requires absolute precision with decimal numbers, you need to use software that implements decimal floating-point arithmetic as opposed to binary floating-point arithmetic."

Erik

On Feb 2, 2018, at 7:22 AM, Σπύρος Αγγελόπουλος <[hidden email]> wrote:

Hi Everyone,

Can anyone verify the results of calculation below ?



var n1:Number=18.9;

var n2:Number=100.0;              

var n3:Number=n1*n2;  //1889.9999999999998





is it bug ?



thanks





spiros











Reply | Threaded
Open this post in threaded view
|

RE: Number Multiplication AS3

Σπύρος Αγγελόπουλος
Eric,

Thanks for the quick response.


Spiros

-----Original Message-----
From: Erik J. Thomas [mailto:[hidden email]]
Sent: Friday, February 2, 2018 6:49 PM
To: [hidden email]
Subject: Re: Number Multiplication AS3

You can sort of fix this issue by rounding, but it just hides the issue and should not be used for financial applications:

var setPrecision:Function = function(number:Number, precision:int) {  precision = Math.pow(10, precision);  return Math.round(number * precision)/precision; }

var number:Number = 10.98813311;
trace(setPrecision(number,1)); //Result is 10.9 trace(setPrecision(number,2)); //Result is 10.98 trace(setPrecision(number,3)); //Result is 10.988 and so on https://stackoverflow.com/questions/632802/how-to-deal-with-number-precision-in-actionscript <https://stackoverflow.com/questions/632802/how-to-deal-with-number-precision-in-actionscript>

Erik

On Feb 2, 2018, at 8:39 AM, Erik J. Thomas <[hidden email]> wrote:

The Number data type, when including a decimal point is a binary floating point number and binary floating point math can result in rounding errors like yours.

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#680 <https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#680>

Adobe says this about Number:

"The Number data type uses the 64-bit double-precision format as specified by the IEEE Standard for Binary Floating-Point Arithmetic (IEEE-754) which means rounding errors can occur... The Number data type uses 52 bits to store the significand, with the result that numbers that require more than 52 bits to represent precisely, such as the fraction 1/3, are only approximations. If your application requires absolute precision with decimal numbers, you need to use software that implements decimal floating-point arithmetic as opposed to binary floating-point arithmetic."

Erik

On Feb 2, 2018, at 7:22 AM, Σπύρος Αγγελόπουλος <[hidden email]> wrote:

Hi Everyone,

Can anyone verify the results of calculation below ?



var n1:Number=18.9;

var n2:Number=100.0;              

var n3:Number=n1*n2;  //1889.9999999999998





is it bug ?



thanks





spiros