Math Operations Leave trailing zeros after operation when result is whole number
Description
Activity
Brad Wood February 25, 2025 at 7:09 PM
I had this workaround in place, since Doubles always left exactly one zero, which would have applied to SOME DB math, but not most of it
if ( result.endsWith( ".0" ) ) {
return result.substring( 0, result.length() - 2 );
}
Brad Wood February 25, 2025 at 7:07 PM
Looks like BD has a method exactly for this called stripTrailingZeros()
https://docs.oracle.com/javase/8/docs/api/java/math/BigDecimal.html#stripTrailingZeros--
I’ll take a look at using this in a few minutes
Brad Wood February 25, 2025 at 7:04 PM
Those are insignificant digits which do not affect the value. All decimal math uses a BigDecimal which defaults to over 30 digits of precision. I have a feeling this is more related to the formatting employed when the BigDecimal is cast to a string based on how its output.
Ah yes, that’s it. The BigDecimal.toString()
will represent as many digits as its been tracking for that number, and the BD class will “grow” what it tracks as-needed. Take this version for example
minutesValid = 1 / 4; // 0.25
expiresSeconds = minutesValid * 60; // 15.00
The first variable has two decimal digits, and as such, the final result also shows precision to 2 digits. Looking at the example in the ticket, we have this:
minutesValid = 1 / 60; // 0.01666666666666666666666666666666667
expiresSeconds = minutesValid * 60; // 1.000000000000000000000000000000000
The first expression takes up 35 digits of precision after the decimal, so once the DB has “grown” the number of decimals its tracking, further match operations against it reflect that same level of precision.
I think we just need to apply some forced rounding on the output since we don’t necessarily care about the amount of internal precision BD was using at the time.
The following code:
minutesValid = 1 / 60; expiresSeconds = minutesValid * 60; writeDump( expiresSeconds );
Produces an
expiresSeconds
value in BoxLang of1.000000000000000000000000000000000
In Lucee and ACF it is rounded to
1