Description
Describe the bug
When using BigDecimal.divide with a DecimalMode, the results are sometimes incorrect.
To Reproduce
Run this code:
@Test
fun testDivision() {
val a = 1.08589200008919E+4
val b = 1.085892300018812E+5
val aOverB = a / b
println("a/b=${aOverB}")
val a2 = BigDecimal.fromDouble(a)
val b2 = BigDecimal.fromDouble(b)
RoundingMode.entries.filter { it != RoundingMode.NONE }.forEach { mode ->
println("mode: $mode")
listOf(5L, 6L, 7L).forEach { precision ->
val a2OverB2 = a2.divide(b2, DecimalMode(precision, mode))
println("\tprec: $precision, a2/b2=${a2OverB2}")
}
println()
}
}
Expected behavior
I expect all results to be similar to the Kotlin Double result, subject to the precision and last digit rounding behavior. Sometimes the result is approximately 10x different from the expected result.
The result from the above code is:
a/b=0.09999997237943192
mode: FLOOR
prec: 5, a2/b2=9.9999E-2
prec: 6, a2/b2=9.99999E-2
prec: 7, a2/b2=9.999997E-2
mode: CEILING
prec: 5, a2/b2=1.0E-2
prec: 6, a2/b2=1.0E-2
prec: 7, a2/b2=9.999998E-2
mode: AWAY_FROM_ZERO
prec: 5, a2/b2=1.0E-2
prec: 6, a2/b2=1.0E-2
prec: 7, a2/b2=9.999998E-2
mode: TOWARDS_ZERO
prec: 5, a2/b2=9.9999E-2
prec: 6, a2/b2=9.99999E-2
prec: 7, a2/b2=9.999997E-2
mode: ROUND_HALF_AWAY_FROM_ZERO
prec: 5, a2/b2=9.9999E-2
prec: 6, a2/b2=1.0E-2
prec: 7, a2/b2=9.999997E-2
mode: ROUND_HALF_TOWARDS_ZERO
prec: 5, a2/b2=9.9999E-2
prec: 6, a2/b2=1.0E-2
prec: 7, a2/b2=9.999997E-2
mode: ROUND_HALF_CEILING
prec: 5, a2/b2=9.9999E-2
prec: 6, a2/b2=1.0E-2
prec: 7, a2/b2=9.999997E-2
mode: ROUND_HALF_FLOOR
prec: 5, a2/b2=9.9999E-2
prec: 6, a2/b2=1.0E-2
prec: 7, a2/b2=9.999997E-2
mode: ROUND_HALF_TO_EVEN
prec: 5, a2/b2=9.9999E-2
prec: 6, a2/b2=1.0E-2
prec: 7, a2/b2=9.999997E-2
mode: ROUND_HALF_TO_ODD
prec: 5, a2/b2=9.9999E-2
prec: 6, a2/b2=1.0E-2
prec: 7, a2/b2=9.999997E-2
Platform
Seen on JVM 17.0.8. I didn't test other platforms but it may exist on other platforms as well.
Using com.ionspin.kotlin:bignum:0.3.10.