(int (exp (gammaln 6))) ;=> 119

Started by kosh, February 17, 2011, 05:48:20 PM

Previous topic - Next topic

kosh

Hello, Lutz.



The problem that does not correctly convert float to integer was found.


newLISP v.10.3.0 on Linux IPv4/6 UTF-8, execute 'newlisp -h' for more info.

> (setq x (exp (gammaln 6)))
120
> (float? x)
true
> (int x)
119
> (bits x)
"1110111"        ; (int "1110111" 0 2) is 119
> (bits 120.0)
"1111000"
> (dump x)
(147342976 387 147335024 -9 1079902207)

Lutz

#1
Floats in newLISP are displayed using a default format string "%1.10g". You see a rounding effect. When formatting for more precision, you see that the value is just under 120.
> (setq x 119.99999999999976)
120
> (format "%5.14f" x)
"119.99999999999976"
>

You can change the default print format for floats using the 'pretty-print' function:
> (pretty-print)
(80 " " "%1.10g")
> (pretty-print 80 " " "%1.15f")
(80 " " "%1.15f")
> (setq x (exp (gammaln 6)))
119.999999999999758
>

kosh

#2
I've understood this problem.

Thanks, Lutz.