Text only | Text with Images

newLISP Fan Club

Forum => newLISP in the real world => Topic started by: HPW on March 20, 2019, 03:34:39 AM

Title: Unexpected result in round
Post by: HPW on March 20, 2019, 03:34:39 AM
Hello,



I get this unexpected result in round:
Code Select

> (round 8.3 -1)
8.300000000000001
> (round 8.30 -1)
8.300000000000001
> (round 8.430 -1)
8.4
> (round 8.4 -1)
8.4
> (round 8.2 -1)
8.199999999999999
>


The doc contains not this behaviour.



Regards

Hans-Peter
Title: Re: Unexpected result in round
Post by: rrq on March 20, 2019, 04:46:08 AM
If you don't mind, I would point you at this fairly in-depth discussion about "representable numbers"

//https://en.wikipedia.org/wiki/Floating-point_arithmetic#Representable_numbers,_conversion_and_rounding

which I think is the issue at hand here.



In short, the numbers you see printed are the nearest representable numbers to the numbers you asked for, given that the rounding is done wrt a decimal base whilst the actual values are held in binary.
Title: Re: Unexpected result in round
Post by: Lutz on March 20, 2019, 07:53:43 AM
If you are concerned about the visual representation use format:


Code Select
> > (round 8.300 -1)
8.300000000000001

> (format "%2.1f" 8.300)
"8.3"
>
Title: Re: Unexpected result in round
Post by: HPW on March 20, 2019, 09:37:06 AM
Hello,



Thanks for the advices.

I am generating nc-code for a nc-drilling mashine.

Format will do the Job.



Regards

Hans-Peter
Text only | Text with Images