resolution of a float?

Started by TedWalther, October 17, 2012, 02:46:49 PM

Previous topic - Next topic

TedWalther

I have been converting some JavaScript code to newLISP.  It is detailed astronomical calculations.  Finally I notice something.  The floats in newLISP are only 10 decimal digits long, while those in JavaScript are 16.  I looked into the IEEE standard; there is 52 bits in which to store an integer without any loss of precision.  That matches the output of JavaScript.



Lutz, are we using IEEE 64 bit floats in newLISP by default?  I was hunting some bugs down for hours now, until I noticed this difference in precision.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence.  Nine months later, they left with a baby named newLISP.  The women of the ivory towers wept and wailed.  \"Abomination!\" they cried.

Lutz

#1
newLISP uses IEEE-754 double floats. The 10 digits seen are only the default display resolution. Use the 'format' function to display more digits. See also in the distribution the file qa-specific-tests/qa-float for some other IEEE floating point specifics.

TedWalther

#2
That is a relief, thanks for explaining.
Cavemen in bearskins invaded the ivory towers of Artificial Intelligence.  Nine months later, they left with a baby named newLISP.  The women of the ivory towers wept and wailed.  \"Abomination!\" they cried.