Re: Floating point error

• Date: Wed, 13 Mar 1996 14:41:56 PST
• From:
  Jay Carlson <nop@k2.ccs.neu.edu>
• cc: moo-cows.PARC@xerox.com
• In-Reply-to: "Your message of \"Tue, 12 Mar 96 15:48:45 PST.\" <199603122348.PAA17846@wasteland.calbbs.com>rnal s"

> It appears that there's a problem with floating point in 1.8.0p1...
> On a MOO running 1.8.0beta3 and on another running beta2:

> ;1.6
> => 1.6

> On a MOO running 1.8.0p1 and on ANOTHER running p1:
> ;1.6
> => 1.6000000000000001
^^^^^1 2345678901234567    17 digits.

I agree that this is probably a bug, but you should be aware that
looking for 17 digits of precision sends you off into the weeds
anyway.

Each decimal digit has log2(10) bits of information in it.  So 17
digits is around 56.5 bits of information.  If we go poke at
/usr/include/float.h, we find out there's only 53 bits of mantissa in
IEEE double-precision floats.  So we're trying to print more
information than we actually have.

More fun:

  ;1.6 == 1.6000000000000001
  => 1
  ;1.6000000000000002
  => 1.6000000000000003

If you care about accuracy, and your application floating point, you
should be careful.  Don't forget that floating point doesn't give you
infinite precision, and that, for example, you almost NEVER want to
use "a == b" on floating point numbers; use "abs(a - b) < .0001" or
whatever is appropriate for your application.  Many of the standard
numerical analysis issues could be magnified by persistence if you're
not careful.

Jay Carlson
nop@nop.com    nop@ccs.neu.edu    nop@kagoona.mitre.org

Flat text is just *never* what you want.   ---stephen p spackman

• Floating point error
• From: Brian Buchanan <brian@wasteland.calbbs.com>

• Prev: Re: Floating point error
• Next: Re: Floating point error
• Index: MOO-cows
• Thread: MOO-cows