Re: floating point numbers / decode_binary()

• Date: Mon, 16 Dec 1996 23:13:41 PST
• From:
  hpa@trantor.zytor.com (H. Peter Anvin)
• Reply-To:
  hpa@storm.net (H. Peter Anvin)
• Content-Transfer-Encoding: 8bit
• Content-Type: text/plain; charset=ISO-8859-1
• Newsgroups: local.moo
• Organization: Where Stormnet meets the Internet
• References: <3.0.1.32.19961216203209.006a04c4@pop.kaiwan.com>

Followup to:  <3.0.1.32.19961216203209.006a04c4@pop.kaiwan.com>
By author:    Brack <slayer@kaiwan.com>
In newsgroup: local.moo
>
> I thought floating point numbers only had a precision up to like 15-17
> digits or so. It seems the largest floating point number I can use is
> 1.79769313486231e+308. But, if I put a real long float number, such as 
> ;1.79769313486231580793728971405303415079
> it will round off to 1.79769313486232e+308. But, I can't re-eval
> 1.79769313486232e+308 because it will give me an overflow. Also if I try
> 1.797693134862315807937289714053034150791 it gives an overflow tb as well.
> 

I presume that is 1.79....e+308 you have there?  It is indeed possible
for the normal output rounding to end up with a number that cannot be
read back in due to overflow; this can be circumvented by using
floatfmt() [or whatever the function is called again] using a very
large precision (such as 9999 - the MOO server will force it down to a
sane number.)  Unfortunately these types of effects are part of the
nature of floating point.

> 
> I also posted on Lambda while this list was down, I noticed if you try
> something like decode_binary("~80~03~90") I correctly get {128, 3, 144} but
> if I try decode_binary("~80~03~90", 1) it comes out as {-128, 3, -112}; I
> was told it was due to using signed integer types rather than unsigned in
> the server code.
> 

This is a sheer bug; this should be avoided by casting to
(unsigned char) before casting to (int):

	*n = (int) (unsigned char) (*ch);

[or whatever...]

	-hpa
-- 
PGP public key available - finger hpa@zytor.com
Key fingerprint: 2047/2A960705 BA 03 D3 2C 14 A8 A8 BD  1E DF FE 69 EE 35 BD 74
"The earth is but one country, and mankind its citizens."  --  Bahá'u'lláh
Just Say No to Morden * The Shadows were defeated -- Babylon 5 is renewed!!

• floating point numbers / decode_binary()
• From: Brack <slayer@kaiwan.com>

• Prev: floating point numbers / decode_binary()
• Next: How to use 8-bit ascii in MOO ?
• Index: MOO-cows
• Thread: MOO-cows