Misunderstanding NaN

[Richard Earnshaw]

On Fri, 2005-05-06 at 22:52, Bryce Schober wrote:
> I'm using the arm-elf gcc 3.4 toolchain from gnuarm.com, which
> includes gcc-3.4.3 and newlib-1.12.0.
> 
> I have some double math that is resulting in a bunch of NaN's. One of
> the offending equations simplifies to: -41 * (3.14159265358979 / 180 /
> 16). In gdb I can:
> 
> (arm-elf-gdb) p -41 * (3.14159265358979 / 180 / 16)
> $3 = nan(0x9d96bbfa6e612)
> 
> My windows calculator calculates -0.04472406208 (some "precision"
> dropped), but gdb gives:
> 
> (arm-elf-gdb) p -0.04472406208
> $1 = nan(0x4abdebfa6e612)
> 
> This is really strange, since I can subtract 1 and gdb gives:
> 
> (arm-elf-gdb) p -0.04472406208 - 1
> $5 = -1.04472406208
> 
> This is really beginning to confuse me. Any ideas on what's going on here?

There are two possible floating point layouts on ARM, one used by old
tools that is layout compatible with the FPA co-processor, and one that
is starting to become more prevalent now that is fully layout-compatible
with IEEE 754 and used on the VFP co-processor.  You need to ensure that
gdb is using the correct one for your application.

GDB tries to work this out automatically for some targets, but it's not
always possible.  You can force the setting with the 'set arm fpu'
command (try: help set arm fpu).

No guarantees that this is your problem though :-)

R.