The IBM long double version of ulp(0) returns 0, because scalbnl(1,-1127) is 0.
Fixing this ASAP.
Is there a publicly available standard that describes IBM extended precision long doubles? The smallest value I can compute is 2^(-1075): 4.94065645841246544177e-324 0x0.00000000000010000000p-1022 The smallest computed value matches the AIX expectations for the, I assume, similar type. I'm going to go with the smallest computed value as ulp of zero. Why? Because the first double represents the number, and if it's mantissa is zero and the exponent is zero, then it's zero. The extra precision from the second double is useless for subnormal values. Therefore the smallest subnormal is identical to that for double.
On Fri, 31 May 2013, carlos at redhat dot com wrote: > Is there a publicly available standard that describes IBM extended precision > long doubles? libgcc/config/rs6000/ibm-ldouble-format in a GCC source tree. > The smallest value I can compute is 2^(-1075): > 4.94065645841246544177e-324 > 0x0.00000000000010000000p-1022 You have an off-by-one error there. It's 0x1p-1074.
Resolved. commit 8b0ccb2d7fd1ec646a622a16bd64e356739ffca3 Author: Carlos O'Donell <carlos@redhat.com> Date: Mon Jun 3 14:49:48 2013 -0400 BZ #15536: Fix ulp for 128-bit IBM long double. In 128-bit IBM long double the precision of the type decreases as you approach subnormal numbers, equaling that of a double for subnormal numbers. Therefore adjust the computation in ulp to use 2^(MIN_EXP - MANT_DIG) which is correct for FP_SUBNORMAL for all types.