This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: correct rounding or not?

       Dear Wilco,

> __sin used to have this:
> 
>   else if (k < 0x3feb6000)
>     {
>       res = do_sin (x, 0, &cor);
>       retval = (res == res + 1.096 * cor) ? res : slow1 (x);

this would correspond to a maximum error of 0.5+0.096/4 = 0.524 ulp.
This corresponds to what I see after about 2.3e9 random tests (5.23e-1).

> >From this I estimated the worst-case ULP as < 0.55. It could well be a bit
> better in reality.

Best regards,
Paul
Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]