[PATCH] math: Fix errno setting for narrowing sqrt functions on underflow [BZ #33393]

[Osama Abdelkader]

On Sun, Oct 19, 2025 at 06:22:38PM +0000, Wilco Dijkstra wrote:
> Hi Osama,
> 
> > The CHECK_NARROW_SQRT macro in math-narrow.h was not setting errno to
> > ERANGE when an underflow exception occurred during narrowing square root
> > operations. While the FE_UNDERFLOW exception flag was correctly set, errno
> > remained unchanged, violating IEEE 754 and POSIX requirements.
> 
> I don't see an issue that needs to be fixed. errno as a concept is highly problematic
> to start with. It's not very useful for math functions, hence most math libraries do
> not set errno at all - and this is perfectly conformant. Note IEEE-754 does not
> mandate anything about errno.
> 
> GLIBC typically sets errno to ERANGE on underflow to zero [1]. And some
> GLIBC functions never set errno.
> 
>        else if ((RET) == 0 && (X) != 0)		\
>  	__set_errno (ERANGE);			\
> +      else if (fetestexcept (FE_UNDERFLOW))	\
> +	__set_errno (ERANGE);			\
> 
> This is incorrect since FE_UNDERFLOW may be set by a different math operation.
> 
> Cheers,
> Wilco
> 
> [1] https://sourceware.org/glibc/manual/2.42/html_node/Math-Error-Reporting.html

Thanks for the review!

You're right about v1 using fetestexcept() - that was incorrect. However, v2 
uses a different approach:

  else if ((RET) != 0 && !isnormal (RET))
      __set_errno (ERANGE);

This checks if the result is subnormal (non-zero but denormal), which 
indicates underflow occurred during the narrowing operation,
without relying on exception flags.

Regarding the policy question: I understand glibc currently only sets errno 
for underflow-to-zero. However, the bug report (BZ #33393) shows a case where:
- Input causes underflow during narrowing
- Result is subnormal (2.938736e-39)
- FE_UNDERFLOW is raised
- math_errhandling indicates errno should be set
- But errno remains 0

Should the behavior be:
a) errno only for exact zero (current behavior)
b) errno for any underflow (zero or subnormal)

If (a) is correct, should we close this bug as "won't fix"? Or is there a 
standards-based reason to prefer (b)?

Best regards,
Osama