ubsan: integer overflow in tc-i386.c:offset_in_range

[Alan Modra]

On Thu, May 22, 2025 at 06:19:11AM +0800, H.J. Lu wrote:
> On Thu, May 22, 2025 at 5:55 AM Alan Modra <amodra@gmail.com> wrote:
> >
> > or $9223372036854775808,%eax
> > runtime error: negation of -9223372036854775808 cannot be represented
> > in type 'offsetT' (aka 'long'); cast to an unsigned type to negate
> > this value to itself
> >
> > Nothing in this function needs a signed type, despite the value under
>                                                      Did you mean "a
> unsigned type"?

No.  offsetT is signed, addressT unsigned.  The code is all bit
twiddling, no expressions are signed comparisons or similar, so I made
the parameter unsigned rather than writing -(addressT) val.

> > test being signed.
> >
> >         * config/tc-i386.c (offset_in_range): Make "val" unsigned.
> >
> > diff --git a/gas/config/tc-i386.c b/gas/config/tc-i386.c
> > index 162e28e0325..e1f30d4b133 100644
> > --- a/gas/config/tc-i386.c
> > +++ b/gas/config/tc-i386.c
> > @@ -2975,7 +2975,7 @@ smallest_imm_type (offsetT num)
> >  }
> >
> >  static offsetT
> > -offset_in_range (offsetT val, int size)
> > +offset_in_range (addressT val, int size)
> >  {
> >    addressT mask;
> >
> >
> > --
> > Alan Modra
> 
> 
> -- 
> H.J.

-- 
Alan Modra