Re: Undo dynamic symbol state after regular object sym type mismatch

[Alan Modra <amodra at gmail dot com>]

On Mon, Jul 24, 2017 at 12:22:34PM +0100, Maciej W. Rozycki wrote:
> Hi Alan,
> 
>  CC-ing `libc-alpha' for any feedback on the dynamic loader issue.
> 
> > >  In an attempt to prevent new MIPS target failures in preparation for the 
> > > 2.29 release I have now looked into it and what the MIPS backend produces 
> > > looks correct to me according to the MIPS psABI.
> > > 
> > >  Specifically, while linking the `indirect5a' executable, in the 
> > > `indirect5a.o' object supplied we have a locally-defined global `bar' 
> > > symbol referred to -- in the case of the o32 ABI -- with an R_MIPS_GOT16 
> > > relocation, which is absolutely fine for a MIPS SVR4 psABI executable.  
> > > In response the MIPS backend creates the mandatory global GOT entry and 
> > 
> > That's fine.
> 
>  If we want to prevent some global symbols from being exported, then not 
> according to the SVR4 MIPS psABI, which requires each global GOT entry to 
> have a corresponding global dynamic symbol table entry.
> 
> > > its corresponding dynsym entry for `bar'.
> > 
> > But in my opinion this is a bug.  A symbol in an executable shouldn't
> > be made dynamic unless it is referenced by a dynamic object, or by
> > --export-dynamic or other similar options.  Since "bar" in the
> > executable is a variable and "bar" in the dynamic library a function,
> > the dynamic library can't be referencing the executable "bar".  The
> > executable GOT entry for "bar" will need a dynamic relocation in a
> > PIE, but that can be a relative relocation not needing a symbol.
> 
>  The MIPS SVR4 psABI uses implicitly relocated GOT, there are no dynamic 
> GOT relocations defined (the only dynamic reloc is R_MIPS_REL32, for 
> static references to global symbols) and dynamic GOT relocation is 
> performed according to the DT_MIPS_GOTSYM dynamic entry: all entries whose 
> index is lower are relocated by the base address and the entries that 
> follow are relocated according to the external part of the dynamic symbol 
> table, which has to be identity mapped to the external part of GOT.  So an 
> external GOT entry by definition is associated with an external dynamic 
> symbol.
> 
>  So if we want to prevent a global symbol from being exported, then (as 
> long as it is locally referenced) it has to be assigned to the local part 
> of the GOT, which in turn means either giving it a local dynamic symbol 
> table entry or (IMO preferably) none at all.

OK, so the MIPS ABI is a little weird, but what you're saying sounds
like the right thing to do.

>  However this will cause 
> trouble for undefined weak symbols in PIE executables, as making these 
> symbols local will make the corresponding GOT entries relocated by the 
> base address and consequently any NULL pointer checks which typically 
> guard the use of such symbols will not work anymore, as already noted in 
> PR ld/21375.

I don't see why this is an issue.  You surely can make undefined weak
symbols global, if you would like them to potentially be resolved by a
definition at run-time not available at link time.  (Undefined weaks
are a special case.  I wasn't careful enough in saying which symbols
should be made dynamic.)  On the other hand, if you want undefined
weaks to *not* be resolved at run-time then you do have a problem to
make sure their value stays zero..

In either case, we're not talking about the indirect5a testcase, where
we have a strong definition of "bar" in both the executable and a
shared library (the definition in the shared library counting as an
implicit reference to allow overriding), and you don't want the
executable definition to be exported because it doesn't match the
shared library symbol type.

>  Note that in the SVR4 MIPS psABI there is no way defined for a GOT entry 
> to remain unchanged in dynamic loading: each is either relocated by the 
> base address or refers to a global dynamic symbol table entry.  
> Contrariwise static references to global symbols are not an issue as 
> R_MIPS_REL32 relocations can be simply omitted, leaving the reference 
> unrelocated in the dynamic load.
> 
>  The only solution to PR ld/21375 I have come up with so far is using a 
> global absolute symbol for GOT entries used for any local references to 
> undefined weak symbols, either by tweaking the attributes of the original 
> symbol (setting SHN_ABS/STB_GLOBAL combined with STV_HIDDEN or 
> STV_INTERNAL; but that breaks strict gABI compliance) or by redirecting to 
> a special reserved symbol, say `__gnu_absolute_zero', whose value is 
> globally zero (and SHN_ABS/STB_GLOBAL/STV_DEFAULT).  The latter approach 
> has the advantage of reducing the number of GOT entries (which is always 
> advantageous) where multiple undefined weak symbols are present.

Another approach is to edit the code, changing it from referencing the
GOT to loading zero by some other means.

>  However this merely shifts the issue with PIE binaries to the dynamic 
> loader, which currently does not tell absolute and regular symbols apart 
> and relocates both kinds by the base address, as noted in glibc BZ 19818.
> 
>  This is PR ld/21805 now, for tracking purposes.  I have experimented with 
> a preliminary fix, and in addition to cleaning indirect.exp test results 
> it indeed makes a number of other test cases crash, that work fine right 
> now.  In the course of this investigation I have also identified what I 
> think are a generic ELF linker bug and an issue in the Cris backend 
> respectively, both of which I'll be sending patches for soon.
> 
>  So I think the correct way to address the problem with the MIPS backend 
> is to:
> 
> 1. Fix glibc BZ 19818, the dynamic loader's absolute symbol bug.
> 
> 2. Bump ABIVERSION up in `ld.so' for correct SHN_ABS support.
> 
> 3. Fix PR ld/21375, the local undefined weak symbol bug.
> 
> 4. Document `--export-dynamic' as the default with MIPS PIE binaries.

OK, in that case you'll need to make ld.so check symbol type before
allowing an override.  Sounds like something that should be done for
all targets..

> 5. Fix PR ld/21805, this issue, by making `--no-export-dynamic' work for
>    SVR4 executables in the first place and then with a PIE executable 
>    produce SHN_ABS symbol GOT references for undefined weak symbols and 
>    make the executable request the corresponding dynamic loader's 
>    ABIVERSION for processing, possibly only where such a SHN_ABS symbol 
>    has indeed been produced.
> 
> 6. Set the defaults for `--export-dynamic'/`--no-export-dynamic' as 
>    documented.
> 
> -- in this order.
> 
>  I am yet reviewing past discussions about our handling of undefined weak 
> symbols in PIE executables, in particular PR ld/2218 and the corresponding 
> thread at: <https://sourceware.org/ml/binutils/2006-01/msg00193.html> and 
> then <https://sourceware.org/ml/binutils/2006-02/msg00023.html>, because 
> it seems we're a bit inconsistent here among targets as far as making them 
> global or local is concerned, and this is not covered by the ELF gABI 
> AFAICT.

Right.  Undefined weaks are a grey area.

>  Personally to keep their handling consistent between the static link and 
> the dynamic load I am leaning towards thinking that weak symbol references 
> from the main executable that are not satisfied at the static link time 
> ought to resolve to zero with the original reference removed, and ones 
> that are satisfied by a dynamic dependency ought to be converted to a 
> global symbol reference.
> 
>  Questions or comments?  I will appreciate further input.  This is 
> obviously too complex an issue for the pending 2.29 binutils release.
> 
>   Maciej

-- 
Alan Modra
Australia Development Lab, IBM