Re: --enable-auto-import extension

[egor duda <deo at logos-m dot ru>]

Hi!

Saturday, 29 June, 2002 Charles Wilson cwilson@ece.gatech.edu wrote:

CW> egor duda wrote:

>> Clean way to handle such situations (other than convincing
>> Microsoft to change a loader) is to add some runtime support for
>> non-zero-offset imports.
>> 
>> The idea is to add a vector of "cygwin internal pseudo-relocation"
>> entries to executable data section and to perform relocations of
>> appropriate data manually at program startup.

CW> But why is this cygwin-specific?  It seems that it's equally applicable 
CW> to mingw (e.g. native) DLLs, just as mingw's gcc can use the current 
CW> auto-import feature, even though MSVC can't understand or use it...

Well, of course it shouldn't be. I was thinking in terms of cygwin,
and, lacking better name, called it "cygwin blah-blah". But
suggestions for more generic name are welcome.

>> If this idea is worthwhile, i think i should add more things to the
>> patch:

CW> Absolutely.  Of course, it would still need to be *rigorously* tested to 
CW> insure that
CW>    a) DLLs built this way could still be linked-to by "regular" code 
CW> that doesn't violate the current limitations.  (e.g. suppose I as the 
CW> cygintl-2.dll maintainer built the next cygintl-2.dll using this spiffy 
CW> new ld.exe; so now, cygintl-2.dll has the extra reloc table.

CW> Q1: will existing code that relied on the OLD cygintl-2.dll (without the
CW> additional reloc table) and does NOT try to access data-with-addend, 
CW> STILL work if I drop in the new DLL?  [e.g. runtime backwards compat]

DLL is not changed in any way! Changes are made in _client_ code. So,
cygintl-*.dll stays as it was, and exports exactly the same symbols as
before. It should be, i believe, byte-to-byte identical to the dll
built with old ld.

CW> Q2: Could I relink old code (that again, does NOT try to access 
CW> data-with-addend) to the new DLL using an OLD ld.exe?  (e.g. an enduser 
CW> of cygintl-2.dll who hasn't updated their binutils)  [e.g linktime 
CW> backwards compat]

If client has no relocations with non-zero addend, i.e. it was
"auto-import-compatible", then nothing changes too. If your old code
could be linked with --enable-auto-import, then my added code is never
called.

CW> Q3: Is the new DLL usable by windows tools, provided a suitable import 
CW> library is generated? (I'm thinking here of mingw folks who build DLLs 
CW> and implibs for use by others with MSVC -- granted, MSVC can't use 
CW> auto-import at all, much less your extention.  But the same linker will 
CW> be used even to build "regular" DLLs...we can't break that.)

DLL is not changing.

>> 1. Make cygreloc generation conditional via --enable-cygwin-reloc or
>> something like that.


CW> At first, yes, it does need to be conditional -- and default to OFF, 
CW> probably...

CW> And, it should probably not be "cygwin" specific.  --enable-data-reloc?

Actually, it's not necessarily data reloc, though with probability of
99%+ it probably-is. Maybe '--enable-runtime-reloc'?

>> 2. If linker creates at least one cygreloc entry, it should emit
>> reference to some external symbol, say 'cygwin_process_cygreloc' so
>> that if object contains non-empty cygreloc vector it'd be
>> guaranteed that it can't be linked with runtime without cygreloc
>> support.

CW> Okay, that takes care of "new style" exe accidentally linking at runtime 
CW> to "old style" DLL.  Still, that leaves compatibility questions about
CW>    existing "old style" EXE ---> "new style" DLL
CW>    linking a "new" old style EXE using the old linker ---> against a 
CW> "new style" DLL

There's no problem. See above.

CW> This chunk of code (in pe-dll.c)

CW> +      if (pe_dll_extra_pe_debug)
CW> +        printf ("creating cygreloc entry for %s (addend=%d)\n",
CW> +               fixup_name, addend);
CW> +      b = make_cygreloc_fixup_entry (name, fixup_name, addend, output_bfd);
CW> +      add_bfd_to_link (b, b->filename, &link_info);

CW> doesn't seem to get called in your example -- but it should, if I 
CW> understand correctly...What's the deal?

Hmm. It should be called, and it called in my case. The resulting
crtest.exe should contain ".data_cygwin_reloc" string. Ah, i see. I
haven't added $(EXE_LDFLAGS) when linking final .exe. Fixed.

CW> Anyway, because I can't see any "creating cygreloc entry..." debug
CW> messages, I'm not quite sure exactly where the cygreloc vector GOES -- 
CW> into the client .o, or into the DLL.  I had assumed the DLL, but your 
CW> point #2 above confuses that issue for me...
(cygreloc -->> addend_reloc?)

It goes to client.

>> 3. Make relocations a bit more flexible by adding type and size
>> (possible 64-bit support?) 


CW> I dunno -- that's a tall order.  This addend-offset problem affects 
CW> structs and arrays -- which come in all SORTS of specific types with 
CW> different field orders and sizes.  Also, what about recursive offsets?

CW> bob = a[2].foo.bar[3].baz ?

Such things are handled by compiler. As far as i understand, all
relocations in coff files are 1) "linear" (base+addend), 2)
independent (i.e. order in which you make them doesn't matter).
I've attached a modified test to demonstrate this.

CW> Granted, fixing 64 bit types (long long), simple arrays, and simple 
CW> structs will go a LONG way to solving the problem in practical terms -- 
CW> but until EVERY case is covered, we still need to detect the failure 
CW> cases and warn at link time (not runtime).

What i was talking about is 64-bit versions of windows where addresses
(and so base symbol values and addends are 64-bit). Or if we want to
add some other types of relocations. Adding type field will make it
possible to add extensions to this feature.

>> Comments?

CW> Nice work so far, but it'll need LOTS of testing and verification, as 
CW> you can well imagine.  Unfortunately, my time will be VERY VERY limited 
CW> over the next six weeks to help with this sort of thing -- or for any 
CW> cygwin-related stuff.  Thesis Defense approaches...

Of course. Luckily, all we have to check is that it doesn't change
binaries for existing code, and then check clients one-by-one.

Egor.            mailto:deo@logos-m.ru ICQ 5165414 FidoNet 2:5020/496.19

Attachment: crtest.c
Description: Binary data

Attachment: crtest.h
Description: Binary data

Attachment: crtest_dll.c
Description: Binary data

Attachment: Makefile
Description: Binary data