Hi, I added i686-pc-mingw32 as target triplet, although this is in fact a platform independ bug (but since dlltool is only useful on win32 ...). A general EXPORTS line in the .def file is: exportedname [[= internalname] @ ordinal [flags]] If the internalname is not given, then normally the same name as the exportedname is used. However, one can specify the -U commandline parameter to dlltool to add an underscore (so that internalname become _exportedname) and one can specify -k, which removes the stdcall @<n> part if any (for example, when exportedname is "foobar@4" then internalname becomes "foobar"). However, the current dlltool implementation contains a bug that allows one to specify the internal name itself in .def file - as should be possible according the documentation of the .def file. Because both -U and -k seem to be kludgy fixes, I think that in the case of an explicitely given internalname, this translation should not take place. I wrote a patch and tested it (it works ;). I will attach the patch. Note that this fix is especially important on mingw because gcc does not add the leading underscore to stdcall functions. That means that a .dll can export the internal name "_foobar@4" while gcc applications try to call "foobar@4". When generating the stub for the import library we NEED a line foobar@4 = _foobar@4 in the .def file. Unless there are ONLY stdcall functions being exported there is no other way to create a correct import library (when there are only stdcall functions then one can the -U option).
Created attachment 177 [details] Patch that fixes the problem that internal names specified in the .def file are ignored.
Hi Carlo, Thanks for locating this bug and submitting a patch. I have looked it over and it seems to be the right thing to do, so I have created a ChangeLog entry for it and checked it in. Unfortunately I do not know your last name... Cheers Nick binutils/ChangeLog 2004-09-02 Carlo <carlo@alinoe.com> PR binutils/351 * dlltool.c (make_one_lib_file): For IDATA6 take the name from exp->internal_name if it is present.
Subject: Re: [PATCH] dlltool doesn't honour aliases in .def files. On Thu, Sep 02, 2004 at 03:53:39PM -0000, nickc at redhat dot com wrote: > for it and checked it in. Unfortunately I do not know your last name... My name is "Carlo Wood" the one and only (http://tinyurl.com/3qf4k) ;-)
Subject: Re: [PATCH] dlltool doesn't honour aliases in .def files. Hi Carlo, >>for it and checked it in. Unfortunately I do not know your last name... > My name is "Carlo Wood" the one and only (http://tinyurl.com/3qf4k) So it is :-) I have updated the ChangeLog entry with your surname. Cheers Nick
Heya, there turns out to be a bug in the patch that I submitted. exp->internal_name is always non-NULL because it is already set to whatever exp->name is in def_exports. As a result, now xlate() is never called for idata$6 entries, disabling the effect of --kill-at and --add-underscore. I propose to fix this by using the same test as is being used in gen_def_file, which generates the '= internalname' part only when exp->internal_name and exp->name are different. So, the code then should become: char const* internal_name = (strcmp (exp->name, exp->internal_name) == 0) ? xlate (exp->name) : exp->internal_name; instead of char const* internal_name = exp->internal_name ? exp->internal_name : exp->name; It is not 100% trivial though, because one should realize that when one does: foo@4 bar@8 = _bar@8 boo@4 = boo@4 and then uses --kill-at, then the internal name used for foo@4 becomes 'foo' (as per --kill-at), the internal of bar@8 becomes _bar@8 because here --kill-at is ignored, but counter intuitive, the internal name of boo@4 becomes 'boo' still. It seems more logical to also not call xlate in that case; then a boolean has to be added to struct export in order to remember if 'internal_name' was given explicitely or not. Which solution does have your preference?
On the mingw-users mailinglist, Aaron W. LaFramboise <aaron77thyme at aaronwl dot com> writes me: > No. The internal name has a well-defined meaning, that is correctly > implemented by the Microsoft tools. The internal name refers to the > name of the symbol within the implementation of the DLL, a name that is > meaningless to users of the DLL. Your patch changes this meaning (but > only within dlltool) to mean some other symbol name, which is incorrect. > It will cause a well-formed .def file to create an incorrect import > library. Specifically, it is incorrect for dlltool to do anything with > the internal name, since neither of the two symbol names emitted by > dlltool is the one referred to by the internal name. (That does not > mean that internal name is generally ignored or unused. It is used, > correctly, by some projects during DLL creation.) Any change, such as > yours, which changes this, is inconsistant with correct operation, and > incompatible with MS's tools. > > I think that you should revert your patch. Although I am not sure I understand what he is talking about exactly, I guess that he knows what he is talking about. Perhaps the whole patch is doing the wrong thing because I misunderstood the meaning of 'internal name'. It is unfortunate that there is no good documentation available about .def files and dlltool etc.
Danny Smith had an even better fix: /* If the user has specified an internal name, use it. Ignore command line name translation options. */ char const * internal_name = exp->internal_name != exp->name ? exp->internal_name : xlate (exp->name); This uses '!=' instead of strcmp(). That will work because exp->internal_name is set literally equal to exp->name when no internal name is given. And it probably solves the counter intuitive foo@4 = foo@4 problem too by STILL ignoring --kill-at in that case :). After some more discussion, I am not convinced that Aaron is right by the way. So now either this patch has to be reverted completely (according to Aaron) or you have to make the above change in order to re-enable to handling of --kill-at again (according to me). I'll leave the decision to you, you are the maintainer after all :p
Oh, that fix *is* already in the CVS tree http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/binutils/dlltool.c.diff?r1=1.52&r2=1.53&cvsroot=src&f=h Sorry, I thought the mingw maintainer had added it. Well, since I am the one who re-opened this PR, I will close it again then now (if that will work).
Ack. Ok reopened again. After a long long discussion on this topic on the mingw-users list (which doesn't have a usable archive for long threads, only single mails) we reached the point that everyone agrees and understands the subject ;). Allow me to include my last post here: On Tue, Sep 07, 2004 at 03:24:12AM -0500, Aaron W. LaFramboise wrote: > Due to some recent confusion on this matter, and the general lack of > clear examples in general, I have created a small example that shows how > the internalname element is correctly used in a DEF file. Ok, thanks. Now I understand the real meaning of 'internal name'. All the confusion came from the fact that I interpreted "exported name" as whatever was visible to the application at linktime: whatever is exported from the .dll.a, and not what is exported from the .dll. As a result, I interpreted 'internal name' as what is exported by the .dll. In order to get rid of any further confusion, let me summarize what means what from now: 'internal name' == symbol name in test.o 'exported name' == symbol exported from test.dll 'imported name' == symbol exported from test.lib (and test.dll.a), and imported by main.o (the final executable). And a .def file contains the syntax: exported_name = internal_name which is only to be used while generating the .dll from its .o. The problem we have then is that gcc doesn't add an underscore to imported stdcall names, while dlltool has no other means then --add-underscore to fix that problem. This means that 1) The meaning of the symbols in the .def are CHANGED into 'imported name' 2) There is no way to fix this problem when there are also non-stdcalls symbols, without an underscore being exported from the .dll. Let me clarify point (1). What --add-underscore does is add an underscore to the name found in the .def file and use the resulting name as exported name. This means that it treats the names in the .def file as 'imported names' and not as exported names. This is probably the main reason for my mistake when trying to understand what is what. Although it isn't really necessary to revert the patch in binutils (because the .def file is "customized" already as it is all the time) I do agree that it isn't very nice to change the meaning of 'internalname' in the .def file just like that, for dlltool. Nevertheless - we need a solution for the missing underscore when compiling with gcc. Like Aaron already said (and Wu too), it is probably best to not break the legacy behaviour we have. A solution should be one that only has effect when a new option is being used. There two possibilities: 1) We add a new option to dlltool that tries to fix the problem with heuristics. 2) We add a new option to dlltool that causes it to treat the exported names as imported names (still) and additionally treat the internal names as exported names, instead of ignoring them. Basically, my patch but with a command line option to enable it. The heuristics could be: - A new option that: Disable --kill-at and --add-underscore (because both treat the names in the .def as imported names). - Ignore the internal names and treat the exported names as exactly that (this means that no more filtering of .def or output of pexports is needed). - Generate imported names by removing leading underscores from exported stdcall symbols. What do you think?
Subject: Re: [PATCH] dlltool doesn't honour aliases in .def files. Nick, are you THE maintainer of dlltool.c ? I'd like to have your opinion on a few things, before I waste time writing a patch that is wrong again ;). I would like to write the patch in steps and get your feedback on each of the steps. Is that ok with you?
Subject: Re: [PATCH] dlltool doesn't honour aliases in .def files. Hi Carlo, > Nick, are you THE maintainer of dlltool.c ? No. I am a maintainer, but not the only one. There are several people who can approve patches to dlltool. > I would like to write the patch in steps and get your feedback > on each of the steps. Is that ok with you? Certainly, but there is one small problem - I am leaving on holiday tomorrow and will not be back for a couple of weeks. In the meantime I would recommend that you post questions and patches to the other binutils mailing list (binutils@soruces.redhat.com) as that is a much more active list. Cheers Nick
Subject: Re: [PATCH] dlltool doesn't honour aliases in .def files. I'll wait till you get back.
Adding the following email here, for archival purposes. On Wed, Sep 29, 2004 at 10:07:41PM +0200, Filip Navara wrote: > I'm working on getting the Wine (www.winehq.com) build tool to generate > only one flavor of .def files (sharable between MSVC and MinGW) that are > later used while linking DLLs. Then I'd like to urge you to use the format that MSVC uses. That means that a __stdcall function either looks like: _foo@16 or foo = _foo@16 Note that from both you can deduce that the original is a __stdcall. Apparently everything is assumed to be a __cdecl at all times (isn't it?). I never looked at __fastcall's. > [snip] > >No idea - I am neither the ld nor the dlltool maintainer. > >But it seems that you are not doing anything that is duplicating > >what I had in mind. However, you better know _exactly_ what you > >are doing... it is confusing enough as it is without patches that > >do things that are not very well thought over :p > > Basically I want to do this in .def files: > EXPORTS > foo=_foo@16 > as in MSVC instead of the MinGW way, where I have to do: > EXPORTS > foo=foo@16 > (note the missing underscore) ok, looks the Right Way to go. But you also should do: EXPORTS _foo@16 instead of EXPORTS foo@16 where appropriate. > It may seem like a little nuisance that can be dealt with using standard > tools (like "sed" or extending the Wine build tool), but I thought I'll > take a stub at fixing binutils to accept the MSVC-style .def files first > before trying to workaround it. I think it goes a lot deeper (than 'sed'). I don't know which posts of me you already read; but just to be sure let me repeat a few things. The correct picture of how things work is this: Notation: GNU/MSVC foo.cc/foo.cpp/foo.h : char __stdcall foobar(int hello); // __stdcall is a macro on GNU that only adds an attribute to the function. foo.o/foo.obj : foobar@4/_foobar@4 // Plus an extra underscore that seems to be always there and never shown except // This is the 'internal name'. foo.def : xyz = foobar@4 / xyz = _foobar@4 // Right-hand-side matches internal name. foo.dll : xyz // Whatever is on the left-hand-side of the '=' sign in the .def file. // This is the 'exported name'. foo.a.dll/foo.lib : xyz@4/_xyz@4 // This is what is *should* be, but no tool will generate this // they see the source file (header declaration) too and there // generate: foobar@4/_foobar@4. Hence, the 'imported names' // also need to be called 'foobar'. So, lets make the .def file more practical: foo.def : foobar = foobar@4 / foobar = _foobar@4 // Right-hand-side matches internal name, left-hand-side // matches the 'exported name'. OR foo.def : foobar@4/_foobar@4 // Matches internal name and exported name. // In practise there will be a mismatch if you use a mixture // of MSVC and GNU though (ie, using foobar@4 on GNU might // still mean that the internal name is _foobar@4). foo.dll.a/foo.lib : foobar@4/_foobar@4 // What I call 'imported name' (although this is what the archive exports). // This is the same as the internal name IF BOTH USE THE SAME HEADER FILE // (declaration). But because the .dll might be compiled with another // compiler there might be a mismatch of underscore. So, we assume that the application source also contains char __stdcall foobar(int hello); and the object files: app.o/app.obj : foobar@4/_foobar@4 // With a possible underscore mismatch with the internal name. // When GNU is used with one and MSVC on the other. This MUST match // the 'imported name' however: .dll.a must contain foobar@4 and // a .lib will contain _foobar@4, as they do (see above). Now - what were MY posts all about in regard of the above? That was about generating the foo.dll.a with dlltool from the .def. Suppose you don't have a foo.dll.a but you DO have a MSVC .def. Then the .def contains: foobar = _foobar@4 (A) OR _foobar@4 (B) and you need to generate a foo.dll.a that contains exported name 'foobar' (A) or '_foobar@4' (B) but imported name 'foobar@4' (see foo.dll.a above). How can the dlltool do this? The way it works AT THE MOMENT is a broken concept imho; it works like this: You have (manually) create a .def file that contains: foobar@4 = whatever OR just foobar@4 Note that this .def file has nothing to do with the original .def file. The left-hand-side is now the 'imported name', horrible. That you use the commandline option (of dlltool) --add-underscore, and it will use '_foobar@4' as 'exported name'. Done. Done? No, not if you also have functions in that .dll that are not __stdcall's. My proposal is now to add a new commandline option to dlltool that will work as follows: 1) You don't have to change the .def anymore. So we have: foobar = _foobar@4 (A) OR _foobar@4 (B) Then dlltool will deduce the 'exported name' by taking the left-hand-side (foobar in case A and _foobar@4 in case B) and construct the 'imported name' by *adding* an underscore to __stdcall functions, as well as adding the @<n> part. This means, that if we have: xyz = _foobar@4 then it will generate _xyz@4 as 'imported name'. And when you have just xyz then it will generate 'xyz' as 'imported name'. [...] Carlo Wood
Bah - that paste is totally screwed up for some reason :/ Here is a new attempt - sorry about that: On Wed, Sep 29, 2004 at 10:07:41PM +0200, Filip Navara wrote: > I'm working on getting the Wine (www.winehq.com) build tool to generate > only one flavor of .def files (sharable between MSVC and MinGW) that are > later used while linking DLLs. Then I'd like to urge you to use the format that MSVC uses. That means that a __stdcall function either looks like: _foo@16 or foo = _foo@16 Note that from both you can deduce that the original is a __stdcall. Apparently everything is assumed to be a __cdecl at all times (isn't it?). I never looked at __fastcall's. > [snip] > >No idea - I am neither the ld nor the dlltool maintainer. > >But it seems that you are not doing anything that is duplicating > >what I had in mind. However, you better know _exactly_ what you > >are doing... it is confusing enough as it is without patches that > >do things that are not very well thought over :p > > Basically I want to do this in .def files: > EXPORTS > foo=_foo@16 > as in MSVC instead of the MinGW way, where I have to do: > EXPORTS > foo=foo@16 > (note the missing underscore) ok, looks the Right Way to go. But you also should do: EXPORTS _foo@16 instead of EXPORTS foo@16 where appropriate. > It may seem like a little nuisance that can be dealt with using standard > tools (like "sed" or extending the Wine build tool), but I thought I'll > take a stub at fixing binutils to accept the MSVC-style .def files first > before trying to workaround it. I think it goes a lot deeper (than 'sed'). I don't know which posts of me you already read; but just to be sure let me repeat a few things. The correct picture of how things work is this: Notation: GNU/MSVC foo.cc/foo.cpp/foo.h : char __stdcall foobar(int hello); // __stdcall is a macro on GNU that only adds an attribute // to the function. foo.o/foo.obj : foobar@4/_foobar@4 // Plus an extra underscore that seems to be always there and // never shown except in hex dumps. // This is the 'internal name'. foo.def : xyz = foobar@4 / xyz = _foobar@4 // Right-hand-side matches internal name. foo.dll : xyz // Whatever is on the left-hand-side of the '=' sign in the .def file. // This is the 'exported name'. foo.a.dll/foo.lib : xyz@4/_xyz@4 // This is what is *should* be, but no tool will generate this // they see the source file (header declaration) too and there // generate: foobar@4/_foobar@4. Hence, the 'imported names' // also need to be called 'foobar'. So, lets make the .def file more practical: foo.def : foobar = foobar@4 / foobar = _foobar@4 // Right-hand-side matches internal name, left-hand-side // matches the 'exported name'. OR foo.def : foobar@4/_foobar@4 // Matches internal name and exported name. // In practise there will be a mismatch if you use a mixture // of MSVC and GNU though (ie, using foobar@4 on GNU might // still mean that the internal name is _foobar@4). foo.dll.a/foo.lib : foobar@4/_foobar@4 // What I call 'imported name' (although this is what the archive // exports). This is the same as the internal name IF BOTH USE THE // SAME HEADER FILE (declaration). But because the .dll might be // compiled with another compiler there might be a mismatch of // underscore. So, we assume that the application source also contains char __stdcall foobar(int hello); and the object files contain: app.o/app.obj : foobar@4/_foobar@4 // With a possible underscore mismatch with the internal name. // When GNU is used with one and MSVC on the other. This MUST match // the 'imported name' however: .dll.a must contain foobar@4 and // a .lib will contain _foobar@4, as they do (see above). Now - what were MY posts all about in regard of the above? That was about generating the foo.dll.a with dlltool from the .def. Suppose you don't have a foo.dll.a but you DO have a MSVC .def. Then the .def contains: foobar = _foobar@4 (A) OR _foobar@4 (B) and you need to generate a foo.dll.a that contains exported name 'foobar' (A) or '_foobar@4' (B) but imported name 'foobar@4' (see foo.dll.a above). How can the dlltool do this? The way it works AT THE MOMENT is a broken concept imho; it works like this: You have (manually) create a .def file that contains: foobar@4 = whatever OR just foobar@4 Note that this .def file has nothing to do with the original .def file. The left-hand-side is now the 'imported name', horrible. That you use the commandline option (of dlltool) --add-underscore, and it will use '_foobar@4' as 'exported name'. Done. Done? No, not if you also have functions in that .dll that are not __stdcall's. My proposal is now to add a new commandline option to dlltool that will work as follows: 1) You don't have to change the .def anymore. So we have: foobar = _foobar@4 (A) OR _foobar@4 (B) Then dlltool will deduce the 'exported name' by taking the left-hand-side (foobar in case A and _foobar@4 in case B) and construct the 'imported name' by *adding* an underscore to __stdcall functions, as well as adding the @<n> part. This means, that if we have: xyz = _foobar@4 then it will generate _xyz@4 as 'imported name'. And when you have just xyz then it will generate 'xyz' as 'imported name'. [...] Carlo Wood
OOPS! I meant this: [...] Then dlltool will deduce the 'exported name' by taking the left-hand-side (foobar in case A and _foobar@4 in case B) and construct the 'imported name' by *removing* the underscore to __stdcall functions, as well as adding the @<n> part. This means, that if we have: xyz = _foobar@4 then it will generate xyz@4 as 'imported name'. And when you have just _foobar@4 then it will generate 'foobar@4' as 'imported name'. Also, when you have: _foobar then it will generate '_foobar' as imported name (not remove the underscore) because it isn't a stdcall.
Recently I needed to import a function from a dll, but under different name in import library. I thought this would work: LIBRARY mydll.dll EXPORTS visible_func = name_from_mydll -or- name_from_mydll = visible_func So I could have my own function named 'name_from_mydll', and still being able to call function from mydll.dll using 'visible_func' as its name. Why currently make_one_lib_file() doesn't use internal_name at all? Well, I understand the terminology that EXPORTS in .def is used for exporting functions, internal_name meaning internal name in dll's source, and we want to use the same file for exporting from original dll and for creating import library, but then there's simply no means to rename imported functions. Could this behaviour be changed, with something like the patch (with corrections from discussion), but controlled with a switch on command line?
This is a common misunderstanding regarding the meaning of the internal name. The internal name has no effect on an import library, and this is the correct behavior, and this is consistant with all existing practice, and it is improper to change the meaning of the internal name to something else. Simply, the internal name affects the name of the symbol internal to the DLL, which is irrelevant to import libraries. This reflects the historical mindset of DEF files being part of the library source, not part of the client source. The traditional import library generator has no way to accomplish what is desired here. I have added the -p option to dlltool that allows hooking functions exported from DLLs, which should suit most needs. Please let me know if there are any problems with this. See the long thread on mingw-users at <http://sourceforge.net/mailarchive/message.php?msg_id=9468480> about this, and an example of the proper meaning of the internal name at <http://sourceforge.net/mailarchive/message.php?msg_id=9470225>. Since sf.net does not appear to archive attachments, I will attach the latter example of proper internal name semantics to this bug.
Created attachment 557 [details] Demonstration of the semantics of the internal name. binutil's present behavior is correct, as demonstrated by this test case.