About the dll search algorithm of dlopen

[Michael Haubenwallner]

(and now with patches attached)

On 08/19/2016 06:37 PM, Michael Haubenwallner wrote:
> On 06/01/2016 10:17 PM, Corinna Vinschen wrote:
>> On Jun  1 16:26, Michael Haubenwallner wrote:
>>> On 06/01/2016 01:09 PM, Corinna Vinschen wrote:
>>>> On Jun  1 08:40, Michael Haubenwallner wrote:
>>>>> Hi,
>>>>>
>>>>> two issues with dlopen here (I'm about to prepare patches):
>>>>>
>>>>> *) The algorithm to combine dll file name variants with the search path
>>>>>    entries needs to be reordered, as in:
>>>>>    - for each dll file name variant:
>>>>>    -   for each search path:
>>>>>    + for each search path entry:
>>>>>    +   for each dll file name variant:
>>>>>          check if useable
> 
>>>> However, if you can speed up the search process ignore the
>>>> question...
> 
> Not sure if there's a speedup actually, ...
> 
>>>
>>> This also depends on whether find_exec really is necessary here.
>>
>> Not as such necessary, it's just the function used to search in a
>> search path.  If you want to change that you have to rewrite the
>> same logic again, just reversed.
>>
>> One way around YA code duplication could be some kind of path iterator
>> class which could be used from find_exec as well as from
>> get_full_path_of_dll.
> 
> ... but:
> 0001.patch is a draft for some new cygwin::pathfinder class, with
> 0002.patch adding the executable's directory as searchpath, and
> 0003.patch to search the PATH environment as well.
> 
> Thoughts?
> Any idea about different template nesting to be useful somewhere else?
> 
>>>>> *) The directory of the current main executable should be searched
>>>>>    after LD_LIBRARY_PATH and before /usr/bin:/usr/lib.
>>>>>    And PATH should be searched before /usr/bin:/usr/lib as well.
>>>>
>>>> Checking the executable path and $PATH are Windows concepts.  dlopen
>>>> doesn't do that on POSIX systems and we're not doing that either.
>>>
>>> Agreed, but POSIX also does have the concept of embedded RUNPATH,
>>> which is completely missing in Cygwin as far as I can see.
>>
>> RPATH and RUNPATH are ELF dynamic loader features, not supported by
>> PE/COFF.
> 
> In any case, to me it does feel quite important to have the (almost) same
> dll search algorithm with dlopen() as with CreateProcess().
> 
>>> However, when dlopen is about to search some path list, I can imagine to
>>> search the list of already loaded dlls first as well, but I'd prefer to
>>> leave this up to LoadLibrary...
>>
>> This problem would only occur if dlopen is not called with a path.  If
>> the given pathname is a plain filename, we could simply add a call to
>> GetModuleHandle and if it succeeds, return the handle (after checking
>> for RTLD_NODELETE).
> 
> But indeed this might lower the need for same search algorithms...
> 
>>> Side note:
>>> We also use some cl.exe/link.exe wrapper that supports LD_PRELOAD,
>>> LD_LIBRARY_PATH, embedded RUNPATH, as well as lazy loading for both
>>> LoadLibrary and CreateProcess: https://github.com/haubi/parity
>>> Basically I'm wondering why Cygwin doesn't provide that (yet?)...
>>
>> We discussed implementing our own dynamic loader once, but gave up due
>> to workload.  Parity is LGPLed and thus can't be included into Cygwin
>> itself.
> 
> 've mentioned Parity more as proof-of-concept than for inclusion,
> and accepting workload as reasoning.
> 
> Let's see if I can live without the ELF dynloader features.
> 
> Thanks!
> /haubi/
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-dlopen-search-each-name-within-one-single-search-dir.patch
Type: text/x-patch
Size: 28707 bytes
Desc: not available
URL: <http://cygwin.com/pipermail/cygwin-developers/attachments/20160819/911236b9/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-dlopen-search-main-executable-s-directory.patch
Type: text/x-patch
Size: 1348 bytes
Desc: not available
URL: <http://cygwin.com/pipermail/cygwin-developers/attachments/20160819/911236b9/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0003-dlopen-search-PATH-environment-variable.patch
Type: text/x-patch
Size: 852 bytes
Desc: not available
URL: <http://cygwin.com/pipermail/cygwin-developers/attachments/20160819/911236b9/attachment-0002.bin>