Bug 23928 - ar crash within LLVMgold.so module
Summary: ar crash within LLVMgold.so module
Status: UNCONFIRMED
Alias: None
Product: binutils
Classification: Unclassified
Component: binutils (show other bugs)
Version: 2.31
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-11-28 08:44 UTC by Werner Lemberg
Modified: 2018-12-12 10:44 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Werner Lemberg 2018-11-28 08:44:14 UTC
ar crashes while calling the LLVMgold.so plugin – this far I was able to the debug the issue...

I've already filed an openSuSE bug report at

  https://bugzilla.opensuse.org/show_bug.cgi?id=1117239

together with an MWE to reproduce the issue.
Comment 1 Nick Clifton 2018-11-28 11:30:56 UTC
Hi Werner,

> ar crashes while calling the LLVMgold.so plugin – this far I was able to the
> debug the issue...
> 
> I've already filed an openSuSE bug report at
> 
>   https://bugzilla.opensuse.org/show_bug.cgi?id=1117239
> 
> together with an MWE to reproduce the issue.

If it is the LLVMgold.so plugin that is actually triggering the crash then
there is nothing that we can do - it is an LLVM problem.  Well unless ar is
calling a function within the plugin and passing it bogus arguments.  But
I think that this is unlikely.

I did try to reproduce the problem on my local machine, but nothing went
wrong - ar completed its task.  I did try modifying the command line
in trigger-ar-crash.sh to include an explicit invocation of the LLVMgold.so
plugin, but it still worked.  It is possible however that the version of
the gold plugin that I have installed is too old.  (It comes from the
llvm-libs-6.0.1-8.fc28.x86_64 rpm available on Fedora 28).  Hence it may
be that this is an openSUSE specific problem...

Are you able to determine more about where the crash is happening and what
exactly is triggering it ?

Cheers
  Nick
Comment 2 Werner Lemberg 2018-11-28 14:50:16 UTC
I installed all necessary debug symbols; however, the very last call is still shown as `??' in the backtrace (deep in dlopened stuff).  What do you recommend to make ar and LLVMgold.so spit out more debugging information?

BTW, what I would like to have for such situations is an option to disable plugins with a command line option, for example `--plugin=""'.

Finally, where can I find a description of the differences between `LLVMgold.so' and gcc's `liblto_plugin'?  Even a more intensive search in the internet didn't show up anything useful...
Comment 3 H.J. Lu 2018-11-28 17:43:31 UTC
Works for me on Fedora 29:

$ ./trigger-ar-crash.sh
ar: /export/home/hjl/bugs/binutils/pr23928/ar-bug/libs2/libz.so.1: no version information available (required by ar)
$ rm libs2/libz*
$ ./trigger-ar-crash.sh
$
Comment 4 Nick Clifton 2018-11-29 11:30:01 UTC
Hi Werner

> I installed all necessary debug symbols; however, the very last call is
> still shown as `??' in the backtrace (deep in dlopened stuff).  

What about the function before the '??'.   That will probably give you a 
clue as to where the problem is occurring.


> What do you
> recommend to make ar and LLVMgold.so spit out more debugging information?

Recompile them with the -g debugging option enabled.  (I am guessing that
they were compiled without it, at least in the environment that you are using).


> BTW, what I would like to have for such situations is an option to disable
> plugins with a command line option, for example `--plugin=""'.

Sadly I do not believe that such an option exists.  You can however capture
the command line being used to invoke the plugin, and then just replay it
with the plugin options removed.


> Finally, where can I find a description of the differences between
> `LLVMgold.so' and gcc's `liblto_plugin'?  Even a more intensive search in
> the internet didn't show up anything useful...

I do not think that such a document exists.  The only real recourse is to
read the source code for each of them, and compare how they work.

Cheers
  Nick
Comment 5 Werner Lemberg 2018-12-11 23:33:43 UTC
> > I installed all necessary debug symbols; however, the very last call is
> > still shown as `??' in the backtrace (deep in dlopened stuff).  
> 
> What about the function before the '??'.   That will probably give you a 
> clue as to where the problem is occurring.

Well, here's the topmost output of `bt full':

#0  0x0000000000010d40 in ?? ()
No symbol table info available.
#1  0x00007ffff7dea1ea in call_init (l=<optimized out>,
                                     argc=argc@entry=5,
                                     argv=argv@entry=0x7fffffffd9b8, 
                                     env=env@entry=0x7fffffffd9e8) at dl-init.c:72
        j = <optimized out>
        jm = <optimized out>
        addrs = <optimized out>
        init_array = <optimized out>
        j = <optimized out>
        jm = <optimized out>
        addrs = <optimized out>
#2  0x00007ffff7dea2d3 in call_init (env=0x7fffffffd9e8,
                                     argv=0x7fffffffd9b8,
                                     argc=5,
                                     l=<optimized out>)
    at dl-init.c:30
        init_array = <optimized out>
        init_array = <optimized out>
        j = <optimized out>
        jm = <optimized out>
        addrs = <optimized out>
#3  _dl_init (main_map=main_map@entry=0x621a90,
              argc=5,
              argv=0x7fffffffd9b8,
              env=0x7fffffffd9e8) at dl-init.c:120
        preinit_array = <optimized out>
        preinit_array_size = <optimized out>
        i = 4
#4  0x00007ffff7dee718 in dl_open_worker (a=a@entry=0x7fffffffb078) at dl-open.c:564
        args = 0x7fffffffb078
        file = <optimized out>
        mode = <optimized out>
        call_map = <optimized out>
        dst = <optimized out>
        new = <optimized out>
        __PRETTY_FUNCTION__ = "dl_open_worker"
        r = <optimized out>
        reloc_mode = <optimized out>
        nmaps = <optimized out>
        l = <optimized out>
        maps = <optimized out>
        relocation_in_progress = <optimized out>
        any_tls = <optimized out>
        first_static_tls = <optimized out>
#5  0x00007ffff7dea0a4 in _dl_catch_error (objname=objname@entry=0x7fffffffb068, 
                                           errstring=errstring@entry=0x7fffffffb070,
                                           mallocedp=mallocedp@entry=0x7fffffffb067, 
                                           operate=operate@entry=0x7ffff7dee410 <dl_open_worker>,
                                           args=args@entry=0x7fffffffb078) at dl-error.c:187
        errcode = 32767
        c = {objname = 0x7fffffffb068,
             errstring = 0x7fffffffb070,
             malloced = 0x7fffffffb067, 
             errcode = 0x7fffffffaf54,
             env = {{__jmpbuf = {140737488335224, -4992220596282808245, 2147483650, 6396560,
                                 140737488345528, 5, -4992220595683022773, -4992238454207240117},
                     __mask_was_saved = -134257408, 
                     __saved_mask = {__val = {4160710744, 0, 140737354097920, 140737354097920, 2,
                                     7, 140737353841376, 140737351926218, 140733193388853,
                                     140737344040720, 140737354097920, 140737488335016, 
                                     140737488335012, 0, 5, 140737354097920}}}}}
        catchp = 0x7ffff7ffdfa0 <data>
        old = <optimized out>
#6  0x00007ffff7dede7b in _dl_open (file=0x619a90 "/usr/bin/../bin/../lib/bfd-plugins/LLVMgold.so", 
                                    mode=-2147483646,
                                    caller_dlopen=<optimized out>,
                                    nsid=-2,
                                    argc=5,
                                    argv=0x7fffffffd9b8,
                                    env=0x7fffffffd9e8) at dl-open.c:649
        args = {file = 0x619a90 "/usr/bin/../bin/../lib/bfd-plugins/LLVMgold.so",
                mode = -2147483646, 
                caller_dlopen = 0x7ffff78be9af,
                caller_dl_open = 0x7ffff70a8eeb <dlopen_doit+91>,
                map = 0x621a90, 
                nsid = 0,
                argc = 5,
                argv = 0x7fffffffd9b8,
                env = 0x7fffffffd9e8}
        objname = 0x7ffff765ef10 ""
        errstring = 0x7ffff7ff6500 ""
        malloced = false
        errcode = <optimized out>
        __PRETTY_FUNCTION__ = "_dl_open"
[...]
#11 0x00007ffff78be9af in try_load_plugin (pname=<optimized out>,
                                           abfd=0x615530,
                                           has_plugin_p=0x7fffffffb37c) at ../../bfd/plugin.c:228
[...]

The address of function #1 looks very suspicious...

> > What do you recommend to make ar and LLVMgold.so spit out more debugging
> > information?
> 
> Recompile them with the -g debugging option enabled.  (I am guessing that
> they were compiled without it, at least in the environment that you are
> using).

Uh, oh.  This definitely costs too much time, sigh.

> > BTW, what I would like to have for such situations is an option to disable
> > plugins with a command line option, for example `--plugin=""'.
> 
> Sadly I do not believe that such an option exists.  You can however capture
> the command line being used to invoke the plugin, and then just replay it
> with the plugin options removed.

`Plugin options removed'?  How shall this work?  As far as I can see,
`ar' automatically uses LLVMgold.so if present – I can't prevent that!
Compiling without plugin support enabled is obviously not helpful in finding the
bug...

> > Finally, where can I find a description of the differences between
> > `LLVMgold.so' and gcc's `liblto_plugin'?  Even a more intensive search in
> > the internet didn't show up anything useful...
> 
> I do not think that such a document exists.

Pfft.

> The only real recourse is to read the source code for each of them, and
> compare how they work.

Honestly, I can't believe that I have to do that.  Sigh.
Comment 6 Nick Clifton 2018-12-12 10:44:48 UTC
Hi Werner,

> Well, here's the topmost output of `bt full':

> #1  0x00007ffff7dea1ea in call_init (l=<optimized out>,
> dl-init.c:72
> #2  0x00007ffff7dea2d3 in call_init (env=0x7fffffffd9e8,
>     at dl-init.c:30
> #3  _dl_init (main_map=main_map@entry=0x621a90,
[etc]

OK, so to me this looks like the failure is occurring inside the dl_open()
C library call.  Which in turn implies that there is something wrong with
the plugin itself.

My guess as to why this is happening is that the plugin that you are using
was compiled against a different set of sources to the ones that were used
to build ar.  So symbols might be in the wrong places and so on.  Are you 
able to determine how the plugin was built ?

I think at this point that you need to refer this bug to either the openSUSE
support people, if the plugin is being built incorrectly, or else the LLVM
bug list, as it currently looks like a bug in the plugin rather than in ar.

Cheers
  Nick