When I was debugging bug #11151, I suddenly couldn't access the markers in libpython anymore. It seems that once the prelink cron job ran, our address computations stop working. I forced-installed the python-libs rpm to reset the prelinking, and things started working again. (diff of good/bad runs to follow)
Created attachment 4519 [details] diff of good/bad runs $ stap -l 'process("/usr/lib64/libpython2.6.so.1.0").mark("*")' -vvvv Before prelink is on the left, after prelink on the right.
(In reply to comment #1) > diff of good/bad runs > +matched probe_name function__return probe_type uprobe at 0x3adacd8394 > +probe_type == uprobe_type, use statement addr: 0x3adacd8394 [...] > +literal_addr_to_sym_addr 0x3adacd8394 > +symbias 0x3adae00000, dwbias 0x3adae00000 > +literal_addr_to_sym_addr ret 0x75b5ad8394 The incoming address looks to be already relocated, but we add a bias to it anyway. The resulting address is of course out of range.
(In reply to comment #2) > (In reply to comment #1) > > diff of good/bad runs > > > +matched probe_name function__return probe_type uprobe at 0x3adacd8394 > > +probe_type == uprobe_type, use statement addr: 0x3adacd8394 > [...] > > +literal_addr_to_sym_addr 0x3adacd8394 > > +symbias 0x3adae00000, dwbias 0x3adae00000 > > +literal_addr_to_sym_addr ret 0x75b5ad8394 > > The incoming address looks to be already relocated, but we add a bias to it > anyway. The resulting address is of course out of range. Could you also post the "before" one (what do the addresses and bias look like in the good case)? We should be storing the addresses "unbiased" in the .probes section and prelink shouldn't relocate them, if we are using literal_addr_to_sym_addr(). literal_addr_to_sym_addr() treats addresses as if given by the user (so raw for ET_EXEC or zero based for ET_DYN).
(In reply to comment #3) > Could you also post the "before" one (what do the addresses and bias look like > in the good case)? It's all there in the diff, but here's the equivalent part extracted... > -matched probe_name function__entry probe_type uprobe at 0xd7cc4 > -probe_type == uprobe_type, use statement addr: 0xd7cc4 [...] > -literal_addr_to_sym_addr 0xd7cc4 > -symbias 0x200000, dwbias 0x200000 > -literal_addr_to_sym_addr ret 0x2d7cc4 > We should be storing the addresses "unbiased" in the .probes section and prelink > shouldn't relocate them, if we are using literal_addr_to_sym_addr(). > literal_addr_to_sym_addr() treats addresses as if given by the user (so raw for > ET_EXEC or zero based for ET_DYN). The "before" case is indeed zero-based, but the "after" case looks raw. Maybe literal_addr_to_sym_addr can look for DT_GNU_PRELINKED and treat such addresses as raw too?
As a workaround, things behave again if you undo the prelink -- simply: $ sudo prelink -u /usr/lib64/libpython2.6.so.1.0
commit 08d1d520616557f6ff7dd023e260ad6577e9e0e8 Author: Mark Wielaard <mjw@redhat.com> Date: Mon Jan 18 09:13:30 2010 +0100 PR11173 Markers get a bad address in prelinked libraries. Our literal_addr_to_sym_addr() function was just wrong. To compensate for raw addresses read from elf (either given by the user or through a mark transformation) we need to know what the elf_bias is (as returned by dwfl_module_getelf) before feeding them to any libdwfl functions. * tapsets.cxx (query_module_dwarf): Always add elf_bias to raw function or statement addresses before calling query_addr(). (query_addr): Don't call literal_addr_to_sym_addr(). * dwflpp.h (literal_addr_to_sym_addr): Removed. * dwflpp.cxx (literal_addr_to_sym_addr): Likewise.