Re: dwarf_output overview

[Mark Wielaard <mjw at redhat dot com>]

On Fri, 2010-08-06 at 15:21 -0700, Roland McGrath wrote:
> > Yes, I saw that example. I need to figure out where/how this is handled
> > in practice in the current scheme. Do you happen to have an actual
> > example of when this kind of situation arises?
> 
> See dwarf_tracker: dwarf_ref_tracker::reference_matched.  
>
> The "plain vanilla circularity" was handled by the "We have a circularity
> on the left-hand side." part.  When the asymmetrical kind came up, I added
> the "Our right-hand side is not in lock-step on a matching circularity."
> part there.

The tracking through stack allocated tracker::reference_match structs
which clear sets under investigation is sometimes a bit hard to follow.
Especially to find the recursive step and how it relates to the actual
die tree walking. I believe I only half understand how it works after
staring at it for some time.

> Where this came up was in dwarfcmp-test.  That is, the comparison running
> on an original dwarf reading a file vs a dwarf_output made from that.
> I don't recall what file the first test case was, but it came up in some of
> the files from the build itself, so 'make check' (run-dwarfcmp-test.sh)
> hit it.

The testwriter -T variants of this test fail with:
terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_S_construct NULL not valid

Because of the throw in entry_copier::populate

> I can't get current GCC to do it in trivial cases off hand.  But the
> situation was that it would emit <pointer_type type=sameref/> several times
> in the same CU.  It might be that current GCC doesn't do that as much, but
> I think it was happening in complex cases where it was sort of
> "understandable".  i.e., it might make sense that for those simple DIEs GCC
> didn't keep track of having emitted them before.  So what I'd say to try to
> see it is play with dwarfcmp-test on src/* files (only the final links work
> so far, not .o's).  If you breakpoint or just comment out the whole second
> part of the tracking in reference_matched (the part maintaining _m_active),
> you should see the problem hit.

Still running, it doesn't seem to trigger in any of the plain C self
tests.

> > It seems to me that if we can somehow embed/hash in the structure/cycles
> > better the number of comparisons we need to do to find duplicate
> > subgraphs should decrease a lot. But I probably need to do some more
> > testing. What would be some good representative samples to look at?
> 
> I don't know about representative.  My methodology has been to start with
> the all-C binaries in the elfutils build itself first.  (Really that's just
> what's easiest to put into 'make check'.)  Second is vmlinux (I used my
> random build, but you can get a whole one in any kernel-debuginfo rpm too),
> which is also just "simple" C code, but sometimes more complex, and also a
> good test in handling a zillion CUs.

Mine seems to not be zillions, but "only" 1393 CUs. But how do you test
such a beast? Are you actually able to run dwarfcmp on your vmlinuz and
see it finish in any reasonable time?

> Third is the C++ binaries in the
> elfutils build (C++ DWARF is *way* more complex in reference graphs
> usually), first dwarflint is a smaller one, then dwarfcmp is pretty hairy.

And do any of these ever finish a dwarfcmp for you?
I tried for a couple of hours, but never saw any of these larger C++
examples finish. Not even a "small" one like dwarflint.

> Our testing plan (see wiki bits) is to use the entire history of Fedora and
> RHEL debuginfo rpms as representative data.  But that's for big test runs
> once we are correctly handling all the local test data readily on hand.
> 
> Beyond the really trivial cycles (like any linked-list struct type),
> I have not found it very easy to come up with small sources that produce
> more interesting DWARF graphs.  But I haven't really tried all that hard.
> I always just settled into some particular corner of weirdness in the
> dwarflint or dwarfcmp binaries that I found by testing, and chugged away on
> debugging handling those particular subtrees.
> 
> If you have any luck coming up with hand-written C/C++ code that produces
> more interesting DWARF trees for test cases, then I will certainly put some
> time into making it easier to maintain those in the elfutils sources to be
> used as test data for 'make check'.

I'll try. If not possible with c/c++ examples having handwritten asm
versions would be nice. Not having simple examples has been the most
frustrating issue for me. Using anything non-trivial quickly makes me
loose track of what I am really trying to figure out...

Cheers,

Mark