Re: dwarf_output overview

[Roland McGrath <roland at redhat dot com>]

> The tracking through stack allocated tracker::reference_match structs
> which clear sets under investigation is sometimes a bit hard to follow.

Right.  Destructors are the cleanest way in C++ to maintain state that is
tied to control flow, but it can be nonobvious stepping through.

> Especially to find the recursive step and how it relates to the actual
> die tree walking. I believe I only half understand how it works after
> staring at it for some time.

Please ask more questions!  I tried to comment the parts that were really
nonobvious at the time.  But certainly I was too deep in it then to notice
how much nontrivial had become obvious to me.  Far less of it is still
obvious to me after a few months.  For anything we tease apart together,
we can add more comments to keep it more clear.

The dwarf_output version of everything is where the real horrible hair was,
and what I was really working on the hardest.  That has the "debug () <<"
bits scattered throughout:
#if 0  // Toggle this to enable massive debugging spew during construction.

> The testwriter -T variants of this test fail with:
> terminate called after throwing an instance of 'std::logic_error'
>   what():  basic_string::_S_construct NULL not valid
> 
> Because of the throw in entry_copier::populate

That is a re-throw, so the actual exception came from deeper in the soup.
In gdb, "catch throw" or "break __cxa_throw" are highly recommended.

> Still running, it doesn't seem to trigger in any of the plain C self
> tests.

Ok.  As well as me not really remembering which cases were which, it's
further complicated by the fact that we are using a newer compiler now.
It could well be that some of the quirks I was seeing no longer show up
in exactly the same ways.  But, my vague recollection is that none of
the really nasty bits had shown up in the C-only cases.

> Mine seems to not be zillions, but "only" 1393 CUs. But how do you test
> such a beast? Are you actually able to run dwarfcmp on your vmlinuz and
> see it finish in any reasonable time?

Yes, well that counts as zillions when it's this slow. ;-)
No, runs on vmlinux have always been slower than is reasonable.
In the current state (as with the C++ examples), it might well
not be terminating.  But it's certainly extremely slow.

> And do any of these ever finish a dwarfcmp for you?
> I tried for a couple of hours, but never saw any of these larger C++
> examples finish. Not even a "small" one like dwarflint.

With the current code, no, I think it doesn't.  This is the state I
described in that big "brain dump" posting from a few months back, which
I've referred you to before.

Note this bit (dwarf_tracker):

      /* XXX not reliable!
	 This match could be predicated on a tentative match of a
	 circular ref inside.  We can't cache that!

This hits on the question of the ref-chasing algorithm.

Because of this being disabled, I expect it to be unreasonably slow.  It
repeats many ref-chasing walks many times.  But, it really appears never
to complete at all.  This I don't understand.  So one tack on this is to
really lay out more formally what the current algorithm is (with no
caching here).  I did expect it to be fairly pessimal, but I also
thought it was correct--i.e., that it would certainly terminate.
This seems not to be so.

Of course, it's a bit questionable how useful it really is to be chasing
down how wrong the current hopelessly-slow mechanism is.  Perhaps it is
more useful to lay out a correct algorithm and then make the implementation
conform to that.  I don't know.

> I'll try. If not possible with c/c++ examples having handwritten asm
> versions would be nice. Not having simple examples has been the most
> frustrating issue for me. Using anything non-trivial quickly makes me
> loose track of what I am really trying to figure out...
   ^s/o//

I understand.  I have really just been too lazy to deal with
hand-assembled DWARF for test cases.  I've always figured that
eventually (i.e. when the writer is done), I'd implement a DWARF->DOM
layer to plug into some XML parser/printer component, and thereafter we
would be able to write nice concise XML trees of DWARF for hand-composed
test cases.  But I've just been working with the data at hand.  In each
case I worked on, I got to where I was only thinking about one
particular subtree and its quirks, and just got to know all the relevant
DIE offsets in my data, etc.  But that does rely on what's apparently a
talent of mine, to ignore reams of nearby information and focus just on
the cross-section of interest.  Not that I don't frequently confuse
myself in the process, of course.


Thanks,
Roland