Re: tracking a dwarf-print --output issue

[Roland McGrath <roland at redhat dot com>]

> things more clear. This is on the dwarf-hacking branch, because on the
> dwarf branch without caching enabled, it will never finish the
> dwarf_output part.

Just to keep clear, note I've rebased roland/dwarf-hacking on top of dwarf
again after the dwarflint merge.  984b82f is what I'm actually using.
None of the libdw code changed, so shouldn't make a difference.

> $ tests/dwarf-print --output src/dwarfcmp
> [... lots of output and then ...]
> /usr/lib/gcc/x86_64-redhat-linux/4.4.4/../../../../include/c
> ++/4.4.4/debug/safe_iterator.h:175:
>     error: attempt to dereference a singular iterator.

I do see this too with my build's src/dwarfcmp.  But I'm using your file
for debugging just to keep us in lock-step.  For reference, as I usually
do, I've done:

	./run.sh tests/dwarf-print --offsets mjw-dwarfcmp > mjw-dwarfcmp.xml

It's helpful to note that "lots of output" is the first compile_unit.
The bug hits in the second CU, which from the dump of the original we see is:

 <compile_unit offset=[0x5058e] producer="GNU C++ 4.4.4 20100726 (Red Hat 4.4.4-13)" language=C_plus_plus name="/home/mark/src/elfutils/libdw/c++/values.cc" ...>

> A "singular iterator" is a "null iterator", one which hasn't been
> initialized to a value, so dereferencing it isn't possible.

Right.  The C++ API representation of a reference attr_value is a
debug_info_entry::(const_)pointer, which is really just a
debug_info_entry::children_type::(const_)iterator.  (In dwarf_edit,
these are the non-const_ ones, since, only there, it's all mutable.)

In dwarf::, this is at base a Dwarf_Die.  
In the others, it is at base some STL container iterator.
In dwarf_edit, it's a std::list iterator with little more wrapping.
In dwarf_output, it's a std::vector iterator with a what looks like
a lot more wrapping but is really just compile-time template hooey
to make the underlying std::vector<die_info_pair *> come out with
a children_type::const_iterator::operator* that yields the debug_info_entry
that is that is pair->first.

> This happens in the wrapped_input_iterator:
> 
> #4  0x000000000040edc0 in
> elfutils::subr::wrapped_input_iterator<std::__debug::vector<elfutils::dwarf_output::die_info_pair*, std::allocator<elfutils::dwarf_output::die_info_pair*> >, elfutils::dwarf_output::debug_info_entry::children_type::deref, elfutils::dwarf_output::debug_info_entry const>::operator-> (
>     this=0x3158738) at /home/mark/src/elfutils/libdw/c++/subr.hh:694
> 
>   template<typename arg_type>
>   inline wrapped_input_iterator (const _base &i, const arg_type &arg)
>     : _base (static_cast<_base> (i)), _m_wrapper (arg)
>     {}
> [...]
>     inline element *operator-> () const
>     {
>       return &(_m_wrapper (_base::operator* ()));
>     }
> 
> So this is just the dereference of i. Which is a vector of
> dwarf_output::die_info_pair pointers. It would be nice if given this
> frame one could easily step into the derefence operator, so one could
> more easily see what is going and what an uninitialized iterator really
> looks like. But I couldn't figure out how to make gdb to that.

When I set a breakpoint on that line (C-x SPC), then "step" took me into

__gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<elfutils::dwarf_output::die_info_pair* const*, std::__norm::vector<elfutils::dwarf_output::die_info_pair*, std::allocator<elfutils::dwarf_output::die_info_pair*> > >, std::__debug::vector<elfutils::dwarf_output::die_info_pair*, std::allocator<elfutils::dwarf_output::die_info_pair*> > >::operator* (this=0x880e78)
    at /usr/lib/gcc/x86_64-redhat-linux/4.4.4/../../../../include/c++/4.4.4/debug/safe_iterator.h:173

which is the _GLIBCXX_DEBUG wrapper.  It a check and then:

	return *_M_current;

step into that is:

__gnu_cxx::__normal_iterator<elfutils::dwarf_output::die_info_pair* const*, std::__norm::vector<elfutils::dwarf_output::die_info_pair*, std::allocator<elfutils::dwarf_output::die_info_pair*> > >::operator* (this=0x880e98)
    at /usr/lib/gcc/x86_64-redhat-linux/4.4.4/../../../../include/c++/4.4.4/bits/stl_iterator.h:699

which is the actual vector iterator dereference operator:

      reference
      operator*() const
      { return *_M_current; }

There:

(gdb) p _M_current
$5 = (
    __gnu_cxx::new_allocator<elfutils::dwarf_output::die_info_pair*>::die_info_pair * const *) 0x87ce60

shows that in the innards of the vector it's just a pointer.

In the bad case, you don't get that far because the _GLIBCXX_DEBUG
wrapper's check bites first.  But you can look before that too:

#0  __gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<elfutils::dwarf_output::die_info_pair* const*, std::__norm::vector<elfutils::dwarf_output::die_info_pair*, std::allocator<elfutils::dwarf_output::die_info_pair*> > >, std::__debug::vector<elfutils::dwarf_output::die_info_pair*, std::allocator<elfutils::dwarf_output::die_info_pair*> > >::operator* (this=0x8831e8)
    at /usr/lib/gcc/x86_64-redhat-linux/4.4.4/../../../../include/c++/4.4.4/debug/safe_iterator.h:173
(gdb) p _M_current
$11 = 
(gdb) 

Ahem.  That there is a pretty-printing fail.  If you're less lazy than me,
got report a gdb bug.  But there is something or other to disable pretty
printing.  I have no idea where it might be documented.  I didn't see
it in "help print" or anywhere.  But I intuit it should be called "raw",
and try:

(gdb) print/r
$12 = {
  _M_current = 0x87dde0
}

which seems to almost work, though I don't know why it didn't print the
field's type normally.  (STL has about three layers of things with a member
called _M_current, so don't get confused.)  At any rate, I don't think you
especially need to grok the innards of STL containers.  It's enough to let
the _GLIBCXX_DEBUG magic tell you what kind of wrong state it's in.

> Although gdb doesn't make it immediately clear this is about the
> attr.second reference() call.

Alex's work on finer-grained line info might eventually improve that, so
you'd see which subexpression (by column number) the call is part of.

> (gdb) whatis attr.second
> type = elfutils::dwarf_output::attr_value
> 
> (gdb) whatis attr.second.reference()
> Cannot evaluate function -- may be inlined

You probably didn't want to do an inferior call as part of examining the
type, anyway!

> I admit that I find this pretty hard to read. My c++ template foo is
> very weak and the round and angle brackets make my head hurt. But this
> is the notation used for template functions. And indeed we have:

The attr_value type dispatch magic is some of the wonkiest contortions for
the C++ type system we have.  I'd never say it was pretty.  The point is
that all the template inline layers just do the right type-checked
down-cast to a value_foo type.

> For the dwarf_output value_reference:
> 
>       struct value_reference : public value_dispatch

Right.  All that variant hooey is just to get you to the right value_*
subclass.

> And there we find out vector subclass that contains die_info_pair*s.
> 
>       class children_type
> 	: public std::vector<die_info_pair *>
> 
> And that has indeed the identity() function we saw at the start:

We don't get that far, because the error happens in operator-> first.
That's really just C++ inanity around operator* (why -> is overloadable
rather than just translated to (*lhs).rhs, I will never know).

> So it seems that what is really going wrong is that there is an
> die_info_pair* element of this vector on which -> identity () doesn't
> work because it is NULL/uninitialized.

Not quite.  value_reference.ref is an iterator into such a vector.
But we don't have any such vector, because the iterator is uninitialized.

> If we look up the stack a bit more we realize that vector is the
> representation of the attributes of the dwarf_output debug_info_entry.

This seems to be a muddle.  There is no vector in the attributes map
representation (it's a std::map, see dwarf_data::attributes_type).
We are indeed here at all because we are walking the attribute map,
as the name prewalk_attrs might imply.  But that's all you have to
think about that at the moment.

> To be continued when I figure out how these uninitialized values are
> represented, so we can catch them earlier when they are put in the
> vector itself.

It doesn't work like that.  _GLIBCXX_DEBUG will give an "attempt to copy
singular iterator" crash if we ever tried to store an uninitialized
iterator object somewhere or pass it around by value.  That's not
what's going on here.

There is no vector.  The reference attribute contains an uninitialized
iterator.  If we back up to frame 4 (prewalk_attr) and look:

(gdb) p attr
$22 = (
    const std::pair<int const, elfutils::dwarf_output::attr_value> &) @0x35cad00: {
  first = 73, 
  second = {
    <attr_value<elfutils::dwarf_output, elfutils::dwarf_output::value>> = {
      _m_value = 0x3158090
    }, <No data fields>}
}
(gdb) p attr.second._m_value
$23 = (elfutils::dwarf_output::value::value_cell_type *) 0x3158090
(gdb) p *$
warning: RTTI symbol not found for class 'elfutils::dwarf_output::copier<elfutils::dwarf>::circular_reference'
$24 = warning: RTTI symbol not found for class 'elfutils::dwarf_output::copier<elfutils::dwarf>::circular_reference'
{
  _vptr.value_dispatch = 0x4d9a90
}

Now we say, "what was that, gdb?"

(gdb) p/a
$25 = warning: RTTI symbol not found for class 'elfutils::dwarf_output::copier<elfutils::dwarf>::circular_reference'
{
  _vptr.value_dispatch = 0x4d9a90 <_ZTVN8elfutils12dwarf_output6copierINS_5dwarfEE18circular_referenceE+16>
}

and C-M-l M-| c++filt RET and we see it's:

$25 = warning: RTTI symbol not found for class 'elfutils::dwarf_output::copier<elfutils::dwarf>::circular_reference'
{
  _vptr.value_dispatch = 0x4d9a90 <vtable for elfutils::dwarf_output::copier<elfutils::dwarf>::circular_reference+16>
}

So GDB couldn't find it, but then when we look up the vptr address, it
found it.  I think I tried to get that bug to happen in some simpler case
and didn't manage it.  So I probably never reported the GDB bug, because I
am a bad person.

Anyway, what we've learned (aside from that it's so distracting to report
GDB bugs!) is that this value_reference is a circular_reference.

(gdb) p (elfutils::dwarf_output::copier<elfutils::dwarf>::circular_reference *) $23
No type "copier<elfutils::dwarf>" within class or namespace "elfutils::dwarf_output".

Sigh.  I don't remember how I beat GDB into submission about that before.

But, anyway, this tells us that the bug is in resolving a circular
reference (no surprise there!).  How that is supposed to work is the heart
of all the real hair in dwarf_output.  Starting to follow all that logic
should generate quite a lot more questions.


Thanks,
Roland