dwarf_output

[Roland McGrath <roland at redhat dot com>]

> My understanding is that dwarf_edit won't do any compression at all. 

Right.  dwarf_edit will stay about like it is.  
It won't really be used for anything in the writer.

I really wrote it just as the simple proof of concept for the idea of
having elfutils::dwarf mirrored by a template-compatible class that can be
copy-constructed from any other such template-compatible class.  In the
future it may serve (perhaps with scriptable language front-end bindings)
for various little programmatic purposes.  Its feature is that it has
"standard" data structures, i.e. all its containers are just mutable STL
container things with no magic going on behind the scenes.  

Because of that, and because the common template interfaces are all the
"logical view" (i.e. separately mutable duplicates of identical things), it
uses huge amounts of memory for nontrivial amounts of DWARF data.  So it
will never be of direct use in general transformation tools that work on
real-world DWARF files.

It continues to provide a useful reference and test, which is why we want
dwarfcmp -T tests to check it.  We have a template interface that covers a
consuming version and a mirror producing version.  Having two
implementations of the producer side helps keep us honest on what the
template interface is.  Each producer side (dwarf_edit, dwarf_output) also
meets the consumer-side interface, so the -T code tests that too.  That
acts as a sort of preemptive dry run of the "read it in, write it out,
dwarfcmp" test cycle for the writer (the dry run tests the data-structure
layers, but not the actual format composition).

> dwarf_output (or rather the collector) will use dwarf_edit to construct 

dwarf_output and the collector are intimately tied together, but do not use
dwarf_edit.  Its possible that some inner classes of dwarf_edit will happen
to be implementations useful inside dwarf_output or the collector, but the
dwarf_edit class itself won't be used at all.

> trees such that shareable sub-trees are referenced through imported_unit 
> DIEs.  That's done recursively, so a referenced tree can itself contain 
> references.  

I think in the actual construction of dwarf_output, all that will be done
is to collect all the unique trees in the collector.  Actually generating
imported_unit entries will be part of the output phase.

Indeed, the construction is by recursive copying of the DIE tree.  That's
how the consolidation of identical copies happens: from the leaves back up
to the whole CU (dwarf_output::compile_unit object).

> I've looked that DIE iterators already implement 
> "flattening" of trees (or un-referencing) including imported unit.

Right, that's the "logical view" I've mentioned.  The producer side classes
(dwarf_edit and dwarf_output) have only the logical view, and not the "raw"
physical view that the libdw-reader class (dwarf) provides via its raw_*
containers.  In dwarf_edit constructed from dwarf, that means you get
automatic explosion of all compressed forms into fat copies (and lots of
memory).  In dwarf_output, it will mean automatic consolidation of
duplicates, so in a sense there is an optimal "physical view" held inside
the collector.

> I spent some time yesterday to make dwarf_output compilable for 
> starters.  It's not doing anything yet, it's just a bunch of "stumps". 
> I don't think it's worth committing, but I've put that on 
> pmachata/dwarf_output branch just in case.  Collector is not there yet.

Thanks.  I'll pick that up and merge bits into roland/dwarf-collector.
I hope to make some more progress on that soon.


Thanks,
Roland