sdt.h vs comdat

[Roland McGrath <roland at redhat dot com>]

There was another issue in sdt.h needing revisiting, that I forgot about in
the "sdt.h revamp".  That's probes in code that goes in COMDAT section,
which happens in some C++ situations.

There is mention of this in PR10512 and testsuite/systemtap.base/cxxclass.cxx
is supposed to be related to that.  But I can't really tell what that
example is intended to test.  No test for a COMDAT case can really be
testing the crux of the COMDAT issue unless it links multiple files together,
which that test does not.

Here's an example:

	$ cat c++-sdt.cc
	#include "includes/sys/sdt.h"
	#include <iostream>

	struct fooclass
	{
	  void method ()
	  {
	    STAP_PROBE2(foo, method, 17, this);
	  }
	  virtual ~fooclass ()
	  {
	#ifndef frob
	    STAP_PROBE2(foo, destructor, 27, this);
	#else
	    std::cout << "fmh\n";
	    STAP_PROBE2(foo, destructor, 666, 666);
	#endif
	  }
	};

	void frob ()
	{
	  fooclass x;
	  std::cout << &fooclass::method << "\n";
	}

	int main () {}
	$ g++ -g -O1 -c c++-sdt.cc
	$ g++ -g -O1 -c c++-sdt.cc -o c++-sdt2.o -Dfrob=diddle -Dmain=loser 
	$ g++ -o c++-sdt c++-sdt{,2}.o
	`.text._ZN8fooclassD2Ev' referenced in section `.probes' of c++-sdt2.o: defined in discarded section `.text._ZN8fooclassD2Ev[_ZN8fooclassD5Ev]' of c++-sdt2.o
	`.text._ZN8fooclassD0Ev' referenced in section `.probes' of c++-sdt2.o: defined in discarded section `.text._ZN8fooclassD0Ev[_ZN8fooclassD5Ev]' of c++-sdt2.o
	collect2: ld returned 1 exit status
	$

So, the current sdt.h doesn't handle it either.  I guess all the talk in
PR10512 was just speculative, since we don't have anything like the
__attribute__ ((section)) way of defining the data mentioned there.

The issue here is that the code for those methods will be emitted 
in each of those two compilations, in COMDAT sections.

	$ eu-readelf -S c++-sdt.o | grep _Z | c++filt
	[ 2] _ZN8fooclassD5Ev     GROUP        0000000000000000 00000048 0000000c  4       47  30  4
	[15] .text._ZN8fooclass6methodEv PROGBITS     0000000000000000 000048dc 00000002  0 AXG    0   0  2
	[18] .text._ZN8fooclassD2Ev PROGBITS     0000000000000000 00004a8c 00000009  0 AXG    0   0  2
	[19] .rela.text._ZN8fooclassD2Ev RELA         0000000000000000 0000ed60 00000018 24       47  18  8
	[22] .text._ZN8fooclassD0Ev PROGBITS     0000000000000000 00004aa0 00000016  0 AXG    0   0  2
	[23] .rela.text._ZN8fooclassD0Ev RELA         0000000000000000 0000ed90 00000030 24       47  22  8
	[26] .rodata._ZTV8fooclass PROGBITS     0000000000000000 00004ae0 00000020  0 AG     0   0 32
	[27] .rela.rodata._ZTV8fooclass RELA         0000000000000000 0000edc0 00000048 24       47  26  8
	[28] .rodata._ZTI8fooclass PROGBITS     0000000000000000 00004b00 00000010  0 AG     0   0 16
	[29] .rela.rodata._ZTI8fooclass RELA         0000000000000000 0000ee08 00000030 24       47  28  8
	[30] .rodata._ZTS8fooclass PROGBITS     0000000000000000 00004b10 0000000a  0 AG     0   0  1
	$ c++filt _ZN8fooclass6methodEv
	fooclass::method()
	$ c++filt _ZN8fooclassD0Ev
	fooclass::~fooclass()
	$

Both files have those sections, but in the linked program only one set will
survive.  Anything the compiler emits associated with that code goes into
COMDAT sections in the same COMDAT group, so they are discarded together.
But our inline asm uses explicit .section directives to select the .probes
(or .note.stapsdt) section, and those don't specify the COMDAT group.  So
we wind up with our probe data fragments referring to addresses in the
COMDAT text sections that have been discarded.  That's what that link error
means.

The right way to handle this is to put our probe sections into the right
COMDAT group.  That way, when the text is discarded, the records of probes
in that text (and hence that we don't actually have any more) are discarded
too.  The trouble is communicating that through the compiler and assembler.

As PR10512 mentions, C/C++ data declarations using __attribute__ ((section))
implicitly do the right thing.  You say section "foo", and the compiler
actually emits COMDAT section "foo" in the COMDAT group of the containing
function.  That's lovely, but there is no way to use a data declaration to
produce a non-allocated section.  So we could theoretically use it for
.probes style (which is an allocated section), but then we'd both go back
to the text bloat that .note.stapsdt avoids, and we'd have to use a
different method of avoiding generating PIC relocs (which might be hard
to manage with C initializers).

In http://sourceware.org/ml/binutils/2010-08/msg00121.html I proposed an
assembler feature for this.  That would let us write:

	.pushsection .note.stapsdt,"?","note"

and that "?" means to make it "G" and append ",whateveritis,comdat"
if the current section was a COMDAT section.  That solves it quite
nicely.  (I don't know yet what gas maintainers think about the idea,
but the implementation was trivial so it might well go in.)

Unfortunately, it is pretty difficult to see how we can write an sdt.h that
could conditionalize its code on what assembler version you're using.  So
making of use of this would have to wait a long time, or be enabled at
configure time (hard-coding a macro setting into the installed sdt.h)
for people whose installed sdt.h will be used only with the right assemblers.

I can also imagine some GCC feature for this in the inline asm handling.
That would be something like a new % syntax in asm strings that
conditionalizes on whether the current code is comdat-bound, and
substitutes in the group name.  But I wouldn't know how to implement that,
and it would require more help from the compiler folks.  If that got done,
it would be reasonable enough for sdt.h to use gcc version conditionals.
That way an installed sdt.h can be compatible with today's compilers as
long as probes are not actually placed in discarded COMDAT text sections
(the status quo for .probes support), but can support even those situations
robustly when using a newer compiler.


Thanks,
Roland