Should strip discard the .ctf section ?

[Rich Felker]

On Mon, Oct 07, 2019 at 04:22:37PM +0100, Nick Alcock wrote:
> On 7 Oct 2019, Rich Felker said:
> 
> > On Sun, Oct 06, 2019 at 01:25:51PM -0400, Hans-Peter Nilsson wrote:
> >> On Sat, 5 Oct 2019, Nick Alcock wrote:
> >> 
> >> ....
> >> > By that point you wouldn't want to strip
> >> > it out by default because too many programs would be using it, any more
> >> > than you'd want to strip out, say, .eh_frame.
> >> ....
> >> 
> >> ITYM .debug_frame.  Keeping .eh_frame is quite necessary for the
> >> program to work correctly, if or when an exception is thrown.
> 
> That was my point. If a program's types are being introspected (by
> itself, or by something else) using libctf, the .ctf section is *also*
> necessary for the program doing the introspection to work correctly. I'm
> trying to get across that this is not necessarily a debugger, maybe a
> correctness issue, and strip cannot tell. I have multiple non-debugger
> use cases lined up (admittedly hypothetical so far). C++ RTTI isn't a
> debugger feature: nor is typeof(). Nor is this, not intrinsically,
> though obviously I hope debuggers find it useful.
> 
> .... hm now I'm wondering if we could implement ctf_dynamic_cast() :)
> 
> > It should be noted that DWARF unwind information has two different
> > sections that can be used, .eh_frame and .debug_frame, depending on
> > whether it's present as part of the loadable program (for exception
> > handling/introspection) or purely as debug information, with
> > corresponding SHF_ALLOC flag status.
> >
> > I suspect the same should be done for CTF. There will be users who
> > want to use it for introspection, and users who want to use it as
> > lightweight debug information.
> 
> Sure, except that these probably have the same content, or at least I
> can't see how they would differ. I hope the current .ctf section is
> useful for both use cases (and if it's not, I want to know, so the next
> format revision, format v4, can fix it so it is!).

Likewise the content of .eh_frame and .debug_frame does not differ. If
you build with unwind information, the content ends up in .eh_frame
which is part of the loadable program segments. If you build without
it, but with -g, the content ends up in .debug_frame. If you build
with neither, it's absent entirely. Debuggers know to use either
.eh_frame or .debug_frame. My view is that CTF should match.

> >                                Forcing the latter to include it in
> > load segments, making it unstrippable, will make many of us unhappy to
> > have it at all.
> 
> I think we already do what you want here. .ctf is already a non-loaded
> section,

If it's a non-loaded section, programs cannot use it to introspect. If
you want to have that functionality, you need a variant where it's
loaded.

> so there is more-or-less zero memory hit from having .ctf
> present unless libctf loads the section in at runtime using bfd. This is
> *why* libctf can load in .ctf sections using bfd, and why the default
> ctf_open() does just that :)

This is not following ELF semantics correctly. Program behavior
absolutely should not depend, ever, on details of the file outside of
the loadable segments. If you're going to keep the CTF data anyway,
there is no cost (except some virtual address space, which I guess
could matter for huge programs on 32-bit archs, but if the CTF is that
huge then disk space matters too) to having it be loadable. But it
should be a choice.

> I have had people asking me to make .ctf a loaded segment, but in
> addition to having the same qualms as you about loading this section in
> and eating memory before it is in widespread use, I can see no way to do

The idea that it "eats memory" is incorrect.

> such a thing without a radical linker rearchitecture. The section is
> usually compressed, so its size depends on its content: and its content
> depends on the content of the ELF string and symbol tables, which GNU ld
> lays out only after the size of all loaded sections is known. So we
> cannot make .ctf a loaded section. At least not yet.

I think this still needs major work then.

> .... I really must submit the spec, since it might make much of this
> stuff clearer. It's not quite complete (no documentation of older file
> formats or the libctf API yet): is that a problem, or can I add that
> incrementally?

Tooling should not be committing to new features for which
compatibility will need to be kept basically-forever until there is a
clear spec and until issues like the above are worked out.

Rich