Should strip discard the .ctf section ?

[Fangrui Song]

On 2019-10-05, Nick Alcock wrote:
>[sorry about the length of this :( ]
>
>On 3 Oct 2019, Fangrui Song said:
>
>> On 2019-10-02, Nick Alcock wrote:
>>>On 2 Oct 2019, Nick Clifton said:
>>>
>>>> Hi Nick,
>>>>
>>>>> The section is basically useless if strip removes it, so I'd vote no.
>>>>> (That's the entire reason for the focus on size in CTF.)
>>>>
>>>> Ok, well I will shelve the idea for now.  But it may have to be revisited
>>>> if we start to get complaints/bug-reports from users.
>>>
>>>If they're complaining, I need to make CTF smaller :P (plans are afoot:
>>>it is expected to shrink :) ).
>>>
>>>But yes, if they want literally no growth@all... one wonders if they
>>>can ever upgrade anything ever, but still. :)
>>
>> I'd vote for yes. Don't invent more customized rules for
>> --strip-debug/--strip-all. Just let them do natural things that meet
>> users' intuition.
>
>I am very much of the opinion that appeals to intuition are meaningless
>when dealing with things as far removed from end-user interfaces as
>this. Nobody is going to run section-stripping tools of any sort who
>hasn't already learned enough stuff in this area that they can probably
>no longer remember how untrained people think (God knows I can't): their
>intuition will mostly be the product of their training and thus quite
>contingent. This sort of thing is not a mass-end-user tool. It's a
>developer tool, and frankly so rarely used even there that it's probably
>better described as a distro-packager tool. Random developers do not
>often run strip, though they might run a compiler with -s or install -s.
>What runs strip is automated machinery like packaging systems. Many of
>these do not allow the behaviour of strip to be customized much (in
>particular, RPM doesn't).
>
>I would agree with rules like 'more regular behaviour is better', except
>when that involves destroying the very utility of the thing you are
>trying to regularize, and that's what I think stripping out the CTF
>section by default would do.
>
>> Add a generic --keep-section option and let user specify
>> --keep-section=.ctf if .ctf is to be retained. See
>> https://llvm.org/docs/CommandGuide/llvm-objcopy.html#cmdoption-llvm-objcopy-keep-section
>
>(I don't really see the relevance of LLVM documentation here: this is
>not LLVM, this is binutils. Also you're citing objcopy documentation,
>not strip, though llvm-strip has a similar option. Binutils strip *does
>not*. Its defaults are a *minimum* set, and as far as I know you cannot
>say "oh actually I didn't want to remove section X after all", except
>for one special-purpose option for source filename sections, which
>doesn't apply to .ctf.)

llvm-objcopy/llvm-strip have the option but objcopy/strip don't.
I propose that binutils implements --keep-section= .
It will solve not only the current .ctf problem, also future problems of
other sections.

It may be that LLVM binary utilities are not particularly welcomed here.
I mention it because its utilities do not want to diverge from binutils
too much. Yes, courtesy of GNU/LLVM collaboration as the Cauldron
discussed.

>I guess I have to go into more detail about *why* I don't think the
>section should be removed by strip by default. There is an argument from
>user-interface consistency and an argument from utility.
>
>The argument from UI consistency is simple: right now you have to
>explicitly pass the -gt flag to GCC turn CTF generation on, and unlike
>-g this is not in the Autoconf default CFLAGS set. So if you turn -gt
>on, it's because you know you're going to use CTF. It's not very helpful
>to the user to have strip -- and thus essentially all packaging systems
>-- strip it straight back out again by default! Stripping it out by
>default will *always* be a choice that annoys the user, until -gt is
>turned on by default, which is not something I would expect to happen
>until CTF becomes widely used. By that point you wouldn't want to strip
>it out by default because too many programs would be using it, any more
>than you'd want to strip out, say, .eh_frame. This is doubly true given
>that binutils strip has no way to say "oh actually I didn't want to
>strip that section out". Are you proposing that CTF users use llvm-strip
>instead?!
>
>The argument from utility is that the whole point of CTF is that it's
>compact enough that it can be left in binaries, so that programs can
>rely on its being present: if leaving it in is not as uncontroversial as
>leaving in .eh_frame, it is too large. Right now it is about 2% of text
>size, i.e. about the same size as .eh_frame: very soon I expect it to be
>much smaller because that 2% is the size before we're doing any type
>deduplication. Is this *really* too high a price to pay for type
>introspection into all C programs on the system? I am biased here
>(obviously) but I'd vote no, given modern disk sizes (RAM sizes are
>irrelevant, since the section is non-loaded).

.eh_frame has the SHF_ALLOC flag. objcopy --strip* and strip should not strip SHF_ALLOC
sections by default.

Whether .ctf should be stripped by objcopy --strip-debug/objcopy
--strip-all/strip --strip-all/strip --strip-debug depends on whether
.ctf has the SHF_ALLOC flag.

A noteworthy omission from the CTF patch series is that there is no test
of the readelf -S/objdump -h output.

>Note: unlike almost all other non-loaded sections this is *not* debug
>information -- it's not SEC_DEBUGGING -- and it's not just meant to be
>loaded by debuggers. It is type-system information and may well be
>loaded by the running binary on itself. So stripping CTF out is not like
>compiling without -g: it's like compiling a C++ program with -fno-rtti.
>It so happens that the first user of this thing in binutils is GDB, but
>that doesn't mean its *only possible* user is debuggers. I honestly
>suspect non-debuggers might find it more useful in the long run, e.g.
>for introspecting shared libraries in plugin systems. Everything from
>good old CORBA though gobject-introspection has its own model of the C
>type system right now, often requiring icky by-hand C parsing in the
>build system or hand-written not-quite-C type systems that reflect what
>the C plugin code is doing: if CTF can be relied upon to be present, all
>that can go away!
>
>If we *do* strip out CTF by default, no-one will ever use it, because
>packaging systems will routinely strip it out into the debuginfo
>packages so it is never there. I've seen how hard it is to avoid
>stripping this sort of thing out when packaging if strip does it by
>default: when I was putting .ctf sections into kernel modules, I had to
>*patch parts of RPM at runtime* to do it, and I had to do this on a
>package-by-package basis. This is not something any sane person is ever
>going to do, so in practice if strip(1) strips out .ctf, nobody who
>wants their program to work when packaged by any major distro is ever
>going to be able to use it, unless they're writing a debugger.

If .ctf does not have the SHF_ALLOC flag, like DWARF .debug_*, I think
the right behavior here is to add --keep-section=.ctf to the packaging
systems where strip/strip -s is used. This does not require fixing more
packages, because you already have to add -gt to CFLAGS.
Then, I don't think stripping .ctf by default will increase the
deployment cost.

>People who really *need* to save that last 2%-or-less of space can
>always remove the .ctf section explicitly with objcopy if they need to
>do so. (Though in practice all such people will just *not pass -gt* at
>compile time. So I suspect nobody will ever do this.)
>
>> What sections are stripped and what are not are pretty complicated now.
>
>So... surely one extra rule is not a devastating cognitive load, I'd
>have thought. Particularly if the increase in binary size is so small
>that you can ignore it (and if it's not, I'm doing something wrong).

Cognitive load and conceptual integrity is a big one.  The rules of
--strip-all/--strip-debug/--strip-unneeded/(strip defaults to
--strip-all when no --strip* is used)/--only-keep-debug are already
sufficiently complicated. I hoped any further rule addition/refinement
could be well considered. Just follow intuition/general principles and
we will need fewer options to cover every use case.