Re: Preventing preemption of 'protected' symbols in GNU ld 2.26 [aka should we revert the fix for 65248]

[Rich Felker <dalias at libc dot org>]

On Tue, Apr 19, 2016 at 12:10:58PM -0700, H.J. Lu wrote:
> On Tue, Apr 19, 2016 at 7:06 AM, Michael Matz <matz@suse.de> wrote:
> > Hi,
> >
> > On Tue, 19 Apr 2016, Richard Biener wrote:
> >
> >> So with all this it sounds that current protected visibility is just
> >> broken and we should forgo with it, making it equal to default
> >> visibility?
> >
> > Like how?  You mean in GCC regarding protected as default visibility?  No,
> > that's just throwing out the baby with the water.  We should make
> > protected do what it was intended to do and accept that not all invariants
> > that are true for default visible symbols are also true for protected
> > symbols, possibly by ...
> >
> >> At least I couldn't decipher a solution that solves all of the issues
> >> with protected visibility apart from trying to error at link-time (or
> >> runtime?) for the cases that are tricky (impossible?) to solve.
> 
> Protected visibility is a useful feature.  But as it stands today,
> it is pretty much useless on x86 as seen in ld and ld.so.  We
> have known this defect for a long time, almost from day 1.  To
> make it truly useful, we need to clearly spell out how and when
> it can be used.  We should enforce its limitation in compiler,
> ld and ld.so so that there is no surprise, either for correctness or
> performance,  at run-time.

I believe protected visibility is useful with the current/correct
semantics. At the request of several people who've seen it, I'm
reposting the following reply I just made to the start of this thread,
since my original CC list seems to have omitted a lot of people
currently involved in the discussion:


As one of the strong advocates for the fix that was made to make
protected visibility work correctly with data symbols, I'd like to
explain why it was the right decision and why it matters. This whole
process is really frustrating to me -- having invested a lot of effort
into getting something important fixed, only to have people come
trying to break it again -- but I'm going to try to be calm and not to
snap at anybody. I was only vaguely aware of this thread until a few
days ago (because I've been trying to make new progress on other
things rather than revisit issues I thought were closed), so I'm just
replying to the beginning of this thread. I'll also try to comment on
particularly important points elsewhere in the thread as I read it,
but I don't want to pile on a bunch of scattered replies that focus on
small details I think others have gotten wrong and ignore the big
picture.

>From a programming standpoint, the semantics of protected visibility
need to be free of arch-specific implementation details. Otherwise
programmers can't use it without hard-coding arch-specific details,
which for practical purposes, means good software can't use it at all.

My original motivation for wanting protected visibility to "just work"
was to be able to use:

	#pragma GCC visibility push(protected)

around the inclusion of a library's public headers when including them
from the implementation files, This is far from being the only usage
case, and I'll expand on more important usage cases below, but it is
an important one because it allows you to eliminate all GOT/PLT cost
of intra-library function calls without any fine-grained maintenance
of which declarations to apply visibility too (and without any
GNUC-specific clutter in the header files themselves).

I understand that some people want protected visibility to avoid the
GOT for data symbols too for the sake of performance, but for my usage
case, the fact that the semantics were wrong for data symbols meant
that my configure check for "does protected visibility work" would
return "no", and the whole optimization would get turned off.

Anyway, let's move past optimization, because it's a distraction.
After all, with the old (broken) behavior of protected data, one
_could_ work around the above problem and still get the performance
benefits for functions without breaking data by explicitly declaring
all data with default visibility. In fact, this is how I solve the
problem in musl libc, where there are only a small number of data
symbols that should be externally accessible, and maintaining a list
of them is managable:

http://git.musl-libc.org/cgit/musl/tree/src/internal/vis.h?id=d1b29c2a54588401494c1a3ac7103c1e91c61fa1

This is done for the sake of compatibility with a wide range of
toolchains including ones with the old/broken behavior for protected
data.

The actual documented purpose of protected visibility is to prevent
other definitions of a symbol from taking precedence over the one in
the library itself. For example, suppose you have the following
situation: mainapp depends on libA which defines symbol foo with
normal visibility, and libA depends on libB, which also defines foo,
but intentionally with protected visibility so that libB always uses
its own definition, not the one from libA. There is no reasonable way
to obtain the desired semantics here without the current/correct
behavior for protected data. Any other approaches I'm aware of would
either allow libB to bind to the wrong definition of foo, or would
prevent another main app which links libB (but not libA) from being
able to use the symbol foo from libB.

On the other hand, there are plenty of other ways to get the
old/broken behavior if desired. The easiest is to simply use hidden
visibility when you don't want the symbol to be accessible outside the
library. If you _do_ want it to be visible to and usable by other
shared libraries, just not main-programs, this can be achieved using a
hidden alias for a default-visibility symbol:

	int foo;
	extern int fast_foo __attribute__((__alias__("foo"),
	                                   __visibility__("hidden")));

I expressed that here explicitly for the sake of clarity but of course
in practice people use things like the glibc macro-maze to do this
kind of binding to hidden aliases.

Rich