[PATCH v2] libio: use PTR_MANGLE/PTR_DEMANGLE for FILE vtables

[Florian Weimer]

On 10/15/2015 05:31 PM, Joseph Myers wrote:
> On Thu, 15 Oct 2015, Florian Weimer wrote:
> 
>> The key point is that in GCC 2.95, you could introduce additional
>> vtables by deriving from std::streambuf (or std::filebuf and the other
>> classes).
>>
>> I think we should proceed with some form of vtable hardening.  I think
>> the main choice we face is:
>>
>>   Do we want backwards compatibility with binaries which use
>>   custom vtables with glibc's libio?
> 
> Does that mean only binaries that passed such derived classes directly to 
> glibc functions, not any binaries that simply made use of standard C++ 
> features?

All binaries which used the old libio-based libstdc++ would be affected.

>> If the answer is “no”, I suggest we apply Kees' patch *and* remove the
>> libio symbols listed only in <libioP.h> (technically an ABI break), so
>> that affected applications break in a well-defined way, instead of
>> crashing with a segmentation fault.
> 
> Would that still keep working (a) C++ binaries, whether using shared or 
> static libstdc++, built with any official GCC release supporting glibc2, 
> where those binaries did not pass C++ classes to C interfaces,

No, they would no longer work.

> and (b) C 
> binaries built against any glibc version (i.e. none of the libioP.h 
> interfaces have ever been used through stdio.h interfaces in glibc 2.0 or 
> later, and none are used by old libstdc++.so)?

Yes, for libioP.h (the private, non-installed header).  We may have
overlooked something, and code which uses <libio.h> directly may be
affected by changes (certainly if we proceed to clean up things and
devirtualize some of the functions).  But it has been suggested on this
thread that struct _IO_FILE is internal (along with the support
functions in <libio.h>) although it is an installed header file.

> (I'm not concerned with binaries built with non-default configurations of 
> GCC 3.0 or later, which restricts compatibility concerns to architectures 
> supported in 2.95 with ABIs still supported in glibc.  I think that means 
> alpha, i386 32-bit, m68k, mips 32-bit, powerpc 32-bit, sparc 32-bit and 
> 64-bit.)

We have libstdc++ compat libraries for ia64, s390 and s390x, too.

Florian