[RFC] libio FILE/vtable hardening ideas against FSOP-style attacks

[Adhemerval Zanella Netto]

On 22/06/26 13:52, Paul Eggert wrote:
> Thanks for your email. Although I'm by no means an expert in that particular area, here's my kneejerk reaction for what it's worth.
> 
> On 2026-06-22 03:42, 류대호 wrote:
>> Does validating the wide vtable path with IO_validate_vtable seem like a reasonable hardening direction?
> 
> Given that we also validate the primary vtable path, validating the wide vtable path sounds like a good idea that would fix a real security issue. I suggest writing up a patchset to address this issue. I assume there would be zero overhead in the typical (non-wide) case.

Agreed, the wide vtable path (_IO_WIDE_JUMPS_FUNC) is a known unguarded
deference and I recall seeing some blog posts using this as an attack
vector (most likely the referenced Hour of Apple).

> 
> 
>> Is exact vtable start- address checking too strict for glibc, or could it be useful in some limited paths? Is a FILE metadata registry fundamentally incompatible with libio, or is it worth exploring as an optional hardening idea?

If I understood correctly, it would be an 'offset % sizeof (struct _IO_jump_t) == 0'
extra check on IO_validate_vtable. It would eliminates sub-vtable shifting and 
enforces method-type-correctness.

Performance-wise, it would be an extra constant modulo for each FILE operation 
(multiply-high + shift + compare / ~3–4 dependent ALU ops), which seems cheap; and we 
can make it even cheaper if we align 'sizeof(struct _IO_jump_t)' vtable entry to a
power of 2 (at the cost of extra RELRO usage).

The gadget set per call site collapses from every function pointer in the array to 
the <=14 legitimate implementations of that one method, and it stays type-correct 
(an overflow dispatch can only land on something that was authored as an overflow
handler).

However, checking the House of Apple 2/3 descriptions online, they both rely 
on the missing wide vtable hardening (so execution could be routed to heap-allocated 
vtable/_IO_codecvt). So it is not clear to me whether, with the wide vtable hardening
in place, we will still have gadgets that can be exploited with similar methods.


Now, I am not sure about the registry-based integrity test. It most likely requires 
adding even more complexity to libio (registry allocation/deallocation, whether it 
would be managed inline or outlined in internal FILE metadata, fork semantics, and 
extra analysis to ensure it does not add extra attack surface).

It will need to review the proposed solution before commenting further.

> 
> My guess is that these would cost too much validation overhead (and be of too little practical effect) to be acceptable in ordinary use. Maybe as an option, though we should get more-expert advice on that.
> 
> If you go ahead with a patch, we'll also need copyright papers signed to satisfy legal requirements before we can accept it. But you should probably wait a bit first, to see what others think about this idea.
>