RFC: GCC plugin to find encrypted function pointer calls in glibc

[Jeff Law]

On 04/29/2016 07:21 PM, Carlos O'Donell wrote:
> On 04/29/2016 06:23 AM, Aldy Hernandez wrote:
>> Since I would prefer not to assume the output of all inline asm's are
>> a demangling operation, I would like to get feedback from the
>> community on what would be preferred.
>
> Awesome looking work!
Agreed.  So one of the higher level questions is where should this (and 
follow-up) plugins live?  This one seems to be fairly glibc specific 
right now(*), so ISTM it ought to live in glibc in the immediate term.

The downside is if we muck things up in GCC-land and break the plugin we 
won't immediately know.  But if the plugin lives on the GCC side it's a 
lot less likely to get used regularly.

And assume that there'll be additional plugins over time where the 
plugin has some domain specific knowledge of constraint/invariant y'all 
want to enforce.  So building a little bit of infrastructure to utilize 
those plugins seems wise.

>
>> My preferred approach is to add an attribute to an inline function that would wrap the asm:
>>
>>     __attribute__((decrypt)) static inline funcp demangler (funcp f)
>>     {
>>         asm("blah");
>>     }
>>
>> This is straightforward, clean, and follows language semantics (not
>> to mention that I already have it implemented into my plugin :)), but
>> Florian made funny faces when I showed it to him, so here I am :).
>
> You can build glibc with gcc 4.7 or newer.
>
> (1) static inline wrapper with function attribute:
>
> For a representative set of architectures, say x86_64, i686, ppc64,
> s390x, aarch64, and arm, does a 4.7 or newer compiler do as good a
> job with a static inline function as it does with the macro that
> evaluates to a bare assembly?
It should, and if it doesn't for a simple case like this, I'd certainly 
treat it like a big.

>
> (2) Asm attribute:
>
> We have function, variable, type, label, and enumerator attributes,
> why not support them on asm statements to mark them as decrypt or
> encrypt functions? This would lend itself to a more natural conversion
> of inline assembly that users may want to write and embed into their
> programs.
>
> It is entirely conceivable that (1) does a good-enough job, and we
> add (2) at the same time to the latest gcc and backport. When we raise
> the supported build compiler version we can switch to (2) if needed.
GCC doesn't have any way to annotate expressions/statements in this way. 
  We can annotate types, functions and variables.

jeff

(*) We have discussed recommending other projects use function pointer 
encryption and if they do, then this plugin would be useful to those 
projects.  But that's future work ;-)