libffi doesn't keep the stack aligned to 16 bytes

[Neil Roberts]

Hi

GCC appears to assume by default that the stack is always aligned to 16
bytes when entering a function. Presumably this is to make it easier to
use the aligned access instructions to load data into SSE
registers. This means that if you have a variable with the aligned(16)
attribute then GCC knows that it doesn't have to add an extra
instruction to align the stack pointer because it will already be
aligned.

However, libffi doesn't appear to respect this. When it calls a function
it allocates space on the stack by decrementing the stack pointer
according to the amount of space required. This means the depending on
the number of arguments the stack may or may not be aligned.

This ends up causes crashes when calling libraries that use -mfpmath=sse
because then GCC will try to store temporary SSE variables on the stack
and it will assume it can use aligned access instructions.

Attached is a test case which demonstates the problem and a patch to fix
it. We are using this patch in MeeGo because all of the libraries are
compiled with SSE math so we were getting crashes with gjs.

Regards,
- Neil

-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-ffi.c
Type: text/x-csrc
Size: 1149 bytes
Desc: Test case
URL: <http://sourceware.org/pipermail/libffi-discuss/attachments/20100701/bdc45a01/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-x86-Align-the-stack-to-16-bytes-before-making-the-ca.patch
Type: text/x-diff
Size: 1215 bytes
Desc: Patch to fix
URL: <http://sourceware.org/pipermail/libffi-discuss/attachments/20100701/bdc45a01/attachment-0001.bin>