i386-tdep.c:i386_extract_return_value()

[Mark Kettenis <kettenis at wins dot uva dot nl>]

Hi All,

Right now, i386-tdep.c:i386_extract_return_value() is a mess.  I
wouldn't be surprised if the current implementation is incorrect for
all current supported targets, even for Linux.  There are two issues
here:

1) How are floating point values returned?  According to the GCC
   configuration files (gcc/config/i386.h) there are two
   possibilities: in st(0) or in the integer registers.  There might
   be a third possibility: in memory.  Most i386 targets return
   floating point values on the stack.  GDB however, only implements
   this for AIX and Linux.  That implementation is lossy right now,
   and probably only works correctly for a native GDB.  Returning in
   the integer registers might work for single precision numbers right
   now (I'm not sure), and maybe for double precision on Linux too
   (but that's obviously never used since Linux uses st(0)).  This
   probably means that all targets besides AIX and Linux lose right
   now.

2) How are integers longer than a word (4 bytes) returned?  Again
   according to GCC, most i386 targets return these in the integer
   registers (up to 12 bytes).  Linux implements this for integers up
   to 8 bytes (which is probably sufficient unless the same code would
   be used for returning extended precision floating point values).  A
   few (mostly SVR4 based systems) seem to return these values in
   memory.  I don't know if that's implemented correctly.  Again GDB
   doesn't handle things correctly for most targets.

How can we resolve this?  My suggestion for issue 1) is to add a new
macro that indicates if floats are returned on in st(0) (GCC
uses TARGET_FLOAT_RETURNS_IN_80387, would that be a good name to use
for GDB too?  Or should we drop the TARGET_ prefix?).  We can easily
do a little better in converting to the right virtual type than the
current code does.  Extended precision floating point numbers can
simply be copied, and for double precision an ISO C conforming
`double' should suffice.  We should probably turn this on for all
targets that use the new register layout and support the FPU
registers, and disable it again for those few targets that do not
return floating point values in st(0).

I think that issue 2) can be solved too.  I think that the code isn't
Linux specific at all.  AFAICS all i386 targets use the same registers
to return the values.  And those targets that do not return those
variables by value shouldn't call i386_extract_return_value() at all
(and if they do, they would lose anyway).

Comments?

Mark