Re: MIPS _mcount-based profiling doesn't work with -abicalls (2.95.3)

[Daniel Jacobowitz <drow at mvista dot com>]

On Wed, Nov 14, 2001 at 03:43:27PM -0500, Rafal Boni wrote:
> [I've CC'ed binutils in case any of you folks there have an opinion, since
>  the things that's biting me really is assembler magic, in a sens...]
> 
> Folks:
> 	I've been doing some work to upgrade the NetBSD MIPS toolchain to
> 	2.95.3 and binutils 2.11.2, and other than some issues I need to
> 	take up with the binutils folks, most things works fine.
> 
> 	However, I've found that executables built with profiling libraries
> 	(which were all rebuilt with the new toolchain) all die way before
> 	doing anything substantial -- somewhere in the startup code.
> 
> 	I tracked the problem down to a bad interaction between the -abicalls
> 	option and the profiling prologue added to functions.
> 
>         Here's what happens:
> 
>                 (1) A function that calls _mcount and uses .cprestore looks
>                     like the following in macroized assembler:
> 
>                         .globl  function
>                     function:
>                         .cpload $25
>                         addiu $sp, $sp, -40
>                         .cprestore 36
>                         ...
>                         sw $gp, 36($sp)
>                         ...
>                         <load $t9 with address of _mcount>
>                         jalr $t9            # _mcount
>                         subu $sp, $sp, 8    # _mcount pops 2 works from stack
> 
> 
>                 (2) The assembler in turn expands the `jalr' bit into 
> 
>                         jalr    $t9         # _mcount
>                         nop                     
>                         lw      $gp, 36($sp)
>                         subu    $sp, $sp, 8
> 
>                 Note that due to the way the .cprestore places the load of 
>                 $gp after the jump (but before the cleanup of $sp), the code
>                 gets the wrong value of $gp.
> 
> 	This is all because the FUNCTION_PROFILER macro includes some hand-
> 	scheduled MIPS assembly with the stack-adjustment put into the delay
> 	slot of the `jal'.
> 
> 	I include here two patches for this problem.  The first is just a 
> 	simple re-schedule of the FUNCTION_PROFILER macro which does the
> 	stack adjust before the `jal' always.  This certainly works, and
> 	is the safer of the two solutions, but may cost some performance
> 	in the non-abicalls call where this is not needed.

This is a solution.  It's not enough of a solution, though.  I'm fairly
certain that GCC's MIPS _mcount ABI originally comes from RiscOS or
something of that time period, from the archaeology I did.  That was
before abicalls and PIC, and the jal had a delay slot to use :)

So this patch is right.  I submitted a similar one several months ago. 
It's wiser to remove .set noreorder and let the assembler cope.

> 	The second patch attempts to keep the hand-scheduled code for the
> 	non-abicalls case and produce the safer code for the abicalls case.
> 
> 	I'd appreciate comments esp. on the second patch -- is this worth
> 	doing, or am I asking for some other case to screw me? 

The second patch is not acceptable.  At least PIC can also expand the
jal.

You'll have another problem.  If you try to compile a nested function
(C monstrosity that they are) you'll die in the inner one.  GCC will
push $v0 onto the stack, jal (which the assembler will try to expand
into a GP restore, from the incorrect stack pointer), and then pop $v0. 
This can also happen with (I think) structure returns.

I suppressed the stack push/pop entirely and changed my C library to
save $v0 in _mcount.  I posted at least the GCC parts of this, too.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer