Re: ucontext functions on powerpc64

[Adhemerval Zanella <adhemerval dot zanella at linaro dot org>]

On 13/02/2019 14:11, A. Wilcox wrote:
> Dear glibc developers,
> 
> I noticed that since glibc 2.5, powerpc32 can use the swapcontext
> syscall on the Linux kernel for most of the ucontext family of functions
> - it was added in kernel 2.6.0.  This was actually made unconditional in
> 2012 for glibc 2.17.
> 
> The powerpc64 architecture does not use the Linux swapcontext syscall.
> I was wondering if anyone knew the reason behind this.  I did a mailing
> list archive search for "ucontext powerpc64" and "swapcontext powerpc64"
> and saw no relevant threads, just threads about adding initial support
> in 2002, and adding VSX extensions.
> 
> For background on this query, I'm trying to add support for ucontext
> functions to an embedded C library.  Some applications are crashing when
> calling setcontext on powerpc64 using the Linux swapcontext syscall.  It
> works fine on powerpc32.  I was wondering if perhaps the syscall is
> somehow unusable for this purpose on powerpc64, since glibc appears to
> be avoiding the syscall.  If not, then it may be a slight performance
> optimisation for glibc to use it instead of using rt_sigreturn.
> 
> I apologise if this may be off-topic; I didn't see any other discussion
> lists that involved glibc developers.
> 
> Thank you for your time.
> 
> Best to you and yours,
> --arw
> 

Mainly because swapcontext does not use rt_sigreturn by default and I am 
almost sure the intended case for rt_sigreturn is also broken.

The glibc getcontext sets the MRS register value (mcontext_t gp_regs[33])
to zero:

sysdeps/unix/sysv/linux/powerpc/powerpc64/getcontext.S
211   /* Zero fill fields that can't be set in user state or are unused.  */
212   std  r0,(SIGCONTEXT_GP_REGS+(PT_MSR*8))(r3)

It will make the swapcontext avoid the rt_sigreturn syscall in most cases:

sysdeps/unix/sysv/linux/powerpc/powerpc64/swapcontext.S
548   ld      r0,(SIGCONTEXT_GP_REGS+(PT_MSR*8))(r31)
549   cmpdi r0,0                    
550   bne     L(do_sigret)

My understanding of the intended usercase for rt_sigreturn (based on code
comments) is to call swapcontext in a signal handler using the third argument 
while using SA_SIGINFO. the ucontext_t will be create by the kernel itself 
and would contain the expected value the MSR.

For kernel syscall the MSR will be used to determine whether or not to 
restore VMX and VSX register. Different than glibc, kernel ucontext_t
(and thus the one allocated on the stack frame for signal handling) 
contains extended state for the upper half of the VSX register.

Now the kernel patch is also not correct and trying call swapcontext with
signal handle ucontext_t issues SEGFAULT with kernel dumping:

[ 4070.047596] test[62308]: bad frame in rt_sigreturn: 0000000000000000 nip 00003fff96166988 lr 00003fff961666dc

I am using a kernel 3.10 for this experiment (CentOS7 on power7) and
the problem seems that sys_rt_sigreturn expects the pt_regs on ninth
argument:

631 /*
632  * Do a signal return; undo the signal stack.
633  */
634 
635 int sys_rt_sigreturn(unsigned long r3, unsigned long r4, unsigned long r5,
636                      unsigned long r6, unsigned long r7, unsigned long r8,
637                      struct pt_regs *regs)
638 {
639         struct ucontext __user *uc = (struct ucontext __user *)regs->gpr[1];

And afaik it is setup by the kernel at the resume from a syscall based on
the stack contents:

arch/powerpc/kernel/entry_64.S
 652 resume_kernel:
 653         /* check current_thread_info, _TIF_EMULATE_STACK_STORE */
 654         CURRENT_THREAD_INFO(r9, r1)

arch/powerpc/include/asm/thread_info.h
 25 #ifdef CONFIG_PPC64
 26 #define CURRENT_THREAD_INFO(dest, sp)   clrrdi dest, sp, THREAD_SHIFT
 27 #else
 28 #define CURRENT_THREAD_INFO(dest, sp)   rlwinm dest, sp, 0, 0, 31-THREAD_SHIFT
 29 #endif

(as a side note recent kernels from 4.18+/f3675644e1723 moved it to rt_sigreturn
with current_pt_regs macro).

So the problem is since we mess with stack frame value (r1), kernel will get a
bogus value for pt_regs and throw an invalid bad frame. In any case, this scenario
is not really supported since swapcontext is *not* async-signal-safe. I will
probably send a patch to just remove the rt_sigreturn path.

Now back to swapcontext on powerpc64 it seem to work on my experiments:

---
[azanella@gcc1-power7 ~]$ cat test.c
#include <stdio.h>
#include <ucontext.h>
#include <assert.h>
#include <unistd.h>
#include <signal.h>
#include <errno.h>
#include <sys/syscall.h>

#define MSR_VEC_LG      25              /* Enable AltiVec */
#define MSR_VSX_LG      23              /* Enable VSX */
#define __MASK(X)       (1UL<<(X))
#define MSR_VEC         __MASK(MSR_VEC_LG)      /* Enable AltiVec */
#define MSR_VSX         __MASK(MSR_VSX_LG)      /* Enable VSX */

static char func_stack[16384];
static ucontext_t ctx, ctxm;

static void
func (void)
{
  printf ("%s\n", __func__);
}

int main (int argc, char *argv[])
{ 
  assert (getcontext (&ctx) == 0);
  ctx.uc_stack.ss_sp = func_stack;
  ctx.uc_stack.ss_size = sizeof (func_stack);
  ctx.uc_link = &ctxm;
#ifdef ENABLE_ALTIVEC
  ctx.uc_mcontext.gp_regs[PT_MSR] |= MSR_VEC;
#endif
  makecontext (&ctx, func, 0);

  //assert (swapcontext (&ctxm, &ctx) == 0);
  long r = syscall (SYS_swapcontext, &ctxm, &ctx, sizeof ctx);
  printf ("%ld (%d)\n", r, errno);

  return 0;
}
---
$ gcc test.c && ./a.out
func
268584584 (0)
$ gcc test.c -DENABLE_ALTIVEC && ./a.out 
func
268584584 (0)
---

The only issue seems to be the returned value which I haven't debug further.
VSX is not supported, at least on glibc, because different than kernel it
does not have the extended size for the upper half of VSX register (check
BZ#6816 [1] for why we chose it) for ucontext_t.

The syscall will be also slower than implement it on userspace, so I don't
see much gain here.

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=6816

Attachment: signature.asc
Description: OpenPGP digital signature