Bug 28899 - FAIL: nptl/tst-cleanupx4
Summary: FAIL: nptl/tst-cleanupx4
Status: RESOLVED FIXED
Alias: None
Product: glibc
Classification: Unclassified
Component: nptl (show other bugs)
Version: 2.36
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-02-16 20:24 UTC by John David Anglin
Modified: 2022-02-28 15:59 UTC (History)
1 user (show)

See Also:
Host: hppa*-*-linux*
Target: hppa*-*-linux*
Build: hppa*-*-linux*
Last reconfirmed:


Attachments
Preprocessed source (29.30 KB, text/plain)
2022-02-16 20:24 UTC, John David Anglin
Details

Note You need to log in before you can comment on or make changes to this bug.
Description John David Anglin 2022-02-16 20:24:47 UTC
Created attachment 13984 [details]
Preprocessed source

FAIL: nptl/tst-cleanupx4

dave@atlas:~/gnu/glibc/objdir/nptl$ cat tst-cleanupx4.out
test 0
clh (2)
clh (1)
clh (3)
global = 12, expected 15
test 1
clh (4)
clh (6)
clh (1)
clh (5)
global = 160, expected 276
test 2
clh (8)
clh (1)
clh (7)
global = 70, expected 120
test 3
clh (2)
clh (10)
clh (1)
clh (9)
global = 288, expected 460

I modified the test to add the noninline attribute to clh() because
I noticed it was inlined into fn0() and fn2().  Still the test is
miscompiled:

(gdb) r
Starting program: /home/dave/gnu/glibc/objdir/elf/ld.so.1 --library-path /home/dave/gnu/glibc/objdir:/home/dave/gnu/glibc/objdir/math:/home/dave/gnu/glibc/objdir/elf:/home/dave/gnu/glibc/objdir/dlfcn:/home/dave/gnu/glibc/objdir/nss:/home/dave/gnu/glibc/objdir/nis:/home/dave/gnu/glibc/objdir/rt:/home/dave/gnu/glibc/objdir/resolv:/home/dave/gnu/glibc/objdir/mathvec:/home/dave/gnu/glibc/objdir/support:/home/dave/gnu/glibc/objdir/crypt:/home/dave/gnu/glibc/objdir/nptl /home/dave/gnu/glibc/objdir/nptl/tst-cleanupx4
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
[Attaching after process 30863 fork to child process 30866]
[New inferior 2 (process 30866)]
[Detaching after fork from parent process 30863]
[Inferior 1 (process 30863) detached]
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
test 0
[New LWP 30867]

Thread 2.2 "ld.so.1" received signal SIG32, Real-time event 32.
[Switching to LWP 30867]
0xf900156c in ?? () from /home/dave/gnu/glibc/objdir/libc.so.6
(gdb) c
Continuing.

Thread 2.2 "ld.so.1" hit Breakpoint 1, clh (arg=0x2) at tst-cleanup4.c:47
47        int val = (long int) arg;
(gdb) bt
#0  clh (arg=0x2) at tst-cleanup4.c:47
#1  0xf8f553c0 in unwind_stop (version=<optimized out>,
    actions=<optimized out>, exc_class=<optimized out>,
    exc_obj=<optimized out>, context=<optimized out>,
    stop_parameter=<optimized out>) at unwind.c:80
#2  0xf9196c70 in ?? () from /lib/hppa-linux-gnu/libgcc_s.so.4
#3  0xf9197300 in _Unwind_ForcedUnwind ()
   from /lib/hppa-linux-gnu/libgcc_s.so.4
#4  0xf8ee319c in _Unwind_ForcedUnwind (exc=0x0, stop=0x0,
    stop_argument=0xf8eb4000) at ../sysdeps/generic/unwind-resume.c:51
#5  0xf8f554f0 in __GI___pthread_unwind (buf=<optimized out>) at unwind.c:130
#6  0xf8f46c78 in __do_cancel () at ../sysdeps/nptl/pthreadP.h:280
#7  sigcancel_handler (sig=32, si=0x0, ctx=<optimized out>)
    at pthread_cancel.c:56
#8  sigcancel_handler (sig=<optimized out>, si=0x0, ctx=<optimized out>)
    at pthread_cancel.c:32
#9  <signal handler called>
#10 0xf8fbfd60 in __GI___libc_read (nbytes=1, buf=0xf86b32c8, fd=3)
    at ../sysdeps/unix/sysv/linux/read.c:26
#11 __GI___libc_read (fd=3, buf=0xf86b32c8, nbytes=1)
    at ../sysdeps/unix/sysv/linux/read.c:24
#12 0x000117a0 in fn_read () at tst-cleanup4.c:67
#13 0x000117dc in fn0 () at tst-cleanup4.c:76
--Type <RET> for more, q to quit, c to continue without paging--
#14 0x0001182c in fn1 () at tst-cleanup4.c:89
#15 0x00011854 in fn2 () at tst-cleanup4.c:100
#16 0x000118ec in tf (a=<optimized out>) at tst-cleanup4.c:112
#17 0xf8f49464 in start_thread (arg=0xf8eb33c0) at pthread_create.c:442
#18 0xf8fd907c in clone () at ../sysdeps/unix/sysv/linux/hppa/clone.S:151
Backtrace stopped: Cannot access memory at address 0xf86b2fec
(gdb) disass fn0
Dump of assembler code for function fn0:
   0x000117cc <+0>:     stw rp,-14(sp)
   0x000117d0 <+4>:     stw,ma r3,40(sp)
   0x000117d4 <+8>:     b,l 0x11768 <fn_read>,rp
   0x000117d8 <+12>:    nop
   0x000117dc <+16>:    ldi 1,r26
   0x000117e0 <+20>:    ldw -54(sp),rp
   0x000117e4 <+24>:    b,l 0x1170c <clh>,r0
   0x000117e8 <+28>:    ldw,mb -40(sp),r3
   0x000117ec <+32>:    ldi 1,r26
   0x000117f0 <+36>:    b,l 0x1170c <clh>,rp
   0x000117f4 <+40>:    copy r20,r3
   0x000117f8 <+44>:    b,l 0x110b4,rp
   0x000117fc <+48>:    copy r3,r26
   0x00011800 <+52>:    nop
End of assembler dump.

The cleanup handler in fn0 is not registered before fn_read is called.
As a result, the cleanup registered using _pthread_cleanup_push in fn1
is on the top of the stack.

Attached .i file for test.  This is code for fn0:
__attribute__((noinline)) void
fn0 (void)
{
  do { struct __pthread_cleanup_frame __clframe __attribute__ ((__cleanup__ (__pthread_cleanup_routine))) = { .__cancel_routine = (clh), .__cancel_arg = ((void *) 1l), .__do_it = 1 };;

  fn_read ();

  __clframe.__do_it = (1); } while (0);
}

Not sure but this might be a compiler bug.
Comment 1 John David Anglin 2022-02-17 00:35:23 UTC
If I replace the LinuxThreads pthread_cleanup_{push,pop} versions with
the new versions in fn1, the sequencing is correct for test 0.
Comment 2 John David Anglin 2022-02-24 17:57:27 UTC
Back traces with libgcc symbols and debug info:

(gdb) bt
#0  clh (arg=0x2) at tst-cleanup4.c:47
#1  0xf91b93b4 in unwind_stop (version=<optimized out>,
    actions=<optimized out>, exc_class=<optimized out>,
    exc_obj=<optimized out>, context=<optimized out>,
    stop_parameter=<optimized out>) at unwind.c:80
#2  0xf9665030 in _Unwind_ForcedUnwind_Phase2 (exc=0x0, context=0x0,
    frames_p=0xf8917b58) at ../../../gcc/libgcc/unwind.inc:171
#3  0xf96656e4 in _Unwind_ForcedUnwind (exc=0x0, stop=0xf9118000,
    stop_argument=0x0) at ../../../gcc/libgcc/unwind.inc:218
#4  0xf914719c in _Unwind_ForcedUnwind (exc=0x0, stop=0x0,
    stop_argument=0xf9118000) at ../sysdeps/generic/unwind-resume.c:51
#5  0xf91b94e4 in __GI___pthread_unwind (buf=<optimized out>) at unwind.c:130
#6  0xf91aac70 in __do_cancel () at ../sysdeps/nptl/pthreadP.h:280
#7  sigcancel_handler (sig=32, si=0x0, ctx=<optimized out>)
    at pthread_cancel.c:56
#8  sigcancel_handler (sig=<optimized out>, si=0x0, ctx=<optimized out>)
    at pthread_cancel.c:32
#9  <signal handler called>
#10 0xf9223d54 in __GI___libc_read (nbytes=1, buf=0xf89172c8, fd=3)
    at ../sysdeps/unix/sysv/linux/read.c:26
#11 __GI___libc_read (fd=3, buf=0xf89172c8, nbytes=1)
    at ../sysdeps/unix/sysv/linux/read.c:24
#12 0x000117a0 in fn_read () at tst-cleanup4.c:67
--Type <RET> for more, q to quit, c to continue without paging--
#13 0x000117e8 in fn0 () at tst-cleanup4.c:76
#14 0x00011858 in fn1 () at tst-cleanup4.c:89
#15 0x0001188c in fn2 () at tst-cleanup4.c:100
#16 0x00011948 in tf (a=<optimized out>) at tst-cleanup4.c:112
#17 0xf91ad45c in start_thread (arg=0xf91173c0) at pthread_create.c:442
#18 0xf923d070 in clone () at ../sysdeps/unix/sysv/linux/hppa/clone.S:151
Backtrace stopped: Cannot access memory at address 0xf8916fec
(gdb) c
Continuing.
clh (2)

Thread 2.2 "ld.so.1" hit Breakpoint 1, clh (arg=0x1) at tst-cleanup4.c:47
47        int val = (long int) arg;
(gdb) bt
#0  clh (arg=0x1) at tst-cleanup4.c:47
#1  0x00011824 in __pthread_cleanup_routine (__frame=<synthetic pointer>)
    at ../sysdeps/nptl/pthread.h:628
#2  fn0 () at tst-cleanup4.c:74
#3  0x00011858 in fn1 () at tst-cleanup4.c:89
#4  0x0001188c in fn2 () at tst-cleanup4.c:100
#5  0x00011948 in tf (a=<optimized out>) at tst-cleanup4.c:112
#6  0xf91ad45c in start_thread (arg=0xf91173c0) at pthread_create.c:442
#7  0xf923d070 in clone () at ../sysdeps/unix/sysv/linux/hppa/clone.S:151
Backtrace stopped: Cannot access memory at address 0xf8916fec
(gdb) c
Continuing.
clh (1)

Thread 2.2 "ld.so.1" hit Breakpoint 1, clh (arg=0x3) at tst-cleanup4.c:47
47        int val = (long int) arg;
(gdb) bt
#0  clh (arg=0x3) at tst-cleanup4.c:47
#1  0x000118cc in __pthread_cleanup_routine (__frame=<synthetic pointer>)
    at ../sysdeps/nptl/pthread.h:628
#2  fn2 () at tst-cleanup4.c:98
#3  0x00011948 in tf (a=<optimized out>) at tst-cleanup4.c:112
#4  0xf91ad45c in start_thread (arg=0xf91173c0) at pthread_create.c:442
#5  0xf923d070 in clone () at ../sysdeps/unix/sysv/linux/hppa/clone.S:151
Backtrace stopped: Cannot access memory at address 0xf8916fec

We have the following code processing cleanups:

__extern_inline void
__pthread_cleanup_routine (struct __pthread_cleanup_frame *__frame)
{
  if (__frame->__do_it)
    __frame->__cancel_routine (__frame->__cancel_arg);
}

and

static _Unwind_Reason_Code
unwind_stop (int version, _Unwind_Action actions,
             _Unwind_Exception_Class exc_class,
             struct _Unwind_Exception *exc_obj,
             struct _Unwind_Context *context, void *stop_parameter)
{
  struct pthread_unwind_buf *buf = stop_parameter;
  struct pthread *self = THREAD_SELF;
  struct _pthread_cleanup_buffer *curp = THREAD_GETMEM (self, cleanup);
  int do_longjump = 0;

  /* Adjust all pointers used in comparisons, so that top of thread's
     stack is at the top of address space.  Without that, things break
     if stack is allocated above the main stack.  */
  uintptr_t adj = (uintptr_t) self->stackblock + self->stackblock_size;

  /* Do longjmp if we're at "end of stack", aka "end of unwind data".
     We assume there are only C frame without unwind data in between
     here and the jmp_buf target.  Otherwise simply note that the CFA
     of a function is NOT within it's stack frame; it's the SP of the
     previous frame.  */
  if ((actions & _UA_END_OF_STACK)
      || ! _JMPBUF_CFA_UNWINDS_ADJ (buf->cancel_jmp_buf[0].jmp_buf, context,
                                    adj))
    do_longjump = 1;

  if (__glibc_unlikely (curp != NULL))
    {
      /* Handle the compatibility stuff.  Execute all handlers
         registered with the old method which would be unwound by this
         step.  */
      struct _pthread_cleanup_buffer *oldp = buf->priv.data.cleanup;
      void *cfa = (void *) (_Unwind_Ptr) _Unwind_GetCFA (context);

      if (curp != oldp && (do_longjump || FRAME_LEFT (cfa, curp, adj)))
        {
          do
            {
              /* Pointer to the next element.  */
              struct _pthread_cleanup_buffer *nextp = curp->__prev;

              /* Call the handler.  */
              curp->__routine (curp->__arg);

              /* To the next.  */
              curp = nextp;
            }
          while (curp != oldp
                 && (do_longjump || FRAME_LEFT (cfa, curp, adj)));

          /* Mark the current element as handled.  */
          THREAD_SETMEM (self, cleanup, curp);
        }
    }

  DIAG_PUSH_NEEDS_COMMENT;
#if __GNUC_PREREQ (7, 0)
  /* This call results in a -Wstringop-overflow warning because struct
     pthread_unwind_buf is smaller than jmp_buf.  setjmp and longjmp
     do not use anything beyond the common prefix (they never access
     the saved signal mask), so that is a false positive.  */
  DIAG_IGNORE_NEEDS_COMMENT (11, "-Wstringop-overflow=");
#endif

  if (do_longjump)
    __libc_unwind_longjmp ((struct __jmp_buf_tag *) buf->cancel_jmp_buf, 1);
  DIAG_POP_NEEDS_COMMENT;

  return _URC_NO_REASON;
}

In the first call to unwind_stop, we have:

(gdb) p *self
$9 = {{header = {multiple_threads = 1, gscope_flag = 0}, __padding = {0x1,
      0x0 <repeats 23 times>}}, list = {next = 0xf9a1f840 <_rtld_local+3024>,
    prev = 0xf9a1f840 <_rtld_local+3024>}, tid = 25464, pid_ununsed = 0, {
    robust_list = {__next = 0xf9117430}, robust_head = {list = 0xf9117430,
      futex_offset = -36, list_op_pending = 0x0}}, cleanup = 0xf8917208,
  cleanup_jmp_buf = 0xf8917048, cancelhandling = 24, flags = 0,
  specific_1stblock = {{seq = 0, data = 0x0} <repeats 32 times>}, specific = {
    0xf911744c, 0x0 <repeats 31 times>}, specific_used = false,
  report_events = false, user_stack = false, stopped_start = false,
  setup_failed = 0, lock = 0, setxid_futex = 0, joinid = 0x0,
  result = 0xffffffff, schedparam = {sched_priority = 0}, schedpolicy = 0,
  start_routine = @0x1801a: 0x118d8 <tf>, arg = 0x0, eventbuf = {eventmask = {
      event_bits = {0, 0}}, eventnum = TD_ALL_EVENTS, eventdata = 0x0},
  nextevent = 0x0, exc = {{{exception_class = 0,
        exception_cleanup = @0xf92d564a: 0xf91b9474 <unwind_cleanup>,
        private_1 = 4180497986, private_2 = 4170281032},
      unwind_exception_align = {0, 0}}}, stackblock = 0xf8917000,
  stackblock_size = 8392704, guardsize = 4096, reported_guardsize = 4096,
  tpp = 0x0, res = {retrans = 0, retry = 0, options = 0, nscount = 0,
    nsaddr_list = {{sin_family = 0, sin_port = 0, sin_addr = {s_addr = 0},
        sin_zero = "\000\000\000\000\000\000\000"}, {sin_family = 0,
        sin_port = 0, sin_addr = {s_addr = 0},
        sin_zero = "\000\000\000\000\000\000\000"}, {sin_family = 0,
--Type <RET> for more, q to quit, c to continue without paging--
        sin_port = 0, sin_addr = {s_addr = 0},
        sin_zero = "\000\000\000\000\000\000\000"}}, id = 0, dnsrch = {0x0,
      0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, defdname = '\000' <repeats 255 times>,
    pfcode = 0, ndots = 0, nsort = 0, ipv6_unavail = 0, unused = 0,
    sort_list = {{addr = {s_addr = 0}, mask = 0}, {addr = {s_addr = 0},
        mask = 0}, {addr = {s_addr = 0}, mask = 0}, {addr = {s_addr = 0},
        mask = 0}, {addr = {s_addr = 0}, mask = 0}, {addr = {s_addr = 0},
        mask = 0}, {addr = {s_addr = 0}, mask = 0}, {addr = {s_addr = 0},
        mask = 0}, {addr = {s_addr = 0}, mask = 0}, {addr = {s_addr = 0},
        mask = 0}}, __glibc_unused_qhook = 0x0, __glibc_unused_rhook = 0x0,
    res_h_errno = 0, _vcsock = 0, _flags = 0, _u = {
      pad = '\000' <repeats 51 times>, _ext = {nscount = 0, nsmap = {0, 0, 0},
        nssocks = {0, 0, 0}, nscount6 = 0, nsinit = 0, nsaddrs = {0x0, 0x0,
          0x0}, __glibc_extension_index = 0}}}, sigmask = {__val = {0, 0,
      2147483648, 1, 4155515848, 1001, 4180505336, 118, 1083179008, 0, 0, 0,
      1076494336, 0, 1076494336, 0, 1952805748, 540019200, 4180505336,
      4178739996, 4189056168, 4294967295, 4188135804, 4189044800, 0,
      4188007991, 0, 0, 4155516504, 4155516416, 0, 1}}, c11 = false,
  cancelstate = 0 '\000', canceltype = 1 '\001', exiting = false,
  exit_lock = 0, tls_state = {strsignal_buf = 0x0, strerror_l_buf = 0x0},
  rseq_area = {cpu_id_start = 0, cpu_id = 4294967294, rseq_cs = {ptr64 = 0,
      ptr = {padding = 0, ptr32 = 0}}, flags = 0},
  end_padding = 0xf9117900 "\371\242\001\230"}
(gdb) p *curp
$10 = {__routine = @0x1803a: 0x1170c <clh>, __arg = 0x2, __canceltype = 0,
  __prev = 0x0}

After the first call to clh, the unwind continues:

238       if (_Unwind_IsExtendedContext (context) && context->by_value[index])
(gdb)
0xf96622c4 in _Unwind_IsExtendedContext (context=<optimized out>)
    at ../../../gcc/libgcc/unwind-dw2.c:217
217               || (context->flags & EXTENDED_CONTEXT_BIT));
(gdb)
216       return (ASSUME_EXTENDED_UNWIND_CONTEXT
(gdb)
250       if (size == sizeof(_Unwind_Ptr))
(gdb)
uw_install_context_1 (current=0xf8917970, target=0xf8917788) at ../../../gcc/libgcc/unwind-dw2.c:1710
1710            return current->cfa - target_cfa - target->args_size;
(gdb)
_Unwind_ForcedUnwind (exc=<optimized out>, stop=@0xf92d5642: 0xf91b9308 <unwind_stop>, stop_argument=<optimized out>) at ../../../gcc/libgcc/unwind.inc:222
222       uw_install_context (&this_context, &cur_context, frames);
(gdb)
fn0 () at tst-cleanup4.c:74
74        pthread_cleanup_push (clh, (void *) 1l);
(gdb)
0x00011818 in __pthread_cleanup_routine (__frame=<optimized out>)
    at ../sysdeps/nptl/pthread.h:628
628         __frame->__cancel_routine (__frame->__cancel_arg);
(gdb)

Thread 5.2 "ld.so.1" hit Breakpoint 1, clh (arg=0x1) at tst-cleanup4.c:47
47        int val = (long int) arg;

I'm far from an expert in the unwind code but I fail to see how the old
and new cleanup mechanisms can be interleaved in the sequence expected by
the test.  self->cleanup would have to be null in first call to unwind_stop
but _pthread_cleanup_push is called in fn1 before fn0 is called.

__attribute__((noinline)) void
fn1 (void)
{
  /* This is the old LinuxThreads pthread_cleanup_{push,pop}.  */
  struct _pthread_cleanup_buffer b;
  _pthread_cleanup_push (&b, clh, (void *) 2l);

  fn0 ();

  _pthread_cleanup_pop (&b, 1);
}
Comment 3 John David Anglin 2022-02-28 15:59:00 UTC
Fixed on master by this commit:

commit 2bbc694df279020a6620096d31c1e05c93966f9b (HEAD -> master, origin/master, origin/HEAD)
Author: John David Anglin <danglin@gcc.gnu.org>
Date:   Mon Feb 28 15:47:38 2022 +0000

    nptl: Fix cleanups for stack grows up [BZ# 28899]

    _STACK_GROWS_DOWN is defined to 0 when the stack grows up.  The
    code in unwind.c used `#ifdef _STACK_GROWS_DOWN' to selct the
    stack grows down define for FRAME_LEFT.  As a result, the
    _STACK_GROWS_DOWN define was always selected and cleanups were
    incorrectly sequenced when the stack grows up.