Bug 27526 - Attaching to threaded process on glibc 2.33: libthread_db fails to initialize with "generic error"
Summary: Attaching to threaded process on glibc 2.33: libthread_db fails to initialize...
Status: RESOLVED FIXED
Alias: None
Product: gdb
Classification: Unclassified
Component: gdb (show other bugs)
Version: HEAD
: P2 normal
Target Milestone: 11.1
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-03-05 16:48 UTC by Simon Marchi
Modified: 2021-06-21 16:39 UTC (History)
4 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Simon Marchi 2021-03-05 16:48:59 UTC
As reported here:

  https://sourceware.org/pipermail/gdb/2021-February/049208.html

On a Linux system with glibc 2.33, attaching to a threaded program gives:

$ pidof gnome-calculator 
1476833
(gdb) attach 1476833
Attaching to process 1476833
[New LWP 1476834]
[New LWP 1476835]
[New LWP 1476836]
LOOKUP nptl_version in libpthread.so.0
LOOKUP nptl_version in libpthread.so.0
LOOKUP _rtld_global in ld-linux-x86-64.so.2
LOOKUP _dl_stack_user in (null)
Error while reading shared library symbols for /usr/lib/libpthread.so.0:
Cannot find user-level thread for LWP 1476836: generic error
0x00007fdb0288b6ed in __GI___poll (fds=0x5581bab7b100, nfds=3, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
29      ../sysdeps/unix/sysv/linux/poll.c: No such file or directory.
(gdb) 

The "LOOKUP" lines come from a print I added in ps_pglobal_lookup to see which symbols libthread_db asks GDB to look up.

The lookup for _dl_stack_user is new with glibc 2.33, I don't see it with glibc 2.31 (I don't have a system with glibc 2.32 handy).  We fail to find a symbol with this name, which fails the loading/initialization of libthread_db.
Comment 1 Simon Marchi 2021-03-05 16:59:45 UTC
The stack trace where the lookup for _dl_stack_user happens:

#0  ps_pglobal_lookup (ph=0x60c000037fe8, obj=0x0, name=0x7fffde34e6bd "_dl_stack_user", sym_addr=0x6170000477c8) at /home/simark/src/binutils-gdb/gdb/proc-service.c:102
#1  0x00007fffde34c7cc in td_mod_lookup (ps=<optimized out>, mod=mod@entry=0x0, idx=idx@entry=54, sym_addr=sym_addr@entry=0x6170000477c8) at td_symbol_list.c:48
#2  0x00007fffde34b131 in __td_ta_stack_user (ta=ta@entry=0x617000047580, plist=plist@entry=0x7fffffffb000) at td_thr_validate.c:32
#3  0x00007fffde349b8c in td_ta_map_lwp2thr (ta_arg=0x617000047580, lwpid=1476836, th=0x7fffffffb0a0) at td_ta_map_lwp2thr.c:190
#4  0x000055555d54fbc2 in thread_from_lwp (stopped=0x61700003a700, ptid=...) at /home/simark/src/binutils-gdb/gdb/linux-thread-db.c:415
#5  0x000055555d554b6b in try_thread_db_load_1 (info=0x60c000037fc0) at /home/simark/src/binutils-gdb/gdb/linux-thread-db.c:912
#6  0x000055555d555837 in try_thread_db_load (library=0x555560734260 "libthread_db.so.1", check_auto_load_safe=false) at /home/simark/src/binutils-gdb/gdb/linux-thread-db.c:1014
#7  0x000055555d55689c in try_thread_db_load_from_sdir () at /home/simark/src/binutils-gdb/gdb/linux-thread-db.c:1091
#8  0x000055555d557450 in thread_db_load_search () at /home/simark/src/binutils-gdb/gdb/linux-thread-db.c:1146
#9  0x000055555d557d90 in thread_db_load () at /home/simark/src/binutils-gdb/gdb/linux-thread-db.c:1203
#10 0x000055555d557e72 in check_for_thread_db () at /home/simark/src/binutils-gdb/gdb/linux-thread-db.c:1246
#11 0x000055555d557fdf in thread_db_new_objfile (objfile=0x614000007040) at /home/simark/src/binutils-gdb/gdb/linux-thread-db.c:1275
...
Comment 2 Tom de Vries 2021-03-07 07:30:08 UTC
Looks like a duplicate of PR25674.
Comment 3 Simon Marchi 2021-03-07 23:08:02 UTC
(In reply to Tom de Vries from comment #2)
> Looks like a duplicate of PR25674.

Even though it's the same error message ("Generic error", not very descriptive), I don't think it's the same root cause.  PR25674 looks like a race condition when attaching, this one is caused by a change in glibc.

See: https://sourceware.org/pipermail/gdb-patches/2021-March/176787.html
Comment 4 crvi 2021-03-22 07:16:07 UTC
Same here in fedora rawhide with glibc-2.33 and gdb-10.1-15.fc35.

Any suggested workarounds ?
Comment 5 Simon Marchi 2021-03-22 13:38:03 UTC
(In reply to crvi from comment #4)
> Same here in fedora rawhide with glibc-2.33 and gdb-10.1-15.fc35.
> 
> Any suggested workarounds ?


I don't see a workaround for this, we'll need a proper fix in GDB.  I'm marking this bug with the 11.1 target milestone, I don't think we'll want to release GDB 11 without a fix for this.
Comment 6 Tom de Vries 2021-03-31 17:00:08 UTC
Patch posted: https://sourceware.org/pipermail/gdb-patches/2021-March/177369.html
Comment 7 Tom de Vries 2021-05-06 13:04:52 UTC
It seems there's a glibc PR27744 - "Support different libpthread/ld.so load orders in libthread_db" with a commit that should fix this on the glibc side.
Comment 8 Simon Marchi 2021-05-06 13:20:09 UTC
I posted a v2 here:

https://sourceware.org/pipermail/gdb-patches/2021-May/178485.html
Comment 9 Joel Brobecker 2021-05-09 23:38:36 UTC
Important fix to have for our next release.
Comment 10 Joel Brobecker 2021-06-20 15:55:34 UTC
Hello,

Now that https://sourceware.org/pipermail/gdb-patches/2021-June/179786.html has been pushed to master, do we have anything left to do for this PR, or can we close?

Thank you!
Comment 11 Simon Marchi 2021-06-21 16:39:38 UTC
(In reply to Joel Brobecker from comment #10)
> Hello,
> 
> Now that https://sourceware.org/pipermail/gdb-patches/2021-June/179786.html
> has been pushed to master, do we have anything left to do for this PR, or
> can we close?
> 
> Thank you!

Yes, let's close it.  If any related problem pops up, it will be a new PR.