This is the mail archive of the
gdb@sourceware.org
mailing list for the GDB project.
Problem debugging multi-threaded application
- From: Arvid Brodin <arvid dot brodin at enea dot com>
- To: gdb at sourceware dot org
- Date: Mon, 21 Jan 2008 19:26:37 +0100
- Subject: Problem debugging multi-threaded application
We have a program, a big beast, which on some platforms cannot be debugged, and I'm trying to figure out why.
The program is supposed to print out some welcome/init messages and then display a prompt. It also starts a bunch of threads (about 200 of them). When run under gdb on some platforms, I only get the [New Thread 0x... (LWP ...)] messages, but no text output. The program seems to run - only I get no text output (or input).
If I invoke gdb under strace, i.e. 'strace -f -o strace-out.txt gdb <my-program>', and then run my program in gdb, I instead get no messages about created threads, but I do get some printouts from my program before it hangs. The strace log among other things contain messages about "--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---" - I don't know if this is important.
If run under gdbtui - 'gdbtui <my-program>' - as soon as I 'run' the program gdbtui exits to the shell with
'[1]+ Stopped gdbtui <my-program>'.
If I comment out either the call to create_thread_event_breakpoint() in gdb/linux-thread-db.c:468 or the call to td_thr_event_enable() in gdb/linux-thread-db.c:716 I get the printouts from my program (although I guess it also messes something up - the program hangs shortly after initialization in this case).
The problem exists with both gdb-6.3.0 and gdb-6.7.1.
The problem exists on these platforms:
---------------
$ uname -r
2.4.21-53.ELsmp
$ /lib/tls/libc.so.6
GNU C Library stable release version 2.3.2, by Roland McGrath et al.
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 3.2.3 20030502 (Red Hat Linux 3.2.3-58).
Compiled on a Linux 2.4.20 system on 2007-04-19.
Available extensions:
GNU libio by Per Bothner
crypt add-on version 2.1 by Michael Glad and others
NPTL 0.60 by Ulrich Drepper
RT using linux kernel aio
The C stubs add-on version 2.1.2.
BIND-8.2.3-T5B
NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
Glibc-2.0 compatibility add-on by Cristian Gafton
Thread-local storage support included.
---------------
and
---------------
$~> uname -r
2.6.5-7.287.3-smp
$~> /lib/tls/libc.so.6
GNU C Library stable release version 2.3.5 (20050720), by Roland McGrath et al.
Copyright (C) 2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Configured for i686-suse-linux.
Compiled by GNU CC version 3.3.3 (SuSE Linux).
Compiled on a Linux 2.6.5 system on 2007-10-10.
Available extensions:
GNU libio by Per Bothner
crypt add-on version 2.1 by Michael Glad and others
Native POSIX Threads Library by Ulrich Drepper et al
GNU Libidn by Simon Josefsson
NoVersion patch for broken glibc 2.0 binaries
BIND-8.2.3-T5B
NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
Thread-local storage support included.
---------------
but not on
---------------
$~> uname -r
2.6.16.46-0.14-smp
$~> /lib/libc.so.6
GNU C Library development release version 2.4 (20070506), by Roland McGrath et al.
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Configured for i686-suse-linux.
Compiled by GNU CC version 4.1.0 (SUSE Linux).
Compiled on a Linux 2.6.16 system on 2007-05-06.
Available extensions:
crypt add-on version 2.1 by Michael Glad and others
GNU Libidn by Simon Josefsson
GNU libio by Per Bothner
NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
NoVersion patch for broken glibc 2.0 binaries
Native POSIX Threads Library by Ulrich Drepper et al
BIND-8.2.3-T5B
Thread-local storage support included.
---------------
The paths to glibc above is taken from 'ldd <path to libthread_db.so.1 as printed by gdb>, i.e. on the systems that fails /lib/tls/libc.so.1 and /lib/tls/libthread_db.so.1 are used, while on the working system the files are found under /lib.
To recapitulate, the program runs fine on all these platforms when not being debugged, and works under gdb on the "newest" one. On the older platforms the symptoms are text input and output does not work when the program is being debugged, unless gdb is run under strace, when it works "better", at least.
I have no idea if this is a bug in gdb, if we do something bad in our program, if it is a bug in nptl, or ...
Any ideas?
--
Arvid Brodin
Enea LCC