RE: gdb and remote debugging with threads

["Srikanth Krishnamohan" <Srikanth dot Krishnamohan at navigon dot com>]

Please ignore my previous message. The problem was that I was setting
the solib-absolute-prefix to non-debug version of the libraries. That
was the reason it failed. 
Setting to debug version of lib solves the problem.

Thanks for your help,


-----Original Message-----
From: Srikanth Krishnamohan 
Sent: Thursday, June 08, 2006 11:02 AM
To: 'Daniel Jacobowitz'
Cc: 'gdb@sourceware.org'
Subject: RE: gdb and remote debugging with threads

Hello Daniel,
   I appreciate your reply. 
I have tried your suggestion before. It did not make any difference. 
I have problems similar to the one discussed in this thread
(http://sources.redhat.com/ml/gdb/2005-04/msg00056.html).

To summarize the problems I have with remote debugging are,

1. In a multi-threaded program linked with shared libraries the
breakpoint in never reached other than the main thread.
2. In the sample program below the breakpoint is reached only once and
it either crashes after that (target=arm) or continues until the program
exits (host=target=i686). 

I'm attaching a more detailed output if it helps (apologize if its too
much information). NOTE that even when host=target=i686 the debugger
does stop in the break point after the first time.

Thanks,
Srikanth

----------------------------------Source and debugging
sessions---------------------
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>


void *thread_func(void *arg) {

   int iThid = *(int *)arg;
   printf("thread_func ID %d\n",iThid);
}

int main(int argc, char **argv)
{
   if (argc < 1) {
     printf("Usage: %s numthreads\n",argv[0]);
     exit(1);
   }
   int retVal = 0,index=0;
   int numThreads = atoi(argv[1]);
   pthread_t thid[numThreads];

  //create all the threads
  for(index = 0;index <numThreads; index ++) {
     printf("Creating thread num %d\n",index);
     retVal = pthread_create(&thid[index],NULL,thread_func,(void
*)&index); 
     if (retVal < 0) {
	printf("Error in creating thread id %d\n",thid[index]);
     } 
     sleep(1);
  }

  for(index = 0;index <numThreads; index ++) {
     pthread_join(thid[index],NULL);
  }

}//end of source

---------------------------------------------------
How they were compiled:

Compile options for host-gdb and target-gdbserver:

1. Host gdb - 
./configure --host=i686-pc-linux-gnu --target=arm-xscale-linux-gnu

2. Target gdb and gdbserver - 
./configure --prefix=/usr/local/arm-linux/arm-linux/
--program-prefix=arm-xscale-linux-gnu- --build=i686-pc-linux-gnu
--target=arm-xscale-linux-gnu arm-xscale-linux-gnu

---------------------------------------------------
//gdb sessions output

//1. debuggin in the target directly using gdb (works as expected)

[root@Linux /opt]#gdb ./thread_tst_xscale
GNU gdb 6.4
Copyright 2005 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for
details.
This GDB was configured as "arm-xscale-linux-gnu"...Using host
libthread_db library "/lib/libthread_db.so.1".

(gdb) b 9
Breakpoint 1 at 0x853c: file thread_tst.c, line 9.
(gdb) r 5
Starting program: /opt/thread_tst_xscale 5
[Thread debugging using libthread_db enabled]
[New Thread 16384 (LWP 811)]
Creating thread num 0
[New Thread 32769 (LWP 814)]
[New Thread 16386 (LWP 815)]
[Switching to Thread 16386 (LWP 815)]

Breakpoint 1, thread_func (arg=0xbefffd88) at thread_tst.c:9
warning: Source file is more recent than executable.
9          printf("thread_func ID %d\n",iThid);
(gdb) c
Continuing.
Creating thread num 1
thread_func ID 0
[Thread 16386 (LWP 815) exited]
[New Thread 32771 (LWP 816)]
[Switching to Thread 32771 (LWP 816)]

Breakpoint 1, thread_func (arg=0xbefffd88) at thread_tst.c:9
9          printf("thread_func ID %d\n",iThid);
(gdb) c
Continuing.
Creating thread num 2
thread_func ID 1
[Thread 32771 (LWP 816) exited]
[New Thread 49156 (LWP 817)]
[Switching to Thread 49156 (LWP 817)]

Breakpoint 1, thread_func (arg=0xbefffd88) at thread_tst.c:9
9          printf("thread_func ID %d\n",iThid);
(gdb) c
Continuing.
Creating thread num 3
thread_func ID 2
[Thread 49156 (LWP 817) exited]
[New Thread 65541 (LWP 818)]
[Switching to Thread 65541 (LWP 818)]

Breakpoint 1, thread_func (arg=0xbefffd88) at thread_tst.c:9
9          printf("thread_func ID %d\n",iThid);
(gdb) c
Continuing.
Creating thread num 4
thread_func ID 3
[Thread 65541 (LWP 818) exited]
[New Thread 81926 (LWP 819)]
[Switching to Thread 81926 (LWP 819)]

Breakpoint 1, thread_func (arg=0xbefffd88) at thread_tst.c:9
9          printf("thread_func ID %d\n",iThid);
(gdb) c
Continuing.
thread_func ID 4
[Thread 81926 (LWP 819) exited]

Program exited with code 05.
(gdb)

---------------------------------------------
//2. Remote debugging of the above program

//host(i686-linux) side session

$ gdb ./thread_tst_xscale
GNU gdb 6.4
Copyright 2005 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for
details.
This GDB was configured as "--host=i686-pc-linux-gnu
--target=arm-xscale-linux-gnu"...
(gdb) target remote 192.168.190.135:5000
Remote debugging using 192.168.190.135:5000
0x40000e50 in ?? ()
(gdb) directory /opt/xxx/nfsroot/opt/
Source directories searched: /opt/xxx/nfsroot/opt:$cdir:$cwd
(gdb) set solib-absolute-prefix /opt/xxx/nfsroot/
(gdb) set solib-search-path /opt/xxx/nfsroot/lib/
(gdb) b 9
Breakpoint 1 at 0x853c: file thread_tst.c, line 9.
(gdb) c
Continuing.

Program received signal SIG32, Real-time event 32.
0x400a75f4 in ?? ()
(gdb) info threads
  1 thread 733  0x400a75f4 in ?? ()
warning: Couldn't restore frame in current thread, at frame 0
0x400a75f4 in ?? ()
(gdb) info b
Num Type           Disp Enb Address    What
1   breakpoint     keep y   0x0000853c in thread_func at thread_tst.c:9
(gdb) c
Continuing.

Program terminated with signal SIGKILL, Killed.
The program no longer exists.
(gdb)


//target(arm-xscale-linux) side session

[root@Linux /opt]#gdbserver :5000 ./thread_tst_xscale 5
Process ./thread_tst_xscale created; pid = 733
Listening on port 5000
Remote debugging from host 192.168.190.117
Creating thread num 0

Child terminated with signal = 9

Child terminated with signal = 0x9
GDBserver exiting
[root@Linux /opt]#

---------------------------------------------

//3. Remote debugging of the above program where host=target=i686 

//host(i686-linux) side session

$ gdb ./thread_tst
GNU gdb Red Hat Linux (6.3.0.0-1.84rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for
details.
This GDB was configured as "i386-redhat-linux-gnu"...Using host
libthread_db library "/lib/libthread_db.so.1".

(gdb) target remote 127.0.0.1:5000
Remote debugging using 127.0.0.1:5000
0x0050a790 in ?? ()
(gdb) set solib-absolute-prefix /lib/
(gdb) set solib-search-path /lib/
(gdb) b 9
Breakpoint 1 at 0x80484e6: file thread_tst.c, line 9.
(gdb) c
Continuing.
[New Thread -1209002304]
[New Thread -1209005136]
[Switching to Thread -1209005136]

Breakpoint 1, thread_func (arg=0xbfa19f08) at thread_tst.c:9
9          printf("thread_func ID %d\n",iThid);
(gdb) info threads
  3 Thread 2147483647  0x00dbf410 in ?? ()
* 2 Thread -1209005136  thread_func (arg=0xbfa19f08) at thread_tst.c:9
  1 Thread -1209002304  0x00dbf410 in ?? ()
(gdb) c
Continuing.

Program exited with code 05.
(gdb)

//target(i686-linux) side session

$ gdbserver :5000 ./thread_tst 5
Process ./thread_tst created; pid = 20542
Listening on port 5000
Remote debugging from host 127.0.0.1
Creating thread num 0
thread_func ID 0
Creating thread num 1
thread_func ID 1
Creating thread num 2
thread_func ID 2
Creating thread num 3
thread_func ID 3
Creating thread num 4
thread_func ID 4

Child exited with retcode = 5

Child exited with status 5
GDBserver exiting


-----Original Message-----
From: Daniel Jacobowitz [mailto:drow@false.org] 
Sent: Wednesday, June 07, 2006 9:05 PM
To: Srikanth Krishnamohan
Cc: gdb@sourceware.org
Subject: Re: gdb and remote debugging with threads

On Wed, Jun 07, 2006 at 04:17:09PM +0200, Srikanth Krishnamohan wrote:
> Hello All,
>    Does gdb 6.4 support remote debugging of a muti-threaded program.
The
> host is i686-linux and target is arm-xscale-linux.

Yes.  It has been supported for years, and I've been giving out the
same advice about it for most of that time.  I apologize for the fact
that I haven't stuck this advice in the manual; obviously I need to do
that.

> Program received signal SIG32, Real-time event 32.
> 0x400a75f4 in ?? ()
> (gdb) c
> Continuing.

If you receive this message, it usually means you did not tell GDB
where to find target libraries, which prevents thread debugging
from initializing.  Search for "set solib-absolute-prefix".

-- 
Daniel Jacobowitz
CodeSourcery