Stepping of multithreaded application

[Tomas Martinec <fyzmat at gmail dot com>]

Hello,

I am developing an Eclipse plugin that allows debugging of a MIPS
virtual machine. The GDB is used as a debugger backend. I have
difficulties with debugging multiprocessor machines (processors are
represented as threads in the GDB).

Stepping of different threads does not seem to work well. When I
change the current thread in the eclipse and apply the step command,
the debugger is stopped in the previous thread. I expect that the
debugger should be stopped at the next line of the newly set thread.
Is my expectation right?

I minimalized the scenario to the GDB/virtual machine only and I got
an example gdb session. The structure of the session is basically
this:

tomas@tomas-laptop:~/workspaceGDB/gdb-7.2/gdb$ ./gdb --interpreter mi2
// use the MI
// connect to the machine
target remote :10001
// bsp_start is the first C function of the bootstrap processor
break bsp_start
// go to the C code
continue
// get information about the other processors/threads
info threads
// do a step with the third processor
-exec-next --thread 3 1
*stopped,reason="signal-received",signal-name="SIGTRAP",signal-meaning="Trace/breakpoint
trap",frame={addr="0x80004354",func="print",args=[{name="text",value="0x0"}],file="drivers/printer.c",fullname="/home/tomas/runtime-EclipseApplication/kalisto/kernel/drivers/printer.c",line="27"},thread-id="1",stopped-threads="all"
// the first processor was stepped instead the third one

Here is the complete session log (without the remote debugging messages):

tomas@tomas-laptop:~/workspaceGDB/gdb-7.2/gdb$ ./gdb --interpreter mi2
=thread-group-added,id="i1"
~"GNU gdb (GDB) 7.2\n"
~"Copyright (C) 2010 Free Software Foundation, Inc.\n"
~"License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>\nThis is free software: you are
free to change and redistribute it.\nThere is NO WARRANTY, to the
extent permitted by law.  Type \"show copying\"\nand \"show warranty\"
for details.\n"
~"This GDB was configured as \"--host=i686-pc-linux-gnu
--target=mips\".\nFor bug reporting instructions, please see:\n"
~"<http://www.gnu.org/software/gdb/bugs/>.\n"
~"The target is assumed to be little endian\n"
(gdb)
set debug remote 1
^done
(gdb)
target remote :10001
~"Remote debugging using :10001\n"
=thread-group-started,id="i1",pid="42000"
=thread-created,id="1",group-id="i1"
~"0xbfc00000 in ?? ()\n"
*stopped,frame={addr="0xbfc00000",func="??",args=[]},thread-id="1",stopped-threads="all"
^done
(gdb)
break bsp_start
~"Breakpoint 1 at 0x80002540: file main.c, line 69.\n"
^done
(gdb)
continue
~"Continuing.\n"
^running
*running,thread-id="all"
(gdb)
~"\nBreakpoint "
~"1, bsp_start () at main.c:69\n"
~"69\t\tprint (\"Initializing Kalisto\\n\");\n"
*stopped,frame={addr="0x80002540",func="bsp_start",args=[],file="main.c",fullname="/home/tomas/runtime-EclipseApplication/kalisto/kernel/main.c",line="69"},thread-id="1",stopped-threads="all"
(gdb)
info threads
=thread-created,id="2",group-id="i1"
~"[New Thread 2]\n"
=thread-created,id="3",group-id="i1"
~"[New Thread 3]\n"
~"  3 Thread 3 (CPU 3)  0x80002654 in ap_start () at main.c:129\n"
~"  2 Thread 2 (CPU 2)  0x80002654 in ap_start () at main.c:129\n"
~"* 1 Thread 1 (CPU 1)  bsp_start () at main.c:69\n"
^done
(gdb)
-exec-next --thread 3 1
^running
*running,thread-id="1"
(gdb)
*running,thread-id="all"
~"[Switching to Thread 1]\n"
*stopped,reason="signal-received",signal-name="SIGTRAP",signal-meaning="Trace/breakpoint
trap",frame={addr="0x80004354",func="print",args=[{name="text",value="0x0"}],file="drivers/printer.c",fullname="/home/tomas/runtime-EclipseApplication/kalisto/kernel/drivers/printer.c",line="27"},thread-id="1",stopped-threads="all"
=thread-selected,id="1"
(gdb)

I have also the same log but with the remote debugging messages. It is
quite long, so I paste only the relevant part:

-exec-next --thread 3 1
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080000000004c030080040040000000000000000000000000000000000054260080\n"
&"Sending packet: $Hg1#e0..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d804008038030080040040000000000000000000000000000000000040250080\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d804008038030080040040000000000000000000000000000000000040250080\n"
&"Sending packet: $Hc1#dc..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $s#73..."
&"Ack\n"
^running
*running,thread-id="1"
(gdb)
&"Packet received: T05thread:00000001;25:44250080;\n"
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080000000004c030080040040000000000000000000000000000000000058260080\n"
&"Sending packet: $Hc0#db..."
&"Ack\n"
&"Packet received: \n"
&"Sending packet: $s#73..."
&"Ack\n"
*running,thread-id="all"
&"Packet received: T05thread:00000001;25:48250080;\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d80400804c250080040040000000000000000000000000000000000048250080\n"
&"Sending packet: $T3#87..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080d80600804c03008004004000000000000000000000000000000000005c260080\n"
&"Sending packet: $Hg1#e0..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d80400804c250080040040000000000000000000000000000000000048250080\n"
&"Sending packet: $Z0,80002540,4#a9..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $Hg3#e2..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000200000000000080d8060080d80600804c03008004004000000000000000000000000000000000005c260080\n"
&"Sending packet: $s#73..."
&"Ack\n"
&"Packet received: T05thread:00000001;25:54430080;\n"
&"Sending packet: $g#67..."
&"Ack\n"
&"Packet received:
00000000000000000000008000000000ec430080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000080d8040080d80400804c250080040040000000000000000000000000000000000054430080\n"
~"[Switching to Thread 1]\n"
&"Sending packet: $z0,80002540,4#c9..."
&"Ack\n"
&"Packet received: OK\n"
&"Sending packet: $m800004c0,40#bc..."
&"Ack\n"
&"Packet received:
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000038030080\n"
*stopped,reason="signal-received",signal-name="SIGTRAP",signal-meaning="Trace/breakpoint
trap",frame={addr="0x80004354",func="print",args=[{name="text",value="0x0"}],file="drivers/printer.c",fullname="/home/tomas/runtime-EclipseApplication/kalisto/kernel/drivers/printer.c",line="27"},thread-id="1",stopped-threads="all"
=thread-selected,id="1"

From the debugging messages I can see, that the GDB steps twice. The
first step is for the first thread - maybe because the first thread
has just raised the breakpoint. Before the second step the GDB tries
to change the current thread in the virtual machine to 0 (means ANY
according to the doc). That is not supported in the virtual machine
and the machine responds so. I believe not supporting the Hc0 command
is not wrong for the virtual machine. Anyway the second step is aimed
for ANY thread and it is applied for the first thread, which both
leads to the unexpected behaviour. I think that the GDB should send
Hc3 command to the simulator before the second step.

I have also tried to find out why the Hc3 command is not send from the
GDB sources. The thread id that will be step is determined in resume
function of target.c. In my case the RESUME_ALL ptid is used. That
ptid leads to the Hc0 command. So far I have not understood why the
resume_ptid is kept in the default RESUME_ALL.

Sorry for quite a story. I have been solving it for some time and
advice would be appreciated. Any idea why the GDB does not send the
Hc3 command? Maybe I just cannot see something simple.

Regards,
Tomas