Toward multicore GDB - Overview

[Stan Shebs <stanshebs at earthlink dot net>]

Ericsson has funded Mentor Graphics to work on several GDB projects, 
under the general umbrella of improving multicore support.  This is a 
high-level view of what we're doing, so as to keep everybody informed.

GDB has long had "multicore support" in the sense that cores are a 
hardware-level concept, so the target operating system often obscures 
them and randomly assigns processes and/or threads to different cores.  
Likewise, for bare-metal targets, it is not too hard to have the 
target-side debugging stub pretend cores are like long-lived threads and 
simply use GDB's existing thread support.  Thanks to some hard work, the 
threads/cores can even be of different architectures.

This all works reasonably for the 2-8 core case that is common today, 
but the next generation of high-performance systems include anywhere 
from 10 to 1,000 cores.  Application areas such as radio signal handling 
(aka cell phone towers) and network packet processing are highly 
parallelizable, and developers are today busily writing programs that 
have outstripped their tools' ability to debug them. Worse, even printf 
loses when a thousand threads are all trying to do output at the same 
time. :-)

The key to debugging these new programs is to shift from working with 
individual threads and processes to handling them in groups.  This was 
presaged years ago by the process/thread set concept defined in the 
High-Performance Debugging Forum (HPDF) spec, and which is today 
implemented in the TotalView debugger.  We are extending this to include 
cores, and call them ptc sets.

The second issue that comes to the fore is that some of GDB's usual 
practices start to break down.  As folks here know, a single step 
command can result in a flurry of duck-feet paddling beneath the 
surface, as GDB single-steps instructions looking for the next source 
line, or tests a breakpoint conditional over and over.  A thousand 
threads on a thousand cores can literally demand GDB's attention all at 
the same instant; at best, debugging becomes unusably slow, at worst 
bugs disappear due to the timing being thrown so far off.

In subsequent messages, I'll go into these two major areas in some more 
depth.


Stan
stan@codesourcery.com