[RFC/WIP PATCH 00/14] I/T sets

[Pedro Alves <pedro at codesourcery dot com>]

Following up on <http://sourceware.org/ml/gdb/2011-11/msg00013.html>.

This patch set seeds an implementation of a general
inferior/process/thread/core set construct, or inferior/thread set
(I/T set, or just itset) for short, which should allow users to refer
to large numbers of inferiors, processes, threads, and cores in a
consistent way and using a flexible syntax.  IPTC sets are a
collection of arbitrarily many processes, threads, and cores.  A set's
definition may consist of a combination enumerated inferiors,
processes, threads, and cores, ranges of these, wildcards, regular
expression matches on names, predicate tests, logical disjunction, and
logical negation.  The full detailed syntax is still a WIP, and not
cast in stone.  This WIP series presents unfinished, but already
useable changes, that are in good enough shape to allow others to
experiment, and perhaps even help.  :-)

Having implemented base chunks of the HPD specified syntax, along with
the '@' extension as explained by Stan in the email linked above, I've
been wondering whether the '@' suffix syntax to specify cores is a
good idea.  The <inferior>.<thread> @ <core> syntax leaves out Ada
tasks, and process ids (pid) and and target thread ids.  It is also
not commutable.  That is, for example, it is not possible to say all
threads in named set "workers" at inferior 1.  Along the idea that we
need an intersection operator somehow, an idea I've been kicking in
the background, is to make all kinds of objects have the same stand,
and require, say, a one letter prefix to identify what kind of object
we're specifying.  E.g, i for inferior, p for process, t for thread, c
for core and a for Ada task.  In this scheme, the '.'  is really a set
intersection operator, and the ',' is the union operator.  I'm very
much inclined to try this route, but I'm also very interested in
learning other's opinions.  Here are examples:

| Syntax                  | Meaning                                           |
|-------------------------+---------------------------------------------------|
| i1.t1                   | inferior 1, thread 1                              |
|-------------------------+---------------------------------------------------|
| p123                    | all (threads cores, etc.) of process with pid 123 |
|-------------------------+---------------------------------------------------|
| i1.t[1-100].c1-3        | threads 1 to 100 of iinferior 1, running on       |
|                         | cores 1 to 3                                      |
|-------------------------+---------------------------------------------------|
| i1.t*.c1-3,workers      | same as above, union with all in the user         |
|                         | defined set "workers"                             |
|-------------------------+---------------------------------------------------|
| workers.~c1-3           | All in the user defined set "workers", except     |
|                         | those on cores 1 to 3                             |
|-------------------------+---------------------------------------------------|
| i1.a1-3                 | Ada tasks 1 to 3 of inferior 1                    |
|-------------------------+---------------------------------------------------|
| workers.i1,crunchers.i2 | All workers of inferior 1, and                    |
|                         | all crunchers of inferior 2                       |
|-------------------------+---------------------------------------------------|

WDYT?

The current implementation, borrowing heavilly from the HPD and the
syntax we first thought out (see Stan's link above), currently
implements the INF.TID form, and as a separate patch, the @core
extension.  The itset framework makes it easy to switch to a scheme as
above though, mostly because I already implemented the @ suffix
support as an intersection of sets.

Borrowing from HPD, either static sets or dynamic sets may be
specified.  If dynamic, membership will be re-evaluated each time the
set is consulted; otherwise, membership is fixed at the time the set
is created.

Commands may ignore a set if it does not make sense for the command.
In many cases the set will act similarly to an iterator, but there is
no guarantee that each member of the set will be handled in any
particular order, and the members may be handled simultaneously if the
target supports that.

For instance, breakpoints will be enhanced to include a trigger set
defining the inferiors, processes, threads and cores to which the
breakpoint is restricted, and another set defining inferiors,
processes, threads and cores that should be suspended when the
breakpoint is triggered.  Breakpoint hits will be analyzed to
determined whether the stopped thread or core fall into the given
trigger set.

A focus command will be added, which will allow the user to select a
particular set as the default context for subsequent GDB operations,
similarly to what the "thread" command does for threads.  As user
feedback for the command line, the prompt will have a way to be
modified to indicate the current focus.  An explicit set added to a
command will always override the current focus.

The following examples exhibit some possible uses of sets:

|-------------------------------------------+----------------------------------|
| Commands                                  | Behaviour                        |
|-------------------------------------------+----------------------------------|
| (gdb) step [.34-59]                       | Single-step the current          |
| <lines stepped over>                      | process's threads whose          |
| (gdb)                                     | numbers are in the range 34      |
|                                           | through 59, inclusive            |
|-------------------------------------------+----------------------------------|
| (gdb) step [@2]                           | Single-step all threads on       |
| <line stepped over>                       | core 2                           |
| (gdb)                                     |                                  |
|-------------------------------------------+----------------------------------|
| (gdb) [crunchers] print -a globvar        | For a set of processes that have |
| [1235] $1 = 45                            | been given the name "crunchers", |
| [1236] $2 = 92                            | print a global value that is in  |
| (gdb)                                     | all of them                      |
|-------------------------------------------+----------------------------------|
| (gdb) [.4] break mfunction                | Set a "traditional"              |
|                                           | thread-specific breakpoint that  |
|                                           | applies to only one thread       |
|-------------------------------------------+----------------------------------|
| (gdb) [!.*] break -stop [wrkrs] mfunction | Set a breakpoint that applies    |
|                                           | to all current threads,          |
|                                           | and suspend all threads in       |
|                                           | the named set "wrkrs" when       |
|                                           | it is hit.                       |
|-------------------------------------------+----------------------------------|


At first, I was trying to follow the HPD spec to the letter, as much
as possible.  I added support for having "[foo] step" step all threads
in [foo] in parallel, but then I noticed this wasn't exactly going to
work.  The problem is that this breaks hardly with GDB's concept of
"selected thread", I think in a fatal way.  Let me clarify.

HPD specifies that:

  [I/T SET] step

steps all threads in the I/T SET.  That is, if there are 3 threads in
I/T SET, all of then are stepped.  However, I don't think this is going to
fly with GDB.  Consider:

 - most execution commands (other than "continue"), actually take two
   sets.  Taking "step" as example, we have:

   1 - the set of threads that is really going to be stepped.
   2 - the set of threads that is allowed to resume freely while
      the stepped threads aren't done with their thing.

     #1 is, on a non-set aware gdb, the currently selected thread.
     #2 is, on a non-set aware gdb, decided by a not very user friendly
        mix of non-stop mode, scheduler-locking and schedule-multiple modes.

   I think that the user will most often want to "focus" on a given set
   of threads, most often a whole inferior, like 

   (gdb) itset [1.*]

   which makes the current scope be [1.*].  With a smart prompt, you'd
   see something like this as prompt:

   [1.*]>

   now say you want to step thread 2, but let all others run (as gdb does
   by default currently).  In HPD, you'd do this:

   [1.*]>    [.2]    step
   ^^^^^^
   prompt   prefix  cmd

   what runs free is left to a global setting, similar to our
   non-stop mode, but specific for what should be resumed, by
   default set to e.g., resume the whole process.

   However, what if the user does:

   [foo]> itset [1.2]
   [1.2]> step

   Here the user has explicitly said it only wants to focus on thread 2.
   What should the step above resume freely?  HPD seems to suggest that
   the global setting should also be respected, which by default means
   the whole process is resumed.

   Let's get back to focusing on the whole process:

   [1.2]> itset [1.*]
   focus set to [1.*]
   [1.*]> step

   And imagine even that we're only debugging inferior 1.
   Now, according to HPD, this would step each and every thread of the
   process.  It's equivalent to issuing:

   [some_default_focus] [1.*]           step
     ^^^^^^^^^^^^^^^^    ^^^
       prompt            explicit set   cmd
    

   But, this conflicts with GDB's concept of a selected thread, and
   will be counter to all of our user's expectations...  Worse, it's
   unimplementable on targets that don't do non-stop, so we'd end up
   with yet another incompatible mode.  I'd very much like to merge
   all modes, not add another!

   Which brings us to the question, where does the concept of gdb's
   selected thread and inferior fit in a i/t set world?

   So, my current thinking (and what's implemented in this series) is
   that we want to define the current focus (as set globally by
   itfocus, or overriden with an itset prefix to any command), as the
   whole set to which a command applies, as before, but, with a twist,
   for execution commands, the set to which a command applies is the
   set of threads which is allowed to be resumed either freely or in
   some command specific maner, not the set over which the command
   will iterate over applying the same operation to all threads.  Let
   me show a few examples:

   (gdb) [1.*] step 

   This steps the selected thread, and lets all other  threads in [1.*] run
   free, until the selected thread stops, or some other event in the [1.*]
   set triggers (like a breakpoint).  This is the equivalent of
   "set schedule-multiple off"+"set non-stop off"+"set scheduler-locking off".

   If the selected thread was not in [1.*] when the command was
   entered, either its an error or the first thread in [1.*] is
   automatically picked as selected thread.  That's undecided yet.

   (gdb) [1.*] step [1.1]

   Step thread 1, and let all threads in [1.*] run free.  The same as
   the example above, but doesn't rely on the selected thread to tell
   which thread to step.

   (gdb) [1.1] step

   Step the selected thread (which was automatically deduced to thread
   1), and let all threads in [1.1] run free, effectivelly, only
   stepping thread 1, and leaving all others as is.

   The same as "set schedule-multiple on" or "set scheduler-locking on/step".

   Then, we can have smarter things like:

   (gdb) [1.*] step [workers]

   to mean, step all worker threads, and let all other threads in
   inferior 1 run free while stepping the workers.

   Note I've used overriding prefixes above, but it's just the same as
   focusing explicitly.  E.g., I could imagine users doing:

   (gdb) itset [serverthreads]
   (gdb) set prompt %ITSET%>
   [serverthreads]> interrupt -a
   all in server threads stop
   [serverthreads]> step [workers]
   * all in the [workers] sep are stepped in parallel, and all others
   in the [serverthreads] set are allowed to run free *

More on this on the patch that adds itsets support to execution
commands.

This series is also available at http://github.com/palves/gdb/tree/itsets-v1
for convenience.

Pedro Alves (14):
      Breakpoints always-inserted and the record target
      Mask software breakpoints from memory writes too
      Flip to set target-async on by default
      Implement all-stop on top of a target running non-stop mode
      Add a small helper to get at a thread's inferior
      Add base itsets support
      Expand %ITSET% in the prompt to the current I/T set.
      Add support for the '@' core operator
      I/T set support for breakpoints - trigger set, and stop set
      Comment out new info breakpoints output, in order to not break the test suite.
      Add I/T set support to most execution commands
      Fix deref of stale pointer
      Make "thread apply all" only loop over threads in the current set
      Fix manythreads.exp test


 gdb/Makefile.in                           |    4 
 gdb/breakpoint.c                          |  332 ++++
 gdb/breakpoint.h                          |   29 
 gdb/cli/cli-decode.c                      |    3 
 gdb/event-top.c                           |   40 +
 gdb/gdbthread.h                           |   29 
 gdb/inf-ptrace.c                          |    4 
 gdb/infcall.c                             |   33 
 gdb/infcmd.c                              |  831 +++++++++--
 gdb/inferior.h                            |    9 
 gdb/infrun.c                              |  859 ++++++++++-
 gdb/itset.c                               | 2248 +++++++++++++++++++++++++++++
 gdb/itset.h                               |   97 +
 gdb/linux-nat.c                           |   26 
 gdb/mem-break.c                           |    8 
 gdb/remote.c                              |   10 
 gdb/target.c                              |   87 +
 gdb/target.h                              |    7 
 gdb/testsuite/gdb.base/jump.exp           |    2 
 gdb/testsuite/gdb.threads/manythreads.exp |   52 +
 gdb/thread.c                              |   19 
 21 files changed, 4355 insertions(+), 374 deletions(-)
 create mode 100644 gdb/itset.c
 create mode 100644 gdb/itset.h

-- 
Pedro Alves