GDB abort on glibc detected file descriptor overflow

[Simon Marchi]

On 2021-09-01 6:37 p.m., Ananthakrishna Sowda (asowda) via Gdb wrote:
> I’m observing abort in GDB 9.2.1 version, and same issue is present in git://sourceware.org/git/binutils-gdb.git tip.
> 
> The full call trace is shown at the end of this message.
> In frame 7,  call to FD_SET is causing buffer overflow when commands from a GDB macro file are processed.
> 
> (gdb) frame 7
> #7  0x000000000076978b in gdb_readline_no_editing (prompt=<optimized out>) at /auto/swtools/prod-builds/src/gdb-9.2.1/gdb/gdb/top.c:850
> 850               FD_SET (fd, &readfds);
> (gdb) p fd
> $1 = 1533
> 
> GDB is processing split dwarf  “.dwp” file for the main executable and processing some “.dwo” files in the workspace, which may have something to do with it.  GDB is opening a bunch of .debug files , one each for every  library  and the open file descriptors  go  past 1024.  This results in buffer overflow when gdb.macros file is opened and processed in frame 7 ( file descriptor 1533).
> 
> The bfd file descriptor caching code which tries to limit no of open descriptors is not effective in this case.
> Does this explanation make  sense? Any ideas to fix this issue are greatly appreciated.

It won't fix the problem, but I think we could start by adding an
assertion before calling FD_SET (everywhere where we do call it):

  gdb_assert (fd < FD_SETSIZE);

Your build happened to catch it, but other builds could just fail
silently or crash in less clear ways.

As for the solution, maybe this code should be converted to use poll or
other more modern APIs to avoid this limit?

It would be interesting if you could show what are the open file
descriptors at this point (list /proc/<pid>/fd), just to see what uses
the most fds.

Simon