When doing a build (at commit ae86f581c2a) and test-run on x86_64 ubuntu 18.04.5, I run into: ... FAIL: gdb.base/gdb-sigterm.exp: expect eof #31 (GDB internal error) ... In more detail: ... /home/vries/gdb/src/gdb/thread.c:95: internal-error: thread_info* inferior_thread(): Assertion `current_thread_ != nullptr' failed.^M A problem internal to GDB has been detected,^M further debugging may prove unreliable.^M Quit this debugging session? (y or n) Recursive internal problem.^M FAIL: gdb.base/gdb-sigterm.exp: expect eof #31 (GDB internal error) ...
Created attachment 12913 [details] gzipped gdb.log
Created attachment 12998 [details] same bug Mac Os Big Sur
*** Bug 27043 has been marked as a duplicate of this bug. ***
Nevermind about the duplicate notification above, it was a mistake.
Created attachment 13229 [details] Reproduced on macOS Mojave 10.14.6 (18G8022)
I encounter a similar bug. Easy to reproduce. On Arch Linux: $ gdb --version GNU gdb (GDB) 10.2 [snip] $ gdb -q -ex run /bin/true Reading symbols from /bin/true... (No debugging symbols found in /bin/true) Starting program: /usr/bin/true [Inferior 1 (process 339) exited normally] ../../gdb/thread.c:95: internal-error: thread_info* inferior_thread(): Assertion `current_thread_ != nullptr' failed. A problem internal to GDB has been detected, further debugging may prove unreliable. This is a bug, please report it. For instructions, see: <https://www.gnu.org/software/gdb/bugs/>. Aborted (core dumped) $ cat ~/.gdbinit define hook-quit set confirm off end python def exit_handler ( event ): if hasattr ( event, 'exit_code' ) and event .exit_code == 0: gdb .execute ( "quit" ) gdb .events .exited .connect ( exit_handler ) end ---- I encounter a similar bug on Ubuntu 20.04. $ gdb --version GNU gdb (Ubuntu 9.2-0ubuntu1~20.04) 9.2 [snip] $ gdb -q -ex run /bin/true Reading symbols from /bin/true... (No debugging symbols found in /bin/true) Starting program: /usr/bin/true [Inferior 1 (process 4134946) exited normally] /build/gdb-OxeNvS/gdb-9.2/gdb/inferior.c:283: internal-error: inferior* find_inferior_pid(int): Assertion `pid != 0' failed. A problem internal to GDB has been detected, further debugging may prove unreliable. This is a bug, please report it. For instructions, see: <http://www.gnu.org/software/gdb/bugs/>. Aborted (core dumped) Possibly related: https://sourceware.org/bugzilla/show_bug.cgi?id=27682
(In reply to Parke from comment #6) > I encounter a similar bug. Easy to reproduce. > > On Arch Linux: > > $ gdb --version > GNU gdb (GDB) 10.2 > [snip] > > $ gdb -q -ex run /bin/true > Reading symbols from /bin/true... > (No debugging symbols found in /bin/true) > Starting program: /usr/bin/true > [Inferior 1 (process 339) exited normally] > ../../gdb/thread.c:95: internal-error: thread_info* inferior_thread(): > Assertion `current_thread_ != nullptr' failed. > A problem internal to GDB has been detected, > further debugging may prove unreliable. > > This is a bug, please report it. For instructions, see: > <https://www.gnu.org/software/gdb/bugs/>. > > Aborted (core dumped) > > $ cat ~/.gdbinit > define hook-quit > set confirm off > end > > python > > def exit_handler ( event ): > if hasattr ( event, 'exit_code' ) and event .exit_code == 0: > gdb .execute ( "quit" ) > > gdb .events .exited .connect ( exit_handler ) > > end > > ---- > > I encounter a similar bug on Ubuntu 20.04. > > $ gdb --version > GNU gdb (Ubuntu 9.2-0ubuntu1~20.04) 9.2 > [snip] > > $ gdb -q -ex run /bin/true > Reading symbols from /bin/true... > (No debugging symbols found in /bin/true) > Starting program: /usr/bin/true > [Inferior 1 (process 4134946) exited normally] > /build/gdb-OxeNvS/gdb-9.2/gdb/inferior.c:283: internal-error: inferior* > find_inferior_pid(int): Assertion `pid != 0' failed. > A problem internal to GDB has been detected, > further debugging may prove unreliable. > > This is a bug, please report it. For instructions, see: > <http://www.gnu.org/software/gdb/bugs/>. > > Aborted (core dumped) > > Possibly related: > https://sourceware.org/bugzilla/show_bug.cgi?id=27682 Although the symptom is the same, the root cause for this particular assertion to fail can be one of many things. So I would suggest filing a new bug, with a title that is specific to what you are doing (e.g. "Assertion failure when using Python exit event handler".
(In reply to Parke from comment #6) [snip] (In reply to Simon Marchi from comment #7) > Although the symptom is the same, the root cause for this particular > assertion to fail can be one of many things. So I would suggest filing a > new bug, with a title that is specific to what you are doing (e.g. > "Assertion failure when using Python exit event handler". Done. https://sourceware.org/bugzilla/show_bug.cgi?id=28352
This seems to fix it: ... diff --git a/gdb/thread.c b/gdb/thread.c index 378c5ee2d13..21d21c9d1c6 100644 --- a/gdb/thread.c +++ b/gdb/thread.c @@ -631,7 +631,8 @@ any_thread_of_inferior (inferior *inf) gdb_assert (inf->pid != 0); /* Prefer the current thread, if there's one. */ - if (inf == current_inferior () && inferior_ptid != null_ptid) + if (inf == current_inferior () && inferior_ptid != null_ptid + && current_thread_ != nullptr) return inferior_thread (); for (thread_info *tp : inf->non_exited_threads ()) ...
(In reply to Tom de Vries from comment #9) > This seems to fix it: > ... > diff --git a/gdb/thread.c b/gdb/thread.c > index 378c5ee2d13..21d21c9d1c6 100644 > --- a/gdb/thread.c > +++ b/gdb/thread.c > @@ -631,7 +631,8 @@ any_thread_of_inferior (inferior *inf) > gdb_assert (inf->pid != 0); > > /* Prefer the current thread, if there's one. */ > - if (inf == current_inferior () && inferior_ptid != null_ptid) > + if (inf == current_inferior () && inferior_ptid != null_ptid > + && current_thread_ != nullptr) > return inferior_thread (); > > for (thread_info *tp : inf->non_exited_threads ()) > ... Full test run on x86_64-linux showed no regressions.
Is it possible that this series here is addressing this bug? https://inbox.sourceware.org/gdb-patches/20220227000051.3336149-1-kevinb@redhat.com/
Alternatively: ... diff --git a/gdb/proc-service.c b/gdb/proc-service.c index dd3d7c9201e..88f07d78236 100644 --- a/gdb/proc-service.c +++ b/gdb/proc-service.c @@ -73,13 +73,9 @@ ps_xfer_memory (const struct ps_prochandle *ph, psaddr_t addr, gdb_byte *buf, size_t len, int write) { scoped_restore_current_inferior restore_inferior; - set_current_inferior (ph->thread->inf); - scoped_restore_current_program_space restore_current_progspace; - set_current_program_space (ph->thread->inf->pspace); - scoped_restore save_inferior_ptid = make_scoped_restore (&inferior_ptid); - inferior_ptid = ph->thread->ptid; + switch_to_thread_no_regs (ph->thread); CORE_ADDR core_addr = ps_addr_to_core_addr (addr); ...
(In reply to Simon Marchi from comment #11) > Is it possible that this series here is addressing this bug? > > https://inbox.sourceware.org/gdb-patches/20220227000051.3336149-1- > kevinb@redhat.com/ Ack, let me check.
(In reply to Tom de Vries from comment #13) > (In reply to Simon Marchi from comment #11) > > Is it possible that this series here is addressing this bug? > > > > https://inbox.sourceware.org/gdb-patches/20220227000051.3336149-1- > > kevinb@redhat.com/ > > Ack, let me check. I build gdb-12.1-release tag, reproduced the problem, then applied the patch series, rebuild, and no more problem.
The master branch has been updated by Kevin Buettner <kevinb@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=522044dc5fa76f9fef70fe746274daf09bbf64fe commit 522044dc5fa76f9fef70fe746274daf09bbf64fe Author: Kevin Buettner <kevinb@redhat.com> Date: Mon Feb 27 16:11:37 2023 -0700 Introduce gdb_exception_forced_quit This commit adds a new exception 'gdb_exception_forced_quit', reason code 'REASON_FORCED_QUIT', return mask 'RETURN_MASK_FORCED_QUIT', and a wrapper for throwing the exception, throw_forced_quit(). The addition of this exception plus supporting code will allow us to recognize that a SIGTERM has been received by GDB and then propagate recognition of that fact to the upper levels of GDB where it can be correctly handled. At the moment, when GDB receives a SIGTERM, it will attempt to exit via a series of calls from the QUIT checking code. However, before it can exit, it must do various cleanups, such as killing or detaching all inferiors. Should these cleanups be attempted while GDB is executing very low level code, such as reading target memory from within ps_xfer_memory(), it can happen that some of GDB's state is out of sync with regard to the cleanup code's expectations. In the case just mentioned, it's been observed that inferior_ptid and the current_thread_ are not in sync; this triggers an assert / internal error. This commit only introduces the exception plus supporting machinery; changes which use this new exception are in later commits in this series. Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=26761 Tested-by: Tom de Vries <tdevries@suse.de> Approved-by: Pedro Alves <pedro@palves.net>
The master branch has been updated by Kevin Buettner <kevinb@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=63509715af867d635ad0e8cfe5a662bfc67b4ddf commit 63509715af867d635ad0e8cfe5a662bfc67b4ddf Author: Kevin Buettner <kevinb@redhat.com> Date: Mon Feb 27 16:11:37 2023 -0700 Handle gdb SIGTERM by throwing / catching gdb_exception_force_quit When a GDB process receives the SIGTERM signal, handle_sigterm() in event-top.c is called. The global variable 'sync_quit_force_run' is set by this signal handler. It does some other things too, but the setting of this global is the important bit for the SIGTERM part of this discussion. GDB will periodically check to see whether a Ctrl-C or SIGTERM has been received. This is performed via use of the QUIT macro in GDB's code. QUIT is defined to invoke maybe_quit(), which will be periodically called during any lengthy operation. This is supposed to ensure that the user won't have to wait too long for a Ctrl-C or SIGTERM to be acted upon. When a Ctrl-C / SIGINT is received, quit_handler() will decide whether to pass the SIGINT onto the inferior or to call quit() which causes gdb_exception_quit to be thrown. This exception (usually) propagates to the top level. Control is then returned to the top level event loop. At the moment, SIGTERM is handled very differently. Instead of throwing an exception, quit_force() is called. This does eventually cause GDB to exit(), but prior to that happening, the inferiors are killed or detached and other target related cleanup occurs. As shown in this discussion between Pedro Alves and myself... https://sourceware.org/pipermail/gdb-patches/2021-July/180802.html https://sourceware.org/pipermail/gdb-patches/2021-July/180902.html https://sourceware.org/pipermail/gdb-patches/2021-July/180903.html ...we found that it is possible for inferior_ptid and current_thread_ to get out of sync. When that happens, the "current_thread_ != nullptr" assertion in inferior_thread() can fail resulting in a GDB internal error. Pedro recommended that we "let the normal quit exception propagate all the way to the top level, and then have the top level call quit_force if sync_quit_force_run is set." However, after the v2 series for this patch set, we tweaked that idea by introducing a new exception for handling SIGTERM. This commit implements the obvious part of Pedro's suggestion: Instead of calling quit_force from quit(), throw_forced_quit() is now called instead. This causes the new exception 'gdb_exception_forced_quit' to be thrown. At the top level, I changed catch_command_errors() and captured_main() to catch gdb_exception_forced_quit and then call quit_force() from the catch block. I also changed start_event_loop() to also catch gdb_exception_forced_quit; while we could also call quit_force() from that catch block, it's sufficient to simply rethrow the exception since it'll be caught by the newly added code in captured_main(). Making these changes fixed the failure / regression that I was seeing for gdb.base/gdb-sigterm.exp when run on a machine with glibc-2.34. However, there are many other paths back to the top level which this test case does not test. I did an audit of all of the try / catch code in GDB in which calls in the try-block might (eventually) call QUIT. I found many cases where gdb_exception_quit and the new gdb_exception_forced_quit will be swallowed. (When using GDB, have you ever hit Ctrl-C and not have it do anything; if so, it could be due to a swallowed gdb_exception_quit in one of the cases I've identified.) The rest of the patches in this series deal with this concern. Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=26761 Tested-by: Tom de Vries <tdevries@suse.de> Approved-by: Pedro Alves <pedro@palves.net>
The master branch has been updated by Kevin Buettner <kevinb@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=b1ffd1124a8c5170a9e06b867a886b1138d28514 commit b1ffd1124a8c5170a9e06b867a886b1138d28514 Author: Kevin Buettner <kevinb@redhat.com> Date: Mon Feb 27 16:11:37 2023 -0700 Catch gdb_exception_error instead of gdb_exception (in many places) As described in the previous commit for this series, I became concerned that there might be instances in which a QUIT (due to either a SIGINT or SIGTERM) might not cause execution to return to the top level. In some (though very few) instances, it is okay to not propagate the exception for a Ctrl-C / SIGINT, but I don't think that it is ever okay to swallow the exception caused by a SIGTERM. Allowing that to happen would definitely be a deviation from the current behavior in which GDB exits upon receipt of a SIGTERM. I looked at all cases where an exception handler catches a gdb_exception. Handlers which did NOT need modification were those which satisifed one or more of the following conditions: 1) There is no call path to maybe_quit() in the try block. I used a static analysis tool to help make this determination. In instances where the tool didn't provide an answer of "yes, this call path can result in maybe_quit() being called", I reviewed it by hand. 2) The catch block contains a throw for conditions that it doesn't want to handle; these "not handled" conditions must include the quit exception and the new "forced quit" exception. 3) There was (also) a catch for gdb_exception_quit. Any try/catch blocks not meeting the above conditions could potentially swallow a QUIT exception. My first thought was to add catch blocks for gdb_exception_quit and then rethrow the exception. But Pedro pointed out that this can be handled without adding additional code by simply catching gdb_exception_error instead. That's what this patch series does. There are some oddball cases which needed to be handled differently, plus the extension languages, but those are handled in later patches. Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=26761 Tested-by: Tom de Vries <tdevries@suse.de> Approved-by: Pedro Alves <pedro@palves.net>
The master branch has been updated by Kevin Buettner <kevinb@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=b940a061c0d549dbe981463414da87cb84a8a9bb commit b940a061c0d549dbe981463414da87cb84a8a9bb Author: Kevin Buettner <kevinb@redhat.com> Date: Mon Feb 27 16:11:37 2023 -0700 Python QUIT processing updates See the previous patches in this series for the motivation behind these changes. This commit contains updates to Python's QUIT handling. Ideally, we'd like to throw gdb_exception_forced_quit through the extension language; I made an attempt to do this for gdb_exception_quit in an earlier version of this patch, but Pedro pointed out that it is (almost certainly) not safe to do so. Still, we definitely don't want to swallow the exception representing a SIGTERM for GDB, nor do we want to force modules written in the extension language to have to explicitly handle this case. Since the idea is for GDB to cleanup and quit for this exception, we'll simply call quit_force() just as if the gdb_exception_forced_quit propagation had managed to make it back to the top level. Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=26761 Tested-by: Tom de Vries <tdevries@suse.de> Approved-By: Pedro Alves <pedro@palves.net>
The master branch has been updated by Kevin Buettner <kevinb@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=53f1f3d4aa7019a159d939b7bd2a27a93d630fda commit 53f1f3d4aa7019a159d939b7bd2a27a93d630fda Author: Kevin Buettner <kevinb@redhat.com> Date: Mon Feb 27 16:11:37 2023 -0700 Guile QUIT processing updates This commit contains QUIT processing updates for GDB's Guile support. As with the Python updates, we don't want to permit this code to swallow the exception, gdb_exception_forced_quit, which is associated with GDB receiving a SIGTERM. I've adopted the same solution that I used for Python; whereever a gdb_exception is caught in try/catch code in the Guile extension language support, a catch for gdb_exception_forced_quit has been added; this catch block will simply call quit_force(), which will cause the necessary cleanups to occur followed by GDB exiting. Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=26761 Tested-by: Tom de Vries <tdevries@suse.de> Approved-By: Pedro Alves <pedro@palves.net>
The master branch has been updated by Kevin Buettner <kevinb@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=363429d593eb1eb17f3f2cf6c247f60fadf660d9 commit 363429d593eb1eb17f3f2cf6c247f60fadf660d9 Author: Kevin Buettner <kevinb@redhat.com> Date: Mon Feb 27 16:11:37 2023 -0700 QUIT processing w/ explicit throw for gdb_exception_forced_quit This commit contains changes which have an explicit throw for gdb_exception_forced_quit, or, in a couple of cases for gdb_exception, but with a throw following a check to see if 'reason' is RETURN_FORCED_QUIT. Most of these are straightforward - it made sense to continue to allow an existing catch of gdb_exception to also catch gdb_exception_quit; in these cases, a catch/throw for gdb_exception_forced_quit was added. There are two cases, however, which deserve a more detailed explanation. 1) remote_fileio_request in gdb/remote-fileio.c: The try block calls do_remote_fileio_request which can (in turn) call one of the functions in remote_fio_func_map[]. Taking the first one, remote_fileio_func_open(), we have the following call path to maybe_quit(): remote_fileio_func_open(remote_target*, char*) -> target_read_memory(unsigned long, unsigned char*, long) -> target_read(target_ops*, target_object, char const*, unsigned char*, unsigned long, long) -> maybe_quit() Since there is a path to maybe_quit(), we must ensure that the catch block is not permitted to swallow a QUIT representing a SIGTERM. However, for this case, we must take care not to change the way that Ctrl-C / SIGINT is handled; we want to send a suitable EINTR reply to the remote target should that happen. That being the case, I added a catch/throw for gdb_exception_forced_quit. I also did a bit of rewriting here, adding a catch for gdb_exception_quit in favor of checking the 'reason' code in the catch block for gdb_exception. 2) mi_execute_command in gdb/mi/mi-main.c: The try block calls captured_mi_execute_command(); there exists a call path to maybe_quit(): captured_mi_execute_command(ui_out*, mi_parse*) -> mi_cmd_execute(mi_parse*) -> get_current_frame() -> get_prev_frame_always_1(frame_info*) -> frame_register_unwind_location(frame_info*, int, int*, lval_type*, unsigned long*, int*) -> frame_register_unwind(frame_info*, int, int*, int*, lval_type*, unsigned long*, int*, unsigned char*) -> value_entirely_available(value*) -> value_fetch_lazy(value*) -> value_fetch_lazy_memory(value*) -> read_value_memory(value*, long, int, unsigned long, unsigned char*, unsigned long) -> maybe_quit() That being the case, we can't allow the exception handler (catch block) to swallow a gdb_exception_quit for SIGTERM. However, it does seem reasonable to output the exception via the mi interface so that some suitable message regarding SIGTERM might be printed; therefore, I check the exception's 'reason' field for RETURN_FORCED_QUIT and do a throw for this case. Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=26761 Tested-by: Tom de Vries <tdevries@suse.de> Approved-By: Pedro Alves <pedro@palves.net>
The master branch has been updated by Kevin Buettner <kevinb@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=80d03917838c16ee0da53a4a8642d5df3bee724e commit 80d03917838c16ee0da53a4a8642d5df3bee724e Author: Kevin Buettner <kevinb@redhat.com> Date: Mon Feb 27 16:11:37 2023 -0700 Introduce set_force_quit_flag and change type of sync_quit_force_run At the moment, handle_sigterm() in event-top.c does the following: sync_quit_force_run = 1; set_quit_flag (); This was used several more times in a later patch in this series, so I'm introducing (at Pedro's suggestion) a new function named 'set_force_quit_flag'. It simply sets sync_quit_force_run and also calls set_quit_flag(). I've revised the later patch to call set_force_quit_flag instead. I noticed that sync_quit_force_run is declared as an int but is being used as a bool, so I also changed its type to bool in this commit. Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=26761 Approved-By: Pedro Alves <pedro@palves.net>
The master branch has been updated by Kevin Buettner <kevinb@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=96e3f4e3c32865fecdb266c2a41532337c75e316 commit 96e3f4e3c32865fecdb266c2a41532337c75e316 Author: Kevin Buettner <kevinb@redhat.com> Date: Mon Feb 27 16:11:37 2023 -0700 Forced quit cases handled by resetting sync_quit_force_run During my audit of the use of gdb_exception with regard to QUIT processing, I found a try/catch in the scoped_switch_fork_info destructor. Static analysis found this call path from the destructor to maybe_quit(): scoped_switch_fork_info::~scoped_switch_fork_info() -> remove_breakpoints() -> remove_breakpoint(bp_location*) -> remove_breakpoint_1(bp_location*, remove_bp_reason) -> memory_validate_breakpoint(gdbarch*, bp_target_info*) -> target_read_memory(unsigned long, unsigned char*, long) -> target_read(target_ops*, target_object, char const*, unsigned char*, unsigned long, long) -> maybe_quit() Since it's not safe to do a 'throw' from a destructor, we simply call set_quit_flag and, for gdb_exception_forced_quit, also set sync_quit_force_run. This will cause the appropriate exception to be rethrown at the next QUIT check. Another case is the try / catch in tui_getc() in tui-io.c. The existing catch swallows the exception. I've added a catch for 'gdb_exception_forced_quit', which also swallows the exception, but also sets sync_quit_force_run and calls set_quit_flag in order to restart forced quit processing at the next QUIT check. This is required because it isn't safe to throw into/through readline. Thanks to Pedro Alves for suggesting this idea. Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=26761 Tested-by: Tom de Vries <tdevries@suse.de> Approved-By: Pedro Alves <pedro@palves.net>
Fix is in upstream sources now.