I got this error while working on a custom GDB stub which sends two initial stop replies and could reproduce it with gdbserver with the following test: #include <pthread.h> #include <stdio.h> #include <unistd.h> void *thread_func(void *) { sleep(1000); } int main() { printf("pid: %u\n", getpid()); pthread_t t; pthread_create(&t, 0, thread_func, 0); pthread_join(t, 0); return 0; } --- $ ./test pid: 13129 ^Z [1]+ Stopped ./test $ ./gdbserver --attach :5555 13129 & [2] 13578 $ Attached; pid = 13129 Listening on port 5555 $ ./gdb -ex "set non-stop on" -ex "target extended-remote :5555" GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word". Remote debugging using :5555 Remote debugging from host 127.0.0.1, port 59942 Reading /.../test from remote target... warning: File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead. Reading /.../test from remote target... Reading symbols from target:/.../test... ../../gdb-13.2/gdb/remote.c:8352: internal-error: wait: Assertion `!async_event_handler_marked (rs->remote_async_inferior_event_token)' failed. A problem internal to GDB has been detected, further debugging may prove unreliable. ----- Backtrace ----- 0x55e67dce9cbb gdb_internal_backtrace_1 ../../gdb-13.2/gdb/bt-utils.c:122 0x55e67dce9cbb _Z22gdb_internal_backtracev ../../gdb-13.2/gdb/bt-utils.c:168 0x55e67e09ea1c internal_vproblem ../../gdb-13.2/gdb/utils.c:396 0x55e67e09ec3a _Z15internal_verrorPKciS0_P13__va_list_tag ../../gdb-13.2/gdb/utils.c:476 0x55e67e1e787e _Z18internal_error_locPKciS0_z ../../gdb-13.2/gdbsupport/errors.cc:58 0x55e67dfaac74 _ZN13remote_target4waitE6ptid_tP17target_waitstatus10enum_flagsI16target_wait_flagE ../../gdb-13.2/gdb/remote.c:8351 0x55e67e04cb78 _Z11target_wait6ptid_tP17target_waitstatus10enum_flagsI16target_wait_flagE ../../gdb-13.2/gdb/target.c:2579 0x55e67dfa2e18 _ZN13remote_target28process_initial_stop_repliesEi ../../gdb-13.2/gdb/remote.c:4579 0x55e67dfacfe6 _ZN13remote_target14start_remote_1Eii ../../gdb-13.2/gdb/remote.c:5020 0x55e67dfad668 _ZN13remote_target12start_remoteEii ../../gdb-13.2/gdb/remote.c:5072 0x55e67dfad668 _ZN13remote_target6open_1EPKcii ../../gdb-13.2/gdb/remote.c:5875 0x55e67e04d211 open_target ../../gdb-13.2/gdb/target.c:853 0x55e67dd1e0bf _Z8cmd_funcP16cmd_list_elementPKci ../../gdb-13.2/gdb/cli/cli-decode.c:2543 0x55e67e05cfcd _Z15execute_commandPKci ../../gdb-13.2/gdb/top.c:690 0x55e67defca94 catch_command_errors ../../gdb-13.2/gdb/main.c:513 0x55e67defcbb0 execute_cmdargs ../../gdb-13.2/gdb/main.c:608 0x55e67defe31b captured_main_1 ../../gdb-13.2/gdb/main.c:1299 0x55e67defefca captured_main ../../gdb-13.2/gdb/main.c:1320 0x55e67defefca _Z8gdb_mainP18captured_main_args ../../gdb-13.2/gdb/main.c:1345 0x55e67dc2850a main ../../gdb-13.2/gdb/gdb.c:32 --------------------- ../../gdb-13.2/gdb/remote.c:8352: internal-error: wait: Assertion `!async_event_handler_marked (rs->remote_async_inferior_event_token)' failed. A problem internal to GDB has been detected, further debugging may prove unreliable. Quit this debugging session? (y or n) Recursive internal problem. Fatal signal: Aborted ----- Backtrace ----- 0x55e67dce9cbb gdb_internal_backtrace_1 ../../gdb-13.2/gdb/bt-utils.c:122 0x55e67dce9cbb _Z22gdb_internal_backtracev ../../gdb-13.2/gdb/bt-utils.c:168 0x55e67de0d98a handle_fatal_signal ../../gdb-13.2/gdb/event-top.c:956 0x7f14ab8f2f0f ??? 0x7f14ab8f2e87 ??? 0x7f14ab8f47f0 ??? 0x55e67dc2611f abort_with_message ../../gdb-13.2/gdb/utils.c:183 0x55e67e09eb2f internal_vproblem ../../gdb-13.2/gdb/utils.c:343 0x55e67e09ec3a _Z15internal_verrorPKciS0_P13__va_list_tag ../../gdb-13.2/gdb/utils.c:476 0x55e67e1e787e _Z18internal_error_locPKciS0_z ../../gdb-13.2/gdbsupport/errors.cc:58 0x55e67dfaac74 _ZN13remote_target4waitE6ptid_tP17target_waitstatus10enum_flagsI16target_wait_flagE ../../gdb-13.2/gdb/remote.c:8351 0x55e67e04cb78 _Z11target_wait6ptid_tP17target_waitstatus10enum_flagsI16target_wait_flagE ../../gdb-13.2/gdb/target.c:2579 0x55e67de9be49 do_target_wait_1 ../../gdb-13.2/gdb/infrun.c:3736 0x55e67deae47a operator() ../../gdb-13.2/gdb/infrun.c:3795 0x55e67deae47a do_target_wait ../../gdb-13.2/gdb/infrun.c:3814 0x55e67deae47a _Z20fetch_inferior_eventv ../../gdb-13.2/gdb/infrun.c:4172 0x55e67dca3ac0 _Z26check_async_event_handlersv ../../gdb-13.2/gdb/async-event.c:337 0x55e67e1e8a9c _Z16gdb_do_one_eventi ../../gdb-13.2/gdbsupport/event-loop.cc:221 0x55e67e05a3c0 _Z20gdb_readline_wrapperPKc ../../gdb-13.2/gdb/top.c:1173 0x55e67e09de5e defaulted_query ../../gdb-13.2/gdb/utils.c:924 0x55e67e09e19a _Z5queryPKcz ../../gdb-13.2/gdb/utils.c:1016 0x55e67e09eaef internal_vproblem ../../gdb-13.2/gdb/utils.c:407 0x55e67e09ec3a _Z15internal_verrorPKciS0_P13__va_list_tag ../../gdb-13.2/gdb/utils.c:476 0x55e67e1e787e _Z18internal_error_locPKciS0_z ../../gdb-13.2/gdbsupport/errors.cc:58 0x55e67dfaac74 _ZN13remote_target4waitE6ptid_tP17target_waitstatus10enum_flagsI16target_wait_flagE ../../gdb-13.2/gdb/remote.c:8351 0x55e67e04cb78 _Z11target_wait6ptid_tP17target_waitstatus10enum_flagsI16target_wait_flagE ../../gdb-13.2/gdb/target.c:2579 0x55e67dfa2e18 _ZN13remote_target28process_initial_stop_repliesEi ../../gdb-13.2/gdb/remote.c:4579 0x55e67dfacfe6 _ZN13remote_target14start_remote_1Eii ../../gdb-13.2/gdb/remote.c:5020 0x55e67dfad668 _ZN13remote_target12start_remoteEii ../../gdb-13.2/gdb/remote.c:5072 0x55e67dfad668 _ZN13remote_target6open_1EPKcii ../../gdb-13.2/gdb/remote.c:5875 0x55e67e04d211 open_target ../../gdb-13.2/gdb/target.c:853 0x55e67dd1e0bf _Z8cmd_funcP16cmd_list_elementPKci ../../gdb-13.2/gdb/cli/cli-decode.c:2543 0x55e67e05cfcd _Z15execute_commandPKci ../../gdb-13.2/gdb/top.c:690 0x55e67defca94 catch_command_errors ../../gdb-13.2/gdb/main.c:513 0x55e67defcbb0 execute_cmdargs ../../gdb-13.2/gdb/main.c:608 0x55e67defe31b captured_main_1 ../../gdb-13.2/gdb/main.c:1299 0x55e67defefca captured_main ../../gdb-13.2/gdb/main.c:1320 0x55e67defefca _Z8gdb_mainP18captured_main_args ../../gdb-13.2/gdb/main.c:1345 0x55e67dc2850a main ../../gdb-13.2/gdb/gdb.c:32 --------------------- A fatal error internal to GDB has been detected, further debugging is not possible. GDB will now terminate. This is a bug, please report it. For instructions, see: <https://www.gnu.org/software/gdb/bugs/>. Remote side has terminated connection. GDBserver will reopen the connection. Listening on port 5555 Aborted (core dumped)
I too have similar issue with gdb12.x version. Attaching the gdb remote stub log GDB -> $QNonStop:1#8d GDB <- $OK#9a GDB -> $qfThreadInfo#bb GDB <- $mp1.1,p1.2#9a GDB -> $qsThreadInfo#c8 GDB <- $l#6c GDB -> $qAttached:1#fa GDB <- $1#31 GDB -> $qXfer:exec-file:read:1:0,1000#4a GDB <- $E01#a6 GDB -> $qXfer:exec-file:read:1:0,1000#4a GDB <- $E01#a6 GDB -> $qTStatus#49 GDB <- $T0#84 GDB -> $qTfV#81 GDB <- $#00 GDB -> $?#3f GDB <- $T05thread:p1.1;#a6 GDB -> $vStopped#55 GDB <- $T05thread:p1.2;#a7 GDB -> $vStopped#55 GDB <- $OK#9a remote.c:8334: internal-error: wait: Assertion `!async_event_handler_marked (rs->remote_async_inferior_event_token)' failed. A problem internal to GDB has been detected, further debugging may prove unreliable.
I was able to reproduce using the OP approach.
I've got the same error in GDB 12.1 in non-stop async mode. Here is the last several lines of remote log: [remote] Sending packet: $vStopped#55 [remote] Packet received: T00thread:177; [remote] Sending packet: $vStopped#55 [remote] Packet received: T00thread:178; [remote] Sending packet: $vStopped#55 [remote] Packet received: OK [remote] wait: enter [remote] wait: exit [remote] wait: enter ../../gdb/remote.c:8334: internal-error: wait: Assertion `!async_event_handler_marked (rs->remote_async_inferior_event_token)' failed. A problem internal to GDB has been detected, further debugging may prove unreliable.
BTW reverting commit 32b1f5e8d6b8ddd3be6e471c26dd85a1dac31dda fixes the issue.
From `git log --grep="assert in remote_target"` This assert was introduced in commit 32b1f5e8d6b8ddd3be6e471c26dd85a1dac31dda Then it was reverted in commit 7719facb311657f8927c12481ccfc22fe84d5589 Then it was committed again in 8d34471f70a84e6db89be40116d1ed0f6af941c0 From GDB BUG #28627 it seems that the last commit was a mistake.
Replacing 'target_can_async_p()' by 'target_is_async_p()' in 'remote_target::queued_stop_reply()' solved the problem for me, but I'm not sure if it is the correct fix. I just tried it when I saw that other calls of 'mark_async_event_handler()' have a 'target_is_async_p()' check.
This change eliminated assertion error for me as well.
Created attachment 15151 [details] Patch suggested in comment6 This change suggested by Christian fixes issue for me.
Thanks for digging into the problem. Can you please send your patch to the gdb-patches mailing list? Reviews happen there.
The master branch has been updated by Simon Marchi <simark@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=635b2dd919b8c58e164b77c396041935fca1d66a commit 635b2dd919b8c58e164b77c396041935fca1d66a Author: Simon Marchi <simon.marchi@efficios.com> Date: Tue Oct 3 22:04:01 2023 -0400 gdb: add assertion when marking the remote async flag As reported in bug 30630 [1], we hit a case where the remote target's async flag is marked while the target is not configured (yet) to work async. This should not happen. It is caught thanks to this assert in remote_target::wait: /* Start by clearing the flag that asks for our wait method to be called, we'll mark it again at the end if needed. If the target is not in async mode then the async token should not be marked. */ if (target_is_async_p ()) rs->clear_async_event_handler (); else gdb_assert (!rs->async_event_handler_marked ()); This is helpful, but I think that we could have caught the problem earlier than that, at the moment we marked the handler. Catching problems earlier makes them easier to debug. [1] https://sourceware.org/bugzilla/show_bug.cgi?id=30630 Change-Id: I7e229c74b04da82bef6a817d5a676be5cf52e833 Approved-By: Andrew Burgess <aburgess@redhat.com>