| Summary: | Can't insert internal breakpoints after inferior dlclose some share objects | ||
|---|---|---|---|
| Product: | gdb | Reporter: | Guinevere Larsen <guinevere> |
| Component: | breakpoints | Assignee: | Not yet assigned to anyone <unassigned> |
| Status: | NEW --- | ||
| Severity: | normal | CC: | aburgess, ssbssa, tromey, woodard |
| Priority: | P2 | ||
| Version: | HEAD | ||
| Target Milestone: | --- | ||
| Host: | Target: | ||
| Build: | Last reconfirmed: | ||
Does dlmopen(LM_ID_NEWLM) load a 2nd CRT in this case?
Because some time ago I noticed a very similar looking problem on Windows if I load & unload a 2nd CRT.
Using this example:
```
#include <windows.h>
#include <stdio.h>
int main()
{
HMODULE mod = LoadLibrary ("ucrtbase.dll");
FreeLibrary (mod);
printf ("hi\n");
return 0;
}
```
This happens when I step through it:
```
C:\src\tests>gdb64 gdb-32079.exe
Reading symbols from gdb-32079.exe...
(gdb) start
Temporary breakpoint 1 at 0x140001581: file gdb-32079.c, line 6.
Starting program: C:\src\tests\gdb-32079.exe
Temporary breakpoint 1, main () at gdb-32079.c:6
6 HMODULE mod = LoadLibrary ("ucrtbase.dll");
(gdb) info sharedlibrary
From To Syms Read Shared Object Library
0x00000000777b1000 0x00000000779584e0 Yes (*) C:\Windows\SYSTEM32\ntdll.dll
0x0000000077691000 0x00000000777aeab8 Yes (*) C:\Windows\system32\kernel32.dll
0x000007fefd691000 0x000007fefd6fb13c Yes (*) C:\Windows\system32\KernelBase.dll
0x000007feff6d1000 0x000007feff76e4bc Yes (*) C:\Windows\system32\msvcrt.dll
(*): Shared library is missing debugging information.
(gdb) maint info breakpoints
Num Type Disp Enb Address What
-4 longjmp master keep n 0x0000000077800fa0 <ntdll!longjmp> inf 1
-4.1 y- 0x0000000077800fa0 <ntdll!longjmp> inf 1
-5 longjmp master keep n 0x000007feff6de540 <msvcrt!longjmp> inf 1
-5.1 y- 0x000007feff6de540 <msvcrt!longjmp> inf 1
(gdb) n
7 FreeLibrary (mod);
(gdb) info sharedlibrary
From To Syms Read Shared Object Library
0x00000000777b1000 0x00000000779584e0 Yes (*) C:\Windows\SYSTEM32\ntdll.dll
0x0000000077691000 0x00000000777aeab8 Yes (*) C:\Windows\system32\kernel32.dll
0x000007fefd691000 0x000007fefd6fb13c Yes (*) C:\Windows\system32\KernelBase.dll
0x000007feff6d1000 0x000007feff76e4bc Yes (*) C:\Windows\system32\msvcrt.dll
0x000007fee6b11000 0x000007fee6c01b04 Yes (*) C:\Windows\system32\ucrtbase.dll
0x000007fee6b01000 0x000007fee6b023f0 Yes (*) C:\Windows\system32\api-ms-win-core-timezone-l1-1-0.dll
0x000007fee6af1000 0x000007fee6af23f0 Yes (*) C:\Windows\system32\api-ms-win-core-file-l2-1-0.dll
0x000007fee6ae1000 0x000007fee6ae23f0 Yes (*) C:\Windows\system32\api-ms-win-core-localization-l1-2-0.dll
0x000007fef05e1000 0x000007fef05e23f0 Yes (*) C:\Windows\system32\api-ms-win-core-synch-l1-2-0.dll
0x000007fee6ad1000 0x000007fee6ad23f0 Yes (*) C:\Windows\system32\api-ms-win-core-processthreads-l1-1-1.dll
0x000007fee6ac1000 0x000007fee6ac23f0 Yes (*) C:\Windows\system32\api-ms-win-core-file-l1-2-0.dll
(*): Shared library is missing debugging information.
(gdb) maint info breakpoints
Num Type Disp Enb Address What
-24 longjmp master keep n 0x0000000077800fa0 <ntdll!longjmp> inf 1
-24.1 y- 0x0000000077800fa0 <ntdll!longjmp> inf 1
-25 longjmp master keep n 0x000007feff6de540 <msvcrt!longjmp> inf 1
-25.1 y- 0x000007feff6de540 <msvcrt!longjmp> inf 1
-26 longjmp master keep n 0x000007fee6b431c0 <ucrtbase!longjmp> inf 1
-26.1 y- 0x000007fee6b431c0 <ucrtbase!longjmp> inf 1
(gdb) n
warning: error removing breakpoint 0 at 0x7fee6b431c0
9 printf ("hi\n");
(gdb) info sharedlibrary
From To Syms Read Shared Object Library
0x00000000777b1000 0x00000000779584e0 Yes (*) C:\Windows\SYSTEM32\ntdll.dll
0x0000000077691000 0x00000000777aeab8 Yes (*) C:\Windows\system32\kernel32.dll
0x000007fefd691000 0x000007fefd6fb13c Yes (*) C:\Windows\system32\KernelBase.dll
0x000007feff6d1000 0x000007feff76e4bc Yes (*) C:\Windows\system32\msvcrt.dll
(*): Shared library is missing debugging information.
(gdb) maint info breakpoints
Num Type Disp Enb Address What
-24 longjmp master keep n 0x0000000077800fa0 <ntdll!longjmp> inf 1
-24.1 y- 0x0000000077800fa0 <ntdll!longjmp> inf 1
-25 longjmp master keep n 0x000007feff6de540 <msvcrt!longjmp> inf 1
-25.1 y- 0x000007feff6de540 <msvcrt!longjmp> inf 1
-26 longjmp master keep n 0x000007fee6b431c0 inf 1
-26.1 y- 0x000007fee6b431c0 inf 1
(gdb) n
Warning:
Cannot insert breakpoint 0.
Cannot access memory at address 0x7fee6b431c0
Command aborted.
(gdb) maint info breakpoints
Num Type Disp Enb Address What
-24 longjmp master keep n 0x0000000077800fa0 <ntdll!longjmp> inf 1
-24.1 y- 0x0000000077800fa0 <ntdll!longjmp> inf 1
-25 longjmp master keep n 0x000007feff6de540 <msvcrt!longjmp> inf 1
-25.1 y- 0x000007feff6de540 <msvcrt!longjmp> inf 1
-26 longjmp master keep n 0x000007fee6b431c0 inf 1
-26.1 y- 0x000007fee6b431c0 inf 1
0 longjmp keep y 0x0000000077800fa0 <ntdll!longjmp> inf 1
stop only in thread 1
0.1 y 0x0000000077800fa0 <ntdll!longjmp> inf 1
0 longjmp keep y 0x000007feff6de540 <msvcrt!longjmp> inf 1
stop only in thread 1
0.1 y 0x000007feff6de540 <msvcrt!longjmp> inf 1
0 longjmp keep y 0x000007fee6b431c0 inf 1
stop only in thread 1
0.1 y 0x000007fee6b431c0 inf 1
```
So there was an additional internal breakpoint set on ucrtbase!longjmp, but it wasn't removed when ucrtbase was unloaded.
I do see multiple instances of libc after dlopen-ing libraries in a new linker namespace (CRTs, as you called them - only spelling it out for others that might be confused like me ;)
It seems that GDB is trying to re-insert shlib event breakpoints into the libc from the second namespace that now no longer exists. We probably need GDB to recognize that that namespace is no longer valid... or we can try to be more generic, and in case we try to insert (certain types of?) internal breakpoints in a PC that has no object associated with it, we just delete the breakpoint instead.
For extra debugging, this is the output of `info shared` and the internal breakpoints added by GDB:
```
(gdb) info shared
From To Syms Read Shared Object Library
0x00007ffff7fc9000 0x00007ffff7ff09f5 Yes /lib64/ld-linux-x86-64.so.2
0x00007ffff7ed2440 0x00007ffff7f49666 Yes (*) /lib64/libm.so.6
0x00007ffff7cf9800 0x00007ffff7e65cfd Yes (*) /lib64/libc.so.6
0x00007ffff7fbc040 0x00007ffff7fbc129 Yes /home/gwenthekween/Documents/upstream-build/gdb/testsuite/outputs
/gdb.base/dlmopen/dlmopen-lib.1.so
0x00007ffff7fb7040 0x00007ffff7fb70f9 Yes /home/gwenthekween/Documents/upstream-build/gdb/testsuite/outputs
/gdb.base/dlmopen/dlmopen-lib-dep.so
0x00007ffff7be0440 0x00007ffff7c57666 Yes (*) /lib64/libm.so.6
0x00007ffff7a07800 0x00007ffff7b73cfd Yes (*) /lib64/libc.so.6
0x00007ffff7fc9000 0x00007ffff7ff09f5 Yes /lib64/ld-linux-x86-64.so.2
0x00007ffff7fa8040 0x00007ffff7fa8129 Yes /home/gwenthekween/Documents/upstream-build/gdb/testsuite/outputs
/gdb.base/dlmopen/dlmopen-lib.1.so
0x00007ffff7cca040 0x00007ffff7cca0f9 Yes /home/gwenthekween/Documents/upstream-build/gdb/testsuite/outputs
/gdb.base/dlmopen/dlmopen-lib-dep.so
0x00007ffff78f1440 0x00007ffff7968666 Yes (*) /lib64/libm.so.6
0x00007ffff7718800 0x00007ffff7884cfd Yes (*) /lib64/libc.so.6
0x00007ffff7fc9000 0x00007ffff7ff09f5 Yes /lib64/ld-linux-x86-64.so.2
0x00007ffff7cc5040 0x00007ffff7cc5129 Yes /home/gwenthekween/Documents/upstream-build/gdb/testsuite/outputs
/gdb.base/dlmopen/dlmopen-lib.2.so
(*): Shared library is missing debugging information.
(gdb) maint info breakpoints
Num Type Disp Enb Address What
-40 shlib events keep n 0x00007ffff7fe7edf <dl_main+6271> inf 1
-40.1 y- 0x00007ffff7fe7edf <dl_main+6271> inf 1
-41 shlib events keep y 0x00007ffff7fe83ba <dl_main+7514> inf 1
-41.1 y 0x00007ffff7fe83ba <dl_main+7514> inf 1
-42 shlib events keep n 0x00007ffff7fd04e5 <_dl_map_object_from_fd+4213> inf 1
-42.1 y- 0x00007ffff7fd04e5 <_dl_map_object_from_fd+4213> inf 1
-43 shlib events keep y 0x00007ffff7fd5177 <dl_open_worker_begin+1191> inf 1
-43.1 y 0x00007ffff7fd5177 <dl_open_worker_begin+1191> inf 1
-44 shlib events keep n 0x00007ffff7fc9f13 <_dl_close_worker+1859> inf 1
-44.1 y- 0x00007ffff7fc9f13 <_dl_close_worker+1859> inf 1
-45 shlib events keep y 0x00007ffff7fca1ca <_dl_close_worker+2554> inf 1
-45.1 y 0x00007ffff7fca1ca <_dl_close_worker+2554> inf 1
-70 longjmp master keep n 0x00007ffff7df79b9 <____longjmp_chk+249> inf 1
-70.1 y- 0x00007ffff7df79b9 <____longjmp_chk+249> inf 1
-71 longjmp master keep n 0x00007ffff7d11b57 <__longjmp_cancel+55> inf 1
-71.1 y- 0x00007ffff7d11b57 <__longjmp_cancel+55> inf 1
-72 longjmp master keep n 0x00007ffff7d11ad3 <__longjmp+163> inf 1
-72.1 y- 0x00007ffff7d11ad3 <__longjmp+163> inf 1
-73 longjmp master keep n 0x00007ffff7b059b9 <____longjmp_chk+249> inf 1
-73.1 y- 0x00007ffff7b059b9 <____longjmp_chk+249> inf 1
-74 longjmp master keep n 0x00007ffff7a1fb57 <__longjmp_cancel+55> inf 1
-74.1 y- 0x00007ffff7a1fb57 <__longjmp_cancel+55> inf 1
-75 longjmp master keep n 0x00007ffff7a1fad3 <__longjmp+163> inf 1
-75.1 y- 0x00007ffff7a1fad3 <__longjmp+163> inf 1
-76 longjmp master keep n 0x00007ffff78169b9 <____longjmp_chk+249> inf 1
-76.1 y- 0x00007ffff78169b9 <____longjmp_chk+249> inf 1
-77 longjmp master keep n 0x00007ffff7730b57 <__longjmp_cancel+55> inf 1
-77.1 y- 0x00007ffff7730b57 <__longjmp_cancel+55> inf 1
-78 longjmp master keep n 0x00007ffff7730ad3 <__longjmp+163> inf 1
-78.1 y- 0x00007ffff7730ad3 <__longjmp+163> inf 1
3 breakpoint keep y 0x000000000040130b /home/gwenthekween/Documents/upstream-build/gdb/testsuite/../../..
/binutils-gdb/gdb/testsuite/gdb.base/dlmopen.c:61 inf 1
breakpoint already hit 2 times
3.1 y 0x000000000040130b /home/gwenthekween/Documents/upstream-build/gdb/testsuite/../../..
/binutils-gdb/gdb/testsuite/gdb.base/dlmopen.c:61 inf 1
(gdb) n
warning: error removing breakpoint 0 at 0x7ffff78169b9
warning: error removing breakpoint 0 at 0x7ffff7730b57
warning: error removing breakpoint 0 at 0x7ffff7730ad3
54 for (dl = 0; dl < 4; ++dl)
```
And the results after closing handle[2] (which closes the linker namespace) are:
```
(gdb) info shared
From To Syms Read Shared Object Library
0x00007ffff7fc9000 0x00007ffff7ff09f5 Yes /lib64/ld-linux-x86-64.so.2
0x00007ffff7ed2440 0x00007ffff7f49666 Yes (*) /lib64/libm.so.6
0x00007ffff7cf9800 0x00007ffff7e65cfd Yes (*) /lib64/libc.so.6
0x00007ffff7fbc040 0x00007ffff7fbc129 Yes /home/gwenthekween/Documents/upstream-build/gdb/testsuite/outputs
/gdb.base/dlmopen/dlmopen-lib.1.so
0x00007ffff7fb7040 0x00007ffff7fb70f9 Yes /home/gwenthekween/Documents/upstream-build/gdb/testsuite/outputs
/gdb.base/dlmopen/dlmopen-lib-dep.so
0x00007ffff7be0440 0x00007ffff7c57666 Yes (*) /lib64/libm.so.6
0x00007ffff7a07800 0x00007ffff7b73cfd Yes (*) /lib64/libc.so.6
0x00007ffff7fc9000 0x00007ffff7ff09f5 Yes /lib64/ld-linux-x86-64.so.2
0x00007ffff7cc5040 0x00007ffff7cc5129 Yes /home/gwenthekween/Documents/upstream-build/gdb/testsuite/outputs
/gdb.base/dlmopen/dlmopen-lib.2.so
(*): Shared library is missing debugging information.
(gdb) maint info breakpoints
Num Type Disp Enb Address What
-40 shlib events keep n 0x00007ffff7fe7edf <dl_main+6271> inf 1
-40.1 y- 0x00007ffff7fe7edf <dl_main+6271> inf 1
-41 shlib events keep y 0x00007ffff7fe83ba <dl_main+7514> inf 1
-41.1 y 0x00007ffff7fe83ba <dl_main+7514> inf 1
-42 shlib events keep n 0x00007ffff7fd04e5 <_dl_map_object_from_fd+4213> inf 1
-42.1 y- 0x00007ffff7fd04e5 <_dl_map_object_from_fd+4213> inf 1
-43 shlib events keep y 0x00007ffff7fd5177 <dl_open_worker_begin+1191> inf 1
-43.1 y 0x00007ffff7fd5177 <dl_open_worker_begin+1191> inf 1
-44 shlib events keep n 0x00007ffff7fc9f13 <_dl_close_worker+1859> inf 1
-44.1 y- 0x00007ffff7fc9f13 <_dl_close_worker+1859> inf 1
-45 shlib events keep y 0x00007ffff7fca1ca <_dl_close_worker+2554> inf 1
-45.1 y 0x00007ffff7fca1ca <_dl_close_worker+2554> inf 1
-70 longjmp master keep n 0x00007ffff7df79b9 <____longjmp_chk+249> inf 1
-70.1 y- 0x00007ffff7df79b9 <____longjmp_chk+249> inf 1
-71 longjmp master keep n 0x00007ffff7d11b57 <__longjmp_cancel+55> inf 1
-71.1 y- 0x00007ffff7d11b57 <__longjmp_cancel+55> inf 1
-72 longjmp master keep n 0x00007ffff7d11ad3 <__longjmp+163> inf 1
-72.1 y- 0x00007ffff7d11ad3 <__longjmp+163> inf 1
-73 longjmp master keep n 0x00007ffff7b059b9 <____longjmp_chk+249> inf 1
-73.1 y- 0x00007ffff7b059b9 <____longjmp_chk+249> inf 1
-74 longjmp master keep n 0x00007ffff7a1fb57 <__longjmp_cancel+55> inf 1
-74.1 y- 0x00007ffff7a1fb57 <__longjmp_cancel+55> inf 1
-75 longjmp master keep n 0x00007ffff7a1fad3 <__longjmp+163> inf 1
-75.1 y- 0x00007ffff7a1fad3 <__longjmp+163> inf 1
-76 longjmp master keep n 0x00007ffff78169b9 inf 1
-76.1 y- 0x00007ffff78169b9 inf 1
-77 longjmp master keep n 0x00007ffff7730b57 inf 1
-77.1 y- 0x00007ffff7730b57 inf 1
-78 longjmp master keep n 0x00007ffff7730ad3 inf 1
-78.1 y- 0x00007ffff7730ad3 inf 1
3 breakpoint keep y 0x000000000040130b /home/gwenthekween/Documents/upstream-build/gdb/testsuite/../../..
/binutils-gdb/gdb/testsuite/gdb.base/dlmopen.c:61 inf 1
breakpoint already hit 3 times
3.1 y 0x000000000040130b /home/gwenthekween/Documents/upstream-build/gdb/testsuite/../../..
/binutils-gdb/gdb/testsuite/gdb.base/dlmopen.c:61 inf 1
```
I came up with this:
```
--- a/gdb/breakpoint.c
+++ b/gdb/breakpoint.c
@@ -8049,6 +8049,7 @@ static void
disable_breakpoints_in_unloaded_shlib (program_space *pspace, const solib &solib)
{
bool disabled_shlib_breaks = false;
+ bool remove_disabled_longjmp_breaks = false;
for (bp_location *loc : all_bp_locations ())
{
@@ -8059,6 +8060,8 @@ disable_breakpoints_in_unloaded_shlib (program_space *pspace, const solib &solib
&& !loc->shlib_disabled
&& (((b->type == bp_breakpoint
|| b->type == bp_jit_event
+ || b->type == bp_longjmp
+ || b->type == bp_longjmp_master
|| b->type == bp_hardware_breakpoint)
&& (loc->loc_type == bp_loc_hardware_breakpoint
|| loc->loc_type == bp_loc_software_breakpoint))
@@ -8071,6 +8074,13 @@ disable_breakpoints_in_unloaded_shlib (program_space *pspace, const solib &solib
to prevent future errors occurring in remove_breakpoints. */
loc->inserted = 0;
+ if (b->type == bp_longjmp
+ || b->type == bp_longjmp_master)
+ {
+ remove_disabled_longjmp_breaks = true;
+ continue;
+ }
+
/* This may cause duplicate notifications for the same breakpoint. */
notify_breakpoint_modified (b);
@@ -8084,6 +8094,12 @@ disable_breakpoints_in_unloaded_shlib (program_space *pspace, const solib &solib
disabled_shlib_breaks = true;
}
}
+
+ if (remove_disabled_longjmp_breaks)
+ for (breakpoint &b : all_breakpoints_safe ())
+ if ((b.type == bp_longjmp || b.type == bp_longjmp_master)
+ && b.first_loc ().shlib_disabled)
+ delete_breakpoint (&b);
}
/* Disable any breakpoints and tracepoints in OBJFILE upon
```
I'm not completely sure if it's the correct approach to fix this, but it works for Windows at least.
If this happens with longjmp breakpoints, it can probably happen with the C++ exception breakpoints as well. I've posted a proposed fix for this bug here: https://inbox.sourceware.org/gdb-patches/cover.1724948606.git.aburgess@redhat.com/ It is inspired by the changes Hannes Domani proposed in: https://sourceware.org/bugzilla/show_bug.cgi?id=32079#c3 But when combined with an earlier patch in the series I don't think the delete_breakpoint call is needed. And as Tom pointed out I handle more b/p types as well as fixing another issue that cropped up during testing. |
As I'm trying to debug what I think is an unrelated issue in gdb.base/dlmopen.exp, I noticed that stepping through the program (using next) GDB gets into a state where it tries to insert some internal breakpoints, and so refuses to let me move forward, but I can't see the number of the breakpoint to delete it. Here's a simplified session: ``` $ ./gdb testsuite/outputs/gdb.base/dlmopen/dlmopen (gdb) start [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Temporary breakpoint 1, main () at /home/gwenthekween/Documents/upstream-build/gdb/testsuite/../../../binutils-gdb/gdb/testsuite/gdb.base/dlmopen .c:36 36 handle[0] = dlmopen (LM_ID_NEWLM, DSO1_NAME, RTLD_LAZY | RTLD_LOCAL); (gdb) n Python Exception <class 'AttributeError'>: module 'gdb' has no attribute '_handle_missing_debuginfo' Python Exception <class 'AttributeError'>: module 'gdb' has no attribute '_handle_missing_debuginfo' 37 assert (handle[0] != NULL); (gdb) print wait_for_gdb = 0 $1 = 0 (gdb) handle SIGALRM ignore Signal Stop Print Pass to program Description SIGALRM No No No Alarm clock (gdb) next <snip. removing unimportant "next"s> 54 for (dl = 0; dl < 4; ++dl) (gdb) 56 fun = dlsym (handle[dl], "inc"); (gdb) 57 assert (fun != NULL); (gdb) 59 fun (42); (gdb) 61 dlclose (handle[dl]); (gdb) warning: error removing breakpoint 0 at 0x7ffff78169b9 warning: error removing breakpoint 0 at 0x7ffff7730b57 warning: error removing breakpoint 0 at 0x7ffff7730ad3 54 for (dl = 0; dl < 4; ++dl) (gdb) Warning: Cannot insert breakpoint 0. Cannot access memory at address 0x7ffff7730ad3 Cannot insert breakpoint 0. Cannot access memory at address 0x7ffff7730b57 Cannot insert breakpoint 0. Cannot access memory at address 0x7ffff78169b9 Command aborted. (gdb) p dl $2 = 2 (gdb) info break No breakpoints, watchpoints, tracepoints, or catchpoints. (gdb) disas /m Dump of assembler code for function main: <snip unimportant disassembly> 54 for (dl = 0; dl < 4; ++dl) 0x00000000004012bc <+294>: movl $0x0,-0x4(%rbp) 0x00000000004012c3 <+301>: jmp 0x401321 <main+395> => 0x000000000040131d <+391>: addl $0x1,-0x4(%rbp) 0x0000000000401321 <+395>: cmpl $0x3,-0x4(%rbp) 0x0000000000401325 <+399>: jle 0x4012c5 <main+303> <snip disassembly continues> (gdb) ``` So, this happened after the last dlmopen-lib.1.so was dlclose'd, we are no longer able to step. If I set a breakpoint after the loop and continue over it, the issue doesn't occur, so it seems to be related to internal "next" breakpoints in some way