| Summary: | cannot reexecute programs using the GDB JIT interface in shared libs | ||
|---|---|---|---|
| Product: | gdb | Reporter: | Török Edwin <edwin+bugs> |
| Component: | breakpoints | Assignee: | Pedro Alves <pedro> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | gdb-prs, laruence, pedro |
| Priority: | P2 | ||
| Version: | 7.1 | ||
| Target Milestone: | 7.12 | ||
| Host: | x86_64-linux-gnu | Target: | x86_64-linux-gnu |
| Build: | x86_64-linux-gnu | Last reconfirmed: | 2016-08-24 00:00:00 |
| Attachments: |
testcase.sh
bp_minus4.c shared.c proposed patch to fix the bug |
||
Created attachment 4472 [details]
testcase.sh
script to compile testcase and run gdb
Created attachment 4473 [details]
bp_minus4.c
the file containing main()
Created attachment 4474 [details]
shared.c
the file for the shared lib, this "implements" the JIT interface.
It doesn't actually implement anything, just declares the __jit* symbols, the
code is a copy of the one in GDB online manual.
LLVM has code that actually registers something with the JIT, but this bug can
be triggered without actually registering any code, hence this simple testcase.
Note that if the file is not compiled as a shared lib, then I can rerun the
program without any errors.
As discussed with rnk on IRC this might be the cause: rnk: edwin, so the problem is the shared library gets loaded at a different address on the next run? rnk: and GDB doesn't update the address of the breakpoint it's trying to set? edwin: rnk: thats very possible edwin: the default is ASLR edwin: so no shared lib will get loaded to same address Note that GDB turns off ASLR by default: gdb64-cvs -nx -ex 'show disable-randomization' -ex 'quit' GNU gdb (GDB) 7.0.50.20091211-cvs ... This GDB was configured as "x86_64-unknown-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Disabling randomization of debuggee's virtual address space is on. (In reply to comment #5) > Note that GDB turns off ASLR by default: > > gdb64-cvs -nx -ex 'show disable-randomization' -ex 'quit' > GNU gdb (GDB) 7.0.50.20091211-cvs > ... > This GDB was configured as "x86_64-unknown-linux-gnu". > For bug reporting instructions, please see: > <http://www.gnu.org/software/gdb/bugs/>. > Disabling randomization of debuggee's virtual address space is on. > Same here. I think it tries to set the breakpoint too early, before the shared lib is loaded. First run: (gdb) b main Breakpoint 1 at 0x4005f8: file bp_minus4.c, line 4. (gdb) r Starting program: /home/edwin/gdbbug/bp_minus4 Breakpoint 1, main () at bp_minus4.c:4 4 return foo(); (gdb) info sharedlibrary From To Syms Read Shared Object Library 0x000000381da00af0 0x000000381da16a24 Yes (*) /lib64/ld-linux-x86-64.so.2 0x00007ffff7dfa510 0x00007ffff7dfa628 Yes ./shared.so 0x000000381de1e730 0x000000381df0292c Yes (*) /lib/libc.so.6 (*): Shared library is missing debugging information. Second run: (gdb) r The program being debugged has been started already. Start it from the beginning? (y or n) y Starting program: /home/edwin/gdbbug/bp_minus4 Warning: Cannot insert breakpoint -2. Error accessing memory address 0x7ffff7dfa5dc: Input/output error. (gdb) info sharedlibrary From To Syms Read Shared Object Library 0x000000381da00af0 0x000000381da16a24 Yes (*) /lib64/ld-linux-x86-64.so.2 (*): Shared library is missing debugging information. And here is /proc/maps for the process $ ps aux|grep gdb edwin 10001 0.2 0.2 37260 10264 pts/1 S+ 18:58 0:00 gdb ./bp_minus4 edwin 10020 0.0 0.0 220 24 pts/1 T 18:59 0:00 /home/edwin/gdbbug/bp_minus4 edwin 10033 0.0 0.0 7288 784 pts/2 S+ 18:59 0:00 grep gdb $ cat /proc/10020/maps 00400000-00401000 r-xp 00000000 fd:02 5389238 /home/edwin/gdbbug/bp_minus4 00600000-00601000 rw-p 00000000 fd:02 5389238 /home/edwin/gdbbug/bp_minus4 381da00000-381da1d000 r-xp 00000000 09:03 260632 /lib/ld-2.10.2.so 381dc1c000-381dc1e000 rw-p 0001c000 09:03 260632 /lib/ld-2.10.2.so 7ffff7ffe000-7ffff7fff000 r-xp 00000000 00:00 0 [vdso] 7ffffffea000-7ffffffff000 rw-p 00000000 00:00 0 [stack] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] I am guessing that bp_jit_event breakpoints must be disabled here
(in addition to bp_breakpoint and bp_hardware_breakpoint):
// gdb/breakpoint.c
static void
disable_breakpoints_in_unloaded_shlib (struct so_list *solib)
{
...
ALL_BP_LOCATIONS (loc, locp_tmp)
{
struct breakpoint *b = loc->owner;
if ((loc->loc_type == bp_loc_hardware_breakpoint
...
&& (b->type == bp_breakpoint || b->type == bp_hardware_breakpoint)
&& solib_contains_address_p (solib, loc->address))
{
loc->shlib_disabled = 1;
[There are likely several more places where bp_jit_event should be treated same
as bp_breakpoint.]
Created attachment 4614 [details]
proposed patch to fix the bug
(In reply to comment #8) This suddenly became important for unladen swallow as soon as we tried to link to LLVM as a shared library. :) I tested my patch with unladen, and it correctly registers JITed code on the first run and any subsequent run. Can someone review and commit this patch for me? Thanks! This is still an issue with GDB 7.1: (gdb) r The program being debugged has been started already. Start it from the beginning? (y or n) y The program being debugged has been started already. Starting program: /home/edwin/clam/git/builds/debug/clambc/.libs/lt-clambc /tmp/foo.cbc Warning: Cannot insert breakpoint -15. Error accessing memory address 0x7ffff7735510: Input/output error. Good news is that GDB 7.1 doesn't crash anymore when trying to rerun the program (it happened quite often with 7.0). Subject: Bug 11094 CVSROOT: /cvs/src Module name: src Changes by: tromey@sourceware.org 2010-03-22 17:36:28 Modified files: gdb : ChangeLog breakpoint.c Log message: 2010-03-22 Reid Kleckner <reid@kleckner.net> PR gdb/11094 * breakpoint.c (disable_breakpoints_in_unloaded_shlib): Add bp_jit_event. (disable_breakpoints_in_shlibs): Likewise. Patches: http://sourceware.org/cgi-bin/cvsweb.cgi/src/gdb/ChangeLog.diff?cvsroot=src&r1=1.11510&r2=1.11511 http://sourceware.org/cgi-bin/cvsweb.cgi/src/gdb/breakpoint.c.diff?cvsroot=src&r1=1.462&r2=1.463 I checked in Reid's fix to cvs trunk. It will show up in 7.2. If you try it and still have problems, please reopen this PR, thanks. this bug still exists in 7.10 and latest 7.11 With master, GDB crashes with a seg fault:
Program received signal SIGSEGV, Segmentation fault.
0x00000000005ee894 in event_location_to_string (location=0x0) at /home/pedro/gdb/mygit/src/gdb/location.c:412
412 if (EL_STRING (location) == NULL)
(top-gdb) bt
#0 0x00000000005ee894 in event_location_to_string (location=0x0) at /home/pedro/gdb/mygit/src/gdb/location.c:412
#1 0x000000000057411a in print_breakpoint_location (b=0x18288e0, loc=0x0) at /home/pedro/gdb/mygit/src/gdb/breakpoint.c:6201
#2 0x000000000057483f in print_one_breakpoint_location (b=0x18288e0, loc=0x182cf10, loc_number=0, last_loc=0x7fffffffd258, allflag=1)
at /home/pedro/gdb/mygit/src/gdb/breakpoint.c:6473
#3 0x00000000005751e1 in print_one_breakpoint (b=0x18288e0, last_loc=0x7fffffffd258, allflag=1) at /home/pedro/gdb/mygit/src/gdb/breakpoint.c:6707
#4 0x000000000057589c in breakpoint_1 (args=0x0, allflag=1, filter=0x0) at /home/pedro/gdb/mygit/src/gdb/breakpoint.c:6947
#5 0x0000000000575aa8 in maintenance_info_breakpoints (args=0x0, from_tty=0) at /home/pedro/gdb/mygit/src/gdb/breakpoint.c:7026
#6 0x00000000004b74c4 in do_cfunc (c=0xedc760, args=0x0, from_tty=0) at /home/pedro/gdb/mygit/src/gdb/cli/cli-decode.c:105
#7 0x00000000004ba420 in cmd_func (cmd=0xedc760, args=0x0, from_tty=0) at /home/pedro/gdb/mygit/src/gdb/cli/cli-decode.c:1888
#8 0x000000000073f089 in execute_command (p=0x16a4426 "", from_tty=0) at /home/pedro/gdb/mygit/src/gdb/top.c:668
#9 0x0000000000624191 in command_handler (command=0x16a4410 "maint info breakpoints") at /home/pedro/gdb/mygit/src/gdb/event-top.c:620
#10 0x000000000073ec9c in read_command_file (stream=0x1715db0) at /home/pedro/gdb/mygit/src/gdb/top.c:481
#11 0x00000000004bd42e in script_from_file (stream=0x1715db0, file=0x7fffffffdca5 "/home/pedro/tmp/pr11094/gdb.tmp") at /home/pedro/gdb/mygit/src/gdb/cli/cli-script.c:1703
#12 0x00000000004be16e in source_script_from_stream (stream=0x1715db0, file=0x7fffffffdca5 "/home/pedro/tmp/pr11094/gdb.tmp",
file_to_open=0x7fffffffdca5 "/home/pedro/tmp/pr11094/gdb.tmp") at /home/pedro/gdb/mygit/src/gdb/cli/cli-cmds.c:578
#13 0x00000000004be22f in source_script_with_search (file=0x7fffffffdca5 "/home/pedro/tmp/pr11094/gdb.tmp", from_tty=1, search_path=0)
at /home/pedro/gdb/mygit/src/gdb/cli/cli-cmds.c:618
#14 0x00000000004be262 in source_script (file=0x7fffffffdca5 "/home/pedro/tmp/pr11094/gdb.tmp", from_tty=1) at /home/pedro/gdb/mygit/src/gdb/cli/cli-cmds.c:628
#15 0x000000000061b0f5 in catch_command_errors_const (command=0x4be23d <source_script(char const*, int)>, arg=0x7fffffffdca5 "/home/pedro/tmp/pr11094/gdb.tmp", from_tty=1)
at /home/pedro/gdb/mygit/src/gdb/main.c:402
#16 0x000000000061c469 in captured_main (data=0x7fffffffd720) at /home/pedro/gdb/mygit/src/gdb/main.c:1122
#17 0x000000000061c50a in gdb_main (args=0x7fffffffd720) at /home/pedro/gdb/mygit/src/gdb/main.c:1159
#18 0x000000000040d38d in main (
During symbol reading, cannot get low and high bounds for subprogram DIE at 24065.
argc=5, argv=0x7fffffffd828) at /home/pedro/gdb/mygit/src/gdb/gdb.c:32
(top-gdb)
This is GDB trying to print the location spec of the jit breakpoint, but that's an internal breakpoint without one. If I add a NULL check, then we see that the JIT breakpoint ended up pending. But that's incorrect. GDB should have managed to recreate the JIT breakpoint's locations for the second run. So the problem is earlier. The problem seems to me to be that we don't discard the breakpoint when the objfile with the JIT is unloaded. The fix seems quite small and safe. Tentatively setting target milestone to 7.12. (In reply to Pedro Alves from comment #17) > Fix posted: > https://sourceware.org/ml/gdb-patches/2016-08/msg00249.html Thanks, it's great. Fixed in master and 7.12. |
When trying to debug a program (for example ClamAV) that uses the GDB JIT inside a shared lib (for example LLVM in libclamavc++.so), then I cannot reexecute the program. This happens when no code was registered with the JIT interface, such as when I simply put a breakpoint on main, and try to run twice, the 2nd run will fail telling me it couldn't reset breakpoint -4 (or -2). I've created a small testcase to show this problem (will attach), here is the output: --- shared.c: #include <stdint.h> typedef enum { JIT_NOACTION = 0, JIT_REGISTER_FN, JIT_UNREGISTER_FN } jit_actions_t; struct jit_code_entry { struct jit_code_entry *next_entry; struct jit_code_entry *prev_entry; const char *symfile_addr; uint64_t symfile_size; }; struct jit_descriptor { uint32_t version; /* This type should be jit_actions_t, but we use uint32_t to be explicit about the bitwidth. */ uint32_t action_flag; struct jit_code_entry *relevant_entry; struct jit_code_entry *first_entry; }; /* GDB puts a breakpoint in this function. */ void __attribute__((noinline)) __jit_debug_register_code() { }; /* Make sure to specify the version statically, because the debugger may check the version before we can set it. */ struct jit_descriptor __jit_debug_descriptor = { 1, 0, 0, 0 }; int foo() { return 4; } --- bp_minus4.c: int foo(); int main() { return foo(); } gcc (Debian 4.4.2-5) 4.4.2 Copyright (C) 2009 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. GNU gdb (GDB) 7.0-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /home/edwin/gdbbug/bp_minus4...done. Breakpoint 1 at 0x4005f8: file bp_minus4.c, line 4. Breakpoint 1, main () at bp_minus4.c:4 4 return foo(); /home/edwin/gdbbug/gdb.tmp:3: Error in sourced command file: Warning: Cannot insert breakpoint -2. Error accessing memory address 0x7ffff7dfa5dc: Input/output error. (gdb) maint info breakpoints Num Type Disp Enb Address What 1 breakpoint keep y 0x00000000004005f8 in main at bp_minus4.c:4 -2 jit events keep y 0x00007ffff7dfa5dc -9 shlib events keep y 0x000000381da0dcc0 <*__GI__dl_debug_state> (gdb) quit A debugging session is active. Inferior 2 [process 18147] will be killed. Quit anyway? (y or n) y