[PATCH 0/3] Fix crash if connection drops in scoped_restore_current_thread's ctor

Simon Marchi simark@simark.ca
Wed Jul 22 19:37:29 GMT 2020 

On 2020-07-10 7:02 p.m., Pedro Alves wrote:
> On 7/9/20 12:31 AM, Pedro Alves wrote:
>> (I have internet again: found a sim card of a different operator that
>> works.  This will do until the communications tower near me is
>> repaired and get I fiber back...)
>>
>> This series fixes the crashes exposed by the
>> gdb.multi/multi-target.exp testcase when run against an Asan-enabled
>> GDB build, initially reported by Simon here:
>>
>>   https://sourceware.org/pipermail/gdb-patches/2020-July/170222.html
>>
>> The first two patches fix the crashes, and we should probably put them
>> in GDB 10.
>>
>> The last patch is a follow up that avoids swallowing exceptions in
>> scoped_restore_current_thread's dtor that I'm thinking would be a bit
>> too invasive to put in GDB 10, I think it could do with a longer
>> baking period in master.
>>
>> Pedro Alves (3):
>>   Fix crash if connection drops in scoped_restore_current_thread's ctor,
>>     part 1
>>   Fix crash if connection drops in scoped_restore_current_thread's ctor,
>>     part 2
>>   Make scoped_restore_current_thread's cdtors exception free (RFC)
> 
> I've now merged patches 1 and 2.  Patch 3 will wait until after the branch
> is cut.
> 

I now see this other ASan failure when running gdb.multi/multi-target.exp, it's in the
attached asan.log.  There are colors, so it's easier to read if you "cat" it in your
terminal.  It looks familiar, because it happens in scoped_restore_current_thread's dtor
(not ctor), but maybe it just happens to be there but could happen at any other point.

It happens when starting test_continue with non-stop on, just after having completed
test_continue with non-stop off.  It's when GDB does "monitor exit".

Unfortunately, the "freed by thread T0 here" stack trace is again truncated, probably
because the stack is too deep for the portion of the stack ASan captures.  But I managed
to attach to GDB with GDB using gdb_interact and capture it (I broke on unpush_and_perror),
here's the equivalent GDB backtrace:

#0  xfree<void> (ptr=0x621004a5d900) at /home/smarchi/src/binutils-gdb/gdb/../gdbsupport/common-utils.h:63
#1  0x0000000001626260 in call_freefun (h=0x20f8da0 <frame_cache_obstack>, old_chunk=0x621004a5d900) at /home/smarchi/src/binutils-gdb/libiberty/obstack.c:103
#2  0x0000000001626c87 in _obstack_free (h=0x20f8da0 <frame_cache_obstack>, obj=0x0) at /home/smarchi/src/binutils-gdb/libiberty/obstack.c:280
#3  0x000000000098ae26 in reinit_frame_cache () at /home/smarchi/src/binutils-gdb/gdb/frame.c:1856
#4  0x0000000001098adf in switch_to_no_thread () at /home/smarchi/src/binutils-gdb/gdb/thread.c:1301
#5  0x0000000000acf544 in switch_to_inferior_no_thread (inf=0x615000244d00) at /home/smarchi/src/binutils-gdb/gdb/inferior.c:626
#6  0x0000000000e7c38c in remote_unpush_target (target=0x6170000c0c00) at /home/smarchi/src/binutils-gdb/gdb/remote.c:5521
#7  0x0000000000e92db6 in unpush_and_perror (target=0x6170000c0c00, string=0x191d400 "Remote communication error.  Target disconnected.") at /home/smarchi/src/binutils-gdb/gdb/remote.c:9101
#8  0x0000000000e930c7 in remote_target::readchar (this=0x6170000c0c00, timeout=2) at /home/smarchi/src/binutils-gdb/gdb/remote.c:9141
#9  0x0000000000e9576f in remote_target::getpkt_or_notif_sane_1 (this=0x6170000c0c00, buf=0x6170000c0c18, forever=0, expecting_notif=0, is_notif=0x0) at /home/smarchi/src/binutils-gdb/gdb/remote.c:9683
#10 0x0000000000e961c9 in remote_target::getpkt_sane (this=0x6170000c0c00, buf=0x6170000c0c18, forever=0) at /home/smarchi/src/binutils-gdb/gdb/remote.c:9790
#11 0x0000000000e95545 in remote_target::getpkt (this=0x6170000c0c00, buf=0x6170000c0c18, forever=0) at /home/smarchi/src/binutils-gdb/gdb/remote.c:9623
#12 0x0000000000e91ba3 in remote_target::remote_read_bytes_1 (this=0x6170000c0c00, memaddr=0x7ffff78bc38d, myaddr=0x7fff7dca59a0 "", len_units=1, unit_size=1, xfered_len_units=0x7fff7dca58b0) at /home/smarchi/src/binutils-gdb/gdb/remote.c:8860
#13 0x0000000000e9240c in remote_target::remote_read_bytes (this=0x6170000c0c00, memaddr=0x7ffff78bc38d, myaddr=0x7fff7dca59a0 "", len=1, unit_size=1, xfered_len=0x7fff7dca58b0) at /home/smarchi/src/binutils-gdb/gdb/remote.c:8987
#14 0x0000000000e9b821 in remote_target::xfer_partial (this=0x6170000c0c00, object=TARGET_OBJECT_MEMORY, annex=0x0, readbuf=0x7fff7dca59a0 "", writebuf=0x0, offset=140737346519949, len=1, xfered_len=0x7fff7dca58b0) at /home/smarchi/src/binutils-gdb/gdb/remote.c:10987
#15 0x000000000104fd3a in raw_memory_xfer_partial (ops=0x6170000c0c00, readbuf=0x7fff7dca59a0 "", writebuf=0x0, memaddr=140737346519949, len=1, xfered_len=0x7fff7dca58b0) at /home/smarchi/src/binutils-gdb/gdb/target.c:918
#16 0x0000000001050425 in memory_xfer_partial_1 (ops=0x6170000c0c00, object=TARGET_OBJECT_MEMORY, readbuf=0x7fff7dca59a0 "", writebuf=0x0, memaddr=140737346519949, len=1, xfered_len=0x7fff7dca58b0) at /home/smarchi/src/binutils-gdb/gdb/target.c:1047
#17 0x0000000001050608 in memory_xfer_partial (ops=0x6170000c0c00, object=TARGET_OBJECT_MEMORY, readbuf=0x7fff7dca59a0 "", writebuf=0x0, memaddr=140737346519949, len=1, xfered_len=0x7fff7dca58b0) at /home/smarchi/src/binutils-gdb/gdb/target.c:1076
#18 0x0000000001050b92 in target_xfer_partial (ops=0x6170000c0c00, object=TARGET_OBJECT_MEMORY, annex=0x0, readbuf=0x7fff7dca59a0 "", writebuf=0x0, offset=140737346519949, len=1, xfered_len=0x7fff7dca58b0) at /home/smarchi/src/binutils-gdb/gdb/target.c:1133
#19 0x0000000001051a7b in target_read_partial (ops=0x6170000c0c00, object=TARGET_OBJECT_MEMORY, annex=0x0, buf=0x7fff7dca59a0 "", offset=140737346519949, len=1, xfered_len=0x7fff7dca58b0) at /home/smarchi/src/binutils-gdb/gdb/target.c:1379
#20 0x0000000001051c59 in target_read (ops=0x6170000c0c00, object=TARGET_OBJECT_MEMORY, annex=0x0, buf=0x7fff7dca59a0 "", offset=140737346519949, len=1) at /home/smarchi/src/binutils-gdb/gdb/target.c:1419
#21 0x0000000001051178 in target_read_memory (memaddr=0x7ffff78bc38d, myaddr=0x7fff7dca59a0 "", len=1) at /home/smarchi/src/binutils-gdb/gdb/target.c:1222
#22 0x00000000004b4731 in amd64_stack_frame_destroyed_p (gdbarch=0x6210027e8510, pc=0x7ffff78bc38d) at /home/smarchi/src/binutils-gdb/gdb/amd64-tdep.c:2909
#23 0x00000000004b4822 in amd64_epilogue_frame_sniffer (self=0x169df00 <amd64_epilogue_frame_unwind>, this_frame=0x621004a5d9e0, this_prologue_cache=0x621004a5d9f8) at /home/smarchi/src/binutils-gdb/gdb/amd64-tdep.c:2924
#24 0x0000000000981048 in frame_unwind_try_unwinder (this_frame=0x621004a5d9e0, this_cache=0x621004a5d9f8, unwinder=0x169df00 <amd64_epilogue_frame_unwind>) at /home/smarchi/src/binutils-gdb/gdb/frame-unwind.c:128
#25 0x000000000098126d in frame_unwind_find_by_frame (this_frame=0x621004a5d9e0, this_cache=0x621004a5d9f8) at /home/smarchi/src/binutils-gdb/gdb/frame-unwind.c:186
#26 0x0000000000983c9d in compute_frame_id (fi=0x621004a5d9e0) at /home/smarchi/src/binutils-gdb/gdb/frame.c:546
#27 0x0000000000984167 in get_frame_id (fi=0x621004a5d9e0) at /home/smarchi/src/binutils-gdb/gdb/frame.c:582
#28 0x0000000001098eef in restore_selected_frame (a_frame_id=..., frame_level=0) at /home/smarchi/src/binutils-gdb/gdb/thread.c:1355
#29 0x00000000010992f8 in scoped_restore_current_thread::restore (this=0x7fff7dca5f30) at /home/smarchi/src/binutils-gdb/gdb/thread.c:1411
#30 0x0000000001099355 in scoped_restore_current_thread::~scoped_restore_current_thread (this=0x7fff7dca5f30, __in_chrg=<optimized out>) at /home/smarchi/src/binutils-gdb/gdb/thread.c:1420
#31 0x0000000000aeab84 in do_target_wait (wait_ptid=..., ecs=0x7fff7dca6290, options=1) at /home/smarchi/src/binutils-gdb/gdb/infrun.c:3670
#32 0x0000000000aecbe3 in fetch_inferior_event () at /home/smarchi/src/binutils-gdb/gdb/infrun.c:3965
#33 0x0000000000aa8097 in inferior_event_handler (event_type=INF_REG_EVENT) at /home/smarchi/src/binutils-gdb/gdb/inf-loop.c:42
#34 0x0000000000eab8b7 in remote_async_inferior_event_handler (data=0x6170000d6a00) at /home/smarchi/src/binutils-gdb/gdb/remote.c:14166
#35 0x00000000004ca110 in check_async_event_handlers () at /home/smarchi/src/binutils-gdb/gdb/async-event.c:295
#36 0x00000000015bef41 in gdb_do_one_event () at /home/smarchi/src/binutils-gdb/gdbsupport/event-loop.cc:194
#37 0x0000000000bfd50e in start_event_loop () at /home/smarchi/src/binutils-gdb/gdb/main.c:356
#38 0x0000000000bfd816 in captured_command_loop () at /home/smarchi/src/binutils-gdb/gdb/main.c:416
#39 0x0000000000c00c25 in captured_main (data=0x7fff7dca65d0) at /home/smarchi/src/binutils-gdb/gdb/main.c:1253
#40 0x0000000000c00cb5 in gdb_main (args=0x7fff7dca65d0) at /home/smarchi/src/binutils-gdb/gdb/main.c:1268
#41 0x0000000000414d9e in main (argc=5, argv=0x7fff7dca6738) at /home/smarchi/src/binutils-gdb/gdb/gdb.c:32


The problem seems to be:

- We create a new frame_info object in restore_selected_frame (by calling find_relative_frame)
- The frame is allocated on the frame_cache_obstack
- In frame_unwind_try_unwinder, we try to find an unwinder for that frame
- While trying unwinders, memory read fails because the remote target closes, because of "monitor exit"
- That calls reinit_frame_cache (as shown above), which resets frame_cache_obstack
- When handling the exception in frame_unwind_try_unwinder, we try to set some things on the frame_info
  object (like *this_cache, which in fact tries to write into frame_info::prologue_cache), but the
  frame_info object is no more, it went away with the obstack.

Simon
-------------- next part --------------
A non-text attachment was scrubbed...
Name: asan.log
Type: text/x-log
Size: 12922 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/gdb-patches/attachments/20200722/03b6c80d/attachment.bin>

More information about the Gdb-patches mailing list