[PATCH] [gdb/python] Throw MemoryError in inferior.read_memory if malloc fails

[Tom Tromey]

>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:

Tom> PR python/31631 reports a gdb internal error when doing:
Tom> ...
Tom> (gdb) python gdb.selected_inferior().read_memory (0, 0xffffffffffffffff)
Tom> utils.c:709: internal-error: virtual memory exhausted.
Tom> A problem internal to GDB has been detected,
Tom> further debugging may prove unreliable.
Tom> ...

Tom> Fix this by throwing a python MemoryError, such that we have instead:
Tom> ...
Tom> (gdb) python gdb.selected_inferior().read_memory (0, 0xffffffffffffffff)
Tom> Python Exception <class 'MemoryError'>:
Tom> Error occurred in Python.
Tom> (gdb)
Tom> ...

I tend to think you will regret opening this door, because I imagine
there are a large number of ways to crash gdb by passing nonsensical
values to Python APIs.

Tom>  @request("readMemory")
Tom>  @capability("supportsReadMemoryRequest")
Tom>  def read_memory(*, memoryReference: str, offset: int = 0, count: int, **extra):
Tom>      addr = int(memoryReference, 0) + offset
Tom> -    buf = gdb.selected_inferior().read_memory(addr, count)
Tom> +    oom = False
Tom> +    try:
Tom> +        buf = gdb.selected_inferior().read_memory(addr, count)
Tom> +    except MemoryError:
Tom> +        oom = True
Tom> +    if oom:
Tom> +        raise DAPException("Out of memory")

This should probably chain the memory error in the except block and
re-throw.  See https://peps.python.org/pep-3134/

However I don't really understand why this is needed.  Isn't the
exception already propagated back to the server thread?

Tom> +      /* We used to use xmalloc, which does this trick to avoid malloc
Tom> +	 returning a nullptr for a valid reason.  Keep doing the same.  */
Tom> +      if (length == 0)
Tom> +	length = 1;

This is most likely a workaround for vendor implementations of malloc
that return NULL for malloc(0).  See
https://www.gnu.org/software/gnulib/manual/html_node/malloc.html

However, here this is not necessary, because a 0-length memory read is
meaningless, and so this case can simply be reported as an error.

Tom