RFA/RFC: Add stack recursion limit to libiberty's demangler [v4]

[Nick Clifton]

Hi Ian,

>>> Shouldn't we make it fool-proof by instead introducing a DMGL_NO_RECURSION_LIMIT

> You don't need my blessing--I wrote that code ages ago--but I agree
> with Richard that in practice it's OK to limit recursion depth by
> default.  Real symbols have very limited recursion requirements.

OK then, here is a fourth revision of the patch.

In this version:

  * The demangler option has been renamed to DMGHL_NO_RECURSE_LIMIT
    and if the option is not present then the limit is enforced.

  * I also found another PR that is fixed by the patch, although I had
    to make sure that the affected code could handle NULL pointers 
    properly afterwards.

  OK to apply ?

Cheers
  Nick


include/ChangeLog
2018-11-29  Nick Clifton  <nickc@redhat.com>

	* demangle.h (DMGL_NO_RECURSE_LIMIT): Define.
        (DEMANGLE_RECURSION_LIMIT): Define

libiberty/ChangeLog
2018-11-29  Nick Clifton  <nickc@redhat.com>

	PR 87681
	PR 87675
	PR 87636
	PR 87350
	PR 87335
	* cp-demangle.h (struct d_info): Add recursion_level field.
	* cp-demangle.c (d_function_type): Add recursion counter.
	If the recursion limit is reached and the check is not disabled,
	then return with a failure result.
	(cplus_demangle_init_info): Initialise the recursion_level field.
        (d_demangle_callback): If the recursion limit is enabled, check
	for a mangled string that is so long that there is not enough
	stack space for the local arrays.
        * cplus-dem.c (struct work): Add recursion_level field.
	(squangle_mop_up): Set the numb and numk fields to zero.
	(work_stuff_copy_to_from): Handle the case where a btypevec or 
	ktypevec field is NULL.
	(demangle_nested_args): Add recursion counter.  If
	the recursion limit is not disabled and reached, return with a
	failure result.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: libiberty-demangler-recursion-limit.4.patch
Type: text/x-patch
Size: 7011 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/binutils/attachments/20181204/838e71da/attachment.bin>