RFA/RFC: Add stack recursion limit to libiberty's demangler

[Nick Clifton]

Hi Ian,

  *sigh* it is always the way.  You post a patch and five minutes later
  you find a bug in it.  In this case it turned out that I had forgotten
  that gcc has its own copy of the libiberty sources, so the bootstrap
  test that I had run did not use the patched sources.  Doh.  When I did
  rerun the bootstrap with the patched sources it failed because I had
  forgotten to use the CP_STATIC_IF_GLIBCPP_V3 macro when declaring the
  new cplus_demangle_set_recursion_limit() function.

  I am attaching a revised patch with this bug fixed, and an updated
  changelog entry as I have found a few more CVE PRs that it fixes.

  Also - Tom and Pedro have raised the issue that the patch introduces
  a new static variable to the library that is not thread safe.  I am
  not sure of the best way to address this problem.  Possibly the
  variable could be made thread local ?  Are there any other static
  variables in libiberty that face the same issue ?
  
Cheers
  Nick

libiberty/ChangeLog
2018-11-29  Nick Clifton  <nickc@redhat.com>

	PR 87681
	PR 87675
	PR 87636
	PR 87335
	* cp-demangle.c (demangle_recursion_limit): New static
	variable.
        (d_function_type): Add recursion counter.  If the recursion
        limit is enabled and reached, return with a failure result.
        (d_demangle_callback): If the recursion limit is enabled, check
	for a mangled string that is so long that there is not enough
	stack space for the local arrays.
        (cplus_demangle_set_recursion): New function.  Sets and/or
	returns the current stack recursion limit.
        * cplus-dem.c (demangle_nested_args): Add recursion counter.  If
	the recursion limit is enabled and reached, return with a
	failure result.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: libiberty-demangler-recursion-limit.2.patch
Type: text/x-patch
Size: 4725 bytes
Desc: not available
URL: <https://sourceware.org/pipermail/binutils/attachments/20181130/a2d4e6d5/attachment.bin>