[PATCH 0/2] Fix gconv reference count overflow in swscanf

[Frédéric Bérat]

This series addresses a gconv module reference counter overflow
triggered by the `swscanf` family of functions.

The issue, originally reported by DJ Delorie, occurs because `swscanf`
utilizes a wide-oriented `FILE` stream allocated on the stack via
`_IO_strfile_readw`. When initialized, `_IO_fwide` implicitly clones the
global locale's gconv configuration and increments the gconv module's
reference counter (`__counter`). Because the stream is on the stack, it
cannot be cleaned up via `fclose()` (which would attempt to `free` the
stack pointer). Consequently, `__gconv_release_step` is never called,
leaking the reference counts. Over enough iterations, the 32-bit counter
overflows, resulting in a fatal abort.

Florian Weimer correctly pointed out that if we open the gconv modules
with `RTLD_NODELETE`, we wouldn't need to track references for
`dlclose`. However, upon further investigation of the codebase, removing
the `__counter` tracking entirely is not viable without introducing
catastrophic memory bugs.

The `__counter` member dictates the execution of `gconv_end()`. Several
complex encodings (such as `UTF-16`) dynamically allocate memory during
`gconv_init()` and attach it to `step->__data`. If we remove
`__counter`, we face an unsolvable dilemma during stream teardown: 1. If
we never call `gconv_end()`, we permanently leak `step->__data` memory
every time a dynamic step array is freed (e.g., from the mmap cache). 2.
If we unconditionally call `gconv_end()`, the stack stream destroys the
`step->__data` state that the global locale (and other concurrent
threads) still rely on, leading to immediate use-after-free conditions.

Therefore, reference counting must be maintained to safely manage the
`step->__data` lifecycle.

This patch series follows Adhemerval Zanella Netto's suggestion to
introduce a targeted internal `fclose` equivalent. We introduce
`_IO_wstrfile_fclose_stack()`, which safely releases the gconv reference
counters and finishes the stream without attempting to deallocate the
`FILE` struct. This new function is then hooked into all 13
implementations of `swscanf` across the tree.

Fred.

--

Frédéric Bérat (2):
  libio: Fix gconv module reference counter overflow in swscanf
  wcsmbs: Add gconv module ref counter overflow test

 libio/iofwide.c                               | 15 ++++++
 libio/iovswscanf.c                            |  4 +-
 libio/libioP.h                                |  1 +
 libio/swscanf.c                               |  2 +-
 .../ieee128-isoc23_swscanf.c                  |  2 +-
 .../ieee128-isoc23_vswscanf.c                 |  4 +-
 .../ieee128-isoc99_swscanf.c                  |  2 +-
 .../ieee128-isoc99_vswscanf.c                 |  4 +-
 .../ldbl-128ibm-compat/ieee128-swscanf.c      |  2 +-
 .../ldbl-128ibm-compat/ieee128-vswscanf.c     |  4 +-
 sysdeps/ieee754/ldbl-opt/nldbl-compat.c       | 12 +++--
 wcsmbs/Makefile                               |  3 +-
 wcsmbs/isoc23_swscanf.c                       |  2 +-
 wcsmbs/isoc23_vswscanf.c                      |  4 +-
 wcsmbs/isoc99_swscanf.c                       |  2 +-
 wcsmbs/isoc99_vswscanf.c                      |  4 +-
 wcsmbs/tst-wcsmbs-clone-overflow.c            | 50 +++++++++++++++++++
 17 files changed, 101 insertions(+), 16 deletions(-)
 create mode 100644 wcsmbs/tst-wcsmbs-clone-overflow.c

-- 
2.54.0