[PATCH 01/15] stdio-common: Don't read real input beyond the field width in scanf [BZ #13988]
Maciej W. Rozycki
macro@orcam.me.uk
Sat Apr 19 10:43:29 GMT 2025
From: Maciej W. Rozycki <macro@redhat.com>
Fix a code pattern that repeats across '__vfscanf_internal' where the
remaining field width of 0 is incorrectly interpreted as no width limit,
which in turn results in reading input beyond the limit requested. The
lack of width limit is indicated by the field width of -1 rather than 0,
set earlier on in the function.
The problematic code pattern is used for both integer and floating-point
conversions, but in the former case a corresponding conditional earlier
on prevents the field width from being 0 when executing the pattern. It
does trigger in the latter case, where the decimal point is a multibyte
character or for multibyte digit characters.
Fix the code pattern by using 'width > 0' comparison, and apply the fix
throughout even to code handling integer conversions so as to interpret
the field width consistently and avoid people's confusion even if width
cannot be 0 at those places.
For multibyte digit characters there is an additional issue that causes
code to push back a partially fetched multibyte character multiple times
as execution proceeds through matching data retrieved against individual
digits that have to be rejected due to the field width limit preventing
the rest of the multibyte character from being retrieved. It is because
code relies on 'ungetc' ignoring a request to push back EOF, however in
the out-of-limit field width condition the data held is not EOF but the
previously retrieved character byte instead.
Fix this issue by artificially assigning EOF to the character byte
storage variable where the out-of-limit field width condition prevents
further processing, and also apply the fix throughout except for the
decimal point/thousands separator case, which uses different code.
Add test cases accordingly.
Referring BZ #13988 as a class bug rather than the specific issue.
---
NB there is something fishy about this code:
1. My understanding of ISO C is that the field width limit is given in
terms of (possibly multibyte) characters rather than bytes ("[...] that
specifies the maximum field width (in characters)") for all the
conversions except for %c, %s, %[ for which the field width limit is
given in bytes ("the extent of the input field is determined on a
byte-by-byte basis"). Our code seems to implement it the other way
round though.
Have I got my understanding backwards? Am I missing something here?
POSIX doesn't help as it's even vaguer ("[...] maximum field width,
which may be measured in characters or bytes dependent [sic] on the
conversion specifier").
Anyway, this fix is meant to bring our implementation to consistency,
so I propose that we apply it regardless of whether we are or aren't
going to consider any change to our interpretation of the field width.
2. It seems suboptimal to me and perhaps non-compliant to ISO C that in an
attempt to match multibyte digit characters we iterate over bytes read
in turns from and then pushed back to input in the case of a matching
failure against each single digit at a time, rather than iterating over
all digits to match against each single byte read from input at a time
and only if no digit has matched then pushing back the last byte read.
I'm not going to address this issue though.
---
localedata/Makefile | 2 +
localedata/tst-scanf-width-digit.c | 58 +++++++++++++++++++++++++++++++++++++
localedata/tst-scanf-width-point.c | 50 +++++++++++++++++++++++++++++++
stdio-common/vfscanf-internal.c | 28 ++++++++++-------
4 files changed, 126 insertions(+), 12 deletions(-)
glibc-scanf-bz13988-number-width.diff
Index: glibc/localedata/Makefile
===================================================================
--- glibc.orig/localedata/Makefile
+++ glibc/localedata/Makefile
@@ -249,6 +249,8 @@ tests = \
tst-mbswcs4 \
tst-mbswcs5 \
tst-mbswcs6 \
+ tst-scanf-width-digit \
+ tst-scanf-width-point \
tst-setlocale \
tst-setlocale2 \
tst-setlocale3 \
Index: glibc/localedata/tst-scanf-width-digit.c
===================================================================
--- /dev/null
+++ glibc/localedata/tst-scanf-width-digit.c
@@ -0,0 +1,58 @@
+/* Verify multibyte digit extending beyond scanf field width.
+ Copyright (C) 2025 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+#include <locale.h>
+#include <stdio.h>
+#include <string.h>
+
+#include <libc-diag.h>
+#include <support/check.h>
+
+#define P1 "\xdb\xb1"
+#define P2 "\xdb\xb2"
+
+static int
+do_test (void)
+{
+ if (setlocale (LC_ALL, "fa_IR.UTF-8") == NULL)
+ FAIL_EXIT1 ("setlocale (LC_ALL, \"fa_IR.UTF-8\")");
+
+ char s[] = P1 P2;
+ FILE *f = fmemopen (s, strlen (s), "r");
+
+ /* Avoid: "warning: 'I' flag used with '%f' gnu_scanf format [-Wformat=]";
+ cf. GCC PR c/119514. */
+ DIAG_PUSH_NEEDS_COMMENT;
+ DIAG_IGNORE_NEEDS_COMMENT (4.9, "-Wformat");
+
+ /* This should succeed parsing a floating-point number, and leave '\xdb',
+ '\xb2' in the input. */
+ double d;
+ int c;
+ TEST_VERIFY_EXIT (fscanf (f, "%I3lf%n", &d, &c) == 1);
+ TEST_VERIFY_EXIT (d == 1.0);
+ TEST_VERIFY_EXIT (c == 2);
+ TEST_VERIFY_EXIT (fgetc (f) == 0xdb);
+ TEST_VERIFY_EXIT (fgetc (f) == 0xb2);
+
+ DIAG_POP_NEEDS_COMMENT;
+
+ return 0;
+}
+
+#include <support/test-driver.c>
Index: glibc/localedata/tst-scanf-width-point.c
===================================================================
--- /dev/null
+++ glibc/localedata/tst-scanf-width-point.c
@@ -0,0 +1,50 @@
+/* Verify multibyte decimal point extending beyond scanf field width.
+ Copyright (C) 2025 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+#include <locale.h>
+#include <stdio.h>
+#include <string.h>
+
+#include <libc-diag.h>
+#include <support/check.h>
+
+#define PD "\xd9\xab"
+
+static int
+do_test (void)
+{
+ if (setlocale (LC_ALL, "ps_AF.UTF-8") == NULL)
+ FAIL_EXIT1 ("setlocale (LC_ALL, \"ps_AF.UTF-8\")");
+
+ char s[] = "1" PD;
+ FILE *f = fmemopen (s, strlen (s), "r");
+
+ /* This should succeed parsing a floating-point number, and leave '\xd9',
+ '\xab' in the input. */
+ double d;
+ int c;
+ TEST_VERIFY_EXIT (fscanf (f, "%2lf%n", &d, &c) == 1);
+ TEST_VERIFY_EXIT (d == 1.0);
+ TEST_VERIFY_EXIT (c == 1);
+ TEST_VERIFY_EXIT (fgetc (f) == 0xd9);
+ TEST_VERIFY_EXIT (fgetc (f) == 0xab);
+
+ return 0;
+}
+
+#include <support/test-driver.c>
Index: glibc/stdio-common/vfscanf-internal.c
===================================================================
--- glibc.orig/stdio-common/vfscanf-internal.c
+++ glibc/stdio-common/vfscanf-internal.c
@@ -1639,7 +1639,7 @@ __vfscanf_internal (FILE *s, const char
++wcdigits[n];
#else
const char *cmpp;
- int avail = width > 0 ? width : INT_MAX;
+ int avail = width >= 0 ? width : INT_MAX;
if (__glibc_unlikely (map != NULL))
mbdigits[n] = digits_extended[n];
@@ -1657,7 +1657,8 @@ __vfscanf_internal (FILE *s, const char
break;
else
{
- if (avail == 0 || inchar () == EOF)
+ if ((avail == 0 && (c = EOF))
+ || inchar () == EOF)
break;
--avail;
}
@@ -1701,7 +1702,7 @@ __vfscanf_internal (FILE *s, const char
++wcdigits[n];
#else
const char *cmpp;
- int avail = width > 0 ? width : INT_MAX;
+ int avail = width >= 0 ? width : INT_MAX;
cmpp = mbdigits[n];
while ((unsigned char) *cmpp == c && avail >= 0)
@@ -1710,8 +1711,8 @@ __vfscanf_internal (FILE *s, const char
break;
else
{
- if (avail == 0 || inchar () == EOF)
- break;
+ if ((avail == 0 && (c = EOF))
+ || inchar () == EOF)
--avail;
}
}
@@ -1757,7 +1758,7 @@ __vfscanf_internal (FILE *s, const char
break;
#else
const char *cmpp = thousands;
- int avail = width > 0 ? width : INT_MAX;
+ int avail = width >= 0 ? width : INT_MAX;
while ((unsigned char) *cmpp == c && avail >= 0)
{
@@ -1766,7 +1767,8 @@ __vfscanf_internal (FILE *s, const char
break;
else
{
- if (avail == 0 || inchar () == EOF)
+ if ((avail == 0 && (c = EOF))
+ || inchar () == EOF)
break;
--avail;
}
@@ -1837,7 +1839,7 @@ __vfscanf_internal (FILE *s, const char
break;
#else
const char *cmpp = thousands;
- int avail = width > 0 ? width : INT_MAX;
+ int avail = width >= 0 ? width : INT_MAX;
while ((unsigned char) *cmpp == c && avail >= 0)
{
@@ -1846,7 +1848,8 @@ __vfscanf_internal (FILE *s, const char
break;
else
{
- if (avail == 0 || inchar () == EOF)
+ if ((avail == 0 && (c = EOF))
+ || inchar () == EOF)
break;
--avail;
}
@@ -2225,7 +2228,7 @@ __vfscanf_internal (FILE *s, const char
}
#else
const char *cmpp = decimal;
- int avail = width > 0 ? width : INT_MAX;
+ int avail = width >= 0 ? width : INT_MAX;
if (! got_dot)
{
@@ -2463,14 +2466,15 @@ __vfscanf_internal (FILE *s, const char
}
#else
const char *cmpp = mbdigits[n];
- int avail = width > 0 ? width : INT_MAX;
+ int avail = width >= 0 ? width : INT_MAX;
while ((unsigned char) *cmpp == c && avail >= 0)
if (*++cmpp == '\0')
break;
else
{
- if (avail == 0 || inchar () == EOF)
+ if ((avail == 0 && (c = EOF))
+ || inchar () == EOF)
break;
--avail;
}
More information about the Libc-alpha
mailing list