This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: [PATCH] Improve check against integer wraparound in hcreate_r [BZ #18240]
- From: Florian Weimer <fw at deneb dot enyo dot de>
- To: Paul Eggert <eggert at cs dot ucla dot edu>
- Cc: Florian Weimer <fweimer at redhat dot com>, GNU C Library <libc-alpha at sourceware dot org>, Adhemerval Zanella <adhemerval dot zanella at linaro dot org>
- Date: Mon, 25 Jan 2016 21:09:24 +0100
- Subject: Re: [PATCH] Improve check against integer wraparound in hcreate_r [BZ #18240]
- Authentication-results: sourceware.org; auth=none
- References: <56A210C4 dot 80609 at redhat dot com> <56A42D78 dot 1030506 at cs dot ucla dot edu>
* Paul Eggert:
> Florian Weimer wrote:
>
>> - if (nel >= SIZE_MAX / sizeof (_ENTRY))
>> + /* This limit is sufficient to avoid unsigned wraparound below,
>> + possibly after truncation to unsigned int. (struct hsearch_data
>> + is part of the public API and uses usigned ints.) */
>> + if (nel >= INT_MAX / sizeof (_ENTRY))
>
> This patch doesn't look right. nel should be bounded by UINT_MAX - 2,
> not by INT_MAX / sizeof (anything). (Not by UINT_MAX, since the code
> computes nel + 1 later; and not by UINT_MAX - 1 since that cannot be
> prime.) Furthermore, calloc will check for size overflow on
> multiplication so hcreate_r need not worry about dividing by sizeof
> (anything). Also, "unsigned" is misspelled in the comment.
>
> How about something like the attached (untested) patch instead?
Fair enough. isprime needs to be fixed as well, like this.
Adhemerval, do we still have time to fix this?
diff --git a/misc/hsearch_r.c b/misc/hsearch_r.c
index 7bc04cf..c73d3ed 100644
--- a/misc/hsearch_r.c
+++ b/misc/hsearch_r.c
@@ -48,7 +48,7 @@ isprime (unsigned int number)
/* no even number will be passed */
unsigned int div = 3;
- while (div * div < number && number % div != 0)
+ while (div * (unsigned long long) div < number && number % div != 0)
div += 2;
return number % div != 0;