[PATCH] Fix nss_db crash when db contains exactly one entry

[Siddhesh Poyarekar]

Hi,

It looks like the hashtable size calculation in nssdb is buggy, since
it returns a hashtable of size 1 when a database has just one entry.
This is easy to reproduce with the following:

* Have exactly one user associated with another group, i.e. exactly
  one line of the type:

mail:x:12:postfix

* Ensure that db is queried first for groups:

cat /etc/nsswitch.conf:

...
passwd:     files
shadow:     files
group:      db files
...

* Make the nssdb:

cd /var/db && make

* Start nscd -d and watch it crash

The crash is inside nssdb and hence with the above configuration,
programs throughout the system will eventually start crashing (sshd,
systemd/init, etc.).  Attached patch ensures that makedb sets the
hashtable size to a prime greater than 3, which is sufficient for the
way the hashtable is populated.

I've verified that this fixes the crash on an F18 test system
(2.16.x).  OK to check-in?

Siddhesh

ChangeLog:

	* nss/makedb.c (is_prime): Ensure that CANDIDATE is greater
        than 3.
-------------- next part --------------
diff --git a/nss/makedb.c b/nss/makedb.c
index 8d7d027..fdcf4c6 100644
--- a/nss/makedb.c
+++ b/nss/makedb.c
@@ -591,10 +591,12 @@ copy_valstr (const void *nodep, const VISIT which, const int depth)
 }
 
 
+/* Check if a number is prime.  We check only odd numbers greater than 10.
+   Enter even numbers and watch the function fail in mysterious ways.  Odd
+   numbers less than 10 return false even if they are prime.  */
 static int
 is_prime (size_t candidate)
 {
-  /* No even number and none less than 10 will be passed here.  */
   size_t divn = 3;
   size_t sq = divn * divn;
 
@@ -605,7 +607,7 @@ is_prime (size_t candidate)
       ++divn;
     }
 
-  return candidate % divn != 0;
+  return (candidate >= divn && candidate % divn != 0);
 }