[RFC] FIPS compliance and other crypt(3) improvements

[Alexandre Oliva]

On May 23, 2012, Roland McGrath <roland@hack.frob.com> wrote:

>> Aha!  Ok, then.  The description for ENOSYS under crypt still fit, but
>> given your comment, I'm now going with EINVAL for this case.

> No, ENOSYS does not fit.  The wording for that particular function may
> not be clear, but elsewhere in the standard it's quite clear that
> ENOSYS is for a function that is not available at all--every call will
> always fail with ENOSYS.

I see.

> EINVAL means it's a malformed argument value, i.e. a bug in the
> calling program.

So this must be the errno code for an invalid DES salt.

> Another sensible alternative is EPERM,

I've made this the errno code for disabled algorithms for FIPS
compliance.

>> +	perror ("mmap 2");

> It should be a hard failure, shouldn't it?

I don't think so.  Without the patch, we'd crash if the second mmap
succeeded, and access (zero-)uninitialized memory otherwise, so both
would fail, although in different ways.  With the patch, we'll report
the same error (EINVAL) regardless of the result of this second mmap.
So it doesn't seem useful to abort the test for this reason.

Now, I've arranged for us to return EINVAL over EPERM when either one
would be reasonable, even though doing the opposite would make the code
simpler and faster, because I decided to favor consistency, i.e., that
crypt(passwd, "*") will return EINVAL regardless of FIPS status, for
with "*" we can tell it was *not* supposed to use DES, and it should
return EPERM for DES only.

My only doubt is whether to pick EINVAL or ENOTSUPP when we encounter an
invalid DES salt, for an invalid DES salt might very well be some
extended algorithm that we don't support (or even know about).  I'm
going with EINVAL because it is appropriate in more cases than ENOTSUPP,
for only some of the invalid DES salts are algorthm extensions that we
don't support.

>> for  ChangeLog
>> 2012-05-15  Alexandre Oliva  <aoliva@redhat.com>

> Note that we now use date-of-commit in the log lines, so you'll need
> to update this when it goes in.

Sure.  Any chance the mailing list moderator could refrain from blocking
patches that have “from” instead of a date?  (assuming this hasn't
already changed since I last tried to post a cvsdiff-generated patch
file).  Then cvsdiff patches would be accepted without manual handling,
and cl2patch would apply them properly, with the current date.

>> -  result |= strcmp ("$1$saltstri$YMyguxXMBpd2TEZ.vS/3q1", cp);
>> +
>> +  /* MD5 is disabled in FIPS mode.  */
>> +  if (cp)
>> +    result |= strcmp ("$1$saltstri$YMyguxXMBpd2TEZ.vS/3q1", cp);
>> 
>> return result;

> Ideally we'd have a way to test the code regardless of the system
> configuration.  But I don't see a straightforward way off hand.

Me neither.  Exposing any alternate entry point would make room for
security-related abuses.

Now, I must confess I'm surprised this FIPS-related restrictions on
crypt are being seriously considered for glibc.  I'd have thought we'd
privilege POSIX-compliant behavior, pushing FIPS password algorithm
rejection to code that uses crypt for actual password checking or
modification, rather than for any code that calls crypt for whatever
reason (e.g., password crackers).

I've implemented your other suggestions and fixes, thanks!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: crypt-badsalt-bz811753.patch
Type: text/x-diff
Size: 7021 bytes
Desc: not available
URL: <http://sourceware.org/pipermail/libc-alpha/attachments/20120605/07b77f0b/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: crypt-fipsalgo-bz811753.patch
Type: text/x-diff
Size: 6657 bytes
Desc: not available
URL: <http://sourceware.org/pipermail/libc-alpha/attachments/20120605/07b77f0b/attachment-0001.bin>
-------------- next part --------------


-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer