NSS error reporting (bug 20532)

[Florian Weimer]

On 08/03/2017 06:28 PM, Florian Weimer wrote:
> I think the three values (enum nss_status return value [1], *errnop [2],
> *herrnop [4]) should behave roughly like this (as expressed in Standard
> ML, I hope anyone reading thus far can understand this):
> 
>   type errno = int
> 
>   datatype h_errno =
>       NETDB_SUCCESS
>     | HOST_NOT_FOUND
>     | TRY_AGAIN
>     | NO_RECOVERY
>     | NO_DATA
>     | NETDB_INTERNAL of errno
> 
>   datatype 'a nss_status =
>       NSS_STATUS_TRYAGAIN of 'a
>     | NSS_STATUS_UNAVAIL of 'a
>     | NSS_STATUS_NOTFOUND
>     | NSS_STATUS_SUCCESS
>     | NSS_STATUS_RETURN
> 
>   (* The hosts database has an associated h_errno variable, and errno is
>      only valid in the NETDB_INTERNAL case.  *)
>   type host_status = h_errno nss_status
> 
>   (* The other databases use errno directly, without h_errno.  *)
>   type other_status = errno nss_status

The above is incorrect because there are two cases for
NSS_STATUS_NOTFOUND: The name does not exist, or the name exists, but
does not have an address of the requested family.  Based on <netdb.h>
and the manual page, applications can tell apart the two cases for
gethostbyname-style functions using the HOST_NOT_FOUND or NO_DATA error
codes in h_errno.  EAI_NONAME and EAI_NODATA convey the same information
for getaddrinfo.

However, this does not seem to be currently implemented for nss_files,
and getaddrinfo is partially incompatible with the nss_dns
implementation (it always returns EAI_NONAME for AF_UNSPEC).  This seems
to be caused by the incorrect error handling in gethosts.

Florian