Calling getaddrinfo() with invalid IPv4 literal

[Florian Weimer]

* Scott Talbert via Libc-help:

> On Wed, 13 May 2020, Scott Talbert wrote:
>
>>>> What is the expected behavior when calling getaddrinfo() with an invalid
>>>> IPv4 literal?  By invalid, I mean a dotted quad where one or more of the
>>>> parts is outside 0-255.
>>>>
>>>> For example, getaddrinfo("1.2.260.5", ...).
>>>>
>>>> Currently, it seems this succeeds, but returns some sort of (rolled over?)
>>>> address.  Previously at some point, it seems this used to return an error.
>>>
>>> You need to provide more information about your system, like glibc
>>> version, applied patches, and what's on the “hosts” line in
>>> /etc/nsswitch.conf.
>>>
>>> I believe stock glibc has failed getaddrinfo queries for the name you
>>> indicate for a long, long time.
>>
>> My bad.  I should have provided more info.  My question pertains
>> mainly to Debian.
>>
>> Debian unstable: glibc 2.30-8
>> nsswitch: hosts:          files myhostname mdns4_minimal
>> [NOTFOUND=return] dns mdns4 mymachines
>> Behavior: return success
>>
>> Debian stretch: glibc 2.24-11+deb9u4
>> nsswitch: hosts:          files mdns4_minimal [NOTFOUND=return] dns
>> myhostname
>> Behavior: return failure
>>
>> Ubuntu 18.04: glibc 2.27-3ubuntu1
>> nsswitch: hosts:          files mdns4_minimal [NOTFOUND=return] dns
>> myhostname
>> Behavior: return success
>
> Okay, so I got to the bottom of this.  The problem turned out to be a
> buggy local nameserver that returns "success" when trying to resolve
> an A record for 1.2.260.5.

Oh indeed, I forgot to mention the policy that these names can leak out
to the DNS.

1.2.0260.5 would be parsed as an IPv4 address literal by getaddrinfo,
though because 0260 == 176, for strange backwards compatibility reasons.

Thanks,
Florian