[RFC PATCH 0/4] fix resolver with IPv6 nameservers and/or rotate option

[Aurelien Jarno <aurelien at aurel32 dot net>]

This patch series is an attempt to fix the longstanding issues in the
resolv code, which happens with an IPv6 nameserver and/or the rotate
option. This issues are not really new, but more people encountered them
because IPv6 is getting more common (yay!) and it seems because IPv6
recursive nameservers are getting more common. Many people tried to fix
this in the past years, including me, but this time I have spent hours
to really understand how each part of the code interact. I don't pretend
to be successful, but I hope that presenting my findings and one
possible solution will help to go in the right direction. That's why I
put a RFC in the subject, why I don't provide changelog entries for now.

First of all a bit of history and code analysis. I *think* the rotate
option itself is broken for a lot of time, in the sense the rotation
itself was not done but without breaking name resolving. On the other
hand having multiple IPv6 nameserver or at least one IPv6 nameserver
and the rotate option enabled as been broken in commit 6f5c3117, and
prevent name resolving.

The __res_state structure contains a main part, including a list of
MAXNS IPv4 nameservers stored in nsaddr_list, and an extension part,
which contains a list of MAXNS nameservers stored in nsaddrs. Due to
the use of a sockaddr_in6 type, they can be either IPv4 or IPv6. Note
also that these are pointers, so memory have to be allocated and freed
in the code. This looks quite complex, but on the other hand the
structure is part of the public header resolv.h, and plenty of functions
taking a struct __res_state as an argument are publically exported (and
not using the GLIBC_PRIVATE symbol versioning), so my guess is that we
can't change that.

When the resolv.conf file is parsed in __res_vinit, the IPv4 nameservers
are filled in nsaddr_list, while the IPv6 nameserver are filled in
nsaddrs, in the extension part. To avoid a complex handling of two set
of nameservers, IPv4 nameservers are copied to the nsaddrs array (after
mallocating the memory for sockaddr_in6) in __libc_res_nsend on the
first initialization. As this structure already contains the IPv6
nameservers, all the question is how the two sets of nameservers
are interleaved:
 - An analysis of the code shows that the nsaddr_list array, storing the
   IPv4 nameserver should be filled in order (which is not the case
   actually), nsaddrs array should be filled by leaving space to later
   store the IPv4 nameservers. This is the purpose of all the map part
   at the beginning of __libc_res_nsend. In order to be able to later
   convert back the nsaddrs index into the nsaddr_list index, the
   nsmap array in the extension part of struct __res_state is used. It
   should be noted there the convention that the MAXNS value represent
   an empty entry, while the MAXNS + 1 value represents an IPv6
   nameserver. The code (and the rotate code) is complexified by the
   fact unallocated entry can be located in the middle of the array. It
   is also complexified by the number of nameserver count variables:
   statp->nscount, EXT(statp).nscount, EXT(statp).nscount6.
 - Instead of the above complex method, commit 6f5c3117 changed the
   indexing of nsaddr_list so that IPv6 entries leave a gap in the IPv4
   nameserver list. That way we don't have any nameserver at the same
   index in both nsaddrs and nsaddr_list. This is much simpler, as it
   means we don't have to keep a map between the two structures. That
   said the commit did only part of the job. This has been partially
   fixed in commit 16b7dc27, but today the code (now wrongly) doing the
   mapping still exists. It causes issues (and completely breaks 
   name resolving with the rotate option) if an IPv6 nameserver is
   located before an IPv4 nameserver in resolv.conf.
   
This patch series is fixing the issue by restoring the original
behaviour, and fixing a few other small things. I prefered this instead
of going in the direction of a simpler code, as I am afraid of side
effects in other parts of the code, and to avoid breaking the public 
interface.

Aurelien Jarno (4):
  resolv: improve comments about nserv and nservall
  resolv: fix nsaddr_list array indexing
  resolv: fix issues when using IPv6 nameservers only
  resolv: fix rotate option

 resolv/res_init.c | 17 +++++++++--------
 resolv/res_send.c |  2 +-
 2 files changed, 10 insertions(+), 9 deletions(-)

-- 
2.0.0.rc0