What to do about libidn?

[Florian Weimer <fweimer at redhat dot com>]

For AI_IDN support in getaddrinfo, we currently bundle a really old copy 
of libidn.

This has several problems:

1. We lack a couple of security fixes.

2. libidn, as an API, is very to use because it has complicated 
preconditions for its input.  This may have been fixed in later upstream 
versions.

3. The tables are fairly large.  On the other hand, we may need the 
Unicode NFC tables for password hashing, too.

4. The IETF more or less replaced IDNA-2003 with a different and 
slightly incompatible standard, IDNA-2008.  There is no version 
negotiation, and some registries tried to implement it with a flag day 
(each registry with a different date, of course).  libidn seems to be 
IDNA-2003 only.

5. There is considerable variance among IDNA-2008 implementation. 
IDNA-2008 is described in terms of a specific Unicode version (5.2). 
The IANA tables were officially updated to Unicode 6.3 in RFC 6452.  I'm 
not sure if actual implementation (in browsers, for example) follow 
these tables because they probably want to use newer Unicode version.

6. Distributions have their own system-wide copy of libidn (which is 
not the one in glibc).  They do not use libidn2 (which seems to be 
required for IDNA-2008 support).  This means that even if we update 
glibc, most applications will not benefit.

7. On the glibc side, IDN only applies to getaddrinfo, is opt-in via 
AI_IDN, and requires a non-ASCII locale.  Everything else sends 
unencoded bytes over the wire via DNS.


What should we do to improve this situation?  I would really like to 
remove AI_IDN, but this is likely not an option.

Should we remove our internal copy and try to dlopen libidn2?  Maybe 
falling back to libidn if libdn2 is unavailable?  Bundle libidn2?  Write 
our own implementation?

Thanks,
Florian