Re: What to do about libidn?

[Zack Weinberg <zackw at panix dot com>]

On Tue, Nov 8, 2016 at 6:52 AM, Florian Weimer <fweimer@redhat.com> wrote:
> For AI_IDN support in getaddrinfo, we currently bundle a really old copy of
> libidn. This has several problems:
>
> 1. We lack a couple of security fixes.
>
> 2. libidn, as an API, is very to use because it has complicated
> preconditions for its input.  This may have been fixed in later upstream
> versions.
>
> 3. The tables are fairly large.  On the other hand, we may need the Unicode
> NFC tables for password hashing, too.
>
> 4. The IETF more or less replaced IDNA-2003 with a different and slightly
> incompatible standard, IDNA-2008.  There is no version negotiation, and some
> registries tried to implement it with a flag day (each registry with a
> different date, of course).  libidn seems to be IDNA-2003 only.
>
> 5. There is considerable variance among IDNA-2008 implementation. IDNA-2008
> is described in terms of a specific Unicode version (5.2). The IANA tables
> were officially updated to Unicode 6.3 in RFC 6452.  I'm not sure if actual
> implementation (in browsers, for example) follow these tables because they
> probably want to use newer Unicode version.
>
> 6. Distributions have their own system-wide copy of libidn (which is not the
> one in glibc).  They do not use libidn2 (which seems to be required for
> IDNA-2008 support).  This means that even if we update glibc, most
> applications will not benefit.
>
> 7. On the glibc side, IDN only applies to getaddrinfo, is opt-in via AI_IDN,
> and requires a non-ASCII locale.  Everything else sends unencoded bytes over
> the wire via DNS.

[...]

I just saw something go by about security problems with blindly
applying IDNA-2008 without additional input validation, too. Can't
find it right now.  cc:ing the libidn(2) maintainer.

> What should we do to improve this situation?  I would really like to remove
> AI_IDN, but this is likely not an option.

I also rather like the idea of dropping AI_IDN.  As a data point,
https://searchcode.com/?q=AI_IDN shows only 39 hits out of "20 billion
lines of code from 7,000,000 projects" - and at least half of those
appear to be implementations and library wrappers.

zw