[11] New NSS module API

[Florian Weimer]

* Replacement for the ERANGE protocol
  - Tremendous overhead for large DNS responses with ERANGE handling.
  - Idea: Introduce deallocation function
* Maybe: generic query function for all databases.
* Convey timeout information for all databases.

NSS functions such as getwpuid produce variable-length results because
there are no immediate restrictions on user name lengths, home directory
path lengths, and so on.  Internally (and externally, with the
getpwuid_r family functions), this is handled by callers allocating a
buffer, the called NSS function (which could be something externally
supplied in an NSS plugin/service module) writes to that buffer up to
the specified size.  If the buffer is not large enough, the function is
supposed to fail in a recognizable way (involving an ERANGE error), and
the caller is expected to retry with a larger buffer (usually doubling
the buffer size—the caller has no knowledge of exact buffer size
requirements).

This resizing is quite costly in practice.  For example, if we have a
DNS response that is about 2,500 bytes, we do a UDP query first, that
gets truncated, so we try over TCP, that doesn't fit into the
bufffer. That triggers the ERANGE error.  Repeat (both UDP + TCP), for
another ERANGE error.  Now we are at a ~2000 bytes buffer, so we still
fail with ERANGE.  The next repeat is at ~4000 bytes, so that succeeds.
That's a truly excessive amount of network traffic.  The same behavior
occurs if you have long lines in /etc/groups or any of the file-based
data sources.

If the plugin function can do its own buffer management and allocate
more memory as needed, we'd just have one UDP query followed by one TCP
query.  The caller-supplied buffer has the advantage that the plugin and
the NSS framework do not need to use the same malloc (different mallocs
occur with static dlopen), but this is easily addressed by the service
module defining a deallocation function in addition to a query function.

What is also missing in most of the current API is timeout
information. For example, if data comes out of LDAP, we don't know for
how long we can cache it according to the organization's policy.

And we currently cannot perform full IPv6-compliant host name lookup
just for IPv6 addresses: the only plugin interface that returns scope
IDs unconditionally performs IPv4 and IPv6 lookups at the same time.

Maybe it makes sense to expose a single, generic query endpoint instead
of multiple functions per service database.

This isn't a small project because we'd have to look at existing plugins
(sssd, mdns, systemd) to see if the new interface meets their needs.  We
could perhaps start with something that is just exposed internally (for
files and dns).