[PATCH 0/7] RFC: Towards deprecating the ERANGE protocol for NSS

[Florian Weimer]

This series started with an attempt to introduce deduplication into
group merging.

I think the group merging code becomes much clearer if there is a way to
copy the intermediate lookup results into their own buffers.  After
that, I wanted to see what an alternative implementation looks like that
dynamically allocates buffers instead telling callers via ERANGE to try
again with a larger buffer.  And avoid using the preprocessor so much.

I think we should move in this direction, eventually extending the
callee-allocates interface to NSS modules as well.  For the legacy *_r
functions, we can add a single-element per-thread cache that at least
makes the retries quite cheap.  This is not yet implemented here, but it
is on my to-do list.

Extending this to other NSS types such as struct passwd isn't too
difficult.  Although the non-group __nss_getXinfo function is a bit of a
stub right now.

I tried first to use function pointers heavily, but I think the version
here, with enum nss_lookup_type, will be easier to maintain.

Eventually, we may consider new interfaces like getgrgidinfo and
getgrnaminfo alongside freegrinfo that work similar to getaddrinfo.

Thanks,
Florian

Florian Weimer (7):
  nscd: Fix most data races in client retry counters (bug 33654)
  nss: Add negative lookup test
  nss: Add __nss_generic_copy and __nss_generic_dup functions
  Extract <stringtable.h> from ldconfig
  nss: Low-level functionality for merging group lists
  nscd: Add __nscd_getgrnam and __nscd_getgrgid client functions
  nss: Add new framework for hiding the ERANGE protocol internally

 elf/Makefile                                  |   4 +-
 elf/cache.c                                   |  28 +--
 elf/cachestrings.c                            | 149 +++++++++++++
 elf/{stringtable.h => cachestrings.h}         |  40 ++--
 elf/stringtable.c                             | 209 -----------------
 elf/{tst-stringtable.c => tst-cachestrings.c} | 128 ++++++-----
 include/set-freeres.h                         |   4 +-
 include/stringtable-skeleton.h                | 128 +++++++++++
 include/stringtable.h                         |  54 +++++
 malloc/set-freeres.c                          |   3 +-
 misc/Makefile                                 |   3 +
 elf/stringtable_free.c => misc/fnv1a.c        |  24 +-
 misc/stringtable_add.c                        | 127 +++++++++++
 misc/stringtable_free.c                       |  43 ++++
 nscd/Makefile                                 |   6 +-
 nscd/nscd_getai.c                             |  10 +-
 nscd/nscd_getgr_r.c                           |  58 ++++-
 nscd/nscd_gethst_r.c                          |  10 +-
 nscd/nscd_getpw_r.c                           |   6 +-
 nscd/nscd_getserv_r.c                         |   7 +-
 nscd/nscd_initgroups.c                        |   5 +-
 nscd/nscd_netgroup.c                          |  11 +-
 nscd/nscd_proto.h                             |  23 +-
 nscd/nscd_use_database.c                      |  57 +++++
 nscd/tst-nscd_use_database.c                  |  53 +++++
 nss/Makefile                                  |  27 ++-
 nss/getXXbyYY_r.c                             |  10 +-
 nss/getaddrinfo.c                             |  12 +-
 nss/getgrgid.c                                |  14 +-
 nss/getgrgid_r.c                              |  20 +-
 nss/getgrnam.c                                |  14 +-
 nss/getgrnam_r.c                              |  23 +-
 nss/getnetgrent_r.c                           |  14 +-
 nss/initgroups.c                              |   9 +-
 nss/nss-lookups.def                           |  26 +++
 nss/nss_database.c                            |   5 +-
 nss/nss_generic.h                             | 132 +++++++++++
 nss/nss_generic_copy.c                        |  90 ++++++++
 nss/nss_generic_dup.c                         |  88 ++++++++
 nss/nss_generic_get.c                         |  61 +++++
 nss/nss_generic_get_r.c                       |  39 ++++
 nss/nss_generic_lookup.c                      |  51 +++++
 nss/nss_generic_next.c                        |  60 +++++
 nss/nss_generic_nscd.c                        |  62 ++++++
 nss/nss_generic_storage.h                     |  29 +++
 nss/nss_getX.c                                |  53 +++++
 nss/nss_getX_r.c                              |  45 ++++
 nss/nss_getXinfo.c                            |  64 ++++++
 nss/nss_getgrXinfo.c                          | 176 +++++++++++++++
 nss/nss_group_members.c                       |  80 +++++++
 nss/nss_group_members.h                       |  61 +++++
 nss/nss_module.c                              |   7 +-
 nss/nsswitch.c                                |   5 -
 nss/nsswitch.h                                |   2 -
 nss/tst-nss-does-not-exist.cc                 | 210 ++++++++++++++++++
 nss/tst-nss-test4.c                           |  12 +-
 nss/tst-nss_generic_copy.c                    | 131 +++++++++++
 nss/tst-nss_generic_dup.c                     | 134 +++++++++++
 nss/tst-nss_group_members.c                   |  77 +++++++
 support/Makefile                              |   1 +
 support/support_stringtable.c                 |  20 ++
 61 files changed, 2593 insertions(+), 461 deletions(-)
 create mode 100644 elf/cachestrings.c
 rename elf/{stringtable.h => cachestrings.h} (53%)
 delete mode 100644 elf/stringtable.c
 rename elf/{tst-stringtable.c => tst-cachestrings.c} (54%)
 create mode 100644 include/stringtable-skeleton.h
 create mode 100644 include/stringtable.h
 rename elf/stringtable_free.c => misc/fnv1a.c (63%)
 create mode 100644 misc/stringtable_add.c
 create mode 100644 misc/stringtable_free.c
 create mode 100644 nscd/nscd_use_database.c
 create mode 100644 nscd/tst-nscd_use_database.c
 create mode 100644 nss/nss-lookups.def
 create mode 100644 nss/nss_generic.h
 create mode 100644 nss/nss_generic_copy.c
 create mode 100644 nss/nss_generic_dup.c
 create mode 100644 nss/nss_generic_get.c
 create mode 100644 nss/nss_generic_get_r.c
 create mode 100644 nss/nss_generic_lookup.c
 create mode 100644 nss/nss_generic_next.c
 create mode 100644 nss/nss_generic_nscd.c
 create mode 100644 nss/nss_generic_storage.h
 create mode 100644 nss/nss_getX.c
 create mode 100644 nss/nss_getX_r.c
 create mode 100644 nss/nss_getXinfo.c
 create mode 100644 nss/nss_getgrXinfo.c
 create mode 100644 nss/nss_group_members.c
 create mode 100644 nss/nss_group_members.h
 create mode 100644 nss/tst-nss-does-not-exist.cc
 create mode 100644 nss/tst-nss_generic_copy.c
 create mode 100644 nss/tst-nss_generic_dup.c
 create mode 100644 nss/tst-nss_group_members.c
 create mode 100644 support/support_stringtable.c


base-commit: 0f7b73f2ed70e783cd02ab77503645b03ee1d332
-- 
2.52.0