Bug 25208 - the new cache cannot be updated when the daemons are started again
Summary: the new cache cannot be updated when the daemons are started again
Status: RESOLVED WONTFIX
Alias: None
Product: glibc
Classification: Unclassified
Component: nscd (show other bugs)
Version: 2.28
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-11-20 09:38 UTC by whzhe
Modified: 2020-02-19 15:19 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:
fweimer: security-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description whzhe 2019-11-20 09:38:45 UTC
when i stop the daemons, i add the group test1 and add test to this group.

[root@localhost whzhe]# id test
uid=1000(test) gid=1000(test) groups=1000(test),1004(test1)

Now i start up the nscd daemons and also check the id test.

[root@localhost whzhe]# id test
uid=1000(test) gid=1000(test) groups=1000(test)

There is no group test1.And i know i can using 'nscd -i group' to delete the cache. 
But i think when i restart up the nscd daemons, the nscd daemons need to update the cache and fix this problem.
Comment 1 Carlos O'Donell 2019-11-20 14:30:35 UTC
(In reply to whzhe from comment #0)
> when i stop the daemons, i add the group test1 and add test to this group.
> 
> [root@localhost whzhe]# id test
> uid=1000(test) gid=1000(test) groups=1000(test),1004(test1)
> 
> Now i start up the nscd daemons and also check the id test.
> 
> [root@localhost whzhe]# id test
> uid=1000(test) gid=1000(test) groups=1000(test)
> 
> There is no group test1.And i know i can using 'nscd -i group' to delete the
> cache. 
> But i think when i restart up the nscd daemons, the nscd daemons need to
> update the cache and fix this problem.

They should have. There is code to add the caches and invlidate them at startup if their last modified date has changed.

What version of glibc is this? Which distribution?

Could you please start nscd with debug level set to 9? And provide the logs.
Comment 2 whzhe 2019-11-21 02:24:34 UTC
(In reply to Carlos O'Donell from comment #1)
> (In reply to whzhe from comment #0)
> > when i stop the daemons, i add the group test1 and add test to this group.
> > 
> > [root@localhost whzhe]# id test
> > uid=1000(test) gid=1000(test) groups=1000(test),1004(test1)
> > 
> > Now i start up the nscd daemons and also check the id test.
> > 
> > [root@localhost whzhe]# id test
> > uid=1000(test) gid=1000(test) groups=1000(test)
> > 
> > There is no group test1.And i know i can using 'nscd -i group' to delete the
> > cache. 
> > But i think when i restart up the nscd daemons, the nscd daemons need to
> > update the cache and fix this problem.
> 
> They should have. There is code to add the caches and invlidate them at
> startup if their last modified date has changed.
> 
> What version of glibc is this? Which distribution?
> 
> Could you please start nscd with debug level set to 9? And provide the logs.

Glibc version is glibc-2.28-9.And i set the debug-level to 9, the log  follows
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring file /etc/passwd for database passwd
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring file `/etc/passwd` (1)
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring directory `/etc` (2)
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring file /etc/group for database group
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring file `/etc/group` (3)
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring directory `/etc` (2)
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring file /etc/hosts for database hosts
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring file `/etc/hosts` (4)
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring directory `/etc` (2)
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring file /etc/resolv.conf for database hosts
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring file `/etc/resolv.conf` (5)
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring directory `/etc` (2)
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring file /etc/services for database services
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring file `/etc/services` (6)
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring directory `/etc` (2)
Wed 20 Nov 2019 08:33:31 PM EST - 111349: monitoring file /etc/netgroup for database netgroup
Wed 20 Nov 2019 08:33:31 PM EST - 111349: disabled inotify-based monitoring for file `/etc/netgroup': No such file or directory
Wed 20 Nov 2019 08:33:31 PM EST - 111349: stat failed for file `/etc/netgroup'; will try again later: No such file or directory
Wed 20 Nov 2019 08:33:31 PM EST - 111349: Access Vector Cache (AVC) started
Wed 20 Nov 2019 08:33:31 PM EST - 111349: handle_request: request received (Version = 2) from PID 111361
Wed 20 Nov 2019 08:33:31 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:31 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:31 PM EST - 111349: handle_request: request received (Version = 2) from PID 111363
Wed 20 Nov 2019 08:33:31 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:31 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:31 PM EST - 111349: handle_request: request received (Version = 2) from PID 111365
Wed 20 Nov 2019 08:33:31 PM EST - 111349:       GETFDGR
Wed 20 Nov 2019 08:33:31 PM EST - 111349: provide access to FD 9, for group
Wed 20 Nov 2019 08:33:31 PM EST - 111349: handle_request: request received (Version = 2) from PID 111374
Wed 20 Nov 2019 08:33:31 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:31 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:31 PM EST - 111349: handle_request: request received (Version = 2) from PID 111385
Wed 20 Nov 2019 08:33:31 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:31 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:31 PM EST - 111349: handle_request: request received (Version = 2) from PID 111385
Wed 20 Nov 2019 08:33:31 PM EST - 111349:       GETFDGR
Wed 20 Nov 2019 08:33:31 PM EST - 111349: provide access to FD 9, for group
Wed 20 Nov 2019 08:33:34 PM EST - 111349: handle_request: request received (Version = 2) from PID 111393
Wed 20 Nov 2019 08:33:34 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:34 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:34 PM EST - 111349: handle_request: request received (Version = 2) from PID 111399
Wed 20 Nov 2019 08:33:34 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:34 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:34 PM EST - 111349: handle_request: request received (Version = 2) from PID 111405
Wed 20 Nov 2019 08:33:34 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:34 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:34 PM EST - 111349: handle_request: request received (Version = 2) from PID 111411
Wed 20 Nov 2019 08:33:34 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:34 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:34 PM EST - 111349: handle_request: request received (Version = 2) from PID 111422
Wed 20 Nov 2019 08:33:34 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:34 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:34 PM EST - 111349: handle_request: request received (Version = 2) from PID 111422
Wed 20 Nov 2019 08:33:34 PM EST - 111349:       GETFDGR
Wed 20 Nov 2019 08:33:34 PM EST - 111349: provide access to FD 9, for group
Wed 20 Nov 2019 08:33:34 PM EST - 111349: handle_request: request received (Version = 2) from PID 111428
Wed 20 Nov 2019 08:33:34 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:34 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:34 PM EST - 111349: handle_request: request received (Version = 2) from PID 111428
Wed 20 Nov 2019 08:33:34 PM EST - 111349:       GETFDGR
Wed 20 Nov 2019 08:33:34 PM EST - 111349: provide access to FD 9, for group
Wed 20 Nov 2019 08:33:36 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:36 PM EST - 111349: handle_request: request received (Version = 2) from PID 111455
Wed 20 Nov 2019 08:33:36 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:36 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:36 PM EST - 111349: handle_request: request received (Version = 2) from PID 111458
Wed 20 Nov 2019 08:33:36 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:36 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:36 PM EST - 111349: handle_request: request received (Version = 2) from PID 111461
Wed 20 Nov 2019 08:33:36 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:36 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:36 PM EST - 111349: handle_request: request received (Version = 2) from PID 111464
Wed 20 Nov 2019 08:33:36 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:36 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:36 PM EST - 111349: handle_request: request received (Version = 2) from PID 111467
Wed 20 Nov 2019 08:33:36 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:36 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:37 PM EST - 111349: handle_request: request received (Version = 2) from PID 111471
Wed 20 Nov 2019 08:33:37 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:37 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:37 PM EST - 111349: handle_request: request received (Version = 2) from PID 111477
Wed 20 Nov 2019 08:33:37 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:37 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:37 PM EST - 111349: handle_request: request received (Version = 2) from PID 111491
Wed 20 Nov 2019 08:33:37 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:37 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:37 PM EST - 111349: handle_request: request received (Version = 2) from PID 111497
Wed 20 Nov 2019 08:33:37 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:37 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:37 PM EST - 111349: handle_request: request received (Version = 2) from PID 111508
Wed 20 Nov 2019 08:33:37 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:37 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:37 PM EST - 111349: handle_request: request received (Version = 2) from PID 111508
Wed 20 Nov 2019 08:33:37 PM EST - 111349:       GETFDGR
Wed 20 Nov 2019 08:33:37 PM EST - 111349: provide access to FD 9, for group
Wed 20 Nov 2019 08:33:40 PM EST - 111349: handle_request: request received (Version = 2) from PID 111524
Wed 20 Nov 2019 08:33:40 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:40 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:40 PM EST - 111349: handle_request: request received (Version = 2) from PID 111530
Wed 20 Nov 2019 08:33:40 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:40 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:40 PM EST - 111349: handle_request: request received (Version = 2) from PID 111533
Wed 20 Nov 2019 08:33:40 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:40 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:40 PM EST - 111349: handle_request: request received (Version = 2) from PID 111540
Wed 20 Nov 2019 08:33:40 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:40 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:40 PM EST - 111349: handle_request: request received (Version = 2) from PID 111546
Wed 20 Nov 2019 08:33:40 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:40 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:40 PM EST - 111349: handle_request: request received (Version = 2) from PID 111557
Wed 20 Nov 2019 08:33:40 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:40 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:40 PM EST - 111349: handle_request: request received (Version = 2) from PID 111557
Wed 20 Nov 2019 08:33:40 PM EST - 111349:       GETFDGR
Wed 20 Nov 2019 08:33:40 PM EST - 111349: provide access to FD 9, for group
Wed 20 Nov 2019 08:33:41 PM EST - 111349: handle_request: request received (Version = 2) from PID 111572
Wed 20 Nov 2019 08:33:41 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:41 PM EST - 111349: provide access to FD 7, for passwd
Wed 20 Nov 2019 08:33:42 PM EST - 111349: handle_request: request received (Version = 2) from PID 111593
Wed 20 Nov 2019 08:33:42 PM EST - 111349:       GETFDPW
Wed 20 Nov 2019 08:33:42 PM EST - 111349: provide access to FD 7, for passwd
Comment 3 whzhe 2019-12-02 06:28:08 UTC
(In reply to Carlos O'Donell from comment #1)
> (In reply to whzhe from comment #0)
> > when i stop the daemons, i add the group test1 and add test to this group.
> > 
> > [root@localhost whzhe]# id test
> > uid=1000(test) gid=1000(test) groups=1000(test),1004(test1)
> > 
> > Now i start up the nscd daemons and also check the id test.
> > 
> > [root@localhost whzhe]# id test
> > uid=1000(test) gid=1000(test) groups=1000(test)
> > 
> > There is no group test1.And i know i can using 'nscd -i group' to delete the
> > cache. 
> > But i think when i restart up the nscd daemons, the nscd daemons need to
> > update the cache and fix this problem.
> 
> They should have. There is code to add the caches and invlidate them at
> startup if their last modified date has changed.
> 
> What version of glibc is this? Which distribution?
> 
> Could you please start nscd with debug level set to 9? And provide the logs.

Could you give me some suggestions? If the latest version of glibc have fixed this problem?
Comment 4 Carlos O'Donell 2019-12-02 18:17:08 UTC
(In reply to whzhe from comment #3)
> (In reply to Carlos O'Donell from comment #1)
> > Could you please start nscd with debug level set to 9? And provide the logs.
> 
> Could you give me some suggestions? If the latest version of glibc have
> fixed this problem?

Please provide your /etc/nsswitch.conf and /etc/nscd.conf?

For example "monitoring file `/etc/group` (3)" tells me the daemon found your /etc/group and is monitoring it for changes.

If you are not changing this file, but instead are altering data in an ldap database e.g. group: files ldap in /etc/nsswitch.conf (which is why I asked for this file), then you are looking at a caching issue between nscd<->ldap.

For example "positive-time-to-live   group           3600" is 1 hour in nscd for group, so you will have to invalidate the cache if you want to see the new data.
Comment 5 whzhe 2019-12-03 02:32:23 UTC
(In reply to Carlos O'Donell from comment #4)
> (In reply to whzhe from comment #3)
> > (In reply to Carlos O'Donell from comment #1)
> > > Could you please start nscd with debug level set to 9? And provide the logs.
> > 
> > Could you give me some suggestions? If the latest version of glibc have
> > fixed this problem?
> 
> Please provide your /etc/nsswitch.conf and /etc/nscd.conf?
> 
> For example "monitoring file `/etc/group` (3)" tells me the daemon found
> your /etc/group and is monitoring it for changes.
> 
> If you are not changing this file, but instead are altering data in an ldap
> database e.g. group: files ldap in /etc/nsswitch.conf (which is why I asked
> for this file), then you are looking at a caching issue between nscd<->ldap.
> 
> For example "positive-time-to-live   group           3600" is 1 hour in nscd
> for group, so you will have to invalidate the cache if you want to see the
> new data.
The /etc/nsswitch.conf is "group:       sss files systemd".
The /etc/nscd.conf 
       "enable-cache            group           yes
        positive-time-to-live   group           3600
        negative-time-to-live   group           60
        suggested-size          group           211
        check-files             group           yes
        persistent              group           yes
        shared                  group           yes
        max-db-size             group           33554432
        auto-propagate          group           yes"
Do you mean i should set "enable-cache" to no?
Comment 6 Carlos O'Donell 2019-12-03 03:38:57 UTC
(In reply to whzhe from comment #5)
> (In reply to Carlos O'Donell from comment #4)
> > (In reply to whzhe from comment #3)
> > > (In reply to Carlos O'Donell from comment #1)
> > > > Could you please start nscd with debug level set to 9? And provide the logs.
> > > 
> > > Could you give me some suggestions? If the latest version of glibc have
> > > fixed this problem?
> > 
> > Please provide your /etc/nsswitch.conf and /etc/nscd.conf?
> > 
> > For example "monitoring file `/etc/group` (3)" tells me the daemon found
> > your /etc/group and is monitoring it for changes.
> > 
> > If you are not changing this file, but instead are altering data in an ldap
> > database e.g. group: files ldap in /etc/nsswitch.conf (which is why I asked
> > for this file), then you are looking at a caching issue between nscd<->ldap.
> > 
> > For example "positive-time-to-live   group           3600" is 1 hour in nscd
> > for group, so you will have to invalidate the cache if you want to see the
> > new data.
> The /etc/nsswitch.conf is "group:       sss files systemd".
> The /etc/nscd.conf 
>        "enable-cache            group           yes
>         positive-time-to-live   group           3600
>         negative-time-to-live   group           60
>         suggested-size          group           211
>         check-files             group           yes
>         persistent              group           yes
>         shared                  group           yes
>         max-db-size             group           33554432
>         auto-propagate          group           yes"
> Do you mean i should set "enable-cache" to no?

No.

I mean to say that you have to run 'nscd -i group' to invalidate the cache if you want your changes to appear *immediately* in nscd regardless of cache +TTL or -TTL. This is part of normal server administration.

When you say "I add the group test1" how are you doing this? In LDAP? In /etc/group?

Restarting the daemon does not clear the cache. The cache is based on time-to-live and the last time a particular entry was added to the cache. You must invalidate the cache if you want to bypass TTL values.
Comment 7 whzhe 2019-12-03 06:33:53 UTC
(In reply to Carlos O'Donell from comment #6)
> (In reply to whzhe from comment #5)
> > (In reply to Carlos O'Donell from comment #4)
> > > (In reply to whzhe from comment #3)
> > > > (In reply to Carlos O'Donell from comment #1)
> > > > > Could you please start nscd with debug level set to 9? And provide the logs.
> > > > 
> > > > Could you give me some suggestions? If the latest version of glibc have
> > > > fixed this problem?
> > > 
> > > Please provide your /etc/nsswitch.conf and /etc/nscd.conf?
> > > 
> > > For example "monitoring file `/etc/group` (3)" tells me the daemon found
> > > your /etc/group and is monitoring it for changes.
> > > 
> > > If you are not changing this file, but instead are altering data in an ldap
> > > database e.g. group: files ldap in /etc/nsswitch.conf (which is why I asked
> > > for this file), then you are looking at a caching issue between nscd<->ldap.
> > > 
> > > For example "positive-time-to-live   group           3600" is 1 hour in nscd
> > > for group, so you will have to invalidate the cache if you want to see the
> > > new data.
> > The /etc/nsswitch.conf is "group:       sss files systemd".
> > The /etc/nscd.conf 
> >        "enable-cache            group           yes
> >         positive-time-to-live   group           3600
> >         negative-time-to-live   group           60
> >         suggested-size          group           211
> >         check-files             group           yes
> >         persistent              group           yes
> >         shared                  group           yes
> >         max-db-size             group           33554432
> >         auto-propagate          group           yes"
> > Do you mean i should set "enable-cache" to no?
> 
> No.
> 
> I mean to say that you have to run 'nscd -i group' to invalidate the cache
> if you want your changes to appear *immediately* in nscd regardless of cache
> +TTL or -TTL. This is part of normal server administration.
> 
> When you say "I add the group test1" how are you doing this? In LDAP? In
> /etc/group?
> 
> Restarting the daemon does not clear the cache. The cache is based on
> time-to-live and the last time a particular entry was added to the cache.
> You must invalidate the cache if you want to bypass TTL values.

Thanks for giving me the solutions.I understand how the cache works.But i think when i stop the deamon, the previously cached information will no longer be useful.There is no need for users to manually invalidate the cache.The deamons invalidate the cache when it starts.What's your opinion?
Comment 8 Carlos O'Donell 2019-12-03 13:50:10 UTC
(In reply to whzhe from comment #7)
> Thanks for giving me the solutions.I understand how the cache works.But i
> think when i stop the deamon, the previously cached information will no
> longer be useful.There is no need for users to manually invalidate the
> cache.The deamons invalidate the cache when it starts.What's your opinion?

My opinion is that removing entries by their proper expiration time is what users expect and is the current behaviour. If you change the data in other locations you must, like with DNS, wait for the other caching layers to timeout their data before new data appears.

If you need shorter timeouts you can change positive-time-to-live and negative-time-to-live, but doing so means the cache will query (maybe over the network) the authoritative source more frequently and there is a cost to that (cpu, network, disk, etc).

The simplest solution is to invalidate the cache with nscd -i group, which clears the cache for nscd only, and that solves your problem.

I'm marking this RESOLVED/WONTFIX for now. If you want to discuss making nscd startup invalidate all caches, please discuss this on libc-alpha@sourceware.org with the developers and distribution maintainers and try to gather consensus on a solution that everyone agrees would work.