Bug 31186 - Some dirent/io tests don't work on xfs
Summary: Some dirent/io tests don't work on xfs
Status: UNCONFIRMED
Alias: None
Product: glibc
Classification: Unclassified
Component: libc (show other bugs)
Version: 2.39
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-12-22 16:18 UTC by H.J. Lu
Modified: 2024-01-05 08:33 UTC (History)
4 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description H.J. Lu 2023-12-22 16:18:50 UTC
When I was building and testing i686 glibc on xfs, I got

FAIL: dirent/list
FAIL: dirent/tst-readdir64-compat
FAIL: io/bug-ftw2
FAIL: io/ftwtest
FAIL: io/tst-ftw-lnk

i686 dirent/list gave

.
.	file number 46282493
..	file number 2187265413
config.log	file number 46282494
readdir: Value too large for defined data type

x86-64 dirent/list gave:

.
.	file number 46282493
..	file number 2187265413
config.log	file number 46282494
sunrpc	file number 4366544490
Comment 1 Adhemerval Zanella 2023-12-29 13:29:04 UTC
I think this a duplicate of B#23960, afaik XFS uses a similar strategy EXT4 uses to define inodes as a 64b-bit hash nad it might overflow depending of the underling FS contents.  Could you check if https://patchwork.sourceware.org/project/glibc/list/?series=18052 improves this?
Comment 2 H.J. Lu 2023-12-29 15:49:14 UTC
(In reply to Adhemerval Zanella from comment #1)
> I think this a duplicate of B#23960, afaik XFS uses a similar strategy EXT4
> uses to define inodes as a 64b-bit hash nad it might overflow depending of
> the underling FS contents.  Could you check if
> https://patchwork.sourceware.org/project/glibc/list/?series=18052 improves
> this?

It doesn't help:

[hjl@gnu-icx-1 build-i686-linux]$ ./dirent/list 
.
readdir: Value too large for defined data type
[hjl@gnu-icx-1 build-i686-linux]$
Comment 3 Adhemerval Zanella 2023-12-29 18:32:21 UTC
(In reply to H.J. Lu from comment #2)
> (In reply to Adhemerval Zanella from comment #1)
> > I think this a duplicate of B#23960, afaik XFS uses a similar strategy EXT4
> > uses to define inodes as a 64b-bit hash nad it might overflow depending of
> > the underling FS contents.  Could you check if
> > https://patchwork.sourceware.org/project/glibc/list/?series=18052 improves
> > this?
> 
> It doesn't help:
> 
> [hjl@gnu-icx-1 build-i686-linux]$ ./dirent/list 
> .
> readdir: Value too large for defined data type
> [hjl@gnu-icx-1 build-i686-linux]$

Hum, is the entry d_ino entry grom getdents also larger than 32-bit in this environment?
Comment 4 H.J. Lu 2023-12-29 18:56:20 UTC
From the first __getdents64 call:

Breakpoint 2, __readdir_unlocked (dirp=dirp@entry=0x5655a5b0)
    at ../sysdeps/unix/sysv/linux/readdir.c:32
32	{
(gdb) next
33	  int saved_errno = errno;
(gdb) 
37	      if (dirp->offset >= dirp->size)
(gdb) 
39		  ssize_t bytes = __getdents64 (dirp->fd, dirp->data,
(gdb) 
41		  if (bytes <= 0)
(gdb) 
51		  dirp->size = bytes;
(gdb) 
54		  dirp->offset = 0;
(gdb) 
78	      if (!in_ino_t_range (inp->dp64.d_ino))
(gdb) 
88		  __set_errno (EOVERFLOW);
(gdb) p inp->dp64.d_ino
$1 = 6442498605
(gdb)
Comment 5 Adhemerval Zanella 2023-12-29 19:25:11 UTC
Right, so XFS essentially does not support non-LFS calls. I am not sure which is the best way to handle it, with my patchset I can also clamp d_ino and not advertise EOVERFLOW.  I am not fully sure about the implications.
Comment 6 Adhemerval Zanella 2023-12-29 19:42:34 UTC
(In reply to Adhemerval Zanella from comment #5)
> Right, so XFS essentially does not support non-LFS calls. I am not sure
> which is the best way to handle it, with my patchset I can also clamp d_ino
> and not advertise EOVERFLOW.  I am not fully sure about the implications.

Scratch that, I just recalled that we always use getdents64 syscall even for non-LFS.  As fair I recall, EXT4 will clamp the values anyway so I think glibc will either need to do something similar or revert to use non-LFS getdents syscall (which I would like to avoid...).
Comment 7 Adhemerval Zanella 2023-12-29 20:00:01 UTC
Could you check if https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/azanella/bz23960-dirent helps?
Comment 8 H.J. Lu 2023-12-29 21:53:07 UTC
(In reply to Adhemerval Zanella from comment #7)
> Could you check if
> https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/azanella/
> bz23960-dirent helps?

It works better.  But I still got

FAIL: dirent/tst-readdir64-compat
FAIL: io/bug-ftw2
FAIL: io/ftwtest
FAIL: io/tst-ftw-lnk

[hjl@gnu-icx-1 build-i686-linux]$ io/tst-ftw-lnk
error: tst-ftw-lnk.c:217: symlink link1-bad failed
error: 1 test failures
[hjl@gnu-icx-1 build-i686-linux]$ io/ftwtest
...
nftw: Value too large for defined data type
[hjl@gnu-icx-1 build-i686-linux]$
Comment 9 Florian Weimer 2023-12-30 12:29:55 UTC
32-bit compatibility on XFS requires using the inode32 option: https://www.man7.org/linux/man-pages/man5/xfs.5.html

(No such option exists with btrfs, by the way, and you need to reformat if you exceed the 32-bit inode space.)

So I think this bug is different from the d_off issue and invalid from a glibc perspective.
Comment 10 Adhemerval Zanella 2024-01-02 18:06:37 UTC
(In reply to Florian Weimer from comment #9)
> 32-bit compatibility on XFS requires using the inode32 option:
> https://www.man7.org/linux/man-pages/man5/xfs.5.html
> 
> (No such option exists with btrfs, by the way, and you need to reformat if
> you exceed the 32-bit inode space.)
> 
> So I think this bug is different from the d_off issue and invalid from a
> glibc perspective.

The dirent/list failure is related, since on my BZ#28824 proposed fix I did not consider that d_ino might overflow.  But the rest of the failures do seem to be related to the XFS inode64, where symlink (io/tst-ftw-lnk) and stat (io/ftwtest) are failing due FS overflow values.

Should we mark this as WONTFIX/NOTABUG?