xfstests suddently can't be installed by running make install on Fedora 28, with make-4.2.1-6. It never happens before. And it's not a xfstests regression issue. I reported to upstream GNU make too and they thought it's glibc problem. More details please check: https://marc.info/?l=fstests&m=153131978019489&w=2 https://marc.info/?l=fstests&m=153180707908075&w=2 The error output as below # make # make install Building include Building lib Building ltp Building src Building log-writes Building perf Building aio-dio-regress Building m4 Building common Building tests /usr/bin/gmake --no-print-directory Q=@ -C include install gmake[1]: Nothing to be done for 'install'. /usr/bin/gmake --no-print-directory Q=@ -C lib install gmake[1]: Nothing to be done for 'install'. /usr/bin/gmake --no-print-directory Q=@ -C ltp install ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio \ fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp ../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp /usr/bin/gmake --no-print-directory Q=@ -C src install Building log-writes Building perf Building aio-dio-regress /usr/bin/gmake -C log-writes install ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes ../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes /usr/bin/gmake -C perf install ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf ../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py \ FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql \ /var/lib/xfstests/src/perf /usr/bin/gmake -C aio-dio-regress install ../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress ../../install-sh -o root -g root -m 755 aio-dio-fcntl-race aio-dio-subblock-eof-read \ aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-hole-filling-race \ aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write \ aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer \ aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race \ /var/lib/xfstests/src/aio-dio-regress ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 \ dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas \ truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature \ alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault \ makeextents itrash rename multi_open_unlink dmiperf unwritten_sync genhashnames \ t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest \ t_truncate_self t_mmap_dio af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate \ fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption \ t_ofd_locks t_locks_execve xfsctl bstat t_mtab getdevicesize \ preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx \ looptest locktest unwritten_mmap bulkstat_unlink_test bulkstat_unlink_test_modified \ t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 \ seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner \ renameat2 t_getcwd e4compact test-nextquota punch-alternating \ attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \ dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon \ mkswap fiemap-tester t_stripealign open_by_handle dbtest fssum \ /var/lib/xfstests/src /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 \ dmerror fill2attr fill2fs fill2fs_check scaleread.sh \ /var/lib/xfstests/src /bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 \ dumpfile /var/lib/xfstests/src /usr/bin/gmake --no-print-directory Q=@ -C m4 install gmake[1]: Nothing to be done for 'install'. /usr/bin/gmake --no-print-directory Q=@ -C common install ../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common ../install-sh -o root -g root -m 644 * /var/lib/xfstests/common /usr/bin/gmake --no-print-directory Q=@ -C tests install /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install /usr/bin/gmake --no-print-directory Q=@ -C /home/git/xfstests-dev/tests/ install ...... ...... (again and again and again ....) ...... ...... Version-Release number of selected component (if applicable): make-4.2.1-7.el8+7.x86_64 glibc-2.27-30.el8+7.x86_64 gcc-8.1.1-1.el8+7.x86_64 How reproducible: 100% Steps to Reproduce: 1. git clone git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git, current HEAD commit is 0804dc173618a1a1fa6d1b916b9b7dcfe72fe1f8. 2. install build dependences. 3. make 4. make install Other reproducers please check: https://marc.info/?l=fstests&m=153180707908075&w=2 Additional info: Paul Smith <psmith@gnu.org> said: If there's a change which introduces a problem with wildcard it is more likely to be related to the GNU libc implementation of glob() or fnmatch(). Compare with different make and glibc: 1) make-4.2.1-4 built by glibc-2.26 works fine with glibc-2.26-27 2) make-4.2.1-7 built by glibc-2.27 reproduce this bug with glibc-2.27-30. 3) make-4.2.1-4 built by glibc-2.26 reproduce this bug with glibc-2.27-30. So glibc-2.26-27 works fine, but glibc-2.27-30 can reproduce this bug.
I can't reproduce it with GNU Make 4.2.1 and master (ba2ea23), neither with 2.27 branch release (8623cfe). The only difference on the mentioned steps to reproduce is I used a prefix path on configuration (--prefix=/tmp/xfstests-dev/install). We recently fixed some glob issues (BZ #1062, BZ #19971, BZ #866, BZ #1062, BZ #22320, BZ #22332), which might be related and we also provide a compat symbol with old semantic related to BZ #866. So I tried to remove the compat version on master to check if it is something to glob implementation, but even forcing GNU make to use the newer implementation I couldn't reproduce the issue. I am not sure which commit id Fedora 28 has used on its deployment or any other out of the, if any, it has used. However if you could which version Fedora 28 is using or if you could check if master branch does help you it would be helpful.
(In reply to Adhemerval Zanella from comment #1) > I can't reproduce it with GNU Make 4.2.1 and master (ba2ea23), neither with > 2.27 branch release (8623cfe). The only difference on the mentioned steps to > reproduce is I used a prefix path on configuration > (--prefix=/tmp/xfstests-dev/install). > > We recently fixed some glob issues (BZ #1062, BZ #19971, BZ #866, BZ #1062, > BZ #22320, BZ #22332), which might be related and we also provide a compat > symbol with old semantic related to BZ #866. So I tried to remove the compat > version on master to check if it is something to glob implementation, but > even forcing GNU make to use the newer implementation I couldn't reproduce > the issue. > > I am not sure which commit id Fedora 28 has used on its deployment or any > other out of the, if any, it has used. However if you could which version > Fedora 28 is using or if you could check if master branch does help you it > would be helpful. We sync from 2.27 here: - Auto-sync with upstream branch release/2.27/master, commit 56170e064e2b21ce204f0817733e92f1730541ea. ... with a few patches applied on top, but not much, since we've started to backport and track release branches as actively as we can. I haven't verified this issue yet either, but it seems oddly suspicious of an interaction between the toolchain and make or some other component that is part of the build. So far nobody has actually debugged the root cause. I'm marking this RESOLVED/INVALID for upstream glibc, given your quick testing. Zorro, if you want to track this for Fedora 28, please open a Fedora bug against glibc for the Fedora release you're having problems with.
Thanks for you comfirming. But my colleague metioned that they've reported this bug before: https://sourceware.org/bugzilla/show_bug.cgi?id=23393 And the change looks like radical. Before, we can do this: # echo abcd > testfile # echo ABCD >> testfile # egrep [a-z] testfile abcd But now, it becomes this: # echo abcd > testfile # echo ABCD >> testfile # egrep [a-z] testfile abcd ABCD I'm afraid that it will break many customer's stripts and tools...
(In reply to Zorro Lang from comment #3) > Thanks for you comfirming. But my colleague metioned that they've reported > this bug before: > https://sourceware.org/bugzilla/show_bug.cgi?id=23393 > > And the change looks like radical. > Before, we can do this: > > # echo abcd > testfile > # echo ABCD >> testfile > # egrep [a-z] testfile > abcd > > But now, it becomes this: > # echo abcd > testfile > # echo ABCD >> testfile > # egrep [a-z] testfile > abcd > ABCD > > I'm afraid that it will break many customer's stripts and tools... It appears you are asking for the following: * Support using a-z and A-Z to mean [:lower:] and [:upper:] * For all locales, even non-English ones. This defeats the purpose of the regular expression statement, and is the reason why [:lower:] and [:upper:] were specified. We could make changes in the English-speaking locales perhaps, argued rationally for conservatism, reverting some ISO 14651 changes, and causing a-z and A-Z not to interleave, which would likely fix your use cases (until you had files with more esoteric names). Do we think that kind of conservative fix might work?
(In reply to Carlos O'Donell from comment #4) > It appears you are asking for the following: > > * Support using a-z and A-Z to mean [:lower:] and [:upper:] > * For all locales, even non-English ones. > > This defeats the purpose of the regular expression statement, and is the > reason why [:lower:] and [:upper:] were specified. > > We could make changes in the English-speaking locales perhaps, argued > rationally for conservatism, reverting some ISO 14651 changes, and causing > a-z and A-Z not to interleave, which would likely fix your use cases (until > you had files with more esoteric names). > > Do we think that kind of conservative fix might work? I guess if we make this change it would be better if we uniformly deviate in all languages from ISO 14651 to solve the issue. However, really this is a bug in all the applications using '[a-z]' etc. because it will cause problems in all sorts of locales that collate other characters in that range, but I understand the problem.
(In reply to Carlos O'Donell from comment #5) > (In reply to Carlos O'Donell from comment #4) > > It appears you are asking for the following: > > > > * Support using a-z and A-Z to mean [:lower:] and [:upper:] > > * For all locales, even non-English ones. > > > > This defeats the purpose of the regular expression statement, and is the > > reason why [:lower:] and [:upper:] were specified. > > > > We could make changes in the English-speaking locales perhaps, argued > > rationally for conservatism, reverting some ISO 14651 changes, and causing > > a-z and A-Z not to interleave, which would likely fix your use cases (until > > you had files with more esoteric names). > > > > Do we think that kind of conservative fix might work? > > I guess if we make this change it would be better if we uniformly deviate in > all languages from ISO 14651 to solve the issue. However, really this is a > bug in all the applications using '[a-z]' etc. because it will cause > problems in all sorts of locales that collate other characters in that > range, but I understand the problem. ... however in making this deviation, we take a step away from our stated goals of harmonization with Unicode and CLDR data, and again add back subtle sorting differences against other systems using Unicode collation or libICU. My opinion is that the scripts in question should just get fixed, and that the differences in collation among systems introduce subtle differences that are more insidious than just the fixes we need for [a-z] and [A-Z].
Rich Felker proposed to use codepoint order only for ranges: https://sourceware.org/bugzilla/show_bug.cgi?id=23393#c13 I think this would be a workable solution. It is POSIX-compliant. (And this bug is really the same as bug 23393.)
(In reply to Florian Weimer from comment #7) > Rich Felker proposed to use codepoint order only for ranges: > > https://sourceware.org/bugzilla/show_bug.cgi?id=23393#c13 > > I think this would be a workable solution. It is POSIX-compliant. > > (And this bug is really the same as bug 23393.) Moving to 23393, closing this as duplicate. *** This bug has been marked as a duplicate of bug 23393 ***
(In reply to Carlos O'Donell from comment #8) > (In reply to Florian Weimer from comment #7) > > Rich Felker proposed to use codepoint order only for ranges: > > > > https://sourceware.org/bugzilla/show_bug.cgi?id=23393#c13 > > > > I think this would be a workable solution. It is POSIX-compliant. > > > > (And this bug is really the same as bug 23393.) > > Moving to 23393, closing this as duplicate. > > *** This bug has been marked as a duplicate of bug 23393 *** Just to certify to actually trigger the issue described on xfstests-dev install one need to set a specific system locale, right? I am trying to understand why I could reproduce it on my environment.
On 07/19/2018 01:42 PM, adhemerval.zanella at linaro dot org wrote: > https://sourceware.org/bugzilla/show_bug.cgi?id=23420 > > --- Comment #9 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> --- > (In reply to Carlos O'Donell from comment #8) >> (In reply to Florian Weimer from comment #7) >>> Rich Felker proposed to use codepoint order only for ranges: >>> >>> https://sourceware.org/bugzilla/show_bug.cgi?id=23393#c13 >>> >>> I think this would be a workable solution. It is POSIX-compliant. >>> >>> (And this bug is really the same as bug 23393.) >> >> Moving to 23393, closing this as duplicate. >> >> *** This bug has been marked as a duplicate of bug 23393 *** > > Just to certify to actually trigger the issue described on xfstests-dev install > one need to set a specific system locale, right? I am trying to understand why > I could reproduce it on my environment. Any locale but C/POSIX. Any locale which uses iso14651_t1_common went from [a-z] == a-z, to [a-z] == aA-zZ.
(In reply to Carlos O'Donell from comment #10) > On 07/19/2018 01:42 PM, adhemerval.zanella at linaro dot org wrote: > > https://sourceware.org/bugzilla/show_bug.cgi?id=23420 > > > > --- Comment #9 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> --- > > (In reply to Carlos O'Donell from comment #8) > >> (In reply to Florian Weimer from comment #7) > >>> Rich Felker proposed to use codepoint order only for ranges: > >>> > >>> https://sourceware.org/bugzilla/show_bug.cgi?id=23393#c13 > >>> > >>> I think this would be a workable solution. It is POSIX-compliant. > >>> > >>> (And this bug is really the same as bug 23393.) > >> > >> Moving to 23393, closing this as duplicate. > >> > >> *** This bug has been marked as a duplicate of bug 23393 *** > > > > Just to certify to actually trigger the issue described on xfstests-dev install > > one need to set a specific system locale, right? I am trying to understand why > > I could reproduce it on my environment. > > Any locale but C/POSIX. > > Any locale which uses iso14651_t1_common went from [a-z] == a-z, to [a-z] == > aA-zZ. Right, I tested with LANG=en_US.UTF-8 and LANGUAGE=en_US but couldn't reproduce it.
(In reply to Adhemerval Zanella from comment #11) > Right, I tested with LANG=en_US.UTF-8 and LANGUAGE=en_US but couldn't > reproduce it. Did you test with glibc from the master branch? We backported the collation data update into Fedora's glibc 2.27, but it's not in upstream glibc 2.27.
(In reply to Florian Weimer from comment #12) > (In reply to Adhemerval Zanella from comment #11) > > > Right, I tested with LANG=en_US.UTF-8 and LANGUAGE=en_US but couldn't > > reproduce it. > > Did you test with glibc from the master branch? We backported the collation > data update into Fedora's glibc 2.27, but it's not in upstream glibc 2.27. I tested with master with my system make (executed through testrun.sh). I am not sure if using DESTDIR along with install rule is tampering with testing.