In the process of updating Ubuntu to glibc 2.31, the following tests fail in the autopkgtest infrastructure: FAIL: elf/tst-dlopen-self-container FAIL: elf/tst-dlopen-tlsmodid-container FAIL: elf/tst-ldconfig-bad-aux-cache FAIL: elf/tst-ldconfig-ld_so_conf-update FAIL: elf/tst-pldd FAIL: nss/tst-nss-db-endgrent FAIL: nss/tst-nss-db-endpwent FAIL: nss/tst-nss-files-hosts-long (full log at https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-focal/focal/amd64/g/glibc/20200310_034140_bacc4@/log.gz) These tests all use test-container.c and fail like so: error: test-container.c:1096: Unable to exec /tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/elf/tst-dlopen-self-container (because our builds run in a chroot, these tests are skipped at build time) A little digging reveals that the missing file is the interpreter, /lib64/ld-linux-x86-64.so.2, and indeed this is not present in testroot.pristine: $ ls testroot.pristine/ bin lib tmp usr What's going on? Well, the code that's supposed to copy this across is: for dso in `$(test-wrapper-env) LD_TRACE_LOADED_OBJECTS=1 \ $(rtld-prefix) \ $(objpfx)testroot.pristine/bin/sh \ | sed -n '/\//{s@.*=> /@/@;s/^[^/]*//;s/ .*//p;}'` ;\ do \ test -d `dirname $(objpfx)testroot.pristine$$dso` || \ mkdir -p `dirname $(objpfx)testroot.pristine$$dso` ;\ $(test-wrapper) cp $$dso $(objpfx)testroot.pristine$$dso ;\ done Which, in the context of this test run became: for dso in ` env LD_TRACE_LOADED_OBJECTS=1 \ /tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/elf/ld-linux-x86-64.so.2 --library-path /tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc:/tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/math:/tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/elf:/tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/dlfcn:/tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/nss:/tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/nis:/tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/rt:/tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/resolv:/tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/mathvec:/tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/support:/tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/nptl \ /tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/testroot.pristine/bin/sh \ | sed -n '/\//{s@.*=> /@/@;s/^[^/]*//;s/ .*//p;}'` ;\ do \ test -d `dirname /tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/testroot.pristine$dso` || \ mkdir -p `dirname /tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/testroot.pristine$dso` ;\ cp $dso /tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/testroot.pristine$dso ;\ done The output of the LD_TRACE_LOADED_OBJECTS=1 .. | sed pipeline is this: /lib/x86_64-linux-gnu/libc.so.6 /tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/elf/ld-linux-x86-64.so.2 So the interpreter gets copied to testroot.pristine/tmp/autopkgtest.YturSE/build.WyJ/src/build-tree/amd64-libc/elf/ld-linux-x86-64.so.2 which is obviously of no use to anyone. I'm not sure I'm competent to suggest a fix. Removing the line containing " $(rtld-prefix)" seems to help and maybe makes sense -- after all, the goal is surely to get files into testroot.pristine that testroot.pristine/bin/sh will look for by default -- but I'm not sure.
(In reply to Michael Hudson-Doyle from comment #0) > $ ls testroot.pristine/ > bin lib tmp usr What's in lib? find testroot.pristine/lib/? > What's going on? Well, the code that's supposed to copy this across is: The intent of this code is to copy any additional required objects from the language runtime. The subsequence "make install" installs everything you need. > I'm not sure I'm competent to suggest a fix. Removing the line containing " > $(rtld-prefix)" seems to help and maybe makes sense -- after all, the goal > is surely to get files into testroot.pristine that testroot.pristine/bin/sh > will look for by default -- but I'm not sure. Can you please run ldd on the tst-dlopen-self-container test? It seems like you make have a disconnected between how the tests are built and what is actually installed.
Does this issue reproduce if you back out the multi-arch path patches (which are not upstream)?
Sorry for the delay, I accidentally destroyed the vm I had been running the tests in. (In reply to Carlos O'Donell from comment #1) > (In reply to Michael Hudson-Doyle from comment #0) > > $ ls testroot.pristine/ > > bin lib tmp usr > > What's in lib? find testroot.pristine/lib/? This looks OK: ubuntu@autopkgtest:/tmp/autopkgtest.aD5exj/build.jIB/real-tree/build-tree/amd64-libc$ find testroot.pristine/lib/ testroot.pristine/lib/ testroot.pristine/lib/x86_64-linux-gnu testroot.pristine/lib/x86_64-linux-gnu/libthread_db.so.1 testroot.pristine/lib/x86_64-linux-gnu/libanl-2.31.so testroot.pristine/lib/x86_64-linux-gnu/librt.so.1 testroot.pristine/lib/x86_64-linux-gnu/libdl-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libmvec-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libmvec.so.1 testroot.pristine/lib/x86_64-linux-gnu/librt-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libpthread-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libnss_files-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libSegFault.so testroot.pristine/lib/x86_64-linux-gnu/libnsl-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libresolv-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libstdc++.so.6 testroot.pristine/lib/x86_64-linux-gnu/libnss_dns.so.2 testroot.pristine/lib/x86_64-linux-gnu/libnss_nisplus.so.2 testroot.pristine/lib/x86_64-linux-gnu/libnss_hesiod.so.2 testroot.pristine/lib/x86_64-linux-gnu/libgcc_s.so.1 testroot.pristine/lib/x86_64-linux-gnu/libm-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libnss_files.so.2 testroot.pristine/lib/x86_64-linux-gnu/libm.so.6 testroot.pristine/lib/x86_64-linux-gnu/libutil-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libnss_nis-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libc.so.6 testroot.pristine/lib/x86_64-linux-gnu/libthread_db-1.0.so testroot.pristine/lib/x86_64-linux-gnu/libpthread.so.0 testroot.pristine/lib/x86_64-linux-gnu/libBrokenLocale-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libnss_dns-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libmemusage.so testroot.pristine/lib/x86_64-linux-gnu/libc-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libnss_nis.so.2 testroot.pristine/lib/x86_64-linux-gnu/libnsl.so.1 testroot.pristine/lib/x86_64-linux-gnu/libutil.so.1 testroot.pristine/lib/x86_64-linux-gnu/libpcprofile.so testroot.pristine/lib/x86_64-linux-gnu/ld-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libresolv.so.2 testroot.pristine/lib/x86_64-linux-gnu/libnss_compat.so.2 testroot.pristine/lib/x86_64-linux-gnu/libBrokenLocale.so.1 testroot.pristine/lib/x86_64-linux-gnu/libnss_compat-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libdl.so.2 testroot.pristine/lib/x86_64-linux-gnu/libnss_nisplus-2.31.so testroot.pristine/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 testroot.pristine/lib/x86_64-linux-gnu/libnss_hesiod-2.31.so testroot.pristine/lib/x86_64-linux-gnu/libanl.so.1 > > What's going on? Well, the code that's supposed to copy this across is: > > The intent of this code is to copy any additional required objects from the > language runtime. > > The subsequence "make install" installs everything you need. > > > I'm not sure I'm competent to suggest a fix. Removing the line containing " > > $(rtld-prefix)" seems to help and maybe makes sense -- after all, the goal > > is surely to get files into testroot.pristine that testroot.pristine/bin/sh > > will look for by default -- but I'm not sure. > > Can you please run ldd on the tst-dlopen-self-container test? ubuntu@autopkgtest:/tmp/autopkgtest.aD5exj/build.jIB/real-tree/build-tree/amd64-libc$ ldd ./elf/tst-dlopen-self-container linux-vdso.so.1 (0x00007ffdf03e8000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc91af78000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc91ad87000) /lib64/ld-linux-x86-64.so.2 (0x00007fc91af90000) > It seems like you make have a disconnected between how the tests are built > and what is actually installed. (In reply to Florian Weimer from comment #2) > Does this issue reproduce if you back out the multi-arch path patches (which > are not upstream)? Do you mean just commenting these two patches out of debian/patches/series: debian/patches/any/local-ldconfig-multiarch.diff debian/patches/any/local-ld-multiarch.diff and trying again?
> (In reply to Florian Weimer from comment #2) > > Does this issue reproduce if you back out the multi-arch path patches (which > > are not upstream)? > > Do you mean just commenting these two patches out of debian/patches/series: > > debian/patches/any/local-ldconfig-multiarch.diff > debian/patches/any/local-ld-multiarch.diff > > and trying again? If that is what you meant, it didn't make any difference.
This has nothing to do with multiarch. The issue is caused by the quick and dirty patch "debian/patches/any/local-rtlddir-cross.diff" that was added to avoid modifying at the dpkg-cross tool, which mangle packages to move libraries from native path to a cross path.
Thanks for tracking this down. Does this mean it's a downstream-only bug?
(In reply to Florian Weimer from comment #6) > Thanks for tracking this down. Does this mean it's a downstream-only bug? Yes, we inherited the same bug in Debian from Ubuntu. After removing that patch a few months ago in Debian glibc 2.31-12, the issue disappeared. Sorry for the delay, I have just discovered that bug.
Thanks. Closing per comment 7.