This is the mail archive of the binutils@sourceware.org mailing list for the binutils project.
Index Nav:	[Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav:	[Date Prev] [Date Next]	[Thread Prev] [Thread Next]
Other format:	[Raw text]
Re: testsuite regressions on trunk, compared to 2.28

From: Christophe Lyon <christophe dot lyon at linaro dot org>
To: Matthias Klose <doko at ubuntu dot com>
Cc: binutils <binutils at sourceware dot org>
Date: Wed, 21 Jun 2017 19:20:24 +0200
Subject: Re: testsuite regressions on trunk, compared to 2.28
Authentication-results: sourceware.org; auth=none
References: <70cfc0c2-5d8b-b7b4-c6ef-6d1905d6e65d@ubuntu.com>
Hi,


On 21 June 2017 at 17:57, Matthias Klose <doko@ubuntu.com> wrote:
> Preparing for the next binutils release, I did a test rebuild on some
> archictures, and comparing test results with those from binutils 2.28,
> configured with
>
> --enable-shared --enable-plugins --enable-threads --with-system-zlib
> --prefix=/usr --enable-deterministic-archives
> --disable-compressed-debug-sections --enable-new-dtags
>
> binutils 2.28 was configured without --enable-new-dtags
>

We run builds of master with different configure options:
--enable-plugins --enable-lto
in cross and native environments.

> I see the following regressions:
>
> amd64:
> Test results, compared with installed binutils:
> W: [ld-x86-64/x86-64.exp] REGRESSION (PASS -> FAIL): Run pr19031
> W: [ld-x86-64/no-plt.exp] REGRESSION (PASS -> FAIL): Run pr20253-2f
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a7 with
> --dynamic-list-data -Bsymbolic and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a6 with
> --dynamic-list-cpp-new -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a5 with
> -Bsymbolic-functions --dynamic-list-cpp-new and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a2 with
> -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-srec/srec.exp] REGRESSION (PASS -> FAIL): S-records
> W: [ld-srec/srec.exp] REGRESSION (PASS -> FAIL): S-records with constructors
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/rpath-1
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/now-3
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/now-4
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/rpath-2
> W: [ld-scripts/crossref.exp] REGRESSION (PASS -> FAIL): NOCROSSREFS 3
> W: [ld-size/size.exp] REGRESSION (PASS -> FAIL): Run size-5a
> W: [ld-size/size.exp] REGRESSION (PASS -> FAIL): Run size-8
> W: [ld-size/size.exp] REGRESSION (PASS -> FAIL): Run size-6
> W: [ld-size/size.exp] REGRESSION (PASS -> FAIL): Run size-5b
> W: [ld-size/size.exp] REGRESSION (PASS -> FAIL): Run size-4a
> W: [ld-size/size.exp] REGRESSION (PASS -> FAIL): Run size-4b
> 15 REGRESSIONS (0.47%).
> 4 progressions (0.12%).
> 3216 tests: 3129 pass (97.29%), 16 fail (0.50%), 57 xfail (1.77%) 1 untested
> (0.03%).
>
> i386:
> Test results, compared with installed binutils:
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/now-4
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/rpath-2
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/rpath-1
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/now-3
> W: [ld-scripts/crossref.exp] REGRESSION (PASS -> FAIL): NOCROSSREFS_TO 3
> W: [ld-scripts/crossref.exp] REGRESSION (PASS -> FAIL): NOCROSSREFS_TO 4
> W: [ld-scripts/crossref.exp] REGRESSION (PASS -> FAIL): NOCROSSREFS 3
> W: [ld-shared/shared.exp] REGRESSION (PASS -> FAIL): shared (non PIC, load offset)
> W: [ld-shared/shared.exp] REGRESSION (PASS -> FAIL): shared (PIC main, non PIC so)
> W: [ld-shared/shared.exp] REGRESSION (PASS -> FAIL): shared (non PIC)
> W: [ld-elfvsb/elfvsb.exp] REGRESSION (PASS -> FAIL): visibility (protected) (PIC
> main, non PIC so)
> W: [ld-elfvsb/elfvsb.exp] REGRESSION (PASS -> FAIL): visibility (hidden_weak)
> (non PIC, load offset)
> W: [ld-elfvsb/elfvsb.exp] REGRESSION (PASS -> FAIL): visibility (protected) (non
> PIC, load offset)
> W: [ld-elfvsb/elfvsb.exp] REGRESSION (PASS -> FAIL): visibility (hidden_weak)
> (non PIC)
> W: [ld-elfvsb/elfvsb.exp] REGRESSION (PASS -> FAIL): visibility (hidden_weak)
> (PIC main)
> W: [ld-elfvsb/elfvsb.exp] REGRESSION (PASS -> FAIL): visibility (hidden_weak)
> (PIC main, non PIC so)
> W: [ld-elfvsb/elfvsb.exp] REGRESSION (PASS -> FAIL): visibility (protected) (non
> PIC)
> W: [ld-elfvsb/elfvsb.exp] REGRESSION (PASS -> FAIL): visibility (hidden_weak)
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a6 with
> --dynamic-list-cpp-new -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a7 with
> --dynamic-list-data -Bsymbolic and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a5 with
> -Bsymbolic-functions --dynamic-list-cpp-new and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a2 with
> -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-i386/i386.exp] REGRESSION (PASS -> FAIL): Run got1
> W: [ld-i386/i386.exp] REGRESSION (PASS -> FAIL): Build gotpc1
> W: [ld-i386/i386.exp] REGRESSION (PASS -> FAIL): Undefined weak symbol (-fPIE )
> W: [ld-i386/i386.exp] REGRESSION (PASS -> FAIL): Undefined weak symbol (-fPIE -pie)
> W: [ld-i386/i386.exp] REGRESSION (PASS -> FAIL): Run pr19031
> W: [ld-srec/srec.exp] REGRESSION (PASS -> FAIL): S-records
> W: [ld-srec/srec.exp] REGRESSION (PASS -> FAIL): S-records with constructors
> W: [ld-gc/gc.exp] REGRESSION (PASS -> FAIL): --gc-sections with KEEP
> W: [ld-size/size.exp] REGRESSION (PASS -> FAIL): Run size-8
> W: [ld-i386/no-plt.exp] REGRESSION (PASS -> FAIL): No PLT (dynamic 1i)
> W: [ld-i386/no-plt.exp] REGRESSION (PASS -> FAIL): Run pr20253-1f
> W: [ld-i386/no-plt.exp] REGRESSION (PASS -> FAIL): Run pr20244-3a
> W: [ld-i386/no-plt.exp] REGRESSION (PASS -> FAIL): Run pr20253-2a
> W: [ld-i386/no-plt.exp] REGRESSION (PASS -> FAIL): No PLT (dynamic 1h)
> W: [ld-i386/no-plt.exp] REGRESSION (PASS -> FAIL): Run pr20253-2c
> 33 REGRESSIONS (1.18%).
> 4 progressions (0.14%).
> 2804 tests: 2750 pass (98.07%), 33 fail (1.18%), 4 xfail (0.14%) 1 untested (0.04%).
>
> arm64:
> Test results, compared with installed binutils:
> I: [ld-elf/shared.exp] progression (FAIL -> PASS): Run pr19579
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a2 with
> -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] XPASS: Run pr19719 fun undefined
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a5 with
> -Bsymbolic-functions --dynamic-list-cpp-new and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a7 with
> --dynamic-list-data -Bsymbolic and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a6 with
> --dynamic-list-cpp-new -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-aarch64/aarch64-elf.exp] REGRESSION (PASS -> FAIL): ld-aarch64/dt_textrel
> W: [ld-gc/gc.exp] REGRESSION (PASS -> FAIL): --gc-sections with KEEP
> 6 REGRESSIONS (0.34%).
> 2 progressions (0.11%).
> 1790 tests: 1748 pass (97.65%), 14 fail (0.78%), 11 xfail (0.61%) 1 untested
> (0.06%).
>
I don't see these (probably because of the configure options), but on
native builds I see:
ld/testsuite/ld-elf/indirect.exp ...
FAIL: Run indirect5 3
FAIL: Run indirect5 4
FAIL: indirect5c dynsym
FAIL: indirect5d dynsym

> armhf:
> Test results, compared with installed binutils:
> W: [ld-gc/gc.exp] REGRESSION (PASS -> FAIL): --gc-sections with KEEP
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a6 with
> --dynamic-list-cpp-new -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] XPASS: Run pr19719 fun undefined
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a5 with
> -Bsymbolic-functions --dynamic-list-cpp-new and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a7 with
> --dynamic-list-data -Bsymbolic and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a2 with
> -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (hidden_undef) (non PIC, load offset)
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (hidden) (non PIC)
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (hidden_undef) (non PIC)
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (protected_undef) (non PIC)
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (hidden_undef) (PIC main, non PIC so)
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (hidden_undef_def) (PIC main, non
> PIC so)
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (protected_undef) (non PIC, load offset)
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (hidden) (PIC main, non PIC so)
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (hidden_undef_def) (non PIC)
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (hidden) (non PIC, load offset)
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (hidden_undef_def) (non PIC, load
> offset)
> W: [ld-elfvsb/elfvsb.exp] XPASS: visibility (protected_undef) (PIC main, non PIC so)
> 5 REGRESSIONS (0.22%).
> 13 progressions (0.57%).
> 2267 tests: 2180 pass (96.16%), 16 fail (0.71%), 50 xfail (2.21%) 1 untested
> (0.04%).
>
In this case in both native and cross-builds, I see:
testsuite/ld-elf/shared.exp ...
FAIL: PR ld/21233 dynamic symbols with section GC (--undefined)
FAIL: PR ld/21233 dynamic symbols with section GC (--require-defined)
FAIL: PR ld/21233 dynamic symbols with section GC (EXTERN)
(I think I reported this some time ago)

In native builds:
ld/testsuite/ld-elf/indirect.exp ...
FAIL: Run indirect5 3
FAIL: Run indirect5 4
FAIL: indirect5c dynsym
FAIL: indirect5d dynsym
FAIL: STM32L4XX erratum : LDM cannot be patched when LDM is too far
from veneer section



> ppc64el:
> Test results, compared with installed binutils:
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/now-3
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/rpath-2
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/now-4
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/rpath-1
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a6 with
> --dynamic-list-cpp-new -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a2 with
> -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a7 with
> --dynamic-list-data -Bsymbolic and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a5 with
> -Bsymbolic-functions --dynamic-list-cpp-new and dlopen on libdl6a.so
> 4 REGRESSIONS (0.25%).
> 4 progressions (0.25%).
> 1588 tests: 1551 pass (97.67%), 7 fail (0.44%), 11 xfail (0.69%) 1 untested (0.06%).
>
> s390x:
> Test results, compared with installed binutils:
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/now-3
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/now-4
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/rpath-2
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/rpath-1
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a2 with
> -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a7 with
> --dynamic-list-data -Bsymbolic and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a6 with
> --dynamic-list-cpp-new -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a5 with
> -Bsymbolic-functions --dynamic-list-cpp-new and dlopen on libdl6a.so
> W: [ld-plugin/plugin.exp] REGRESSION (PASS -> FAIL): plugin 3 with source lib
> W: [ld-plugin/plugin.exp] REGRESSION (PASS -> FAIL): load plugin 3 with source
> W: [ld-plugin/plugin.exp] REGRESSION (PASS -> FAIL): plugin ignore lib with source
> W: [ld-plugin/plugin.exp] REGRESSION (PASS -> FAIL): plugin set symbol
> visibility with source
> W: [ld-plugin/plugin.exp] REGRESSION (PASS -> FAIL): plugin 2 with source lib
> W: [ld-plugin/plugin.exp] REGRESSION (PASS -> FAIL): plugin claimfile replace
> file with source
> W: [ld-plugin/plugin.exp] REGRESSION (PASS -> FAIL): load plugin 2 with source
> W: [ld-plugin/plugin.exp] REGRESSION (PASS -> FAIL): plugin claimfile replace
> lib with source
> W: [ld-plugin/plugin.exp] REGRESSION (PASS -> FAIL): plugin claimfile replace
> symbol with source
> W: [ld-plugin/plugin.exp] REGRESSION (PASS -> FAIL): plugin claimfile resolve
> symbol with source
> 14 REGRESSIONS (0.93%).
> 4 progressions (0.26%).
> 1513 tests: 1458 pass (96.36%), 25 fail (1.65%), 10 xfail (0.66%) 1 untested
> (0.07%).
>
>
> sparc64:
> Test results, compared with installed binutils:
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/rpath-1
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/rpath-2
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/now-4
> I: [ld-elf/elf.exp] progression (FAIL -> PASS): ld-elf/now-3
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a7 with
> --dynamic-list-data -Bsymbolic and dlopen on libdl6a.so
> I: [ld-elf/shared.exp] progression (FAIL -> PASS): Build libpr16496b.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a6 with
> --dynamic-list-cpp-new -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a2 with
> -Bsymbolic-functions and dlopen on libdl6a.so
> W: [ld-elf/shared.exp] REGRESSION (PASS -> FAIL): Run dl6a5 with
> -Bsymbolic-functions --dynamic-list-cpp-new and dlopen on libdl6a.so
> W: [ld-elfvsb/elfvsb.exp] REGRESSION (PASS -> FAIL): visibility (hidden_weak)
> I: [ld-elfvsb/elfvsb.exp] change (XPASS -> XFAIL): visibility (hidden_weak) (non
> PIC)
> I: [ld-elfvsb/elfvsb.exp] change (XPASS -> XFAIL): visibility (hidden_weak) (non
> PIC, load offset)
> W: [ld-elfvsb/elfvsb.exp] REGRESSION (PASS -> FAIL): visibility (hidden_weak)
> (PIC main)
> I: [ld-elfvsb/elfvsb.exp] change (XPASS -> XFAIL): visibility (hidden_weak) (PIC
> main, non PIC so)
> 6 REGRESSIONS (0.37%).
> 5 progressions (0.31%).
> 3 changes (0.19%).
> 1605 tests: 1546 pass (96.32%), 31 fail (1.93%), 8 xfail (0.50%) 1 untested (0.06%).
>
>
> powerpc: no regressions
>
Follow-Ups:
- Re: testsuite regressions on trunk, compared to 2.28
  - From: Jiong Wang
References:
- testsuite regressions on trunk, compared to 2.28
  - From: Matthias Klose
Index Nav:	[Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav:	[Date Prev] [Date Next]	[Thread Prev] [Thread Next]