Re: ld testsuite use of notarget

[Alan Modra <amodra at gmail dot com>]

On Tue, Jul 10, 2018 at 04:22:56PM +0100, Maciej W. Rozycki wrote:
> On Tue, 10 Jul 2018, Alan Modra wrote:
> 
> > 2) Tests might be used as a template for a new test, carrying over
> >    notarget and/or xfail.  Inappropriate notarget entries are silent.
> 
>  Well, not anymore, after my commit 8a8c916960b0 ("LD/testsuite: 
> run_dump_test: Report UNSUPPORTED for excluded targets").  Now you get 
> UNSUPPORTED, which is not quite silent, though you do need to reach for 
> the .sum or .log file.

Yes, but the test isn't run at all.  You don't get any indication
whether the test passes or fails for a notarget entry.

>From the ChangeLog:
	* testsuite/ld-elf/compress1a.d: Likewise, and adjust targets.
	* testsuite/ld-elf/compressed1a.d: Likewise.
	* testsuite/ld-elf/compressed1d.d: Likewise.
	* testsuite/ld-elf/compressed1e.d: Likewise.
	* testsuite/ld-elf/eh5.d: Likewise.
	* testsuite/ld-elf/group8a.d: Likewise.
	* testsuite/ld-elf/linkonce2.d: Likewise.
	* testsuite/ld-elf/orphan-9.d: Likewise.
	* testsuite/ld-elf/pr17550a.d: Likewise.
	* testsuite/ld-elf/pr17550b.d: Likewise.
	* testsuite/ld-elf/pr17550c.d: Likewise.
	* testsuite/ld-elf/pr19162.d: Likewise.
	* testsuite/ld-elf/pr21884.d: Likewise.
	* testsuite/ld-gc/pr19167.d: Likewise.
	* testsuite/ld-gc/pr20022.d: Likewise.
	* testsuite/ld-gc/start.d: Likewise.
	* testsuite/ld-elf/orphan-11.d: Delete notarget.
	* testsuite/ld-elf/orphan-12.d: Likewise.
	* testsuite/ld-gc/stop.d: Likewise.
	* testsuite/ld-scripts/empty-address-3a.d: Likewise.
	* testsuite/ld-scripts/empty-address-3b.d: Likewise.

All of those tests needed some adjustment to notarget when converting
to xfail, and I'd say in the great majority of cases that was due to
blindly copying notarget from some other similar test.  If we hadn't
been using notarget that wouldn't have happened.

> > diff --git a/ld/testsuite/ld-elf/hash.d b/ld/testsuite/ld-elf/hash.d
> > index d31168c664..593d48739b 100644
> > --- a/ld/testsuite/ld-elf/hash.d
> > +++ b/ld/testsuite/ld-elf/hash.d
> > @@ -2,7 +2,7 @@
> >  #readelf: -d -s -D
> >  #ld: -shared --hash-style=gnu
> >  #target: *-*-linux* *-*-gnu* arm*-*-uclinuxfdpiceabi
> > -#notarget: mips*-*-*
> > +#xfail: mips*-*-*
> >  
> >  #...
> >   +0x[0-9a-z]+ +\(GNU_HASH\) +0x[0-9a-z]+
> 
>  This is not a bug, we do not support the GNU hash for the MIPS target 
> due to psABI restrictions on dynsym table ordering.  So I'd rather left 
> it with #notarget for an accurate UNSUPPORTED result.

I think it is better as an xfail.  xfail doesn't necessarily mean you
have a bug, just that the test is expected to fail.  Suppose in some
future patch someone enabled GNU hash for mips targets.  Then an XPASS
on this test indicates a bug!  (My point 3 in why notarget is a bad
idea.)

I'll add the following comment.
# GNU hash is not supported for MIPS targets due to psABI restrictions
# on dynsym table ordering.

> > diff --git a/ld/testsuite/ld-gc/pr20022.d b/ld/testsuite/ld-gc/pr20022.d
> > index 6600f4a476..681c68c48b 100644
> > --- a/ld/testsuite/ld-gc/pr20022.d
> > +++ b/ld/testsuite/ld-gc/pr20022.d
> > @@ -2,7 +2,7 @@
> >  #ld: --gc-sections -e _start tmpdir/pr20022.so
> >  #readelf: -SsW
> >  #target: *-*-linux* *-*-gnu* arm*-*-uclinuxfdpiceabi
> > -#notarget: frv-*-linux* metag-*-linux* mips*-*-*
> > +#xfail: frv-*-* metag-*-* mips*-*-*
> >  # Skip on targets without dynamic relocations in .text section.
> >  
> >  #...
> 
>  I think this test case is broken, I fail to see from PR ld/20022 why it 
> insists on using `.text' for relocated data rather than `.data' (and then 
> it would work for all targets).  There's nothing specific to `.text' here 
> AFAICT.

True, except the test uses -e _start and it's unusual for an entry
point to be in .data.  I'll see how that goes.

>  This is a variant test for `fill16.d'.  So I'd rather kept the 
> distinction as noted in the comment, though we might use the new #skip 
> tag instead.

Oops.  Yes, I'll put this one and fill.d back to as they were but
using #skip.

-- 
Alan Modra
Australia Development Lab, IBM