Bug 18067 - many failures in testsuite
Summary: many failures in testsuite
Status: NEW
Alias: None
Product: gdb
Classification: Unclassified
Component: testsuite (show other bugs)
Version: 7.8
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-03-01 22:15 UTC by yuri
Modified: 2015-07-31 11:46 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments
check log (12.22 KB, text/x-log)
2015-03-01 22:15 UTC, yuri
Details
sudo Ubuntu 14.04 (177.68 KB, text/plain)
2015-07-22 12:48 UTC, Ciro Santilli
Details

Note You need to log in before you can comment on or make changes to this bug.
Description yuri 2015-03-01 22:15:38 UTC
Created attachment 8158 [details]
check log

I ran it for 7.8.2 on Arch linux, and results aren't assuring gdb is ok.

Problematic results:
# of unexpected failures        616
# of unexpected successes       1

It appears gdb.threads, gdb.python are broken, among others.
Comment 1 Keith Seitz 2015-03-17 16:50:57 UTC
The real test log would probably be more enlightening than the .sum file, but from what I can see in the .sum you supplied, there appears to be something quite suspicious about your build/test environment.

Can you attach the gdb.log of:
$ make check RUNTESTFLAGS=dw2-basic.exp

This is showing:
ERROR: remote_download to host of ./gdb.dwarf2/file1.txt to /home/yuri/gdb-7.8.2/gdb/testsuite/gdb.dwarf2/file.txt: cp: cannot stat './gdb.dwarf2/file1.txt': No such file or directory

This same error appears in several places, along with:
FAIL: gdb.dwarf2/gdb-index.exp: touch binary

This is all very odd. What directory are you trying to run tests in? Is it read-only?

A huge chunk of failures is in attach-into-signal.exp. That has, in the past, been one of a handful of racy tests. Try running it alone and see if it passes.

We'll worry about the other big problem area, python, when we get the remote_download issues resolved.
Comment 2 Ciro Santilli 2015-07-22 12:40:05 UTC
I was getting 388 failures on tag binutils-2_25, Ubuntu 14.04 GCC 4.8 the large majority related to attach.

Then I run with:

    sudo make check RUNTESTFLAGS="--ignore bigcore.exp"

and it fell down to 36 only!

I ignore bigcore.exp because it slows my machine too much: https://sourceware.org/bugzilla/show_bug.cgi?id=18704

So it seems that the inability to attach without sudo was the cause of most failures.

We should really document that somewhere.

@Keith:

- why don't we mark failing tests with XFAIL so as to unclutter the tests?
- shouldn't we document about the sudo on the testsuite/README ?
- I am unable to find the buildbot results for a given commit to compare, e.g. binutils-2_25... is that possible? Not built-in it seems: https://groups.google.com/a/chromium.org/forum/#!topic/infra-dev/T_7S9HXLWlo

Attaching .sum and .log.
Comment 3 Ciro Santilli 2015-07-22 12:48:18 UTC
Created attachment 8448 [details]
sudo Ubuntu 14.04
Comment 4 Ciro Santilli 2015-07-23 08:56:33 UTC
ptrace may be done without sudo, it depends on the ptrace_scope kernel parameter: http://askubuntu.com/questions/41629/after-upgrade-gdb-wont-attach-to-process
Comment 5 Pedro Alves 2015-07-23 09:41:00 UTC
> why don't we mark failing tests with XFAIL so as to unclutter the tests?

We do - though it's KFAIL for known gdb failures - but it takes someone to actually do the leg work.  All help appreciated.

> - shouldn't we document about the sudo on the testsuite/README ?

I'd welcome a patch.  As you found out, it's the ptrace_scope thing that's at play.  Running gdb under sudo isn't a good idea.

> - I am unable to find the buildbot results for a given commit to compare, e.g. binutils-2_25... is that possible? Not built-in it seems: 

Find the git repo for results of the builder you want the results for, then look for a results commit whose subject indicates it tested the gdb commit hash you wanted.  Note that most builders are testing master, some the 7.10 branch, so you naturally won't find results for hashes only on other branches.
Comment 6 Ciro Santilli 2015-07-31 11:46:05 UTC
(In reply to Pedro Alves from comment #5)
> > why don't we mark failing tests with XFAIL so as to unclutter the tests?
> 
> We do - though it's KFAIL for known gdb failures - but it takes someone to
> actually do the leg work.  All help appreciated.
> 

OK!

> > - shouldn't we document about the sudo on the testsuite/README ?
> 
> I'd welcome a patch.  As you found out, it's the ptrace_scope thing that's
> at play.  Running gdb under sudo isn't a good idea.
> 

Patch at: https://sourceware.org/ml/gdb-patches/2015-07/msg00932.html

> > - I am unable to find the buildbot results for a given commit to compare, e.g. binutils-2_25... is that possible? Not built-in it seems: 
> 
> Find the git repo for results of the builder you want the results for, then
> look for a results commit whose subject indicates it tested the gdb commit
> hash you wanted.  Note that most builders are testing master, some the 7.10
> branch, so you naturally won't find results for hashes only on other
> branches.

Ah, after reading the wiki a bit more I've found it, thanks: http://gdb-build.sergiodj.net/cgit