The linaro CI consistently reports a progression on test-case gdb.base/startup-with-shell.exp. AFAICU, this happens because: - the FAIL is listed in an xfails.xfail file, and - the FAIL was fixed. I think the FAIL was fixed by: ... commit 52498004a34dfa4bc55fbf791a4ed4e81599a436 Author: Andrew Burgess <aburgess@redhat.com> Date: Wed Jan 17 09:53:16 2024 +0000 gdb/testsuite: handle long filenames in gdb.base/startup-with-shell.exp I got a report of a failure from Linaro's CI testing for the test gdb.base/startup-with-shell.exp. ... Can we update the xfails.xfail file?
Hi Tom, Yes, this annoys me as well, but the problem is a bit more complicated. Our CI automatically updates xfails file, which includes both failed test in the baseline and flaky tests. The problem with this test, is that it consistently fails in our post-commit CI jobs, but tends to pass in the pre-commit jobs. Since we are using baseline results from the post-commit CI to compare pre-commit CI results against -- we get this "progression". We have it on our TODO list to fix this problem, but it requires digging into the GDB testsuite logic to understand where the different behavior is coming from.
(In reply to Maxim Kuvyrkov from comment #1) > Hi Tom, > > Yes, this annoys me as well, but the problem is a bit more complicated. Our > CI automatically updates xfails file, which includes both failed test in the > baseline and flaky tests. > > The problem with this test, is that it consistently fails in our post-commit > CI jobs, but tends to pass in the pre-commit jobs. Since we are using > baseline results from the post-commit CI to compare pre-commit CI results > against -- we get this "progression". > Hi Maxim, I see, interesting. > We have it on our TODO list to fix this problem, but it requires digging > into the GDB testsuite logic to understand where the different behavior is > coming from. Understood. I managed to find the failure mode: ... (gdb) PASS: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: inferior started print argv[1] $1 = 0xfffeda96 "/home/tcwg-buildslave/workspace/tcwg_gnu_4/abe/builds/armv8l-unknown-linux-gnueabihf/armv8l-unknown-linux-gnueabihf/gdb-gdb.git~master/gdb/testsuite/outputs/gdb.base/startup-with-shell/*.unique-extens"... (gdb) FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: first argument not expanded ... at https://ci.linaro.org/job/tcwg_gdb_check--master-arm-build/lastSuccessfulBuild/artifact/artifacts/00-sumfiles/gdb.log.xz . I'm changing the component from external to testsuite. I think the fix should be the same as in the commit mentioned in comment 0: ... diff --git a/gdb/testsuite/gdb.base/startup-with-shell.exp b/gdb/testsuite/gdb.base/startup-with-shell.exp index d5ec503dceb..670fab6a543 100644 --- a/gdb/testsuite/gdb.base/startup-with-shell.exp +++ b/gdb/testsuite/gdb.base/startup-with-shell.exp @@ -66,6 +66,7 @@ with_test_prefix "startup_with_shell = on; run_args = *.unique-extension" { with_test_prefix "startup_with_shell = off; run_args = *.unique-extension" { initial_setup_simple "off" "$unique_file_dir/*.unique-extension" + gdb_test_no_output "set print characters unlimited" gdb_test "print argv\[1\]" "\\\$$decimal = $hex \"$unique_file_dir/\\\*\.unique-extension\"" \ "first argument not expanded" } ...
https://sourceware.org/pipermail/gdb-patches/2024-February/206826.html
The master branch has been updated by Tom de Vries <vries@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=d82ede20df36eb8acfc9e67969e396e3a8fe1d45 commit d82ede20df36eb8acfc9e67969e396e3a8fe1d45 Author: Tom de Vries <tdevries@suse.de> Date: Mon Feb 26 15:59:47 2024 +0100 [gdb/testsuite] Further handle long filenames in gdb.base/startup-with-shell.exp In commit 52498004a34 ("gdb/testsuite: handle long filenames in gdb.base/startup-with-shell.exp") we fixed a FAIL reported by the Linaro CI: ... (gdb) print argv[1] $1 = 0xfffed978 "<snip>/startup-with-shell/unique-file.unique-e"... (gdb) FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = on; \ run_args = *.unique-extension: first argument expanded ... PR testsuite/31410 reports a similar failure: ... (gdb) print argv[1] $1 = 0xfffeda96 "<snip>/startup-with-shell/*.unique-extens"... (gdb) FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; \ run_args = *.unique-extension: first argument not expanded ... Fix this in the same way, using "set print characters unlimited". Tested on x86_64-linux. Approved-By: Tom Tromey <tom@tromey.com> Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31410
Fixed.
Thank you!