Bug 23720 - Build directory outside of Source Tree - Test Suite not executed correctly if executed with runtest
Summary: Build directory outside of Source Tree - Test Suite not executed correctly if...
Status: UNCONFIRMED
Alias: None
Product: binutils
Classification: Unclassified
Component: binutils (show other bugs)
Version: 2.32
: P2 minor
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-09-28 09:49 UTC by bbarbisch
Modified: 2018-09-28 09:49 UTC (History)
0 users

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description bbarbisch 2018-09-28 09:49:31 UTC
Description
If the binutils test suite is used to verify a build outside of the source directory tree using runtest (objdir in site.exp points to the external build directory), the tests do not use the newly build binaries but fall back to a already exiting (older) binary.


Steps to reproduce
- build binutils (./configure and make check)
- copy binutils/testsuite to a destination outside of the binutils source tree
- copy the site.exp to the copied testuite directory 
- check site.exp (objdir should point to the build directory)
- execute runtest within the copied testsuite directory

Expected Result:
- tests are executed
- tests are using the newly generated binaries within the build directory

Current Result:
- tests are executed
- tests do not use the newly generated binaries but fall back to the system binaries

Root cause:
the file testsuites/config/default.exp initializes the paths to the generated binaries. It locates these binaries within $base_dir which points to the current working direcotry (of the test execution). Instead of $base_dir the variable $objdir should be used (which points the directory which contains the binaries)


Additional info:
- this seems to be an issue only if you try to execute the test suite with runtest and the objdir path pointing to an external directory. It does not seem to be an issue if binutils is build in an external build directory and the tests are executed using 'make check'.
- one can see that the wrong binaries are used if their versions differ (e.g. system version: 2.30; build version 2.32) because '--version' outputs can be found in the generated log files
- use case for this scenario: one wants to execute the test suite against a already built/existing/deployed version of binutils
- this seems to be also an issue for
-- ld
-- gas