Bug 18378

Summary: Implement GOLD backend support for IBM z Systems
Product: binutils Reporter: Andreas Krebbel <krebbel1>
Component: goldAssignee: Cary Coutant <ccoutant>
Status: CLOSED FIXED    
Severity: enhancement CC: eduard.beutel, ian, koriakin, krebbel
Priority: P2    
Version: 2.26   
Target Milestone: ---   
Host: Target:
Build: Last reconfirmed:
Bug Depends on: 18975, 18959    
Bug Blocks:    
Attachments: patch #1
patch #2 - SHT_HASH entry size
patch #3 - 64-bit archive map support
patch #4 - the main course
patch #4 - the main course (with 32-bit PLT fix)
patch #5 - fixes
patch #6 - more fixes
patch #7 - PLTOFF fix
patch #8 - debug_msg test fixes
patch #9 - add attribute aligned(1) to tests where necessary
patch #10 - reenable scripts disabled by patch #4 and fixed by patch #9

Description Andreas Krebbel 2015-05-05 18:42:31 UTC
Enable the GOLD linker for IBM z Systems by implementing the
target-dependent parts in GOLD/Binutils.  The goal is to pass the GOLD
Binutils testsuite on a z machine.
Comment 1 eduard.beutel 2015-05-20 20:11:25 UTC
Hi Andreas,
Do you have any idea how I could get access to a z machine for development purposes?
Regards, Eduard
Comment 2 Andreas.Krebbel 2015-05-21 06:17:11 UTC
Hi,

I'm glad to read that you are interested in this bounty :)

You can register here to get access to a z community server:

http://www-03.ibm.com/systems/z/os/linux/support/community.html

However, you have to be aware that the ressources on these machines are 
limited. Also the servers will time out after a certain period of time so 
please backup your work to your local system often. If you need more 
ressources or need to extend the expiration period I can try to contact 
the responsible people for you.

Mit  freundlichem Gruß / Kind regards,
Andreas Krebbel

***********************************************************
IBM Deutschland Research & Development GmbH
Linux on zSeries Development & Service, Dept. D3303
Schönaicherstr. 220, 71032 Böblingen
Vorsitzender des Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

Office: 06/122 --- Phone: +49-(0)7031-16-1089
External mail: krebbel1@de.ibm.com



From:
"eduard.beutel at yahoo dot com" <sourceware-bugzilla@sourceware.org>
To:
Andreas Krebbel1/Germany/IBM@IBMDE
Date:
05/20/2015 10:11 PM
Subject:
[Bug gold/18378] Implement GOLD backend support for IBM z Systems



https://sourceware.org/bugzilla/show_bug.cgi?id=18378

eduard.beutel at yahoo dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |eduard.beutel at yahoo 
dot com

--- Comment #1 from eduard.beutel at yahoo dot com ---
Hi Andreas,
Do you have any idea how I could get access to a z machine for development
purposes?
Regards, Eduard
Comment 3 eduard.beutel 2015-05-26 11:38:11 UTC
Thanks for the info. 
I did what you suggested, haven't heard back until now.
I posted the request with a link to this page.
I'll try running debian on the hercules emulator.
Comment 4 Andreas Krebbel 2015-05-26 13:26:57 UTC
(In reply to eduard.beutel from comment #3)
Starting in June there will be a new way of getting access to z Systems hardware. I'll post the information here as soon as I know more.
Comment 5 Cary Coutant 2015-06-04 18:48:07 UTC
Who should I assign this to?
Comment 6 Andreas Krebbel 2015-06-24 07:18:17 UTC
(In reply to Andreas Krebbel from comment #0)
IBM has created a bounty on this item at:
https://www.bountysource.com/issues/14306991-implement-gold-backend-support-for-ibm-z-systems

Part of the success criteria is that all required patches for this feature are accepted and integrated into the Binutils source repository.
Comment 7 Marcin Kościelnicki 2015-09-14 02:06:46 UTC
Created attachment 8605 [details]
patch #1
Comment 8 Marcin Kościelnicki 2015-09-14 02:07:10 UTC
Created attachment 8606 [details]
patch #2 - SHT_HASH entry size
Comment 9 Marcin Kościelnicki 2015-09-14 02:07:30 UTC
Created attachment 8607 [details]
patch #3 - 64-bit archive map support
Comment 10 Marcin Kościelnicki 2015-09-14 02:07:57 UTC
Created attachment 8608 [details]
patch #4 - the main course
Comment 11 Marcin Kościelnicki 2015-09-14 02:16:36 UTC
Here's my attempt. I get a full testsuite pass on qemu-system-s390x with 64-bit gentoo.

Two bugs have been found along the way: #18959 (in gold, affecting all arches) and #18960 (in glibc, affecting likely only s390).  The first patch in the series is my attempt at #18959.

I had to disable 4 tests since they use symbol values that are not a multiple of 2. Such values cannot be loaded by larl instruction that gcc uses to get addresses of all symbols, and hence can never pass on s390.  This is the same issue that causes #18960.

Aside of the usual target-specific code, I had to add support for 64-bit archive maps (/SYM64/) and 64-bit SHT_HASH entries (s390 seems to be the only architecture using these).

Although in theory everything should work for 32-bit systems too, I haven't tested it yet (compiling stuff on qemu has been painful enough without having to bootstrap a multilib system). Unfortunately I haven't heard back after asking for a test machine.

Does that look OK?
Comment 12 Andreas Krebbel 2015-09-14 08:35:45 UTC
(In reply to Marcin Kościelnicki from comment #11)
> Does that look OK?

Great! Looks good to me from a first glance, but I'll need some time to actually do a review. Could you please post the patches on the binutils mailing list to allow other maintainers have a look? Thanks!
Comment 13 Marcin Kościelnicki 2015-09-14 10:01:03 UTC
Created attachment 8609 [details]
patch #4 - the main course (with 32-bit PLT fix)

One fix added: I found a bug in handling of PLT entries with plt_offset == 0x10000 for 32-bit target.
Comment 14 Marcin Kościelnicki 2015-09-14 10:12:49 UTC
I've posted the patches on the mailing list.
Comment 15 Andreas Krebbel 2015-09-14 10:41:42 UTC
(In reply to Marcin Kościelnicki from comment #13)
> Created attachment 8609 [details]
> patch #4 - the main course (with 32-bit PLT fix)
> 
> One fix added: I found a bug in handling of PLT entries with plt_offset ==
> 0x10000 for 32-bit target.

Thanks! I've found your github repo of the S/390 gold port: https://github.com/koriakin/binutils-gdb.git   Is this supposed to be up-to-date so that I can base my review and test on that?
Comment 16 Marcin Kościelnicki 2015-09-14 10:43:03 UTC
(In reply to Andreas Krebbel from comment #15)
> (In reply to Marcin Kościelnicki from comment #13)
> > Created attachment 8609 [details]
> > patch #4 - the main course (with 32-bit PLT fix)
> > 
> > One fix added: I found a bug in handling of PLT entries with plt_offset ==
> > 0x10000 for 32-bit target.
> 
> Thanks! I've found your github repo of the S/390 gold port:
> https://github.com/koriakin/binutils-gdb.git   Is this supposed to be
> up-to-date so that I can base my review and test on that?

Yeah, it is. If you have commit 28817dbedbfbd1acfc68be35508d697924d36848, you're good to go.
Comment 17 Marcin Kościelnicki 2015-09-16 21:15:21 UTC
I got the new shiny testing machine and found a few more issues. I've fixed a few already and will resend the whole patchset once I get both 32-bit and 64-bit testsuite pass on RHEL. Amazing how much the compiler differences between gentoo and RHEL can affect the output...
Comment 18 Marcin Kościelnicki 2015-09-17 01:48:22 UTC
Created attachment 8616 [details]
patch #5 - fixes

Here come the fixes from testing. Now I have only the following test failures:

- on 31-bit: debug_msg test shows wrong line for SometimesInlined in odr_violation2.o (29 instead of 27). It seems gcc emits two .loc directives at the beginning of the function, for line 27 and then 29. gold's DWARF parser picks the second one (29), while addr2line picks the first (27). I'm not sure what to do about this one.
- on 64-bit: 5 tests (the ones that involve linker scripts) fail due to bug 18975.
Comment 19 Marcin Kościelnicki 2015-09-21 02:09:44 UTC
Created attachment 8622 [details]
patch #6 - more fixes

Two more 31-bit fixes found in testing.
Comment 20 Marcin Kościelnicki 2015-09-21 04:25:03 UTC
Created attachment 8623 [details]
patch #7 - PLTOFF fix
Comment 21 Marcin Kościelnicki 2015-09-27 19:05:11 UTC
Created attachment 8638 [details]
patch #8 - debug_msg test fixes

debug_msg fixes, see https://sourceware.org/ml/binutils/2015-09/msg00257.html
Comment 22 Marcin Kościelnicki 2015-09-27 19:06:43 UTC
Created attachment 8639 [details]
patch #9 - add attribute aligned(1) to tests where necessary

Fixes tests involving unaligned symbols from binary format or linker scripts, see https://sourceware.org/ml/binutils/2015-09/msg00288.html
Comment 23 Marcin Kościelnicki 2015-09-27 19:07:58 UTC
Created attachment 8640 [details]
patch #10 - reenable scripts disabled by patch #4 and fixed by patch #9

Turns out there's a way to make these tests work on S/390, but I missed it at first. Should be squashed into patch #4 before merge, like other S/390 fixes.
Comment 24 Marcin Kościelnicki 2015-09-27 19:10:54 UTC
I have added a few more test fixes (no changes to linker itself). Now the funny script and binary tests no longer need to be skipped. The only test failures remaining for me (31-bit and 64-bit, gentoo and RHEL) are caused by #18975 (and they only affect 64-bit RHEL).
Comment 25 Marcin Kościelnicki 2015-10-29 01:02:32 UTC
The patches have been merged to binutils-gdb master.
Comment 26 Andreas Krebbel 2015-10-29 19:09:18 UTC
Resolved per comment above
Comment 27 Andreas Krebbel 2015-10-29 19:09:50 UTC
Closing