bios/core/ldlinux.elf has

Contents of section .prefix:
 0000 007c0000 00000000 5c960000 00000000  .|......\.......
 0010 00000200  

(gdb) p/x *prefix
$9 = {pfx_start = 0x7c00, pfx_compressed = 0x0, pfx_cdatalen = 0x965c, 
  pfx_checksum = 0x0, pfx_maxlma = 0x20000}
(gdb) 

Does it look normal?

pfx_compressed, which is __pm_code_lma, is wrong:

                section .prefix         nowrite progbits align=16
pfx_start       dd _start               ; Start of raw chunk
pfx_compressed  dd __pm_code_lma        ; Start of compressed chunk
pfx_cdatalen    dd lzo_data_size        ; Pointer to compressed size field
%if IS_ISOLINUX
pfx_checksum    dd bi_length            ; File length and checksum fields
%else
pfx_checksum    dd 0                    ; No checksum
%endif
pfx_maxlma      dd MaxLMA               ; Maximum size

00009770 B __pm_code_lma

(In reply to H.J. Lu from comment #2)
> pfx_compressed, which is __pm_code_lma, is wrong:
> 
>                 section .prefix         nowrite progbits align=16
> pfx_start       dd _start               ; Start of raw chunk
> pfx_compressed  dd __pm_code_lma        ; Start of compressed chunk
> pfx_cdatalen    dd lzo_data_size        ; Pointer to compressed size field
> %if IS_ISOLINUX
> pfx_checksum    dd bi_length            ; File length and checksum fields
> %else
> pfx_checksum    dd 0                    ; No checksum
> %endif
> pfx_maxlma      dd MaxLMA               ; Maximum size
> 
> 00009770 B __pm_code_lma

bios/core/ldlinux.elf is built with -pie.  The value of __pm_code_lma
won't be known until run-time:

Relocation section '.rel.dyn' at offset 0x1ba34 contains 1435 entries:
 Offset     Info    Type            Sym.Value  Sym. Name
...
00000004  00010d01 R_386_32          00009770   __pm_code_lma
00008f62  00010d01 R_386_32          00009770   __pm_code_lma

Peter, am I missing something?

Created attachment 8939 [details]
Try this syslinux patch

Please try this and verify if it generates a working syslinux.

On January 30, 2016 9:08:32 AM PST, "hjl.tools at gmail dot com" <sourceware-bugzilla@sourceware.org> wrote:
>https://sourceware.org/bugzilla/show_bug.cgi?id=19538
>
>--- Comment #3 from H.J. Lu <hjl.tools at gmail dot com> ---
>(In reply to H.J. Lu from comment #2)
>> pfx_compressed, which is __pm_code_lma, is wrong:
>> 
>>                 section .prefix         nowrite progbits align=16
>> pfx_start       dd _start               ; Start of raw chunk
>> pfx_compressed  dd __pm_code_lma        ; Start of compressed chunk
>> pfx_cdatalen    dd lzo_data_size        ; Pointer to compressed size
>field
>> %if IS_ISOLINUX
>> pfx_checksum    dd bi_length            ; File length and checksum
>fields
>> %else
>> pfx_checksum    dd 0                    ; No checksum
>> %endif
>> pfx_maxlma      dd MaxLMA               ; Maximum size
>> 
>> 00009770 B __pm_code_lma
>
>bios/core/ldlinux.elf is built with -pie.  The value of __pm_code_lma
>won't be known until run-time:
>
>Relocation section '.rel.dyn' at offset 0x1ba34 contains 1435 entries:
> Offset     Info    Type            Sym.Value  Sym. Name
>...
>00000004  00010d01 R_386_32          00009770   __pm_code_lma
>00008f62  00010d01 R_386_32          00009770   __pm_code_lma
>
>Peter, am I missing something?

ldlinux.elf shouldn't be build pie, so that would explain why.

On January 30, 2016 12:53:18 PM PST, hpa at zytor dot com <sourceware-bugzilla@sourceware.org> wrote:
>https://sourceware.org/bugzilla/show_bug.cgi?id=19538
>
>--- Comment #5 from hpa at zytor dot com <hpa at zytor dot com> ---
>On January 30, 2016 9:08:32 AM PST, "hjl.tools at gmail dot com"
><sourceware-bugzilla@sourceware.org> wrote:
>>https://sourceware.org/bugzilla/show_bug.cgi?id=19538
>>
>>--- Comment #3 from H.J. Lu <hjl.tools at gmail dot com> ---
>>(In reply to H.J. Lu from comment #2)
>>> pfx_compressed, which is __pm_code_lma, is wrong:
>>> 
>>>                 section .prefix         nowrite progbits align=16
>>> pfx_start       dd _start               ; Start of raw chunk
>>> pfx_compressed  dd __pm_code_lma        ; Start of compressed chunk
>>> pfx_cdatalen    dd lzo_data_size        ; Pointer to compressed size
>>field
>>> %if IS_ISOLINUX
>>> pfx_checksum    dd bi_length            ; File length and checksum
>>fields
>>> %else
>>> pfx_checksum    dd 0                    ; No checksum
>>> %endif
>>> pfx_maxlma      dd MaxLMA               ; Maximum size
>>> 
>>> 00009770 B __pm_code_lma
>>
>>bios/core/ldlinux.elf is built with -pie.  The value of __pm_code_lma
>>won't be known until run-time:
>>
>>Relocation section '.rel.dyn' at offset 0x1ba34 contains 1435 entries:
>> Offset     Info    Type            Sym.Value  Sym. Name
>>...
>>00000004  00010d01 R_386_32          00009770   __pm_code_lma
>>00008f62  00010d01 R_386_32          00009770   __pm_code_lma
>>
>>Peter, am I missing something?
>
>ldlinux.elf shouldn't be build pie, so that would explain why.

What I genuinely can't remember offhand is whether or not it is a combination of pic and non-pic objects...

(In reply to hpa@zytor.com from comment #5)
> 
> ldlinux.elf shouldn't be build pie, so that would explain why.

It was changed to PIE by

commit 38e58635d3868c23537fc5dce87b152a52df34ad
Author: chandramouli narayanan <mouli@linux.intel.com>
Date:   Mon Jun 25 12:45:26 2012 -0700

    The make files have undergone changes to support both i386 and x86_64 platforms.
    Pertinent changes are sprinkled through *.mk and Makefile in the syslinux tree.
    
    Build architecture is specified at the command line via ARCH macro. This is to enable whether
    one is building for the native platform or cross-building for a different architecture. Currently,
    the only supported architectures via ARCH macro are i386 and x86_64.
    
    There are some modules (e.g. memdump) that don't build under the EFI environment. To go forward
    with efi boot support, a separate EFI_BUILD=1 macro has been added and is required to be
    specified while building. This is a temporary solution for the current build environment.
    
    example build:
    
    On a x86_64 platform, the following cross-builds syslinux for i386
    To build for i386: make EFI_BUILD=1 ARCH=i386
    
    On a x86_64 platform, the following builds syslinux for x86_64
    To build natively: make EFI_BUILD=1
    
    Changes specific to GCC and LD options:
    
    Architecture specific flags and include/search directories are set up for GCC and LD commands.
    Internally, GCC options such as stack-boundary, PIC and machine architecture are set up
    based on ARCH macro. While stack-boundary=2 is fine for i386, it does not work for x86_64
    and so it is left to the default value. Using -fPIE with gcc does not work with ld on x86_64,
    but -fPIC works fine. Similarly, target and elf options are set up for linking
    as approrpriate for architecture.
    
    Assumption about gnu-efi install path:
    
    Since the EFI build supports both EFI32 and EFI64, the following assumptions
    are made with regard to the installed location of gnu-efi on the build platform.
    1. gnu-efi-ia32: assumed installed in /usr/local
    [ EFI-32 include files in /usr/local/include/efi and gnu-efi library in /usr/local/lib/ ]
    2. gnu-efi-x86_64: assumed installed in /usr
    [ EFI-64 include files in /usr/include/efi and gnu-efi library in /usr/lib64/ ]
    
    Remanants of the unused old i386-only files, if any, need to be pruned.

(In reply to hpa@zytor.com from comment #6)
>
> >
> >ldlinux.elf shouldn't be build pie, so that would explain why.
> 
> What I genuinely can't remember offhand is whether or not it is a
> combination of pic and non-pic objects...

It may be used for EFI.

Created attachment 8941 [details]
Please try this simpler patch

It is a syslinux bug.  Please reopen it if it isn't the case.

On January 30, 2016 2:18:51 PM PST, "hjl.tools at gmail dot com" <sourceware-bugzilla@sourceware.org> wrote:
>https://sourceware.org/bugzilla/show_bug.cgi?id=19538
>
>--- Comment #8 from H.J. Lu <hjl.tools at gmail dot com> ---
>(In reply to hpa@zytor.com from comment #6)
>>
>> >
>> >ldlinux.elf shouldn't be build pie, so that would explain why.
>> 
>> What I genuinely can't remember offhand is whether or not it is a
>> combination of pic and non-pic objects...
>
>It may be used for EFI.

But that is not built in core/bios.

(In reply to hpa@zytor.com from comment #11)
> 
> But that is not built in core/bios.

core/Makefile has

# Set up the NASM and LD options for the architecture
NASM_ELF = "unknown"
LD_PIE = "unknown"
ifeq ($(ARCH),i386)
        NASM_ELF = elf 
        LD_PIE = -pie
endif
ifeq ($(ARCH),x86_64)
        NASM_ELF = elf64
        #LD_PIE = --pic-executable
        LD_PIE = 
endif

On January 30, 2016 4:19:57 PM PST, "hjl.tools at gmail dot com" <sourceware-bugzilla@sourceware.org> wrote:
>https://sourceware.org/bugzilla/show_bug.cgi?id=19538
>
>--- Comment #12 from H.J. Lu <hjl.tools at gmail dot com> ---
>(In reply to hpa@zytor.com from comment #11)
>> 
>> But that is not built in core/bios.
>
>core/Makefile has
>
># Set up the NASM and LD options for the architecture
>NASM_ELF = "unknown"
>LD_PIE = "unknown"
>ifeq ($(ARCH),i386)
>        NASM_ELF = elf 
>        LD_PIE = -pie
>endif
>ifeq ($(ARCH),x86_64)
>        NASM_ELF = elf64
>        #LD_PIE = --pic-executable
>        LD_PIE = 
>endif

And that is true for most of the modules.  Let me look into it later.

However, part of the weirdness is that it is a partly pic and partly non-pic object.

(In reply to H.J. Lu from comment #9)
> Created attachment 8941 [details]
> Please try this simpler patch

With this patch build finishes, but produced binaries break extlinux/isolinux loading.
Thanks for trying.

So ldlinux.elf on BIOS is an odd duck.  It contains bits that are relocated and bits that are not relocated, but the two components cross-call, and both have symbols that need to be exported.

2.25 generates output that has the unrelocated values (that is, what would be exported from the linker script without -pie) in the output sections, and R_386_RELATIVE relocations in .rel.dyn.

At the same time, I also observe that with 2.25, there seem to be R_386_RELATIVE annotations for locations which reference ABS symbols, which seems very odd; fortunately that particular code is all executed before relocation is performed.

I'm going to muck with the linker script to see if I can improve the situation.

I have tried removing -pie (which shouldn't actually be necessary); however,
when I do so, ld no longer generates the sections which are necessary to export symbols to dynamic libraries.  The command line looks like:

ld -m elf_i386  -Bdynamic -Bsymbolic -Bsymbolic-functions \
        -E --hash-style=gnu -T /home/hpa/syslinux/git/core/i386/syslinux.ld -M -o ldlinux.elf ldlinux.o \
        --start-group libcom32.a --whole-archive /home/hpa/syslinux/git/bios/com32/lib/libcom32core.a libldlinux.a --end-group \
        > ldlinux.map

... and I will attach the linker script.

Created attachment 8966 [details]
Linker script

Note that despite the use of -pie the only reason for including that option at this time is to export the necessary symbols.  However, this appears to have been there from the start in order to actually generate the dynamic output sections.

This is confusing to me, because the two would seem unrelated to me.

Adding KEEP to the relevant sections have appeared to make no difference.

-pie is needed only if output has been PIC.  Otherwise, you don't
need -pie and all symbol addresses will be fixed at link-time.

What I'm telling you is that without either -pie or at least one undefined symbol *and* a shared library specified to the linker, all the dynamic sections disappear.

(In reply to hpa@zytor.com from comment #20)
> What I'm telling you is that without either -pie or at least one undefined
> symbol *and* a shared library specified to the linker, all the dynamic
> sections disappear.

Why do you need dynamic sections? Is output PIC at load-time?

The output file exports symbols to other ELF modules which are loaded (ldlinux.elf includes the dynamic linker.)

So to summarize:

1. ldlinux.elf is an executable which does not depend on any other object
2. It does not require relocation
3. ldlinux.elf *includes* an ELF dynamic linker as well as a core library
4. The symbols from the core library should be exported to dynamic ELF
   objects which are loaded later

... and when linking with "-Bdynamic" without "-pie", ld removes all the sections related to dynamic linking when there are no undefined symbols which need resolution, with the result that loading the dynamic modules fail.

(In reply to hpa@zytor.com from comment #24)
> ... and when linking with "-Bdynamic" without "-pie", ld removes all the
> sections related to dynamic linking when there are no undefined symbols
> which need resolution, with the result that loading the dynamic modules fail.

How many pieces are there at run-time? Please keep -pie and

1. Get the current binutils-2_26-branch branch from git.
2. Configure it with --disable-x86-relax-relocations
3. Find the smallest output which is broken by binutils 2.26.
4. Upload ALL inputs for binutils 2.26.
5. Upload the good output generated by binutils 2.25.
6. Upload the bad output generated by binutils 2.26.

When compiled in that fashion I cannot reproduce the original problem.

However, I will shortly upload a test case for -Bdynamic without -pie.

Created attachment 8967 [details]
Test case for -Bdynamic without -pie

This test case shows how dynamic symbols are not exported without -pie, despite -Bdynamic.

When building *without* --disable-x86-relax-relocations then the test case I submitted exhibits the problem that causes syslinux to fail to build: you can see offset 0x30 (xyzzy_ptr) of with-pie/test1.bin is zero, rather than 0x1f as would be expected.

When compiled with --disable-x86-relax-relocations the expected value is obtained.

However, it would be better to not need the -pic at all.

Created attachment 8968 [details]
A patch for syslinux

Please try this patch for syslinux.

tools/relocs.c isn't actually used, so it won't make any difference (I deleted it from my private tree to verify, together with several other unused files.)

I have now removed the unused files from the Syslinux git tree.

(In reply to hpa@zytor.com from comment #24)
> ... and when linking with "-Bdynamic" without "-pie", ld removes all the
> sections related to dynamic linking when there are no undefined symbols
> which need resolution, with the result that loading the dynamic modules fail.

To get dynamic sections, you have to either use -pie or -shared.  -Bdynamic
is unrelated.  Please use -pie.

(In reply to hpa@zytor.com from comment #26)
> When compiled in that fashion I cannot reproduce the original problem.
> 

Please get the current binutils-2_26-branch branch from git and
find the smallest output which is broken by binutils 2.26.  Do

1. Configure binutils 2.26 with --disable-x86-relax-relocations
2. Upload ALL inputs for binutils 2.26 and linker command line.
3. Upload the good output generated by binutils 2.26.

and do

1. Configure binutils 2.26 with --enable-x86-relax-relocations
2. Upload ALL inputs for binutils 2.26 and linker command line.
3. Upload the bad output generated by binutils 2.26.

I will compare them to figure out what goes wrong.

Created attachment 8970 [details]
A patch

Please try this.

With that patch, syslinux builds but is non-functional; the header looks correct but there is a problem somewhere else.

I have uploaded the entire build to:

http://www.zytor.com/~hpa/syslinux/syslinux-ld.tar.xz

The directory "git" was build with binutils.git git ID 8e50258985f01270b142d0537d7b80e789e4d7d7 plus your patch.

The directory "git.ref" was build with binutils-2.25-15.fc23.x86_64 from Fedora, and works.

(In reply to hpa@zytor.com from comment #35)
> With that patch, syslinux builds but is non-functional; the header looks
> correct but there is a problem somewhere else.

The header is correct without my linker patch.

> I have uploaded the entire build to:
> 
> http://www.zytor.com/~hpa/syslinux/syslinux-ld.tar.xz
> 
> The directory "git" was build with binutils.git git ID
> 8e50258985f01270b142d0537d7b80e789e4d7d7 plus your patch.
> 
> The directory "git.ref" was build with binutils-2.25-15.fc23.x86_64 from
> Fedora, and works.

I need the linker command line for creating ldlinux.elf.

On February 10, 2016 3:33:00 PM PST, "hjl.tools at gmail dot com" <sourceware-bugzilla@sourceware.org> wrote:
>https://sourceware.org/bugzilla/show_bug.cgi?id=19538
>
>--- Comment #36 from H.J. Lu <hjl.tools at gmail dot com> ---
>(In reply to hpa@zytor.com from comment #35)
>> With that patch, syslinux builds but is non-functional; the header
>looks
>> correct but there is a problem somewhere else.
>
>The header is correct without my linker patch.
>
>> I have uploaded the entire build to:
>> 
>> http://www.zytor.com/~hpa/syslinux/syslinux-ld.tar.xz
>> 
>> The directory "git" was build with binutils.git git ID
>> 8e50258985f01270b142d0537d7b80e789e4d7d7 plus your patch.
>> 
>> The directory "git.ref" was build with binutils-2.25-15.fc23.x86_64
>from
>> Fedora, and works.
>
>I need the linker command line for creating ldlinux.elf.

You can just do:

rm -f bios/core/ldlinux.*
make bios |& tee make.log

That will give you the linker command line, as well as the postprocessing.

On February 10, 2016 3:33:00 PM PST, "hjl.tools at gmail dot com" <sourceware-bugzilla@sourceware.org> wrote:
>https://sourceware.org/bugzilla/show_bug.cgi?id=19538
>
>--- Comment #36 from H.J. Lu <hjl.tools at gmail dot com> ---
>(In reply to hpa@zytor.com from comment #35)
>> With that patch, syslinux builds but is non-functional; the header
>looks
>> correct but there is a problem somewhere else.
>
>The header is correct without my linker patch.
>
>> I have uploaded the entire build to:
>> 
>> http://www.zytor.com/~hpa/syslinux/syslinux-ld.tar.xz
>> 
>> The directory "git" was build with binutils.git git ID
>> 8e50258985f01270b142d0537d7b80e789e4d7d7 plus your patch.
>> 
>> The directory "git.ref" was build with binutils-2.25-15.fc23.x86_64
>from
>> Fedora, and works.
>
>I need the linker command line for creating ldlinux.elf.

I suspect my changes to the linker script to make the LMA symbols absolute might have fixed the header problem, but the small test case should still show it.

(In reply to hpa@zytor.com from comment #38)
>
> I suspect my changes to the linker script to make the LMA symbols absolute
> might have fixed the header problem, but the small test case should still
> show it.

We may have more than one issues here:

1. Without -pie, no dynamic symbols.   I tried the small testcase with
binutils 2.25.2:

[hjl@gnu-6 without-pie]$ /export/build/gnu/binutils-misc/build-x86_64-linux/ld/ld-new -m elf_i386 -Ttext 0 -Bdynamic -Bsymbolic -E -o test1.elf test1.o
[hjl@gnu-6 without-pie]$ /export/build/gnu/binutils-misc/build-x86_64-linux/ld/ld-new -v
GNU ld (GNU Binutils) 2.25.2
[hjl@gnu-6 without-pie]$  nm -Dn test1.elf 
nm: test1.elf: no symbols

There is no dynamic symbol either.

[hjl@gnu-6 without-pie]$ 

2. With -pie, still doesn't work. Please tell me what is wrong in
the binary generated by 2.26.

On February 10, 2016 4:15:39 PM PST, "hjl.tools at gmail dot com" <sourceware-bugzilla@sourceware.org> wrote:
>https://sourceware.org/bugzilla/show_bug.cgi?id=19538
>
>--- Comment #39 from H.J. Lu <hjl.tools at gmail dot com> ---
>(In reply to hpa@zytor.com from comment #38)
>>
>> I suspect my changes to the linker script to make the LMA symbols
>absolute
>> might have fixed the header problem, but the small test case should
>still
>> show it.
>
>We may have more than one issues here:
>
>1. Without -pie, no dynamic symbols.   I tried the small testcase with
>binutils 2.25.2:
>
>[hjl@gnu-6 without-pie]$
>/export/build/gnu/binutils-misc/build-x86_64-linux/ld/ld-new -m
>elf_i386 -Ttext
>0 -Bdynamic -Bsymbolic -E -o test1.elf test1.o
>[hjl@gnu-6 without-pie]$
>/export/build/gnu/binutils-misc/build-x86_64-linux/ld/ld-new -v
>GNU ld (GNU Binutils) 2.25.2
>[hjl@gnu-6 without-pie]$  nm -Dn test1.elf 
>nm: test1.elf: no symbols
>
>There is no dynamic symbol either.
>
>[hjl@gnu-6 without-pie]$ 
>
>2. With -pie, still doesn't work. Please tell me what is wrong in
>the binary generated by 2.26.

Yes, 1 really should be a separate enhancement PR.

On February 10, 2016 4:15:39 PM PST, "hjl.tools at gmail dot com" <sourceware-bugzilla@sourceware.org> wrote:
>https://sourceware.org/bugzilla/show_bug.cgi?id=19538
>
>--- Comment #39 from H.J. Lu <hjl.tools at gmail dot com> ---
>(In reply to hpa@zytor.com from comment #38)
>>
>> I suspect my changes to the linker script to make the LMA symbols
>absolute
>> might have fixed the header problem, but the small test case should
>still
>> show it.
>
>We may have more than one issues here:
>
>1. Without -pie, no dynamic symbols.   I tried the small testcase with
>binutils 2.25.2:
>
>[hjl@gnu-6 without-pie]$
>/export/build/gnu/binutils-misc/build-x86_64-linux/ld/ld-new -m
>elf_i386 -Ttext
>0 -Bdynamic -Bsymbolic -E -o test1.elf test1.o
>[hjl@gnu-6 without-pie]$
>/export/build/gnu/binutils-misc/build-x86_64-linux/ld/ld-new -v
>GNU ld (GNU Binutils) 2.25.2
>[hjl@gnu-6 without-pie]$  nm -Dn test1.elf 
>nm: test1.elf: no symbols
>
>There is no dynamic symbol either.
>
>[hjl@gnu-6 without-pie]$ 
>
>2. With -pie, still doesn't work. Please tell me what is wrong in
>the binary generated by 2.26.

In the test case, byte 0x30 of test1.bin should be 0x1f.  In the output I believe I sent it is 0x00 from ld 2.26.

Created attachment 8974 [details]
A new patch

Please try this.

It is a dup.  Please try users/hjl/pr19615/binutils-2_26-branch.

*** This bug has been marked as a duplicate of bug 19615 ***

With that branch, Syslinux builds and runs correctly.