Bug 28902 - ld: `not found for insert` error has a weird ordering
Summary: ld: `not found for insert` error has a weird ordering
Status: RESOLVED FIXED
Alias: None
Product: binutils
Classification: Unclassified
Component: ld (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: 2.41
Assignee: Alan Modra
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-02-17 01:48 UTC by Fangrui Song
Modified: 2023-05-13 11:35 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed: 2022-02-28 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Fangrui Song 2022-02-17 01:48:23 UTC
echo 'SECTIONS { .byte : { BYTE(0) } } INSERT AFTER .data;' > insert-data.lds
echo 'SECTIONS { .text : {*(.text)} .data : {*(.data)}}' > a.lds
echo 'SECTIONS { .byte : { BYTE(0) } } INSERT AFTER .note.gnu.build-id;' > insert-note.lds
echo '.globl _start; _start:; .data; .byte 0' | gcc -c -x assembler - -o a.o
exit

ld.bfd a.o -T insert-data.lds -T a.lds  # ok
ld.bfd a.o -T a.lds -T insert-data.lds  # .data not found for insert

# Remark: The order is puzzling. If ld processes -T in order, one will expect that `-T a.lds -T insert-data.lds` works and the other order fails.

ld.bfd a.o -T insert-note.lds --build-id  # ok
ld.bfd a.o -T a.lds -T insert-note.lds --build-id  # .note.gnu.build-id not found for insert
ld.bfd a.o -T a.lds -dT insert-note.lds --build-id  # ok

# Remark: Why -T fails while -dT works is puzzling. If you specify --verbose, the used linker script is exactly the same.
Comment 1 Fangrui Song 2022-02-17 01:57:17 UTC
(I don't have a Fedora machine and I know very little about systemd.)
I was asked why lld doesn't support -dT. I have dug up and found

https://github.com/systemd/package-notes/blob/main/rpm/macros.package-notes-srpm#L52

%_package_note_flags    %[%_package_note_status?"-Wl,%["%_package_note_linker" != "lld"?"-dT":"-T"],%{_package_note_file}":""]

According to Debian Code Search (
https://codesearch.debian.net/search?q=%5B-%5D%5B-%5Ddefault-script&literal=0&page=1), no project uses `--default-script` or `Wl,-dT`.

> ld.bfd a.o -T a.lds -dT insert-note.lds --build-id  # ok

I suspect this is the reason https://github.com/systemd/package-notes/blob/main/rpm/macros.package-notes-srpm#L52 uses -dT. But this appears as an ill advice to me, working around a problem with an obscure option, instead of making the ld maintainer aware of the problem.
Comment 2 Nick Clifton 2022-02-28 14:03:52 UTC
Hi Fangrui,  
 
> ld.bfd a.o -T insert-data.lds -T a.lds  # ok
> ld.bfd a.o -T a.lds -T insert-data.lds  # .data not found for insert
> 
> # Remark: The order is puzzling. If ld processes -T in order, one will
> expect that `-T a.lds -T insert-data.lds` works and the other order fails.

I have not dug deeply into this, but I would guess that this happens because
the script parser is stack based, so it pulls out and processes insert-data.lds before a.lds.   But because semantic processing happens before syntactic processing the INSERT in insert-data.lds has already stopped the default linker script from being parsed, so the only possible definition of .data comes from a.lds.

I expect that using multiple -T options combined with INSERT directives is a rare thing, which is why no-one has noticed this odd behaviour before.  Personally I think that the safest thing to do would be to just document the behaviour, but not make any changes to the code.



> ld.bfd a.o -T insert-note.lds --build-id  # ok
> ld.bfd a.o -T a.lds -T insert-note.lds --build-id  # .note.gnu.build-id not
> found for insert

This makes sense.  The a.lds script does not define a .note.gnu.build-id section, so there is nowhere for the INSERT AFTER in insert-note.lds to attach to.

When a.lds is omitted the default linker script is used instead and this does define a .note.gnu.build-id section, which is why the INSERT AFTER works.


> ld.bfd a.o -T a.lds -dT insert-note.lds --build-id  # ok
> 
> # Remark: Why -T fails while -dT works is puzzling. If you specify
> --verbose, the used linker script is exactly the same.

It seems that -T and -dT are incompatible.  In my tests it appears that the -dT option is entirely ignored if -T is also used.  This is a bug, but I am concerned that fixing it might break scripts which unknowingly rely upon this behaviour.  So maybe it should be considered a documentation bug that the incompatibility is not mentioned.

So overall I am inclined to think that these are both documentation bugs.  Do you agree ?

Cheers
  Nick
Comment 3 Fangrui Song 2022-03-09 00:51:58 UTC
Hi Nick, sorry for my belated reply. I was on a trip.

> > ld.bfd a.o -T insert-data.lds -T a.lds  # ok
> > ld.bfd a.o -T a.lds -T insert-data.lds  # .data not found for insert
> > 
> > # Remark: The order is puzzling. If ld processes -T in order, one will
> > expect that `-T a.lds -T insert-data.lds` works and the other order fails.
> 
> 
> I have not dug deeply into this, but I would guess that this happens because
> the script parser is stack based, so it pulls out and processes insert-data.lds before a.lds.   But because semantic processing happens before syntactic processing the INSERT in insert-data.lds has already stopped the default linker script from being parsed, so the only possible definition of .data comes from a.lds.
> 
> I expect that using multiple -T options combined with INSERT directives is a rare thing, which is why no-one has noticed this odd behaviour before.  Personally I think that the safest thing to do would be to just document the behaviour, but not make any changes to the code.

The order issue can be fixed by postponing the effects of INSERT after regular SECTIONS. Is it feasible?

> 
> > ld.bfd a.o -T insert-note.lds --build-id  # ok
> > ld.bfd a.o -T a.lds -T insert-note.lds --build-id  # .note.gnu.build-id not
> > found for insert
> 
> 
> This makes sense.  The a.lds script does not define a .note.gnu.build-id section, so there is nowhere for the INSERT AFTER in insert-note.lds to attach to.
> 
> When a.lds is omitted the default linker script is used instead and this does define a .note.gnu.build-id section, which is why the INSERT AFTER works.
> 
> 
> > ld.bfd a.o -T a.lds -dT insert-note.lds --build-id  # ok
> > 
> > # Remark: Why -T fails while -dT works is puzzling. If you specify
> > --verbose, the used linker script is exactly the same.
> 
> 
> It seems that -T and -dT are incompatible.  In my tests it appears that the -dT option is entirely ignored if -T is also used.  This is a bug, but I am concerned that fixing it might break scripts which unknowingly rely upon this behaviour.  So maybe it should be considered a documentation bug that the incompatibility is not mentioned.

From Debian Code Search, https://codesearch.debian.net/search?q=%5B-%5DdT&literal=0 (-dT) has no result.
--default-script seems unused as well, so I think changing the behavior of -dT probably does not matter.
Making -dT work in the presence of -T probably makes more sense.

> So overall I am inclined to think that these are both documentation bugs.  Do you agree ?

-dT/--default-script seems unused (except the recent .note.package) and INSERT is super rare as well. I think there are some changes which may be worth making to make the behavior more reasonable and make the documentation more explainable to a reader. The current behaviors may be a bit difficult to describe in the documentation...
Comment 4 Nick Clifton 2022-03-16 14:06:02 UTC
(In reply to Fangrui Song from comment #3)
Hi Fangrui,

 
> The order issue can be fixed by postponing the effects of INSERT after
> regular SECTIONS. Is it feasible?

Hmmm, maybe, but I would be very reluctant to try.  This feels like it
would create lots of new problems.  Given that one way works - even if
it is not the intuitive way - I would prefer to leave things as they
are and just document the behaviour.

If you want to create a patch to change the behaviour then please go
ahead, but I have said before, I think that updating the documentation
would be the safest way to go.
Comment 5 Sourceware Commits 2023-05-13 11:19:29 UTC
The master branch has been updated by Alan Modra <amodra@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=31f149015865401c760e9d76157734996fbab770

commit 31f149015865401c760e9d76157734996fbab770
Author: Alan Modra <amodra@gmail.com>
Date:   Sat May 13 14:50:23 2023 +0930

    PR28902, -T script with INSERT ordering
    
    The answer to PR28902 may be deduced from the existing INSERT
    documentation that says the default script is parsed after the -T
    INSERT script, if you assume (correctly) that nothing special is done
    when inserting into -T scripts overriding the default script.  In both
    cases INSERT handling looks for the specified output section later on
    the internal list of parsed script commands.  This isn't obvious
    though, so make the ordering explicit, and mention that section
    assignments are the same too.
    
            PR 28902
            * ld.texi (INSERT): Specify ordering when -T is used both to
            override the default script and to augment.
Comment 6 Alan Modra 2023-05-13 11:35:15 UTC
Fixed