Bug 12565 - NOLOAD sections empty
Summary: NOLOAD sections empty
Status: WAITING
Alias: None
Product: binutils
Classification: Unclassified
Component: ld (show other bugs)
Version: 2.22
: P2 minor
Target Milestone: ---
Assignee: unassigned
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-03-10 17:56 UTC by Jonathan Larmour
Modified: 2014-09-24 14:38 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:


Attachments
A patch (475 bytes, patch)
2011-03-15 22:21 UTC, H.J. Lu
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Larmour 2011-03-10 17:56:33 UTC
It appears that the fix for bug #5522 (fixed by H.J. Lu between 2.18 and 2.19 in Jan 2008) is not quite right.

When bringing over an existing linker script (which worked before 2.19), it happened to mark the DWARF debug sections as NOLOAD, e.g.:

    .debug_info         0 (NOLOAD) : { *(.debug_info .gnu.linkonce.wi.*) }  \

The use of NOLOAD is redundant, but also shouldn't be an issue. However the fix for bug #5522 causes the .debug_info section to be filled with 0's instead of data. Analysis with readelf or objdump shows that it in other respects the section has the correct flags and properties.

This is not a high priority for me personally as I can trivially just remove the (NOLOAD), however someone who needs to use NOLOAD for contents that /would/ otherwise be loadable, may be surprised to find that the section is always completely zeroed. This goes beyond what NOLOAD should be doing. If they didn't want any part of the section in the image, they could just use /DISCARD/, so it removes the point of NOLOAD.

I can confirm this problem still exists in CVS today 2011-03-10.

I expect you can see the problem on any platform by using a slightly modified linker script, adding (NOLOAD) into the .debug_info line (this section just makes it easy to demonstrate the problem). readelf -x .debug_info will either show all 0s, or proper data.
Comment 1 H.J. Lu 2011-03-14 13:52:35 UTC
Please provide a testcase.
Comment 2 Jonathan Larmour 2011-03-15 19:52:27 UTC
Yes here's a definite way to reproduce. I can definitely confirm it's platform-independent in fact as I've reproduced it with native Linux. Here it is in annoying detail purely for completeness. Take shortcuts where you like:

1. Build binutils for native linux (I used CVS of 2011-03-10), and put the bin/ directory at the head of your PATH:

mkdir -p /tmp/b12565/build
cd /tmp/b12565/build
~/src/binutils/src/configure --prefix=/tmp/b12565/install
make all install
export PATH=/tmp/b12565/install/bin:$PATH

2. Create a helloworld app just for testing:
mkdir /tmp/b12565/c
cd /tmp/b12565/c
cat > helloworld.c <<EOF
#include <stdio.h>
int main(int argc, char *argv[])
{
  printf("Hello world!\n");
  return 0;
}
EOF

3. Build it with full verbosity, and with -g:
gcc -g -o helloworld helloworld.c -v -Wl,-v --save-temps

4. As a baseline, look at its .debug_info section:
readelf -x .debug_info helloworld | less

Notice it has useful stuff in.

5. Extract the default linker script with:
ld --verbose | sed -n '/^======/,/^======/p' |egrep -v ^====== > script.ld 

6. Modify the .debug_info line of script.ld to add NOLOAD:
  .debug_info     0 (NOLOAD) : { *(.debug_info .gnu.linkonce.wi.*) }

7. Cut'and'paste the link line (invoking ld) you will have seen in step (3) but appending -T./script.ld

8. Look at .debug_info again, and you'll see it is zeroed:
readelf -x .debug_info helloworld | less


Again, as a reminder, I'm just using .debug_info as an example section which isn't meant to be loaded. The problem is not specific to it of course.
Comment 3 H.J. Lu 2011-03-15 22:21:57 UTC
Created attachment 5312 [details]
A patch

Although the linker script is bad, this patch help it.
Comment 4 Jonathan Larmour 2011-03-15 22:37:58 UTC
(In reply to comment #3)
> Created attachment 5312 [details]
> A patch
> 
> Although the linker script is bad, this patch help it.

Sorry, I was trying to emphasise earlier that using .debug_info is just an example. It applies to any section, not just debug sections.

For example, change helloworld.c to:
#include <stdio.h>
const char foobar[]  __attribute__((section(".foobar"))) = "foobar";
int main(int argc, char *argv[])
{
  printf("Hello world!\n");
  return 0;
}

And edit script.ld to include:
  .foobar        0 : { *(.foobar) }
and you see useful data. All well and good.

But if you change that line to:
  .foobar        0 (NOLOAD) : { *(.foobar) }
then the section gets removed entirely from the binary (which is admittedly going even further than just zeroing the contents). Objdump reports:
 25 .foobar       00000007  0000000000000000  0000000000000000  00200000  2**0
                  ALLOC, READONLY
and readelf -S reports:
  [26] .foobar           NOBITS           0000000000000000  00200000
       0000000000000007  0000000000000000   A       0     0     1

Just because some data isn't meant to be loaded into memory (by using NOLOAD), should not affect whether that data is present in the binary.
Comment 5 Jonathan Larmour 2011-03-15 22:39:05 UTC
(In reply to comment #4)
> 
> And edit script.ld to include:
>   .foobar        0 : { *(.foobar) }
> and you see useful data. All well and good.

To be clear, I mean useful data by running:
readelf -x .foobar helloworld
Comment 6 Nick Clifton 2011-06-30 14:37:18 UTC
Hi H.J., Hi Alan,

I have been looking at PR 12565, and I have to say that I do not 
understand the linker's behaviour for NOLOAD sections on ELF based 
targets.  What is the point of having a section that cannot be loaded 
and that does not have any contents ?

Also, as far as I can see, this behaviour is not documented anywhere. 
Do you know of any applications that rely upon this feature ?

Cheers
   Nick
Comment 7 H.J. Lu 2011-06-30 14:47:49 UTC
(In reply to comment #6)
> Hi H.J., Hi Alan,
> 
> I have been looking at PR 12565, and I have to say that I do not 
> understand the linker's behaviour for NOLOAD sections on ELF based 
> targets.  What is the point of having a section that cannot be loaded 
> and that does not have any contents ?
> 
> Also, as far as I can see, this behaviour is not documented anywhere. 
> Do you know of any applications that rely upon this feature ?
> 

NOLOAD means "don't load into memory", which maps to NOBITS.
Otherwise, it will be loaded into memory.
Comment 8 davem 2011-06-30 15:36:49 UTC
On Thu, Jun 30, 2011 at 3:37 PM, Nick Clifton <nickc@redhat.com> wrote:

> Hi H.J., Hi Alan,
>
> I have been looking at PR 12565, and I have to say that I do not understand
> the linker's behaviour for NOLOAD sections on ELF based targets.  What is
> the point of having a section that cannot be loaded and that does not have
> any contents ?
>
> Also, as far as I can see, this behaviour is not documented anywhere. Do
> you know of any applications that rely upon this feature ?
>

I thought it was for bss segments.

I've been using that flag to allow me to overly bss sections on top of other
sections that are copied to other memory locations at runtime. If I didn't
use NOLOAD then the linker would complain about overlapping sections.

http://devkitpro.svn.sourceforge.net/viewvc/devkitpro/trunk/buildscripts/dkarm-eabi/crtls/ds_arm9.ld?revision=4706&view=markupfor
example.

Dave
Comment 9 davem 2011-06-30 15:40:01 UTC
On Thu, Jun 30, 2011 at 3:37 PM, Nick Clifton <nickc@redhat.com> wrote:
>
> Hi H.J., Hi Alan,
>
> I have been looking at PR 12565, and I have to say that I do not understand the linker's behaviour for NOLOAD sections on ELF based targets.  What is the point of having a section that cannot be loaded and that does not have any contents ?
>
> Also, as far as I can see, this behaviour is not documented anywhere. Do you know of any applications that rely upon this feature ?

I thought it was for bss segments.

I've been using that flag to allow me to overly bss sections on top of
other sections that are copied to other memory locations at runtime.
If I didn't use NOLOAD then the linker would complain about
overlapping sections.

http://devkitpro.svn.sourceforge.net/viewvc/devkitpro/trunk/buildscripts/dkarm-eabi/crtls/ds_arm9.ld?revision=4706&view=markup
for example.

Dave
Comment 10 Ian Lance Taylor 2011-06-30 16:39:39 UTC
Nick Clifton <nickc@redhat.com> writes:

> I have been looking at PR 12565, and I have to say that I do not
> understand the linker's behaviour for NOLOAD sections on ELF based
> targets.  What is the point of having a section that cannot be loaded
> and that does not have any contents ?
>
> Also, as far as I can see, this behaviour is not documented
> anywhere. Do you know of any applications that rely upon this feature
> ?

If you look in the libgloss linker scripts you will see a bunch of uses
of NOLOAD.

I'm not sure whether any of them are really necessary, but they are
certainly there.

Ian
Comment 11 Doug Kwan 2011-06-30 18:24:45 UTC
The ARM linux kernel also uses NOLOAD.  I added support for that in
gold to link the kernel.

-Doug

On Thu, Jun 30, 2011 at 9:07 AM, Ian Lance Taylor <iant@google.com> wrote:
> Nick Clifton <nickc@redhat.com> writes:
>
>> I have been looking at PR 12565, and I have to say that I do not
>> understand the linker's behaviour for NOLOAD sections on ELF based
>> targets.  What is the point of having a section that cannot be loaded
>> and that does not have any contents ?
>>
>> Also, as far as I can see, this behaviour is not documented
>> anywhere. Do you know of any applications that rely upon this feature
>> ?
>
> If you look in the libgloss linker scripts you will see a bunch of uses
> of NOLOAD.
>
> I'm not sure whether any of them are really necessary, but they are
> certainly there.
>
> Ian
>
Comment 12 Jonathan Larmour 2011-06-30 21:24:57 UTC
(In reply to comment #6)
> 
> I have been looking at PR 12565, and I have to say that I do not 
> understand the linker's behaviour for NOLOAD sections on ELF based 
> targets.  What is the point of having a section that cannot be loaded 
> and that does not have any contents ?

For the avoidance of doubt, my example was for the case when it does have contents. In which case there are various potential uses, such as for metadata about an executable or object, but where that metadata may not be from an input section defined with 'nobits' (or where you can't guarantee every input section is). You can understand some more applications for it in linker scripts when you consider compiling executables with -ffunction-sections -fdata-sections.

> Also, as far as I can see, this behaviour is not documented anywhere. 

It is documented in ld.texinfo in the 'Output Section Type' node:
@item NOLOAD
The section should be marked as not loadable, so that it will not be
loaded into memory when the program is run.

Jifl
Comment 13 Alan Modra 2011-07-01 00:03:32 UTC
See http://sourceware.org/ml/binutils/2010-09/msg00245.html, the link in that email and followup posts over the next couple of months.
Comment 14 Alan Modra 2011-07-01 00:03:38 UTC
On Thu, Jun 30, 2011 at 03:37:43PM +0100, Nick Clifton wrote:
> Hi H.J., Hi Alan,
> 
> I have been looking at PR 12565, and I have to say that I do not
> understand the linker's behaviour for NOLOAD sections on ELF based
> targets.  What is the point of having a section that cannot be
> loaded and that does not have any contents ?
> 
> Also, as far as I can see, this behaviour is not documented
> anywhere. Do you know of any applications that rely upon this
> feature ?

I spent quite a lot of time wrestling with NOLOAD before
http://sourceware.org/ml/binutils/2010-09/msg00245.html.  I believe we
have fairly strong historical precedent for the current behaviour on
ELF, ie. that NOLOAD makes a .bss style section, noload, alloc, no
contents.  As others have replied, people make use of it this way.
See also http://sourceware.org/ml/binutils/2010-10/msg00433.html
Comment 15 Jonathan Larmour 2011-07-01 08:57:41 UTC
(In reply to comment #14)
> 
> I spent quite a lot of time wrestling with NOLOAD before
> http://sourceware.org/ml/binutils/2010-09/msg00245.html.  I believe we
> have fairly strong historical precedent for the current behaviour on
> ELF, ie. that NOLOAD makes a .bss style section, noload, alloc, no
> contents.

But before H.J.'s change in bug #5522 it did include contents, which is why I reported this change of behaviour in the first place, so I'm not sure what you mean about the precedent. The documented behaviour was to include contents prior to your (and H.J.'s) change.

I hadn't noticed your change before as I've only been using 2.20.1 which doesn't include it.

> As others have replied, people make use of it this way.

We have made use of it with contents, as per the documentation. If you want something that doesn't include contents, wouldn't it be better to make a new output section type rather than changing the behaviour of an existing one? Otherwise it could be confusing. If you want something different from the previous behaviour of NOLOAD it would be better to remove NOLOAD so people can decide what they want to do with their linker scripts once they see the error. Personally, I had thought when I submitted this bug that I had found an isolated instance, but it turns out there are a bunch of linker scripts floating around my company which rely on NOLOAD's old behaviour.

libgloss currently relies on it including contents too.

Jifl
Comment 16 Nick Clifton 2011-07-01 12:45:37 UTC
Hi Jifl,

>> Also, as far as I can see, this behaviour is not documented anywhere.
>
> It is documented in ld.texinfo in the 'Output Section Type' node:
> @item NOLOAD
> The section should be marked as not loadable, so that it will not be
> loaded into memory when the program is run.

But that does not document the fact that the NOLOAD section will also be 
without any contents.  This was the point that I was trying to get at.

Cheers
   Nick
Comment 17 Nick Clifton 2011-07-01 12:52:14 UTC
Hi H.J.

>> Also, as far as I can see, this behaviour is not documented anywhere.
>> Do you know of any applications that rely upon this feature ?
>
> NOLOAD means "don't load into memory", which maps to NOBITS.
> Otherwise, it will be loaded into memory.

OK, I now get that NOLOAD is effectively a marker for .bss type sections.

But it does seem to me that the linker should be aware that users might 
think that NOLOAD means: "do not load the contents of this section from 
the file into the running image", rather than its real meaning (for ELF 
based targets) of "do not give this section any space in the output 
file; if present in the running image, initialise its contents with zero".

Hence I now think that we should update the documentation to more 
properly describe the behaviour of NOLOAD and that we should fix the 
linker so that it issues a warning message whenever it throws away 
non-zero contents of an input section being mapped to a NOLOAD output 
section.

Cheers
   Nick
Comment 18 Nick Clifton 2011-07-01 13:10:20 UTC
Hi Ian,

> If you look in the libgloss linker scripts you will see a bunch of uses
> of NOLOAD.
>
> I'm not sure whether any of them are really necessary, but they are
> certainly there.

Apart from a few .bss sections and one target which uses them to get the 
addresses of its hardware I/O ports, all of the rest are for debugging 
sections.  Which brings us back to the original point of this PR.  In 
theory it makes sense to mark debug sections as NOLOAD, since you do not 
need them in the running executable.  But, for ELF based targets, this 
must not be done because NOLOAD will also obliterate their contents on disk.

Maybe we need a new ELF section type, eg SHT_GNU_NOLOAD, which can be 
used to mark sections which *do* have contents on disk but which should 
not be loaded into running memory.  Or maybe we could use SHT_NOTE instead.

At the moment the linker identifies debugging sections by their name and 
arranges for them to be excluded from any loadable segments.  But it 
would be much cleaner if we could just examine the section type and if 
it was SHT_GNU_NOLOAD then decide not to map it to a loadable segment.

Cheers
   Nick
Comment 19 Nick Clifton 2011-07-01 13:14:34 UTC
Hi Doug,

> The ARM linux kernel also uses NOLOAD.

Do you know if this entirely for .bss style sections, or maybe for 
establishing a region of memory mapped hardware or I/O ports ?

The point I am interested in is if we changed the current behaviour of 
NOLOAD so that it *did* preserve the contents of any input sections (if 
those contents were non-zero) would this break anything ?  I am pretty 
certain that it would, so I am not going to create a patch to make this 
change.  But I do wonder if there any applications out there that are 
relying upon NOLOAD to actually get rid of the contents of input sections.

Cheers
   Nick
Comment 20 Jonathan Larmour 2011-07-01 23:22:43 UTC
Hi Nicl,

(In reply to comment #16) 
> >
> > It is documented in ld.texinfo in the 'Output Section Type' node:
> > @item NOLOAD
> > The section should be marked as not loadable, so that it will not be
> > loaded into memory when the program is run.
> 
> But that does not document the fact that the NOLOAD section will also be 
> without any contents.  This was the point that I was trying to get at.

Sorry I had thought you were asking whether there was any documentation on NOLOAD at all. Evidently it was documented as including contents up to and including 2.20.1. As per Alan's changes described in comment #14, it is only then that what it said about contents was removed, becoming its current more ambiguous state. Here's what it additionally said in 2.20.1:
-=-=-=-=-=-=-=-=-
   The linker normally sets the attributes of an output section based on
the input sections which map into it.  You can override this by using
the section type.  For example, in the script sample below, the `ROM'
section is addressed at memory location `0' and does not need to be
loaded when the program is run.  The contents of the `ROM' section will
appear in the linker output file as usual.
     SECTIONS {
       ROM 0 (NOLOAD) : { ... }
       ...
     }
-=-=-=-=-=-=-=-=-
The last sentence is the most relevant. Practice in the ldscripts of libgloss and elsewhere fits with that.

Just to reiterate, if you want to replace NOLOAD with something which strips contents, personally I think I should be able to deal with that in other ways. But please don't keep NOLOAD but change its meaning from pre-2.21. That makes problems harder to find. New behaviour should use a new name e.g. ALLOCONLY or something.
Comment 21 Christian Bruel 2014-09-24 14:38:51 UTC
hello,

A little up. For 2.24.51.0.3

We have another usage of NOLOAD with a port of the remote proc framework.
If the behavior to clear a NOLOAD section is confirmed, an update of documentation would indeed help. This one was quite confusing, in particular after a change in behavior