Bug 10478 - "File format not recognized" for some solaris libraries
Summary: "File format not recognized" for some solaris libraries
Status: RESOLVED FIXED
Alias: None
Product: binutils
Classification: Unclassified
Component: binutils (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: unassigned
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-08-04 11:45 UTC by Niki W. Waibel
Modified: 2009-09-28 09:46 UTC (History)
1 user (show)

See Also:
Host: sparc-sun-solaris2.10
Target: sparc-sun-solaris2.10
Build: sparc-sun-solaris2.10
Last reconfirmed:


Attachments
file recognized okay (39.49 KB, application/octet-stream)
2009-08-04 11:46 UTC, Niki W. Waibel
Details
file format not recognized (146.98 KB, application/octet-stream)
2009-08-04 11:48 UTC, Niki W. Waibel
Details
Accept sparc binaries with a sh_link field set to SHN_BEFORE or SHN_AFTER (2.10 KB, patch)
2009-09-09 11:47 UTC, Nick Clifton
Details | Diff
same as previous patch (2.17 KB, patch)
2009-09-09 14:55 UTC, Nick Clifton
Details | Diff
sol10/x86 okay (32.19 KB, application/octet-stream)
2009-09-17 19:27 UTC, Niki W. Waibel
Details
sol10/x86 not okay (135.42 KB, application/octet-stream)
2009-09-17 19:30 UTC, Niki W. Waibel
Details
Update previous patch to work with x86 binaries (1.15 KB, patch)
2009-09-25 10:31 UTC, Nick Clifton
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Niki W. Waibel 2009-08-04 11:45:09 UTC
it seems there's problems reading the format of some solaris 10 libraries:
===
$ uname -a
SunOS m5000-1.dev.nlatlu.wipro.com 5.10 Generic_139555-08 sun4u sparc
SUNW,SPARC-Enterprise Solaris
$ size --version
GNU size (Linux/GNU Binutils) 2.19.51.0.14.20090722
Copyright 2008 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later version.
This program has absolutely no warranty.
$ size /usr/lib/libC.so.*
   text    data     bss     dec     hex filename
  72246    2707    1049   76002   128e2 /usr/lib/libC.so.3
size: /usr/lib/libC.so.5: File format not recognized
===

this happens not just with the size tool, also with objdump, ...

will attach libC.so.3 and libC.so.5
Comment 1 Niki W. Waibel 2009-08-04 11:46:52 UTC
Created attachment 4111 [details]
file recognized okay
Comment 2 Niki W. Waibel 2009-08-04 11:48:17 UTC
Created attachment 4112 [details]
file format not recognized
Comment 3 Niki W. Waibel 2009-08-04 13:22:40 UTC
by the way:
===
$ /opt/binutils/2.18.50.0.4/bin/size /usr/lib/libC.so.*
   text    data     bss     dec     hex filename
  72246    2707    1049   76002   128e2 /usr/lib/libC.so.3
 313309   21201   59788  394298   6043a /usr/lib/libC.so.5
===
/opt/binutils/2.18.50.0.6/bin/size /usr/lib/libC.so.*
   text    data     bss     dec     hex filename
  72246    2707    1049   76002   128e2 /usr/lib/libC.so.3
/opt/binutils/2.18.50.0.6/bin/size: /usr/lib/libC.so.5: File format not recognized
===
... unfortunately version 2.18.50.0.5 i've not available for testing.
Comment 4 Alan Modra 2009-08-05 00:25:17 UTC
This is the reason for the failure:
  [13] .exception_ranges PROGBITS        000412ec 0412ec 007aac 00   A 65280   0  4
When reading an ELF file BFD does some basic sanity checks in elf_object_p.
Why does .exception_ranges have an invalid sh_link value of 65280?
Comment 5 Niki W. Waibel 2009-09-08 12:53:34 UTC
is this going to be fixed in binutils? it is definitely a big impact on solaris,
as some programs just do not compile (using gcc/binutils). libC.so.5 is just one
example of such a library.
Comment 6 Nick Clifton 2009-09-08 13:49:11 UTC
Hi Niki,

> is this going to be fixed in binutils?

If you can find an explanation for the why the .exception_ranges section has an
 invalid sh_link value of 65280 then we will fix the binutils.  But at the
moment the tools are quite reasonably rejecting the file as not being a valid
sparc-elf binary, because of this discrepancy.

My guess is that Solaris is using the sh_link field for some non-standard
purpose.  But what purpose ?

Cheers
  Nick
Comment 7 Niki W. Waibel 2009-09-08 14:50:30 UTC
the following files can't be used by binutils:
===
find /usr/sfw/lib /usr/lib -type f | grep "\.so\." | while read i; do size $i
1>/dev/null 2>&1 || echo $i; done
/usr/sfw/lib/libMagick++.so.5.0.47
/usr/sfw/lib/libosp.so.3.0.0
/usr/lib/sparcv9/libCrun.so.1
/usr/lib/sparcv9/libCstd.so.1
/usr/lib/sparcv9/libiostream.so.1
/usr/lib/sparcv9/libicui18n.so.2
/usr/lib/sparcv9/libicui18n.so.3
/usr/lib/sparcv9/libicuio.so.3
/usr/lib/sparcv9/libicule.so.2
/usr/lib/sparcv9/libicule.so.3
/usr/lib/sparcv9/libiculx.so.3
/usr/lib/sparcv9/libicutoolutil.so.2
/usr/lib/sparcv9/libicutu.so.3
/usr/lib/sparcv9/libicuuc.so.2
/usr/lib/sparcv9/libicuuc.so.3
/usr/lib/sparcv9/libustdio.so.2
/usr/lib/sparcv9/libfru.so.1
/usr/lib/sparcv9/libsun_fc.so.1
/usr/lib/sparcv9/libdmi.so.1
/usr/lib/sparcv9/libdmici.so.1
/usr/lib/sparcv9/libdmimi.so.1
/usr/lib/cpu/sparcv8plus/libCstd_isa.so.1
/usr/lib/libC.so.5
/usr/lib/libCrun.so.1
/usr/lib/libCstd.so.1
/usr/lib/libExbridge.so.1
/usr/lib/libiostream.so.1
/usr/lib/cc-ccr/lib/libccr.so.1
/usr/lib/libcctagent.so.1
/usr/lib/patch/libsdbndbm.so.1
/usr/lib/libwsreg.so.1
/usr/lib/libicui18n.so.2
/usr/lib/libicui18n.so.3
/usr/lib/libicuio.so.3
/usr/lib/libicule.so.2
/usr/lib/libicule.so.3
/usr/lib/libiculx.so.3
/usr/lib/libicutoolutil.so.2
/usr/lib/libicutu.so.3
/usr/lib/libicuuc.so.2
/usr/lib/libicuuc.so.3
/usr/lib/libustdio.so.2
/usr/lib/libfru.so.1
/usr/lib/libaspell.so.15.0.2
/usr/lib/libpspell.so.15.0.2
/usr/lib/libsun_fc.so.1
/usr/lib/libdmi.so.1
/usr/lib/libdmici.so.1
/usr/lib/libdmimi.so.1
===
this is a pretty large amount. gcc/binutils should be able to use them to
compile happily on solaris. as i am not a binutils developer the only thing i
can do is upload those "invalid" libraries.
Comment 8 Ian Lance Taylor 2009-09-09 03:16:04 UTC
If SHF_LINK_ORDER is set in the flags field, then the sh_link field is permitted
to contain the special values SHN_BEFORE (65280) and SHN_AFTER (65281). 
SHN_BEFORE means that this section should come before all other sections that it
is combined with (e.g., it should be the first .text section) and SHN_AFTER
means it should come after all other sections.  See also the SHF_ORDERED linker
flag, which has similar semantics for SHN_BEFORE and SHN_AFTER.

This actually seems somewhat broken, as it will fail in an object file with 
more than 65280 sections.  There doesn't seem to be any mechanism to decide
whether the sh_link field should be treated as SHN_BEFORE/SHN_AFTER or a section
index.  But, whatever.
Comment 9 Niki W. Waibel 2009-09-09 07:43:34 UTC
shall i upload more unrecognized files for reference?
Comment 10 Nick Clifton 2009-09-09 11:47:09 UTC
Created attachment 4182 [details]
Accept sparc binaries with a sh_link field set to SHN_BEFORE or SHN_AFTER
Comment 11 Nick Clifton 2009-09-09 11:50:19 UTC
Hi Niki,

  Ian's explanation was all that I needed.

  Please try out the uploaded patch.  It appears to work for the libC.so.5 file
you supplied, at least as far as the informational binutils programs go (size,
objdump, readelf etc).  I have not tried actually linking with one or more of
these files, so maybe you would like to try that yourself.

Cheers
  Nick
Comment 12 Niki W. Waibel 2009-09-09 12:34:57 UTC
nick, can you provide a patchfile against 2.19.51.0.14? i get failures using the
file provided:
===
patching file bfd/elf.c
Hunk #1 succeeded at 1537 (offset 23 lines).
patching file bfd/elfcode.h
Hunk #1 FAILED at 760.
1 out of 1 hunk FAILED -- saving rejects to file bfd/elfcode.h.rej
patching file binutils/readelf.c
Hunk #1 succeeded at 3872 (offset -1 lines).
Hunk #2 succeeded at 3928 (offset -1 lines).
Hunk #3 succeeded at 3948 (offset -1 lines).
Hunk #4 succeeded at 4380 (offset 37 lines).
Hunk #5 FAILED at 4394.
1 out of 5 hunks FAILED -- saving rejects to file binutils/readelf.c.rej
===
Comment 13 Niki W. Waibel 2009-09-09 14:49:38 UTC
oh! just learned about 2.20.51.0.1 to which your patch applies smoothly.

... and, yes, your patch fixes all issues. thank you!
Comment 14 Nick Clifton 2009-09-09 14:55:42 UTC
Created attachment 4184 [details]
same as previous patch
Comment 15 Nick Clifton 2009-09-09 14:57:56 UTC
Hi Niki,

  Ok, I will commit the patch to the mainline sources and the new
about-to-be-released 2.20 branch.

  I'll close the PR for now.  If it turns out that there are issues linking
against libraries which use the SHN_BEFORE and SHN_AFTER values then it can be
reopened.

Cheers
  Nick
Comment 16 Sourceware Commits 2009-09-09 15:04:44 UTC
Subject: Bug 10478

CVSROOT:	/cvs/src
Module name:	src
Changes by:	nickc@sourceware.org	2009-09-09 15:03:54

Modified files:
	bfd            : ChangeLog elf.c elfcode.h 
	binutils       : ChangeLog readelf.c 

Log message:
	PR 10478:
	* elf.c (bfd_section_from_shdr): Do not reject sparc binaries with
	section headers containing sh_link values of SHN_BEFORE or
	SHN_AFTER.
	* elfcode.h (elf_object_p): Likewise.
	
	readelf.c (get_elf_section_flags): Add support for SHF_EXCLUDE and
	SHF_ORDERED.
	(process_section_headers): Warn about out of range sh_link
	values.  When displaying detailed section header information
	annote the SHN_BEFORE and SHN_AFTER values.

Patches:
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/bfd/ChangeLog.diff?cvsroot=src&r1=1.4771&r2=1.4772
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/bfd/elf.c.diff?cvsroot=src&r1=1.490&r2=1.491
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/bfd/elfcode.h.diff?cvsroot=src&r1=1.95&r2=1.96
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/binutils/ChangeLog.diff?cvsroot=src&r1=1.1528&r2=1.1529
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/binutils/readelf.c.diff?cvsroot=src&r1=1.456&r2=1.457

Comment 17 Sourceware Commits 2009-09-09 15:06:58 UTC
Subject: Bug 10478

CVSROOT:	/cvs/src
Module name:	src
Branch: 	binutils-2_20-branch
Changes by:	nickc@sourceware.org	2009-09-09 15:05:52

Modified files:
	bfd            : ChangeLog elf.c elfcode.h 
	binutils       : ChangeLog readelf.c 

Log message:
	PR 10478:
	* elf.c (bfd_section_from_shdr): Do not reject sparc binaries with
	section headers containing sh_link values of SHN_BEFORE or
	SHN_AFTER.
	* elfcode.h (elf_object_p): Likewise.
	
	readelf.c (get_elf_section_flags): Add support for SHF_EXCLUDE and
	SHF_ORDERED.
	(process_section_headers): Warn about out of range sh_link
	values.  When displaying detailed section header information
	annote the SHN_BEFORE and SHN_AFTER values.

Patches:
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/bfd/ChangeLog.diff?cvsroot=src&only_with_tag=binutils-2_20-branch&r1=1.4761.2.5&r2=1.4761.2.6
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/bfd/elf.c.diff?cvsroot=src&only_with_tag=binutils-2_20-branch&r1=1.490&r2=1.490.2.1
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/bfd/elfcode.h.diff?cvsroot=src&only_with_tag=binutils-2_20-branch&r1=1.95&r2=1.95.4.1
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/binutils/ChangeLog.diff?cvsroot=src&only_with_tag=binutils-2_20-branch&r1=1.1521.2.4&r2=1.1521.2.5
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/binutils/readelf.c.diff?cvsroot=src&only_with_tag=binutils-2_20-branch&r1=1.454.2.1&r2=1.454.2.2

Comment 18 Niki W. Waibel 2009-09-17 19:23:33 UTC
same issue on i386-pc-solaris2.10!!!
will upload libC.so.3 and libCso.5 from solaris10/x86.
===
$ uname -a
SunOS x4150-1 5.10 Generic_139556-08 i86pc i386 i86pc Solaris
$ size /usr/lib/libC.so.*
   text    data     bss     dec     hex filename
  63629    5448    1049   70126   111ee /usr/lib/libC.so.3
size: /usr/lib/libC.so.5: File format not recognized
===
Comment 19 Niki W. Waibel 2009-09-17 19:27:00 UTC
Created attachment 4201 [details]
sol10/x86 okay
Comment 20 Niki W. Waibel 2009-09-17 19:30:40 UTC
Created attachment 4202 [details]
sol10/x86 not okay
Comment 21 Niki W. Waibel 2009-09-21 15:52:27 UTC
any idea related to this? it should be as easy to fix as for solaris/sparc ...
Comment 22 Nick Clifton 2009-09-25 10:31:11 UTC
Created attachment 4229 [details]
Update previous patch to work with x86 binaries
Comment 23 Nick Clifton 2009-09-25 10:32:42 UTC
Hi Niki,

  OK, please try out the newly uploaded patch (pr10478.x86.patch).  This should
be applied on top of the previously patched sources, if you are using them, or
just straight to the current mainline binutils code if you are using that.

  Let me know how it goes.

Cheers
  Nick
Comment 24 Niki W. Waibel 2009-09-26 00:35:56 UTC
looks very good now:
===
$ uname -a
SunOS x4150-1.dev.nlatlu.wipro.com 5.10 Generic_139556-08 i86pc i386 i86pc Solaris
$ size /usr/lib/libC.so.*
   text    data     bss     dec     hex filename
  63629    5448    1049   70126   111ee /usr/lib/libC.so.3
 304158   52968   59804  416930   65ca2 /usr/lib/libC.so.5
===
Comment 25 Sourceware Commits 2009-09-28 09:45:47 UTC
Subject: Bug 10478

CVSROOT:	/cvs/src
Module name:	src
Changes by:	nickc@sourceware.org	2009-09-28 09:45:33

Modified files:
	bfd            : ChangeLog elf.c elfcode.h 
	binutils       : ChangeLog readelf.c 

Log message:
	PR 10478
	* elf.c (bfd_section_from_shdr): Allow SHN_BEFORE and SHN_AFTER
	section link values in x86 binaries.
	* elfcode.h (elf_object_p): Likewise.
	* readelf.c (get_elf_section_flags): Allow SHN_BEFORE and
	SHN_AFTER section link values in x86 binaries.
	(process_section_headers): Likewise.

Patches:
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/bfd/ChangeLog.diff?cvsroot=src&r1=1.4797&r2=1.4798
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/bfd/elf.c.diff?cvsroot=src&r1=1.492&r2=1.493
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/bfd/elfcode.h.diff?cvsroot=src&r1=1.97&r2=1.98
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/binutils/ChangeLog.diff?cvsroot=src&r1=1.1540&r2=1.1541
http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/binutils/readelf.c.diff?cvsroot=src&r1=1.463&r2=1.464

Comment 26 Nick Clifton 2009-09-28 09:46:29 UTC
Hi Niki,

  Great - I have checked the patch in along with these changelog entries.

Cheers
  Nick

bfd/ChangeLog
	PR 10478
	* elf.c (bfd_section_from_shdr): Allow SHN_BEFORE and SHN_AFTER
	section link values in x86 binaries.
	* elfcode.h (elf_object_p): Likewise.

binutils/ChangeLog
	PR 10478
	* readelf.c (get_elf_section_flags): Allow SHN_BEFORE and
	SHN_AFTER section link values in x86 binaries.
	(process_section_headers): Likewise.