Bug 24000 - couple of ELF compression testsuite fails with uclibc library
Summary: couple of ELF compression testsuite fails with uclibc library
Status: RESOLVED FIXED
Alias: None
Product: elfutils
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Mark Wielaard
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-12-17 14:12 UTC by selva
Modified: 2019-02-20 06:04 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed: 2019-02-18 00:00:00


Attachments
Attached nm.o file (45.50 KB, application/x-object)
2019-01-07 08:30 UTC, selva
Details
Uclibc full testsuite log (1.73 KB, text/plain)
2019-01-19 08:30 UTC, selva
Details
config.log (57.61 KB, text/x-log)
2019-02-01 09:35 UTC, selva
Details
Handle malloc(0) specially when decompressing (887 bytes, patch)
2019-02-18 18:51 UTC, Mark Wielaard
Details | Diff
Handle malloc(0) specially when decompressing (1.26 KB, patch)
2019-02-18 21:44 UTC, Mark Wielaard
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description selva 2018-12-17 14:12:54 UTC
When elfutils is compiled against uclibc i got the below memory exhaust with eu-elflint

eu-elflint src/nm.o 
invalid machine flags: 0x5000000
eu-elflint: memory exhausted
Comment 1 Mark Wielaard 2019-01-06 21:06:33 UTC
Could you attach the src/nm.o file?
Comment 2 selva 2019-01-07 08:30:22 UTC
Created attachment 11512 [details]
Attached nm.o file
Comment 3 Mark Wielaard 2019-01-13 21:22:35 UTC
Sorry, I have no idea what is going on.
It looks like a normal arm ELF file:

$ eu-readelf -h nm.o
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF32
  Data:                              2's complement, little endian
  Ident Version:                     1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              REL (Relocatable file)
  Machine:                           ARM
  Version:                           1 (current)
  Entry point address:               0
  Start of program headers:          0 (bytes into file)
  Start of section headers:          106424 (bytes into file)
  Flags:                             0x5000000
  Size of this header:               52 (bytes)
  Size of program header entries:    0 (bytes)
  Number of program headers entries: 0
  Size of section header entries:    40 (bytes)
  Number of section headers entries: 31
  Section header string table index: 28

0x5000000 (which your eu-elflint complains about) is EF_ARM_EABI_VER5, which the arm backend hook machine_flag_check knows about. So eu-elflint really should have no complaints:

$ eu-elflint nm.o 
No errors

Maybe your installation isn't able to correctly load the ebl backends?
Comment 4 selva 2019-01-19 08:30:44 UTC
Created attachment 11548 [details]
Uclibc full testsuite log

Attaching the full test suite log.
Comment 5 Mark Wielaard 2019-01-23 19:46:54 UTC
(In reply to selva from comment #4)
> Created attachment 11548 [details]
> Uclibc full testsuite log
> 
> Attaching the full test suite log.

BTW. It is easier to just attache the tests/test-suite.log.
But I am afraid there still isn't enough to figure out what is going wrong.
Maybe the config.log could show a bit more.

Please run one of the segfaulting tests under gdb and get a backtrace to show what is really going on.
Comment 6 selva 2019-02-01 09:31:22 UTC
(In reply to Mark Wielaard from comment #5)
> (In reply to selva from comment #4)
> > Created attachment 11548 [details]
> > Uclibc full testsuite log
> > 
> > Attaching the full test suite log.
> 
> BTW. It is easier to just attache the tests/test-suite.log.
> But I am afraid there still isn't enough to figure out what is going wrong.
> Maybe the config.log could show a bit more.
> 
> Please run one of the segfaulting tests under gdb and get a backtrace to
> show what is really going on.

Mmm., Tried to run the test with gdb but i get "no stack" Please share me the steps to get the backtrace if you have any.

# ./run-elfputzdata.sh 
Segmentation fault
# 
# gdb --args bash ./run-elfputzdata.sh 
GNU gdb (GDB) 7.10.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "armv7l-unkown-linux-uclibcgnueabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from bash...done.
(gdb) run
Starting program: /bin/bash ./run-elfputzdata.sh
/root/elfutils-0.173/tests/test-subr.sh: line 64:  3679 Segmentation fault      diff -u $outfile -
[Inferior 1 (process 3670) exited with code 0213]
(gdb) 
(gdb) backtrace 
No stack.
Comment 7 selva 2019-02-01 09:35:17 UTC
Created attachment 11582 [details]
config.log

Attached the config.log.
Comment 8 Mark Wielaard 2019-02-09 21:52:28 UTC
(In reply to selva from comment #6)
> (In reply to Mark Wielaard from comment #5)
> > (In reply to selva from comment #4)
> > > Created attachment 11548 [details]
> > > Uclibc full testsuite log
> > > 
> > > Attaching the full test suite log.
> > 
> > BTW. It is easier to just attache the tests/test-suite.log.
> > But I am afraid there still isn't enough to figure out what is going wrong.
> > Maybe the config.log could show a bit more.
> > 
> > Please run one of the segfaulting tests under gdb and get a backtrace to
> > show what is really going on.
> 
> Mmm., Tried to run the test with gdb but i get "no stack" Please share me
> the steps to get the backtrace if you have any.

Best to figure out which of the tests in run-elfputzdata.sh actually fails and then run that by hand on the failing test file (bunzip2ed first).

e.g.

cp tests/testfile4.bz2 .
bunzip2 testfile4.bz2
LD_LIBRARY_PATH=backends:libelf:libdw gdb --args tests/elfputzdata gnu testfile4
(gdb) run
Comment 9 Mark Wielaard 2019-02-09 22:04:23 UTC
(In reply to selva from comment #7)
> Created attachment 11582 [details]
> config.log
> 
> Attached the config.log.

>  $ ./configure --target=armv7l-unkown-linux-uclibcgnueabi --host=armv7l-unkown-linux-uclibcgnueabi --build=x86_64-linux-gnu --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libdir=/usr/lib --libexecdir=/usr/lib --sysconfdir=/etc --datadir=/usr/share --localstatedir=/var --mandir=/usr/share/man --infodir=/usr/share/info --includedir=/usr/include --disable-nls --disable-Werror --enable-largefile --with-biarch --disable-symbol-versioning

That looks like a cross build. It might work, but I never tried myself.
It might be easier to make sure a native build works first.

> =====================================================================
>         elfutils: 0.173 (eu_version: 173)
> =====================================================================

Please try with the latest release: 0.175 

>  RECOMMENDED FEATURES (should all be yes)
>    gzip support                       : yes
>    bzip2 support                      : yes
>    lzma/xz support                    : no
>    libstdc++ demangle support         : yes
>    File textrel check                 : yes
>    Symbol versioning                  : no

Please enable lzma/xz support by installing xz-devel.
Don't disable symbol versioning, without it binary compatibility is broken.
Comment 10 selva 2019-02-18 12:09:10 UTC
(In reply to Mark Wielaard from comment #8)
> (In reply to selva from comment #6)
> > (In reply to Mark Wielaard from comment #5)
> > > (In reply to selva from comment #4)
> > > > Created attachment 11548 [details]
> > > > Uclibc full testsuite log
> > > > 
> > > > Attaching the full test suite log.
> > > 
> > > BTW. It is easier to just attache the tests/test-suite.log.
> > > But I am afraid there still isn't enough to figure out what is going wrong.
> > > Maybe the config.log could show a bit more.
> > > 
> > > Please run one of the segfaulting tests under gdb and get a backtrace to
> > > show what is really going on.
> > 
> > Mmm., Tried to run the test with gdb but i get "no stack" Please share me
> > the steps to get the backtrace if you have any.
> 
> Best to figure out which of the tests in run-elfputzdata.sh actually fails
> and then run that by hand on the failing test file (bunzip2ed first).
> 
> e.g.
> 
> cp tests/testfile4.bz2 .
> bunzip2 testfile4.bz2
> LD_LIBRARY_PATH=backends:libelf:libdw gdb --args tests/elfputzdata gnu
> testfile4
> (gdb) run

Thanks a lot for sharing testing steps.
I got "out of memory" error for testfile4 only 
other tests like testfile12,testfileppc64,testfileppc32 are okay.


# cp tests/testfile4.bz2 ./
# bunzip2 testfile4.bz2 
# 
# LD_LIBRARY_PATH=backends:libelf:libdw gdb --args tests/elfputzdata gnu testfil
e4
GNU gdb (GDB) 7.10.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "armv7l-unkown-linux-uclibcgnueabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from tests/elfputzdata...done.
(gdb) run
Starting program: /root/elfutils-0.173/tests/elfputzdata gnu testfile4
Cannot compress 1 .interp
Cannot compress 2 .note.ABI-tag
Cannot compress 3 .hash
Cannot compress 4 .dynsym
Cannot compress 5 .dynstr
Cannot compress 6 .gnu.version
Cannot compress 7 .gnu.version_r
Cannot compress 8 .rel.got
Cannot compress 9 .rel.plt
Cannot compress 10 .init
Cannot compress 11 .plt
Cannot compress 12 .text
Cannot compress 13 .fini
Cannot compress 14 .rodata
Cannot compress 15 .data
Cannot compress 16 .eh_frame
Cannot compress 17 .gcc_except_table
Cannot compress 18 .ctors
Cannot compress 19 .dtors
Cannot compress 20 .got
Cannot compress 21 .dynamic
Lets compress 22 .sbss, size: 0
elf_[un]compress_gnu failed for section 22: out of memory
[Inferior 1 (process 5398) exited with code 0377]
(gdb) backtrace 
No stack.
(gdb) quit
# 
# 
# LD_LIBRARY_PATH=backends:libelf:libdw gdb --args tests/elfputzdata elf testfil
e4
GNU gdb (GDB) 7.10.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "armv7l-unkown-linux-uclibcgnueabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from tests/elfputzdata...done.
(gdb) run       
Starting program: /root/elfutils-0.173/tests/elfputzdata elf testfile4
Cannot compress 1 .interp
Cannot compress 2 .note.ABI-tag
Cannot compress 3 .hash
Cannot compress 4 .dynsym
Cannot compress 5 .dynstr
Cannot compress 6 .gnu.version
Cannot compress 7 .gnu.version_r
Cannot compress 8 .rel.got
Cannot compress 9 .rel.plt
Cannot compress 10 .init
Cannot compress 11 .plt
Cannot compress 12 .text
Cannot compress 13 .fini
Cannot compress 14 .rodata
Cannot compress 15 .data
Cannot compress 16 .eh_frame
Cannot compress 17 .gcc_except_table
Cannot compress 18 .ctors
Cannot compress 19 .dtors
Cannot compress 20 .got
Cannot compress 21 .dynamic
Lets compress 22 .sbss, size: 0
elf_[un]compress failed for section 22: out of memory
[Inferior 1 (process 5403) exited with code 0377]
(gdb) backtrace 
No stack.
(gdb) quit
#
Comment 11 selva 2019-02-18 12:15:58 UTC
(In reply to Mark Wielaard from comment #9)
> (In reply to selva from comment #7)
> > Created attachment 11582 [details]
> > config.log
> > 
> > Attached the config.log.
> 
> >  $ ./configure --target=armv7l-unkown-linux-uclibcgnueabi --host=armv7l-unkown-linux-uclibcgnueabi --build=x86_64-linux-gnu --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libdir=/usr/lib --libexecdir=/usr/lib --sysconfdir=/etc --datadir=/usr/share --localstatedir=/var --mandir=/usr/share/man --infodir=/usr/share/info --includedir=/usr/include --disable-nls --disable-Werror --enable-largefile --with-biarch --disable-symbol-versioning
> 
> That looks like a cross build. It might work, but I never tried myself.
> It might be easier to make sure a native build works first.
> 
> > =====================================================================
> >         elfutils: 0.173 (eu_version: 173)
> > =====================================================================
> 
> Please try with the latest release: 0.175 
> 
> >  RECOMMENDED FEATURES (should all be yes)
> >    gzip support                       : yes
> >    bzip2 support                      : yes
> >    lzma/xz support                    : no
> >    libstdc++ demangle support         : yes
> >    File textrel check                 : yes
> >    Symbol versioning                  : no
> 
> Please enable lzma/xz support by installing xz-devel.
> Don't disable symbol versioning, without it binary compatibility is broken.


As per the following commit disabled symbol versioning for uclibc
https://sourceware.org/git/?p=elfutils.git;a=commit;h=bafacacaf7659a4933604662daba26a480b29a8d

with uclibc,the test suite hangs in middle when compiled with Symbol versioning
Comment 12 Mark Wielaard 2019-02-18 18:51:24 UTC
Created attachment 11613 [details]
Handle malloc(0) specially when decompressing

(In reply to selva from comment #10)
> Lets compress 22 .sbss, size: 0
> elf_[un]compress failed for section 22: out of memory

That is interesting. It looks like your malloc implementation returns NULL when calling malloc (0). That seems valid, but somewhat unfortunate.

Could you try the following patch?
Comment 13 Mark Wielaard 2019-02-18 18:54:38 UTC
(In reply to selva from comment #11)
> (In reply to Mark Wielaard from comment #9)
> > Don't disable symbol versioning, without it binary compatibility is broken.
> 
> As per the following commit disabled symbol versioning for uclibc
> https://sourceware.org/git/?p=elfutils.git;a=commit;
> h=bafacacaf7659a4933604662daba26a480b29a8d
> 
> with uclibc,the test suite hangs in middle when compiled with Symbol
> versioning

hmmm. I think that commit might have been a mistake in hindsight.
But lets first see what happens with the special malloc (0) patch.
Comment 14 Mark Wielaard 2019-02-18 21:44:13 UTC
Created attachment 11614 [details]
Handle malloc(0) specially when decompressing

Sorry, that patch was clearly untested. This one is.
Comment 15 selva 2019-02-19 10:40:03 UTC
(In reply to Mark Wielaard from comment #14)
> Created attachment 11614 [details]
> Handle malloc(0) specially when decompressing
> 
> Sorry, that patch was clearly untested. This one is.

Thanks,With this patch test passed.

PASS: run-elfputzdata.sh
Comment 16 Mark Wielaard 2019-02-19 15:39:54 UTC
Thanks for testing. I pushed this:

commit ff7fe986679b0811591ef6e40cf485fa9f037e07
Author: Mark Wielaard <mark@klomp.org>
Date:   Tue Feb 19 16:27:40 2019 +0100

    libelf: Handle zero size decompressed data.
    
    This is a corner case that will most likely never occur in practice,
    but we have several testcases that compress and decompress zero sized
    data. In that case during decompression we might malloc a buffer of
    size zero. It is allowed for malloc to return NULL in that case. But
    we do need a non-NULL buffer to return and set as result. So make sure
    to always at least allocate one byte. Also make sure that we don't
    allocate a zero sized conversion buffer for the data.
    
    https://sourceware.org/bugzilla/show_bug.cgi?id=24000
    
    Signed-off-by: Mark Wielaard <mark@klomp.org>

Lets close this bug.

Could you please open a new bug in case you still see other testsuite failures?
Comment 17 selva 2019-02-20 06:04:01 UTC
(In reply to Mark Wielaard from comment #16)
> Thanks for testing. I pushed this:
> 
> commit ff7fe986679b0811591ef6e40cf485fa9f037e07
> Author: Mark Wielaard <mark@klomp.org>
> Date:   Tue Feb 19 16:27:40 2019 +0100
> 
>     libelf: Handle zero size decompressed data.
>     
>     This is a corner case that will most likely never occur in practice,
>     but we have several testcases that compress and decompress zero sized
>     data. In that case during decompression we might malloc a buffer of
>     size zero. It is allowed for malloc to return NULL in that case. But
>     we do need a non-NULL buffer to return and set as result. So make sure
>     to always at least allocate one byte. Also make sure that we don't
>     allocate a zero sized conversion buffer for the data.
>     
>     https://sourceware.org/bugzilla/show_bug.cgi?id=24000
>     
>     Signed-off-by: Mark Wielaard <mark@klomp.org>
> 
> Lets close this bug.
> 
> Could you please open a new bug in case you still see other testsuite
> failures?

Thanks, opened new one.
https://sourceware.org/bugzilla/show_bug.cgi?id=24240