When I compile wxWidgets 2.8.12 into single library, ld use huge memory, I have to use 64bit Linux to do cross compile: CFLAGS='-pipe' CXXFLAGS='-pipe' LDFLAGS='-pipe -s -static' \ ${HOME}/vcs/svn/wxWidgets/branches/WX_2_8_BRANCH/configure --prefix=/tmp/w32 \ --host=i686-w64-mingw32 --with-odbc --with-opengl --enable-shared \ --enable-unicode --enable-mediactrl --enable-official_build \ --disable-compat26 --enable-abi-incompatible-features --enable-monolithic When make finished, 454 object files generate one 12M file wxmsw28u_gcc.dll. During the building, after ld running 25 minutes, the VM physical memory (3GB) was exhausted, the host have 4 Xeon 3.20GHz CPU, and the other VMs were idle. Maybe there have memory leaks in ld ? *) 4 minutes Cpu(s): 18.2%us, 5.9%sy, 0.0%ni, 75.1%id, 0.8%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 3095296k total, 1354524k used, 1740772k free, 420k buffers Swap: 7811064k total, 0k used, 7811064k free, 284988k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1110 dongshen 20 0 989m 983m 892 R 96 32.5 3:50.05 ld *) 8 minutes Cpu(s): 20.8%us, 4.5%sy, 0.0%ni, 73.9%id, 0.7%wa, 0.0%hi, 0.2%si, 0.0%st Mem: 3095296k total, 2531320k used, 563976k free, 420k buffers Swap: 7811064k total, 0k used, 7811064k free, 500396k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1110 dongshen 20 0 1926m 1.9g 892 R 97 63.5 8:01.56 ld *) 18 minutes Cpu(s): 19.1%us, 7.1%sy, 0.0%ni, 73.5%id, 0.0%wa, 0.0%hi, 0.2%si, 0.0%st Mem: 3095296k total, 3072352k used, 22944k free, 32k buffers Swap: 7811064k total, 0k used, 7811064k free, 290136k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1110 dongshen 20 0 2658m 2.6g 892 R 99 87.8 18:02.34 ld *) 25 minutes Cpu(s): 22.9%us, 16.6%sy, 0.0%ni, 28.7%id, 29.4%wa, 0.2%hi, 2.2%si, 0.0%st Mem: 3095296k total, 3075624k used, 19672k free, 16k buffers Swap: 7811064k total, 201056k used, 7610008k free, 11748k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1110 dongshen 20 0 3086m 2.8g 444 R 99 95.9 25:00.40 ld
Now, ld running 6 hours cpu time, use 4.6GB memory: VmPeak: 4776608 kB VmSize: 4734656 kB VmLck: 0 kB VmHWM: 3020260 kB VmRSS: 2982304 kB VmData: 4728000 kB VmStk: 100 kB VmExe: 912 kB VmLib: 1504 kB VmPTE: 9232 kB Threads: 1 Cpu(s): 7.9%us, 11.5%sy, 0.0%ni, 44.4%id, 33.3%wa, 0.2%hi, 2.7%si, 0.0%st Mem: 3095296k total, 3073516k used, 21780k free, 16k buffers Swap: 7811064k total, 3905532k used, 3905532k free, 9484k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1110 dongshen 20 0 4623m 2.8g 196 D 49 96.1 358:15.13 ld root@x64-ora-01:/home/dongsheng/gcc-4.6-windows-linux# ./bin/i686-w64-mingw32-g++ -v Using built-in specs. COLLECT_GCC=./bin/i686-w64-mingw32-g++ COLLECT_LTO_WRAPPER=/home/dongsheng/gcc-4.6-windows-linux/libexec/gcc/i686-w64-mingw32/4.6.1/lto-wrapper Target: i686-w64-mingw32 Configured with: /home/dongsheng/vcs/svn/gcc/branches/gcc-4_6-branch/configure --prefix=/home/dongsheng/gcc-4.6-windows-linux --with-sysroot=/home/dongsheng/gcc-4.6-windows-linux --target=i686-w64-mingw32 --disable-multilib --enable-languages=c,c++ Thread model: win32 gcc version 4.6.1 20110408 (prerelease) (GCC) root@x64-ora-01:/home/dongsheng/gcc-4.6-windows-linux# ./i686-w64-mingw32/bin/ld --version GNU ld (GNU Binutils) 2.21.51.20110408 Copyright 2011 Free Software Foundation, Inc. This program is free software; you may redistribute it under the terms of the GNU General Public License version 3 or (at your option) a later version. This program has absolutely no warranty.
Hi Dongsheng, Have you tried linking with the --reduce-memory-overheads option specified ? It is entirely possible that there are memory leaks in the linker, but without a lot of analysis it is hard to say where they might be. It does sound as though you are producing a really big DLL, so maybe it is just a case that you need lots of memory to perform the link. Have you tried using earlier versions of the linker to see if this problem is something that has only recently been introduced ? Cheers Nick
(In reply to comment #2) > Hi Dongsheng, > > Have you tried linking with the --reduce-memory-overheads option specified ? > > It is entirely possible that there are memory leaks in the linker, but > without a lot of analysis it is hard to say where they might be. > > It does sound as though you are producing a really big DLL, so maybe it is > just a case that you need lots of memory to perform the link. Have you tried > using earlier versions of the linker to see if this problem is something that > has only recently been introduced ? > > Cheers > Nick When I use '-Wl,--no-keep-memory,--reduce-memory-overheads', only reduce about 200MB memory usage. I'm not very care about the memory usage, I'm more concern the infinite loop during link because it blocked from generate the final .DLL file. *) 25 minutes Cpu(s): 16.2%us, 8.0%sy, 0.0%ni, 75.0%id, 0.0%wa, 0.0%hi, 0.7%si, 0.0%st Mem: 3095296k total, 3073056k used, 22240k free, 32k buffers Swap: 7811064k total, 0k used, 7811064k free, 321800k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1159 dongshen 20 0 2627m 2.6g 896 R 104 86.7 25:35.69 ld
Hi Dongsheg, Ok - we are going to need a testcase that reproduced this problem then. I appreciate that this might be difficult as I assume that you will not be able to reduce the size/number of object files involved. If you can place a compressed tarball somewhere where I can download it and give me a full linker command line, I will try reproducing the problem myself. Cheers Nick
Created attachment 5661 [details] Compile gcc 4.6 for mingw-w64 target
(In reply to comment #5) > Created attachment 5661 [details] > Compile gcc 4.6 for mingw-w64 target Sorry, the working directory size is 1.2GB, it's too hard to upload. Maybe you can build yourself: 1) build mingw-w64 cross compiler If you use Debian 6.0 i686 or amd64 box, I can upload the compiler. See gcc-4.6-windows-x86.sh for more details. 2) Checkout wxWidgets 2.8 branch (http://svn.wxwidgets.org/svn/wx/wxWidgets/branches/WX_2_8_BRANCH) 3) Configure CFLAGS='-pipe' CXXFLAGS='-pipe' LDFLAGS='-pipe -s -static' \ ${HOME}/vcs/svn/wxWidgets/branches/WX_2_8_BRANCH/configure --prefix=/tmp/w32 --host=i686-w64-mingw32 \ --with-odbc --with-opengl --enable-shared --enable-unicode --enable-mediactrl --enable-official_build --disable-compat26 --enable-abi-incompatible-features ... Configured wxWidgets 2.8.12 for `i686-w64-mingw32' Which GUI toolkit should wxWidgets use? msw Should wxWidgets be compiled into single library? no Should wxWidgets be compiled in debug mode? no Should wxWidgets be linked as a shared library? yes Should wxWidgets be compiled in Unicode mode? yes What level of wxWidgets compatibility should be enabled? wxWidgets 2.4 no wxWidgets 2.6 no Which libraries should wxWidgets use? jpeg builtin png builtin regex builtin tiff builtin zlib builtin odbc yes expat builtin libmspack no sdl no 4) build make -j2
Hi Dongsheng, I am sorry - I am currently unable to build the mingw-w64 cross compiler using the gcc-4.6-windows-x86.sh script. It fails building the CRT with: configure: error: Please check if the mingw-w64 header set and the build/host option are set properly. make: *** No targets specified and no makefile found. Stop. [2011-04-12 10:03:38] Build Mingw-w64 crt failed I suspected that something like this might happen. :-( Are you able to run the linker under GDB and maybe find out where the infinite loop is happening ? Cheers Nick
(In reply to comment #7) > Hi Dongsheng, > > I am sorry - I am currently unable to build the mingw-w64 cross compiler > using the gcc-4.6-windows-x86.sh script. It fails building the CRT with: > > configure: error: Please check if the mingw-w64 header set and the build/host > option are set properly. > make: *** No targets specified and no makefile found. Stop. > [2011-04-12 10:03:38] Build Mingw-w64 crt failed > > I suspected that something like this might happen. :-( > > Are you able to run the linker under GDB and maybe find out where the infinite > loop is happening ? > > Cheers > Nick I have no enough to run the gdb to debug ld now. I can not know why your build failed without the exact error messages, but I think this may be caused by your Linux not have the proper version of gmp, mpfr and mpc develop packages. Packages necessary for building GCC 4.6: http://gcc.gnu.org/install/prerequisites.html GMP 4.3.2 (or later) MPFR 2.4.2 (or later) MPC 0.8.1 (or later) FC14: http://download.fedora.redhat.com/pub/fedora/linux/releases/14/Fedora/source/SRPMS/ gmp-4.3.1-7.fc14.src.rpm mpfr-2.4.2-1.fc13.src.rpm libmpc-0.8.1-1.fc13.src.rpm FC15: http://download.fedora.redhat.com/pub/fedora/linux/releases/test/15-Alpha/Fedora/source/SRPMS/ gmp-4.3.2-3.fc15.src.rpm mpfr-3.0.0-4.fc15.src.rpm libmpc-0.8.3-0.3.svn855.fc15.src.rpm RHEL6: http://ftp.scientificlinux.org/linux/scientific/6/SRPMS/vendor/ gmp-4.3.1-7.el6.src.rpm mpfr-2.4.1-6.el6.src.rpm If the answer is yes, then you must use FC15 or upgrade your gmp, mpfr and mpc develop packages yourself.
CVSROOT: /cvs/src Module name: src Changes by: ktietz@sourceware.org 2011-04-13 12:53:36 Modified files: ld : ChangeLog deffile.h deffilep.y pe-dll.c Log message: 2011-04-13 Kai Tietz <ktietz@redhat.com> PR binutils/12658 * deffile.h (def_file_add_export): Add is_dup argument. (def_file_add_import): Likewise. * deffilep.y (are_names_equal): New helper. (cmp_export_elem): New helper. (find_export_in_list): Add search routine for exports. (def_file_add_export): Check for duplicates. (cmp_import_elem): New helper. (find_import_in_list): Add search routine for imports. (def_file_add_import): Check for duplicates. (def_exports): Handle duplicates. (def_imports): Likewise. * pe-dll.c (process_def_file_and_drectve): Likewise. (pe_implied_import_dll): Likewise. Patches: http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/ChangeLog.diff?cvsroot=src&r1=1.2310&r2=1.2311 http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/deffile.h.diff?cvsroot=src&r1=1.14&r2=1.15 http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/deffilep.y.diff?cvsroot=src&r1=1.33&r2=1.34 http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/pe-dll.c.diff?cvsroot=src&r1=1.135&r2=1.136
Hi Dongsheng, Please could you try the latest FSF mainline sources with Kai's patch in them, and see if this has any affect on your problem ? Cheers Nick
(In reply to comment #10) > Hi Dongsheng, > > Please could you try the latest FSF mainline sources with Kai's patch in them, > and see if this has any affect on your problem ? > > Cheers > Nick In 32bit Linux, still out of memory: /home/dongsheng/gcc-4.6-windows-linux/lib/gcc/i686-w64-mingw32/4.6.1/../../../../i686-w64-mingw32/bin/ld: out of memory allocating 82956108 bytes after a total of 2268610560 bytes collect2: ld returned 1 exit status make: *** [/home/dongsheng/tmp/wx/lib/wxmsw28u_gcc.dll] Error 1 PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4039 oracle 20 0 3024m 2.9g 724 R 100 89.6 18:50.58 ld VmPeak: 3097084 kB VmSize: 3097084 kB VmLck: 0 kB VmHWM: 3041600 kB VmRSS: 3022700 kB VmData: 3094432 kB VmStk: 100 kB VmExe: 1036 kB VmLib: 1472 kB VmPTE: 3036 kB
(In reply to comment #11) > (In reply to comment #10) > > Hi Dongsheng, > > > > Please could you try the latest FSF mainline sources with Kai's patch in them, > > and see if this has any affect on your problem ? > > > > Cheers > > Nick > > In 32bit Linux, still out of memory: It seems that Kai's patch save about 300M+ memory on 64 bit Linux box (My new Scientific Linux 6 VM, 1G physical memory). But it seems that still do a infinite loop: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ DATA COMMAND 15602 dongshen 20 0 4246m 768m 144 D 4.3 77.0 47:49.43 4.1g ld VmPeak: 4348140 kB VmSize: 4348140 kB VmLck: 0 kB VmHWM: 892440 kB VmRSS: 787316 kB VmData: 4341348 kB VmStk: 100 kB VmExe: 928 kB VmLib: 1620 kB VmPTE: 8508 kB VmSwap: 3554152 kB # pmap -d 15602 15602: /home/dongsheng/gcc-4.6-windows-linux/lib/gcc/i686-w64-mingw32/4.6.1/../../../../i686-w64-mingw32/bin/ld --sysroot=/home/dongsheng/gcc-4.6-windows-linux -m i386pe --shared -Bstatic -e _DllMainCRTStartup@12 --enable-auto-image-base -o /home/dongsheng/tmp/wx/lib/wxmsw28u_gcc.dll -s -s /home/dongsheng/gcc-4.6-windows-linux/lib/gcc/i686-w64-mingw32/4.6.1/../../../../i686-w64-mingw32/lib/../lib/dllcrt2.o /home/dongsheng/gcc-4.6-windows-linux/lib/gcc/i686-w64-mingw32/4.6.1/crtbegin.o -L/home/dongsheng/tmp/wx/lib Address Kbytes Mode Offset Device Mapping 0000000000400000 928 r-x-- 0000000000000000 0fd:00000 ld 00000000006e7000 12 rw--- 00000000000e7000 0fd:00000 ld 00000000006ea000 20 rw--- 0000000000000000 000:00000 [ anon ] 0000000000801000 3494196 rw--- 0000000000000000 000:00000 [ anon ] 00007f016d265000 1052 rw--- 0000000000000000 000:00000 [ anon ] 00007f016d38c000 835028 rw--- 0000000000000000 000:00000 [ anon ] 00007f01a0302000 1832 rw--- 0000000000000000 000:00000 [ anon ] 00007f01a04cd000 1816 rw--- 0000000000000000 000:00000 [ anon ] 00007f01a0694000 1848 rw--- 0000000000000000 000:00000 [ anon ] 00007f01a0863000 1848 rw--- 0000000000000000 000:00000 [ anon ] 00007f01a0a32000 1908 rw--- 0000000000000000 000:00000 [ anon ] 00007f01a0c10000 420 rw--- 0000000000000000 000:00000 [ anon ] 00007f01a0c79000 1492 r-x-- 0000000000000000 0fd:00000 libc-2.12.so 00007f01a0dee000 2048 ----- 0000000000175000 0fd:00000 libc-2.12.so 00007f01a0fee000 16 r---- 0000000000175000 0fd:00000 libc-2.12.so 00007f01a0ff2000 4 rw--- 0000000000179000 0fd:00000 libc-2.12.so 00007f01a0ff3000 20 rw--- 0000000000000000 000:00000 [ anon ] 00007f01a0ff8000 8 r-x-- 0000000000000000 0fd:00000 libdl-2.12.so 00007f01a0ffa000 2048 ----- 0000000000002000 0fd:00000 libdl-2.12.so 00007f01a11fa000 4 r---- 0000000000002000 0fd:00000 libdl-2.12.so 00007f01a11fb000 4 rw--- 0000000000003000 0fd:00000 libdl-2.12.so 00007f01a11fc000 120 r-x-- 0000000000000000 0fd:00000 ld-2.12.so 00007f01a1225000 20 rw--- 0000000000000000 000:00000 [ anon ] 00007f01a126a000 1724 rw--- 0000000000000000 000:00000 [ anon ] 00007f01a1419000 4 rw--- 0000000000000000 000:00000 [ anon ] 00007f01a141a000 4 r---- 000000000001e000 0fd:00000 ld-2.12.so 00007f01a141b000 4 rw--- 000000000001f000 0fd:00000 ld-2.12.so 00007f01a141c000 4 rw--- 0000000000000000 000:00000 [ anon ] 00007fffc12ad000 96 rw--- 0000000000000000 000:00000 [ stack ] 00007fffc13c4000 4 r-x-- 0000000000000000 000:00000 [ anon ] ffffffffff600000 4 r-x-- 0000000000000000 000:00000 [ anon ] mapped: 4348536K writeable/private: 4341860K shared: 0K
Here is the stack trace (After ld running 8 minutes, use 1.4GB memory): # gdb /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld 28412 GNU gdb (GDB) Red Hat Enterprise Linux (7.1-29.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld...done. Attaching to program: /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld, process 28412 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 _bfd_pei_swap_sym_in (abfd=0x4c5eda20, ext1=0x5474af34, in1=0x7fff02619200) at peigen.c:115 115 in->n_type = H_GET_16 (abfd, ext->e_type); Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.7.el6_0.5.x86_64 (gdb) bt #0 _bfd_pei_swap_sym_in (abfd=0x4c5eda20, ext1=0x5474af34, in1=0x7fff02619200) at peigen.c:115 #1 0x0000000000446106 in handle_COMDAT (abfd=0x4c5eda20, hdr=<value optimized out>, name=<value optimized out>, section=0x55422a28, flags_ptr=0x7fff02619ebc) at /home/dongsheng/vcs/git/binutils/bfd/coffcode.h:897 #2 styp_to_sec_flags (abfd=0x4c5eda20, hdr=<value optimized out>, name=<value optimized out>, section=0x55422a28, flags_ptr=0x7fff02619ebc) at /home/dongsheng/vcs/git/binutils/bfd/coffcode.h:1253 #3 0x0000000000431c0a in make_a_section_from_file (abfd=0x4c5eda20) at /home/dongsheng/vcs/git/binutils/bfd/coffgen.c:132 #4 coff_real_object_p (abfd=0x4c5eda20) at /home/dongsheng/vcs/git/binutils/bfd/coffgen.c:222 #5 coff_object_p (abfd=0x4c5eda20) at /home/dongsheng/vcs/git/binutils/bfd/coffgen.c:301 #6 0x00000000004320ce in bfd_check_format_matches (abfd=0x4c5eda20, format=<value optimized out>, matching=0x0) at /home/dongsheng/vcs/git/binutils/bfd/format.c:211 #7 0x000000000041b9df in ldfile_try_open_bfd (attempt=0x7fff0261c97b "monodll_msw_notebook.o", entry=0x160ea90) at /home/dongsheng/vcs/git/binutils/ld/ldfile.c:178 #8 0x000000000041c10c in ldfile_open_file (entry=0x160ea90) at /home/dongsheng/vcs/git/binutils/ld/ldfile.c:443 #9 0x0000000000410465 in load_symbols (entry=0x160ea90, place=0x7fff0261a040) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:2678 #10 0x0000000000410e52 in open_input_bfds (s=0x160ea90, force=0) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:3239 #11 0x000000000041215a in lang_process () at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:6471 #12 0x00000000004168e6 in main (argc=536, argv=0x7fff0261a288) at /home/dongsheng/vcs/git/binutils/ld/ldmain.c:461 (gdb) q A debugging session is active. Inferior 1 [process 28412] will be detached. Quit anyway? (y or n) y Detaching from program: /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld, process 28412
Here is the updated stack trace (After ld running 25 minutes, use 3.6GB memory): PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ DATA COMMAND 28412 dongshen 20 0 3685m 860m 228 D 22.9 86.3 25:03.09 3.6g ld # gdb /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld 28412 GNU gdb (GDB) Red Hat Enterprise Linux (7.1-29.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld...done. Attaching to program: /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld, process 28412 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 0x00000000004092e3 in match_simple_wild (pattern=0x161aa90 ".gnu.lto_*", name=0xbbcc5538 ".text$_ZNK12wxWindowBase9IsEnabledEv") at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:368 368 if (pattern[0] != name[0] || pattern[1] != name[1] Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.7.el6_0.5.x86_64 (gdb) bt #0 0x00000000004092e3 in match_simple_wild (pattern=0x161aa90 ".gnu.lto_*", name=0xbbcc5538 ".text$_ZNK12wxWindowBase9IsEnabledEv") at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:368 #1 0x000000000040b2d5 in walk_wild_section_specs1_wild1 (ptr=0x161a7a0, file=0x1613260, callback=0x409960 <gc_section_callback>, data=0x0) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:591 #2 0x0000000000410a11 in walk_wild (s=0x161a7a0, callback=0x409960 <gc_section_callback>, data=0x0) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:908 #3 0x0000000000410b67 in lang_gc_sections_1 (s=0x161a7a0) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:6241 #4 0x0000000000410b1d in lang_gc_sections_1 (s=0x1605cb0) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:6250 #5 0x0000000000412348 in lang_gc_sections () at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:6263 #6 lang_process () at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:6549 #7 0x00000000004168e6 in main (argc=536, argv=0x7fff0261a288) at /home/dongsheng/vcs/git/binutils/ld/ldmain.c:461 (gdb) q A debugging session is active. Inferior 1 [process 28412] will be detached. Quit anyway? (y or n) y Detaching from program: /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld, process 28412
Here is the updated stack trace (After ld running CPU time 36 minutes, use 3.7GB memory): PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ DATA COMMAND 28412 dongshen 20 0 3714m 844m 248 R 7.0 84.7 36:20.98 3.6g ld VmPeak: 3803368 kB VmSize: 3803368 kB VmLck: 0 kB VmHWM: 892824 kB VmRSS: 860116 kB VmData: 3796576 kB VmStk: 100 kB VmExe: 928 kB VmLib: 1620 kB VmPTE: 7448 kB VmSwap: 2936724 kB # gdb /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld 28412 GNU gdb (GDB) Red Hat Enterprise Linux (7.1-29.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld...done. Attaching to program: /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld, process 28412 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 0x000000000040bf93 in compare_section (sort=<value optimized out>, asec=0x52e6b5e8, bsec=0x242e1758) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:454 454 ret = strcmp (bfd_get_section_name (asec->owner, asec), Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.7.el6_0.5.x86_64 (gdb) bt #0 0x000000000040bf93 in compare_section (sort=<value optimized out>, asec=0x52e6b5e8, bsec=0x242e1758) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:454 #1 0x000000000040c0c7 in wild_sort_fast (ptr=0x1618140, sec=0x1618410, section=0x52e6b5e8, file=<value optimized out>, output=<value optimized out>) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:499 #2 output_section_callback_fast (ptr=0x1618140, sec=0x1618410, section=0x52e6b5e8, file=<value optimized out>, output=<value optimized out>) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:531 #3 0x000000000040b2f0 in walk_wild_section_specs1_wild1 (ptr=0x1618140, file=0x160e8d0, callback=0x40bff0 <output_section_callback_fast>, data=0x16057e0) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:594 #4 0x0000000000410a11 in walk_wild (s=0x1618140, callback=0x40bff0 <output_section_callback_fast>, data=0x16057e0) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:908 #5 0x000000000041208d in wild (s=0x1618140, target=0x0, os=0x16057e0) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:2835 #6 map_input_to_output_sections (s=0x1618140, target=0x0, os=0x16057e0) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:3533 #7 0x0000000000412057 in map_input_to_output_sections (s=0x16057e0, target=0x0, os=0x0) at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:3555 #8 0x0000000000412451 in lang_process () at /home/dongsheng/vcs/git/binutils/ld/ldlang.c:6560 #9 0x00000000004168e6 in main (argc=536, argv=0x7fff0261a288) at /home/dongsheng/vcs/git/binutils/ld/ldmain.c:461 (gdb) q A debugging session is active. Inferior 1 [process 28412] will be detached. Quit anyway? (y or n) y Detaching from program: /home/dongsheng/gcc-4.6-windows-linux/i686-w64-mingw32/bin/ld, process 28412
Ho Dongsheng, Hmm, I am not sure what else to suggest. You could try turning off section garbage collection (--no-gc-sections) as this appears to be turned on. If you really have 1.2 Gb of object files then I am not sure if you can reasonably expect the linker to be anything but slow when you only 1G of real memory available. I would strongly urge trying the link on a bigger machine. Cheers Nick
(In reply to comment #16) > Ho Dongsheng, > > Hmm, I am not sure what else to suggest. You could try turning off section > garbage collection (--no-gc-sections) as this appears to be turned on. > > If you really have 1.2 Gb of object files then I am not sure if you can > reasonably expect the linker to be anything but slow when you only 1G of real > memory available. I would strongly urge trying the link on a bigger machine. > > Cheers > Nick I do not care about the real time, I'm only aware the CPU (user + sys) time. I don not think when I link object files to multiple .DLL files only use no more than 10 minutes, but link these objects files to single .DLL file not finished after 6 hours CPU time is acceptable.
If I link wxWidgets 2.8 into multiple .DLL files, the link speed is very fast, here is the details (the last 2 columns were reported by ld '--stats'): DLL SIZE FILE NAME LD TIME LD DATA SIZE 1991K wxbase28u_gcc.dll 42.56s 524M 214K wxbase28u_net_gcc.dll 0.29s 49M 194K wxbase28u_odbc_gcc.dll 0.20s 32M 244K wxbase28u_xml_gcc.dll 0.15s 18M 831K wxmsw28u_adv_gcc.dll 1.06s 102M 481K wxmsw28u_aui_gcc.dll 0.62s 82M 5028K wxmsw28u_core_gcc.dll 349.60s 1694M 109K wxmsw28u_dbgrid_gcc.dll 0.18s 39M 108K wxmsw28u_gl_gcc.dll 0.22s 44M 632K wxmsw28u_html_gcc.dll 0.95s 78M 190K wxmsw28u_media_gcc.dll 0.29s 32M 166K wxmsw28u_qa_gcc.dll 0.24s 28M 1226K wxmsw28u_richtext_gcc.dll 1.64s 102M 571K wxmsw28u_xrc_gcc.dll 0.68s 60M 11740K - 398.68s 2884M The total time is about 400s, why when they link into single .DLL file use so much memory (4.6GB) ? and not finished after 6 hours CPU (sys + user) time ?
(In reply to comment #18) > If I link wxWidgets 2.8 into multiple .DLL files, the link speed is very fast, > here is the details (the last 2 columns were reported by ld '--stats'): > > DLL SIZE FILE NAME LD TIME LD DATA SIZE > 1991K wxbase28u_gcc.dll 42.56s 524M > 214K wxbase28u_net_gcc.dll 0.29s 49M > 194K wxbase28u_odbc_gcc.dll 0.20s 32M > 244K wxbase28u_xml_gcc.dll 0.15s 18M > 831K wxmsw28u_adv_gcc.dll 1.06s 102M > 481K wxmsw28u_aui_gcc.dll 0.62s 82M > 5028K wxmsw28u_core_gcc.dll 349.60s 1694M > 109K wxmsw28u_dbgrid_gcc.dll 0.18s 39M > 108K wxmsw28u_gl_gcc.dll 0.22s 44M > 632K wxmsw28u_html_gcc.dll 0.95s 78M > 190K wxmsw28u_media_gcc.dll 0.29s 32M > 166K wxmsw28u_qa_gcc.dll 0.24s 28M > 1226K wxmsw28u_richtext_gcc.dll 1.64s 102M > 571K wxmsw28u_xrc_gcc.dll 0.68s 60M > > 11740K - 398.68s 2884M > > The total time is about 400s, why when they link into single .DLL file use so > much memory (4.6GB) ? and not finished after 6 hours CPU (sys + user) time ? PS: I checked the total object files, they are 999,944KB.
One question: when memory exhausted during linking, can it use swap file instead? I means to use hardisk to compensate the lack of memory space. Here it doesn't care about the linking speed. It on earth the memory is not cheap. Thanks -Sunworld
Hi Sunworld, > One question: when memory exhausted during linking, can it use swap file > instead? No and yes ... "no" as in, "no the linker cannot be changed to use the swap file when it runs out of memory", but "yes" as in "yes, the swap file will be used if there is insufficient memory available". Ie using the swap file is not a feature that needs to be added to the linker. Rather it is a feature that is already present in the operating system and one which is used automatically. Cheers Nick
Hi Dongshen > The total time is about 400s, why when they link into single .DLL file use so > much memory (4.6GB) ? and not finished after 6 hours CPU (sys + user) time ? I do not know, and without a way to reproduce the problem myself, I cannot find out. Sorry, but all I can suggest is that you either track down where the memory is being used up yourself, or else try using a machine with more physical memory on which to link the DLL. Cheers Nick
Hmm, does the gcc option '-fno-keep-inline-dllexport' help here?
Me helped flag --reduce-memory-overheads
Not current