Summary: | cross-compiling for ARM target fails at linking ld.so.new if THUMB used | ||
---|---|---|---|
Product: | glibc | Reporter: | Piotr Oniszczuk <piotr.oniszczuk> |
Component: | libc | Assignee: | Not yet assigned to anyone <unassigned> |
Status: | UNCONFIRMED --- | ||
Severity: | normal | CC: | drepper.fsp |
Priority: | P2 | ||
Version: | 2.31 | ||
Target Milestone: | --- | ||
Host: | Target: | ||
Build: | Last reconfirmed: | ||
Attachments: | glibc config.log from failed arm thumb build |
Look at the command line used to build libc-do-syscall.S, the exact macros defined at various points when building it, etc. - that's where __libc_do_syscall should be defined, if the compiler defined __thumb__ when building that file. If the objects built from libc-do-syscall.S do define __libc_do_syscall, you instead have some issue with how objects were selected to go into ld.so. @joseph, I'm afraid Your proposal exceeds my skills with reading glibc code so starting will need some more guidance.... here is one from many builds of libc-do-syscall.S: arm-minimyth-linux-gnueabi-gcc ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S -c -I../include -I/home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/nptl -I/home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build -I../sysdeps/unix/sysv/linux/arm/le -I../sysdeps/unix/sysv/linux/arm -I../sysdeps/arm/nptl -I../sysdeps/unix/sysv/linux/include -I../sysdeps/unix/sysv/linux -I../sysdeps/nptl -I../sysdeps/pthread -I../sysdeps/gnu -I../sysdeps/unix/inet -I../sysdeps/unix/sysv -I../sysdeps/unix/arm -I../sysdeps/unix -I../sysdeps/posix -I../sysdeps/arm/le -I../sysdeps/arm/include -I../sysdeps/arm -I../sysdeps/wordsize-32 -I../sysdeps/ieee754/flt-32 -I../sysdeps/ieee754/dbl-64 -I../sysdeps/arm/nofpu -I../sysdeps/ieee754/soft-fp -I../sysdeps/ieee754 -I../sysdeps/generic -I.. -I../libio -I. -nostdinc -isystem /home/piotro/minimyth-dev/images/build/usr/lib/gcc/arm-minimyth-linux-gnueabi/9.3.0/include -isystem /home/piotro/minimyth-dev/images/build/usr/lib/gcc/arm-minimyth-linux-gnueabi/9.3.0/include-fixed -isystem /home/piotro/minimyth-dev/images/main/usr/include -D_LIBC_REENTRANT -include /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/libc-modules.h -DMODULE_NAME=libpthread -include ../include/libc-symbols.h -DPIC -DSHARED -DTOP_NAMESPACE=glibc -DASSEMBLER -Werror=undef -Wa,--noexecstack -o /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/nptl/libc-do-syscall.os -MD -MP -MF /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/nptl/libc-do-syscall.os.dt -MT /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/nptl/libc-do-syscall.os may you pls hint me how to proceed with above? That command is missing the compiler options such as -march=armv7. You should put such options in CC, not CFLAGS, so that they are used for building .S files rather than just .c files. hmm, I scanned build.log and it looks like missig target compiler options (-march=armv7 -mfloat-abi=soft) are missed for _all_ invocations of arm-minimyth-linux-gnueabi-gcc for *.S files. Other source files (i.e. .c) are build with correct target compiler options (pls see below). It pretends like compiler options are not propagated for building .S files. As .c files (like below) are build with correct compiler options - I suspect flags for CC are set correctly.... You mention "You should put such options in CC". Looking on configure --help, i see CFLAGS is configure var. for setting desired option for C compiler. So what i'm missing here? arm-minimyth-linux-gnueabi-gcc C_name.c -c -std=gnu11 -fgnu89-inline -pipe -pipe -march=armv7 -mfloat-abi=soft -O2 -fno-lto -Wall -Wwrite-strings -Wundef -fmerge-all-constants -frounding-math -fno-stack-protector -Wstrict-prototypes -Wold-style-definition -fmath-errno -fPIC -ftls-model=initial-exec -DCOMPLOCALEDIR='"/usr/lib/locale"' -DLOCALE_ALIAS_PATH='"/usr/share/locale"' -Iprograms -I../include -I/home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/locale -I/home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build -I../sysdeps/unix/sysv/linux/arm/le -I../sysdeps/unix/sysv/linux/arm -I../sysdeps/arm/nptl -I../sysdeps/unix/sysv/linux/include -I../sysdeps/unix/sysv/linux -I../sysdeps/nptl -I../sysdeps/pthread -I../sysdeps/gnu -I../sysdeps/unix/inet -I../sysdeps/unix/sysv -I../sysdeps/unix/arm -I../sysdeps/unix -I../sysdeps/posix -I../sysdeps/arm/le -I../sysdeps/arm/include -I../sysdeps/arm -I../sysdeps/wordsize-32 -I../sysdeps/ieee754/flt-32 -I../sysdeps/ieee754/dbl-64 -I../sysdeps/arm/nofpu -I../sysdeps/ieee754/soft-fp -I../sysdeps/ieee754 -I../sysdeps/generic -I.. -I../libio -I. -nostdinc -isystem /home/piotro/minimyth-dev/images/build/usr/lib/gcc/arm-minimyth-linux-gnueabi/9.3.0/include -isystem /home/piotro/minimyth-dev/images/build/usr/lib/gcc/arm-minimyth-linux-gnueabi/9.3.0/include-fixed -isystem /home/piotro/minimyth-dev/images/main/usr/include -D_LIBC_REENTRANT -include /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/libc-modules.h -DMODULE_NAME=libc -include ../include/libc-symbols.h -DPIC -DSHARED -DTOP_NAMESPACE=glibc -o /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/locale/C_name.os -MD -MP -MF /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/locale/C_name.os.dt -MT /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/locale/C_name.os one more comment: whole (exactly the same building procedure with exactly the same sources + scripts) works ok for: i386, x86_64, aarch64 and non-THUMB ARM targets. Only change to trigger issue on ARM is change compiler options from: working: -marm -march=armv5te+fp -mfloat-abi=softfp to non-working: -mthumb -march=armv7 -mfloat-abi=soft CFLAGS is for C compiler, CXXFLAGS is for C++ compiler, ASFLAGS is for assembler. Options wanted for all languages should go in CC rather than in a language-specific variable. Ok. So when you are writing: "Options wanted for all languages should go in CC" - what exactly do you mean in terms of configure supplied params? On Tue, 5 May 2020, piotr.oniszczuk at gmail dot com via Glibc-bugs wrote:
> Ok. So when you are writing: "Options wanted for all languages should go in CC"
> - what exactly do you mean in terms of configure supplied params?
CC="<target>-gcc -mthumb -march=armv7 -mfloat-abi=soft", for example.
(In reply to joseph@codesourcery.com from comment #8) > On Tue, 5 May 2020, piotr.oniszczuk at gmail dot com via Glibc-bugs wrote: > > > Ok. So when you are writing: "Options wanted for all languages should go in CC" > > - what exactly do you mean in terms of configure supplied params? > > CC="<target>-gcc -mthumb -march=armv7 -mfloat-abi=soft", for example. Ah yes! I already tries exactly this and this makes glibc build failing quite early at assembling with assembler :-( On Tue, 5 May 2020, piotr.oniszczuk at gmail dot com via Glibc-bugs wrote:
> https://sourceware.org/bugzilla/show_bug.cgi?id=25920
>
> --- Comment #9 from Piotr Oniszczuk <piotr.oniszczuk at gmail dot com> ---
> (In reply to joseph@codesourcery.com from comment #8)
> > On Tue, 5 May 2020, piotr.oniszczuk at gmail dot com via Glibc-bugs wrote:
> >
> > > Ok. So when you are writing: "Options wanted for all languages should go in CC"
> > > - what exactly do you mean in terms of configure supplied params?
> >
> > CC="<target>-gcc -mthumb -march=armv7 -mfloat-abi=soft", for example.
>
> Ah yes! I already tries exactly this and this makes glibc build failing quite
> early at assembling with assembler :-(
So look at that failure, because putting the options in CC is the right
way to build glibc with custom compiler options.
One other thing to note: you have to use armv7-a, not armv7; v7-R and v7-M
aren't supported for normal GNU/Linux (with MMU) userspace.
@joseph, I'm really appreciating for Your help, but i have feeling we are in workaround mode instead of finding root cause and fix (if this is bug) or wait (if this is expected side-effect of WiP). My logic is following (pls correct me where i'm wrong): 1.Output from statically build multi-lib cross-compiler gcc via "-print-multi-lib" shows _all_ supported by compiler target compilation flags. 2.As we need always to provide desired target compilation flags - list of flag sets from p1. is not specific per se. It is just _required_ config to define desired result from multiple equal choices. 3.As long as I'm using target compilation flags derived from statically build multi-lib cross-compiler - all should be ok without any additional "non-standard flags" provided to glibc. 4.I have 4 of of 4 target architectures working this way with one having issue on subset of architecture tunables (-marm vs. -mthumb). So for 4 cases I have 3.5 working & standardised solution (single sources+build system with only 2 variable sets to select target + tunables). As I still don't know: is this bug or WiP - I'm assuming it is bug. Assuming above: we may start to workarround bug by modify 4to3.5 working procedure to move forward with missing just 0.5 subset - or we may rather trying to fix problem. I'm preferring to go with fix route. So back to key Q (for me): assuming my logic at p1..p4 is correct - do we have here bug or WiP issue? On Thu, 7 May 2020, piotr.oniszczuk at gmail dot com via Glibc-bugs wrote: > 3.As long as I'm using target compilation flags derived from statically build > multi-lib cross-compiler - all should be ok without any additional > "non-standard flags" provided to glibc. Only if (a) those flags are included in CC so they apply to all compilations and (b) the flags are actually valid ones for GNU/Linux. -march=armv7 is not a valid flag for GNU/Linux; GNU/Linux (as opposed to uClinux for no-MMU systems) only supports A-profile cores, not M-profile or R-profile cores, and -march=armv7 is the common subset of A-profile, M-profile and R-profile. Any compiler that tries to build a multilib for GNU/Linux with -march=armv7 is incorrectly configured. > As I still don't know: is this bug or WiP - I'm assuming it is bug. There's a bug in your compiler: it's configured with a multilib that would be valid for bare-metal / RTOS / uClinux configurations, but is not valid for GNU/Linux. (Actually, it has five such multilibs, all of which should be removed. Furthermore, I don't know what default CPU your compiler was configured with, so don't know whether the @mthumb@mfloat-abi=soft multilib is a valid one or not; note that glibc cannot be built as Thumb-1, only as Thumb-2.) If you have problems *with a valid configuration properly specified in CC*, you'll need to give more details of what those problems are. I suspect the "failing quite early at assembling with assembler" was a problem with a bad -march=armv7 option, but without exact error messages and corresponding command lines we can't advise further. @joseph May thx for hinting me about valid subset of GCC multi-libs for GNU/Linux. I was missing this information. I looked a bit inside of glibc building infra and in my case following patch allows to successfully cross-compile for armv7-a targets: [https://github.com/warpme/minimyth2/blob/master/script/devel/glibc/files/glibc-2.31-fix-building-arm-thumb.patch][1]. With above patch I tested cross-compilation for following list of targets where glibc cross-compiles with success: -marm -march=armv5te+fp -mfloat-abi=softfp -mthumb -march=armv7-a -mfloat-abi=soft -mthumb -march=armv7-a+fp -mfloat-abi=softfp -mthumb -march=armv7-a+simd -mfloat-abi=softfp -mthumb -march=armv7ve+simd -mfloat-abi=softfp Will above prove-of-concept be enough to develop proper fix? (probably it should go also to configure.ac)... If you wish to argue for all CFLAGS going in ASFLAGS (or e.g. all -m options), the right place to change would be the code in Makeconfig that instead just puts -g and -fdebug-prefix-map options from CFLAGS in ASFLAGS. Note that at that point CFLAGS contains many options that are specific to C compilation (such as -std=gnu11, for example), so it might be necessary to save a copy of relevant flags earlier in Makeconfig / refactor how those variables are built up. Any such proposal would best be sent to libc-alpha rather than in the discussion of a bug report, and would need to include an analysis of the history of why ASFLAGS only includes certain CFLAGS settings at present. It's quite likely the reasons are historical and it would be safe for nearly all options in CFLAGS to be passed in ASFLAGS, but we need to try to understand what the reasons were and what might break first. |
Created attachment 12501 [details] glibc config.log from failed arm thumb build Hi, I’m trying to build cross-gcc for arm family. Whole multistep process i'm using of building 9.3gcc cross-gcc works ok for: i386, x86_64, aarch64 and arm - but fails in arm for thumb case. gcc9.3.0 cross-compiler I’m using to build glibc is build with multi-lib and provides following multi-libs: arm/v5te/softfp;@marm@march=armv5te+fp@mfloat-abi=softfp arm/v5te/hard;@marm@march=armv5te+fp@mfloat-abi=hard thumb/nofp;@mthumb@mfloat-abi=soft thumb/v7/nofp;@mthumb@march=armv7@mfloat-abi=soft thumb/v7+fp/softfp;@mthumb@march=armv7+fp@mfloat-abi=softfp thumb/v7+fp/hard;@mthumb@march=armv7+fp@mfloat-abi=hard thumb/v7-r+fp.sp/softfp;@mthumb@march=armv7-r+fp.sp@mfloat-abi=softfp thumb/v7-r+fp.sp/hard;@mthumb@march=armv7-r+fp.sp@mfloat-abi=hard thumb/v7-a/nofp;@mthumb@march=armv7-a@mfloat-abi=soft thumb/v7-a+fp/softfp;@mthumb@march=armv7-a+fp@mfloat-abi=softfp thumb/v7-a+fp/hard;@mthumb@march=armv7-a+fp@mfloat-abi=hard thumb/v7-a+simd/softfp;@mthumb@march=armv7-a+simd@mfloat-abi=softfp thumb/v7-a+simd/hard;@mthumb@march=armv7-a+simd@mfloat-abi=hard thumb/v7ve+simd/softfp;@mthumb@march=armv7ve+simd@mfloat-abi=softfp thumb/v7ve+simd/hard;@mthumb@march=armv7ve+simd@mfloat-abi=hard thumb/v8-a/nofp;@mthumb@march=armv8-a@mfloat-abi=soft thumb/v8-a+simd/softfp;@mthumb@march=armv8-a+simd@mfloat-abi=softfp thumb/v8-a+simd/hard;@mthumb@march=armv8-a+simd@mfloat-abi=hard Buildiding glibc 2.31 for ARM works ok for target: -marm -march=armv5te+fp -mfloat-abi=softfp Build fails however (with the same error) for any ARM target with THUMB i.e.: -mthumb -march=armv7 -mfloat-abi=soft build error is following: arm-minimyth-linux-gnueabi-gcc -Wl,--as-needed -pipe -pipe -march=armv7 -mfloat-abi=soft -O2 -fno-lto -nostdlib -nostartfiles -shared -o /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/elf/ld.so.new \ -Wl,-z,combreloc -Wl,-z,relro -Wl,--hash-style=both -Wl,-z,defs -Wl,-z,now \ /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/elf/librtld.os -Wl,--version-script=/home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/ld.map \ -Wl,-soname=ld-linux.so.3 \ -Wl,-defsym=_begin=0 /home/piotro/minimyth-dev/images/build/usr/lib/gcc/arm-minimyth-linux-gnueabi/9.3.0/../../../../arm-minimyth-linux-gnueabi/bin/ld: /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/elf/librtld.os: in function `init_tls': rtld.c:(.text+0x520): undefined reference to `__libc_do_syscall' /home/piotro/minimyth-dev/images/build/usr/lib/gcc/arm-minimyth-linux-gnueabi/9.3.0/../../../../arm-minimyth-linux-gnueabi/bin/ld: /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/elf/librtld.os: in function `dl_main': rtld.c:(.text+0x2426): undefined reference to `__libc_do_syscall' /home/piotro/minimyth-dev/images/build/usr/lib/gcc/arm-minimyth-linux-gnueabi/9.3.0/../../../../arm-minimyth-linux-gnueabi/bin/ld: /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/elf/librtld.os: in function `_dl_lookup_symbol_x': (.text+0x798c): undefined reference to `__libc_do_syscall' /home/piotro/minimyth-dev/images/build/usr/lib/gcc/arm-minimyth-linux-gnueabi/9.3.0/../../../../arm-minimyth-linux-gnueabi/bin/ld: /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/elf/librtld.os: in function `_dl_relocate_object': (.text+0x913e): undefined reference to `__libc_do_syscall' /home/piotro/minimyth-dev/images/build/usr/lib/gcc/arm-minimyth-linux-gnueabi/9.3.0/../../../../arm-minimyth-linux-gnueabi/bin/ld: /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/elf/librtld.os: in function `_dl_fixup': (.text+0xabac): undefined reference to `__libc_do_syscall' /home/piotro/minimyth-dev/images/build/usr/lib/gcc/arm-minimyth-linux-gnueabi/9.3.0/../../../../arm-minimyth-linux-gnueabi/bin/ld: /home/piotro/minimyth-dev/script/devel/glibc/work/main.d/glibc-2.31_build/elf/librtld.os:(.text+0xafc0): more undefined references to `__libc_do_syscall' follow collect2: error: ld returned 1 exit status As whole (exactly the same building procedure with exactly the same sources) works ok for: i386, x86_64, aarch64 and non-THUMB ARM targets - failed ARM builds for targets with THUMB I’m considering as bug in glibc. glibc config.log from failed arm thumb build is attached.