Bug 30779 - support gprofng on musl
Summary: support gprofng on musl
Status: SUSPENDED
Alias: None
Product: binutils
Classification: Unclassified
Component: gprofng (show other bugs)
Version: 2.41
: P3 enhancement
Target Milestone: ---
Assignee: Vladimir Mezentsev
URL:
Keywords:
: 31960 (view as bug list)
Depends on:
Blocks:
 
Reported: 2023-08-18 07:34 UTC by Sam James
Modified: 2024-11-05 16:04 UTC (History)
4 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed:
Project(s) to access:
ssh public key:


Attachments
correct patch (236 bytes, patch)
2024-07-08 09:35 UTC, cqwrteur
Details | Diff
use wchar.h instead (236 bytes, patch)
2024-07-08 11:29 UTC, cqwrteur
Details | Diff
just exclude all things here (458 bytes, patch)
2024-07-08 12:17 UTC, cqwrteur
Details | Diff
finally get this work (481 bytes, patch)
2024-07-08 12:32 UTC, cqwrteur
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sam James 2023-08-18 07:34:35 UTC
```
/bin/sh ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.././gprofng/src -I..  -U_ASM -DLOCALEDIR=\"/usr/local/share/locale\" -DSYSCONFDIR=\"/usr/local/etc\" -DLIBDIR=\"/usr/local/lib\" -I.. -I../../.././gprofng/src -I../../.././gprofng/src/../common -I../../.././gprofng/src/../../include -I../../.././gprofng/src/../../opcodes -I../../bfd -I../../.././gprofng/src/../../bfd -I../../.././gprofng/../zlib  -Wall -pthread -Wno-switch -g -O2     -MT Dbe.lo -MD -MP -MF $depbase.Tpo -c -o Dbe.lo ../../.././gprofng/src/Dbe.cc &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I../../.././gprofng/src -I.. -U_ASM -DLOCALEDIR=\"/usr/local/share/locale\" -DSYSCONFDIR=\"/usr/local/etc\" -DLIBDIR=\"/usr/local/lib\" -I.. -I../../.././gprofng/src -I../../.././gprofng/src/../common -I../../.././gprofng/src/../../include -I../../.././gprofng/src/../../opcodes -I../../bfd -I../../.././gprofng/src/../../bfd -I../../.././gprofng/../zlib -Wall -pthread -Wno-switch -g -O2 -MT Dbe.lo -MD -MP -MF .deps/Dbe.Tpo -c ../../.././gprofng/src/Dbe.cc -o Dbe.o
In file included from ../../.././gprofng/src/Exp_Layout.h:31,
                 from ../../.././gprofng/src/MemorySpace.h:28,
                 from ../../.././gprofng/src/DbeSession.h:50,
                 from ../../.././gprofng/src/Dbe.cc:31:
../../.././gprofng/src/Data_window.h:56:3: error: 'off64_t' does not name a type; did you mean 'off_t'?
   56 |   off64_t get_fsize ()          { return fsize; }
      |   ^~~~~~~
      |   off_t
In file included from ../../.././gprofng/src/LoadObject.h:27,
                 from ../../.././gprofng/src/Dbe.cc:40:
../../.././gprofng/src/Stabs.h:90:28: error: 'ino64_t' has not been declared
   90 |     Stab_status read_stabs(ino64_t srcInode, Module *module, Vector<ComC*> *comComs, bool readDwarf = false);
      |                            ^~~~~~~
../../.././gprofng/src/LoadObject.h:194:3: error: 'ino64_t' does not name a type; did you mean 'int64_t'?
  194 |   ino64_t inode;                // inode number of segment file
      |   ^~~~~~~
      |   int64_t
In file included from ../../.././gprofng/src/Function.h:28,
                 from ../../.././gprofng/src/Dbe.cc:45:
../../.././gprofng/src/SourceFile.h:59:3: error: 'ino64_t' does not name a type; did you mean 'int64_t'?
   59 |   ino64_t
      |   ^~~~~~~
      |   int64_t
../../.././gprofng/src/SourceFile.h:106:3: error: 'ino64_t' does not name a type; did you mean 'int64_t'?
  106 |   ino64_t srcInode;         // Inode number of source file
      |   ^~~~~~~
      |   int64_t
make[5]: *** [Makefile:844: Dbe.lo] Error 1
```

This is with musl-1.2.4 which drops the LFS types because it always has native LFS.

I suggest using -D_FILE_OFFSET_BITS=64 via AC_SYS_LARGEFILE and then always using ino_t and friends, which will work on glibc too.
Comment 1 Sam James 2023-08-18 07:41:10 UTC
Downstream report: https://bugs.gentoo.org/912284
Full build log (xz compressed): https://bugs.gentoo.org/attachment.cgi?id=867845

I can give more information if needed.
Comment 2 Vladimir Mezentsev 2023-08-25 04:29:40 UTC
I don't use musl.
But I fixed couple bugs in gprofng to build on x86_64-pc-linux-musl.
I did on my machine:
% wget https://distfiles.adelielinux.org/adelie/1.0/iso/rc2/adelie-rootfs-mini-x86_64-1.0-rc2.txz
% sudo tar xf adelie-rootfs-mini-x86_64-1.0-rc2.txz
% sudo systemd-nspawn --directory=. -M mez_musl
Spawning container mez_musl on /data2/vmezents/musl.
Press ^] three times within 1s to kill container.
-sh: 0: can't access tty; job control turned off

/sbin/apk add bash && bash
/sbin/apk add build-tools m4 bison flex gmp-dev texinfo
cd /home && mkdir bld && cd bld
../binutils-gdb.git/configure --prefix=`pwd`/INSTALL  --enable-multilib --disable-gdb --disable-gdbsupport --disable-gdbserver

I fixed the compiler errors (open64, fgetpo64 and fsetpos64 are redefined) in gprofng.
But I don't see errors about off64_t or ino64_t.

I use this system:
% uname -a
Linux mez_musl 5.4.17-2136.318.7.1.el8uek.x86_64 #2 SMP Thu Apr 13 17:33:39 PDT 2023 x86_64 GNU/Linux


My questions are:
 How can I simply use/download musl-1.2.4 ?
 Is the bug on Arm or on Intel ?
Comment 3 Sam James 2023-08-25 14:48:33 UTC
(In reply to Vladimir Mezentsev from comment #2)
> [...]
> My questions are:
>  How can I simply use/download musl-1.2.4 ?

Unfortunately, I think Adelie is a bit out of date in general at the moment. Alpine edge should have the latest musl or I can give you some instructions for a Gentoo docker container.

The bug will need >=musl-1.2.4 to reproduce.

>  Is the bug on Arm or on Intel ?

Intel in this case (amd64/x86_64).
Comment 4 John Doe 2023-09-25 13:18:29 UTC
> I fixed the compiler errors (open64, fgetpo64 and fsetpos64 are redefined) in gprofng.

This seems to be missing in git upstream.
Comment 5 Sam James 2023-12-16 04:17:51 UTC
Any update here? It'd be great to have this one fixed before 2.42.
Comment 6 Vladimir Mezentsev 2023-12-21 06:40:16 UTC
We have the other bugs against gprofng-musl:
29477 Build failure on musl: collector_module.h:78:13: error: duplicate member ‘pwrite’ 
29593 error: '__u64' undeclared (first use in this function) for aarch64-linux-musl host
30893 gprofng uses syscall + perf_event (hwcdrv) and ptrace (gpcollect) without any check and doc
 
I can fix build issues.
But I cannot fix the run time problem before 2.42.
The known problem is getcontext() is absent in libc on musl.
Comment 7 Kurt Goebel 2023-12-21 14:32:15 UTC
    Thank you for responding to musl issues!

     Good to let them know where we are

     I'm sure they appreciate your explanation

     Kurt


On 12/21/23 1:40 AM, vladimir.mezentsev at oracle dot com wrote:
> https://urldefense.com/v3/__https://sourceware.org/bugzilla/show_bug.cgi?id=30779__;!!ACWV5N9M2RV99hQ!K8Dln1WWQeCCQikDkZ4YfU3DWeoY0g0f8XOHvICdnaL4Erz9EFGO4EsNZ38vVxWPFab06Fkq_ttgUVKBML7aVmprh5kKeByd$
>
> Vladimir Mezentsev <vladimir.mezentsev at oracle dot com> changed:
>
>             What    |Removed                     |Added
> ----------------------------------------------------------------------------
>               Status|NEW                         |ASSIGNED
>             See Also|                            |https://urldefense.com/v3/__https://sourceware.org/bugz__;!!ACWV5N9M2RV99hQ!K8Dln1WWQeCCQikDkZ4YfU3DWeoY0g0f8XOHvICdnaL4Erz9EFGO4EsNZ38vVxWPFab06Fkq_ttgUVKBML7aVmprhwhmEanT$
>                     |                            |illa/show_bug.cgi?id=29477,
>                     |                            |https://urldefense.com/v3/__https://sourceware.org/bugz__;!!ACWV5N9M2RV99hQ!K8Dln1WWQeCCQikDkZ4YfU3DWeoY0g0f8XOHvICdnaL4Erz9EFGO4EsNZ38vVxWPFab06Fkq_ttgUVKBML7aVmprhwhmEanT$
>                     |                            |illa/show_bug.cgi?id=29593
>
> --- Comment #6 from Vladimir Mezentsev <vladimir.mezentsev at oracle dot com> ---
> We have the other bugs against gprofng-musl:
> 29477 Build failure on musl: collector_module.h:78:13: error: duplicate member
> ‘pwrite’
> 29593 error: '__u64' undeclared (first use in this function) for
> aarch64-linux-musl host
> 30893 gprofng uses syscall + perf_event (hwcdrv) and ptrace (gpcollect) without
> any check and doc
>
> I can fix build issues.
> But I cannot fix the run time problem before 2.42.
> The known problem is getcontext() is absent in libc on musl.
>
Comment 8 Sourceware Commits 2023-12-23 05:27:14 UTC
The master branch has been updated by Vladimir Mezentsev <vmezents@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=576d2c97d8de26d1ab5a2058e8ba4ffd00b8096d

commit 576d2c97d8de26d1ab5a2058e8ba4ffd00b8096d
Author: Vladimir Mezentsev <vladimir.mezentsev@oracle.com>
Date:   Thu Dec 21 21:33:58 2023 -0800

    gprofng: fix build problems on linux-musl
    
    ino64_t, off64_t, fpos64_t, stat64, __u64 are not defined on linux-musl.
    Fixed by declaring these types for linux-musl.
    
    2023-12-21  Vladimir Mezentsev  <vladimir.mezentsev@oracle.com>
    
            PR gprofng/30779
            PR gprofng/29593
            * common/gp-defs.h: Define ino64_t, off64_t, fpos64_t for linux-musl.
            * libcollector/unwind.c: Define __u64 for linux-musl.
            * src/util.h: Define dbe_stat_t.
            * src/ClassFile.cc: Use dbe_stat_t instead of "struct stat64".
            * src/Dbe.cc: Likewise.
            * src/DbeFile.cc: Likewise.
            * src/DbeFile.h: Likewise.
            * src/DbeSession.cc: Likewise.
            * src/Experiment.cc: Likewise.
            * src/checks.cc: Likewise.
            * src/util.cc: Likewise.
Comment 9 Vladimir Mezentsev 2024-01-11 03:27:59 UTC
Has the build problem been fixed in your environment?
Comment 10 Adam Baxter 2024-01-13 06:29:29 UTC
Hi Vladimir - I also hit this, I can confirm this patch fixes *this specific compilation error* but I cannot confirm that everything builds correctly as I am hitting other environment-specific problems.
Comment 11 Sam James 2024-07-08 09:06:55 UTC
*** Bug 31960 has been marked as a duplicate of this bug. ***
Comment 12 Sam James 2024-07-08 09:25:34 UTC
*** Bug 31960 has been marked as a duplicate of this bug. ***
Comment 13 cqwrteur 2024-07-08 09:27:48 UTC
(In reply to Sam James from comment #12)
> *** Bug 31960 has been marked as a duplicate of this bug. ***

This patch is WRONG.


there is no __musl__ related macros in musl libc. musl does not want you to detect it. The right way is to do what my fast_io is doing, include<stdio.h> then use __NEED___isoc_va_list to hack it around.
Comment 14 cqwrteur 2024-07-08 09:28:44 UTC
(In reply to cqwrteur from comment #13)
> (In reply to Sam James from comment #12)
> > *** Bug 31960 has been marked as a duplicate of this bug. ***
> 
> This patch is WRONG.
> 
> 
> there is no __musl__ related macros in musl libc. musl does not want you to
> detect it. The right way is to do what my fast_io is doing, include<stdio.h>
> then use __NEED___isoc_va_list to hack it around.

https://github.com/trcrsired/fast_io/blob/ff0515ed342cd08a73d6917c0ae52be629db82cb/include/fast_io_legacy_impl/c/impl.h#L1141
Comment 15 Sam James 2024-07-08 09:29:35 UTC
See gprofng/configure.ac:175 which sets _MUSL_LIBC.
Comment 16 cqwrteur 2024-07-08 09:31:04 UTC
(In reply to Sam James from comment #15)
> See gprofng/configure.ac:175 which sets _MUSL_LIBC.

it does not set it for me.
Comment 17 cqwrteur 2024-07-08 09:35:01 UTC
Created attachment 15605 [details]
correct patch

i have put a patch here.

This is exactly the reason why musl libc is a crap. Everyone should just move on to LLVM libc.
Comment 18 cqwrteur 2024-07-08 11:21:08 UTC
(In reply to cqwrteur from comment #16)
> (In reply to Sam James from comment #15)
> > See gprofng/configure.ac:175 which sets _MUSL_LIBC.
> 
> it does not set it for me.

patch still not working out
Comment 19 cqwrteur 2024-07-08 11:29:45 UTC
Created attachment 15606 [details]
use wchar.h instead

i should use wchar.h instead not stdio.h since stdio also defines some 64 file ops
Comment 20 cqwrteur 2024-07-08 11:31:02 UTC
eelf_x86_64.c: In function 'gldelf_x86_64_get_script':
eelf_x86_64.c:2692:42: error: 'return' with no value, in function returning non-void [-Wreturn-mismatch]
 2692 |              && link_info.separate_code) return
      |                                          ^~~~~~
eelf_x86_64.c:155:1: note: declared here
  155 | gldelf_x86_64_get_script (int *isfile)
      | ^~~~~~~~~~~~~~~~~~~~~~~~
make[4]: *** [Makefile:1617: eelf_x86_64.o] Error 1
make[4]: *** Waiting for unfinished jobs....
  CXX    dwarf2/abbrev.o

then you have another one
Comment 21 cqwrteur 2024-07-08 11:32:34 UTC
/home/cqwrteur/toolchains_build/binutils-gdb/gprofng/src/../../include/libiberty.h:112:14: error: conflicting declaration of C function 'char* basename(const char*)'
  112 | extern char *basename (const char *) ATTRIBUTE_RETURNS_NONNULL ATTRIBUTE_NONNULL(1);
      |              ^~~~~~~~
In file included from /home/cqwrteur/toolchains_build/binutils-gdb/gprofng/src/collctrl.cc:34:
/home/cqwrteur/toolchains/x86_64-pc-linux-gnu/x86_64-linux-musl/x86_64-linux-musl/include/libgen.h:9:7: note: previous declaration 'char* basename(char*)'
    9 | char *basename(char *);
      |       ^~~~~~~~
eelf_x86_64.c: In function 'gldelf_x86_64_get_script':
eelf_x86_64.c:2692:42: error: 'return' with no value, in function returning non-void [-Wreturn-mismatch]
 2692 |              && link_info.separate_code) return
      |                                          ^~~~~~
eelf_x86_64.c:155:1: note: declared here
  155 | gldelf_x86_64_get_script (int *isfile)
      | ^~~~~~~~~~~~~~~~~~~~~~~~
make[4]: *** [Makefile:1617: eelf_x86_64.o] Error 1
make[4]: *** Waiting for unfinished jobs....
/home/cqwrteur/toolchains_build/binutils-gdb/gprofng/src/util.cc: In function 'void* dbe_stat_on_thread(void*)':
/home/cqwrteur/toolchains_build/binutils-gdb/gprofng/src/util.cc:911:12: error: 'stat64' was not declared in this scope; did you mean 'fstat64'?
  911 |   int st = stat64 (path, &(wt_info->statbuf));
      |            ^~~~~~
      |            fstat64
/home/cqwrteur/toolchains_build/binutils-gdb/gprofng/src/util.cc: In function 'int dbe_stat_internal(const char*, dbe_stat_t*, bool)':
/home/cqwrteur/toolchains_build/binutils-gdb/gprofng/src/util.cc:1107:12: error: 'stat64' was not declared in this scope; did you mean 'fstat64'?
 1107 |   int st = stat64 (path, sbuf);
      |            ^~~~~~
      |            fstat64

THIS IS UTTERLY CRAP.
Comment 22 cqwrteur 2024-07-08 11:45:03 UTC
(In reply to Sam James from comment #15)
> See gprofng/configure.ac:175 which sets _MUSL_LIBC.

it does not really work out. a lot of things are not defined like fopen64, fstat64. you really should not use them tbh. i don't understand why do you even use these non standard extensions
Comment 23 cqwrteur 2024-07-08 11:45:42 UTC
(In reply to cqwrteur from comment #22)
> (In reply to Sam James from comment #15)
> > See gprofng/configure.ac:175 which sets _MUSL_LIBC.
> 
> it does not really work out. a lot of things are not defined like fopen64,
> fstat64. you really should not use them tbh. i don't understand why do you
> even use these non standard extensions

also I don't think this would only happen with musl, probably mlibc, uclibc-ng and many others as well.
Comment 24 cqwrteur 2024-07-08 11:46:45 UTC
/home/cqwrteur/toolchains_build/binutils-gdb/gprofng/src/Experiment.cc:234:12: error: 'fopen64' was not declared in this scope; did you mean 'open64'?
  234 |       fh = fopen64 (fname, NTXT ("r"));
      |            ^~~~~~~
      |            open64
/home/cqwrteur/toolchains_build/binutils-gdb/gprofng/src/Experiment.cc:244:9: error: 'fseeko64' was not declared in this scope; did you mean 'fseeko'?
  244 |         fseeko64 (fh, offset, SEEK_SET);
      |         ^~~~~~~~
      |         fseeko
/home/cqwrteur/toolchains_build/binutils-gdb/gprofng/src/Experiment.cc: In member function 'void Experiment::ExperimentFile::close()':
/home/cqwrteur/toolchains_build/binutils-gdb/gprofng/src/Experiment.cc:285:16: error: 'ftello64' was not declared in this scope; did you mean 'ftello'?
  285 |       offset = ftello64 (fh);
      |                ^~~~~~~~
      |                ftello
Comment 25 cqwrteur 2024-07-08 12:17:58 UTC
Created attachment 15608 [details]
just exclude all things here
Comment 26 cqwrteur 2024-07-08 12:32:26 UTC
Created attachment 15609 [details]
finally get this work
Comment 27 Vladimir Mezentsev 2024-07-08 16:54:45 UTC
>> i should use wchar.h instead not stdio.h since stdio also defines some 64 file ops


Can you set _MUSL_LIBC in gprofng/configure.ac instead of gp-defs.h.



>> it does not really work out. a lot of things are not defined like fopen64,
>> fstat64. you really should not use them tbh.
>>  i don't understand why do you even use these non standard extensions

It's probably time to clean up libcollector and remove these functions.
Comment 28 Vladimir Mezentsev 2024-07-11 04:20:12 UTC
>> Created attachment 15609 [details]
>> finally get this work

The 2.43 release will be on Sunday 14th July.
If your fix works with MUSL_LIBC, could you push your fixes to sourceware.org/git/binutils-gdb.git

Thank you,
-Vladimir
Comment 29 Sam James 2024-08-17 00:47:24 UTC
Vladimir, I was thinking: while we should cleanup the use of the LFS aliases, given that gprofng needs glibc-specific functionality (AFAIK) for runtime tracing with LD_PRELOAD, maybe we should disable gprofng by default in configure for *musl* targets?

It might reduce the friction for some people.
Comment 30 Vladimir Mezentsev 2024-08-21 02:22:55 UTC
Let's disable the gprofng build for musl.
I don't have a musl machine to test this fix:
% git diff
diff --git a/configure b/configure
index 7823f2c3784..93d286bb3b7 100755
--- a/configure
+++ b/configure
@@ -3146,6 +3146,8 @@ fi
 
 if test "$enable_gprofng" = "yes"; then
   case "${target}" in
+    *-musl*)
+      ;;
     x86_64-*-linux* | i?86-*-linux* | aarch64-*-linux* | riscv64-*-linux*)
     configdirs="$configdirs gprofng"
     ;;
diff --git a/configure.ac b/configure.ac
index a390639bfa9..6fd9e77f20a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -414,6 +414,8 @@ enable_gprofng=$enableval,
 enable_gprofng=yes)
 if test "$enable_gprofng" = "yes"; then
   case "${target}" in
+    *-musl*)
+      ;;
     x86_64-*-linux* | i?86-*-linux* | aarch64-*-linux* | riscv64-*-linux*)
     configdirs="$configdirs gprofng"
     ;;


I suggest to rename this bug as "support gprofng on musl"
and update 29477 and 29593 as "suspended" or "waiting".
Comment 31 Sam James 2024-08-21 05:24:41 UTC
This sounds like a good plan.

I will test the patch.
Comment 32 Sam James 2024-08-21 07:16:52 UTC
Thanks, the patch works (and LGTM).
Comment 33 Sourceware Commits 2024-08-26 21:19:57 UTC
The master branch has been updated by Vladimir Mezentsev <vmezents@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=abb27dd124fd8d8a879ff667776bbf22422f464e

commit abb27dd124fd8d8a879ff667776bbf22422f464e
Author: Vladimir Mezentsev <vladimir.mezentsev@oracle.com>
Date:   Fri Aug 23 13:43:18 2024 -0700

    Disable gprofng build for *musl*
    
    I disable gprofng until gprofng is ported to musl.
    
    gprofng/ChangeLog
    2024-08-22  Vladimir Mezentsev  <vladimir.mezentsev@oracle.com>.
    
            PR gprofng/30779
            PR gprofng/29593
            PR gprofng/29477
            * configure.ac: Disable gprofng build for *musl*.
            * configure: Rebuild.