Bug 25509

I have basically 2 possible solutions:

- elfutils will provide a stub client library (libdebuginfod-stub.so) which will then do dlopen for the real libdebuginfod.so during run-rime
- one can do execv of debuginfod-find which is a standalone tool that can download a debuginfo

(In reply to Martin Liška from comment #0)
> In openSUSE, we do face a problem with cyclic dependencies. Many core
> packages like gcc, glibc, elfutils or binutils depend on each other and
> create a cycle. The cycle should contain a reasonable amount of packages.
> 
> When debuginfod was added to elfutils, we would have a huge bunch of
> dependencies caused by libhttpmicro and libsqlite. These have very many
> transitive dependencies.

debuginfod the tool has dependencies on libhttpmicro, libsqlite, libdw and debuginfod-client.

debuginfod-client depends on libcurl (which pulls in a lot of the crypto stuff for https support).

libdw depends on debuginfod-client, but only indirectly through ldopen.

It does make sense to put these in different (output/sub) packages.
Which is actually what the sample elfutils.spec file in config/ does.

That doesn't "split" the inputs/dependencies of the source build though. Which is what you seem most concerned with.

> So that I was forced to split elfutils into 2
> packages: elfutils and elfutils-debuginfod. The later contains all the new
> packages and is not part of the boostrap cycle.
> 
> What's more problematic is that there are (and will be) tools that want to
> utilize libdebuginfod such as Binutils. As mentioned, the tool is in the
> bootstrap cycle and so that can't depend on elfutils-debuginfod.

Do you have a pointer to the spec files you are using now?

> So the question is how to unbreak all these dependencies for future core
> packages?

The GNU Guix people also had some concerns about all the new dependencies and also suggested splitting the package:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=38803

Indeed, merely splitting the package up does not solve problematic incoming dependencies.

IMHO we should encourage distros to have a bootstrap-oriented build of elfutils, possibly prepared thusly:

- have the elfutils configury have a -debuginfod mode (--without-debuginfod?) that, instead of just skipping building all that stuff, builds a no-op stub instead

- expose that --without-debuginfod configure flag to another spec file (or a --with-* flagged version of the main one) to generate a set of elfutils-bootstrap-* subrpms

- have the normal non-bootstrap spec file variant Obsolete/Conflict with the bootstrap siblings

- then these elfutils-bootstrap* rpms should be functional and satisfy all your bootstrapping needs

- and after bootstrapping is done, nuke the elfutils-bootstrap rpms from your repo, and build the real thing

(In reply to Mark Wielaard from comment #2)
> (In reply to Martin Liška from comment #0)
> > In openSUSE, we do face a problem with cyclic dependencies. Many core
> > packages like gcc, glibc, elfutils or binutils depend on each other and
> > create a cycle. The cycle should contain a reasonable amount of packages.
> > 
> > When debuginfod was added to elfutils, we would have a huge bunch of
> > dependencies caused by libhttpmicro and libsqlite. These have very many
> > transitive dependencies.
> 
> debuginfod the tool has dependencies on libhttpmicro, libsqlite, libdw and
> debuginfod-client.
> 
> debuginfod-client depends on libcurl (which pulls in a lot of the crypto
> stuff for https support).
> 
> libdw depends on debuginfod-client, but only indirectly through ldopen.
> 
> It does make sense to put these in different (output/sub) packages.
> Which is actually what the sample elfutils.spec file in config/ does.
> 
> That doesn't "split" the inputs/dependencies of the source build though.
> Which is what you seem most concerned with.
> 
> > So that I was forced to split elfutils into 2
> > packages: elfutils and elfutils-debuginfod. The later contains all the new
> > packages and is not part of the boostrap cycle.
> > 
> > What's more problematic is that there are (and will be) tools that want to
> > utilize libdebuginfod such as Binutils. As mentioned, the tool is in the
> > bootstrap cycle and so that can't depend on elfutils-debuginfod.
> 
> Do you have a pointer to the spec files you are using now?

You can find the 2 spec files here:
https://build.opensuse.org/package/show/Base:System/elfutils

> 
> > So the question is how to unbreak all these dependencies for future core
> > packages?
> 
> The GNU Guix people also had some concerns about all the new dependencies
> and also suggested splitting the package:
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=38803

> # Look for libmicrohttpd, libcurl, libarchive, sqlite for debuginfo server
> # minimum versions as per rhel7.  Single --enable-* option arranges to build
> # both client and server.

Maybe this should not be the case? If libdebuginfod was enabled on its own as long as libcurl is available, this would make things a lot simpler. At least for Arch Linux, curl is part of the base system while libmicrohttpd is not.

So the tools in the bootstrap cycle would only need to depend on curl (which itself can be bootstrapped without support for most things including SSL) and that would be sufficient to build everything that links to libdebuginfod.so

This would be independently useful alongside making a stub loader.

(In reply to Eli Schwartz from comment #5)
> > # Look for libmicrohttpd, libcurl, libarchive, sqlite for debuginfo server
> > # minimum versions as per rhel7.  Single --enable-* option arranges to build
> > # both client and server.
> 
> Maybe this should not be the case? If libdebuginfod was enabled on its own
> as long as libcurl is available, this would make things a lot simpler. At
> least for Arch Linux, curl is part of the base system while libmicrohttpd is
> not.

This part wouldn't affect the suse/fedora folks, because one would already place the debuginfod server into a separate subpackage, so its prereqs don't affect the buildroot.

So there are two ideas here:

- Split --disable-debuginfod which currently disables building
  debuginfod (the server), libdebuginfod (the library) and
  debuginfod-find (the helper binaries) in two:
  --disable-debuginfod to disable building of the server.
  --disable-libdebuginfod to disable building of the library and helper binary.

- Introduce a dummy-libdebuginfod which has the same interface as
  libdebuginfod, so applications (during bootstrapping) can link to
  it. But which would not link against libcurl and all functions would
  simply fail.

I assume that --disable-libdebuginfod would mean you cannot have --enable-debuginfod (because debuginfod links against libdebuginfod). And that --enable-dummy-libdebuginfod would imply --disable-libdebuginfod because they provide the same library. I assume that debuginfod-find would be build if either --enable-libdebuginfod or --enable-dummy-libdebuginfod is given (in the last case it would simply not do anything useful, but would show the dummy "works").

> 
> - Introduce a dummy-libdebuginfod which has the same interface as
>   libdebuginfod, so applications (during bootstrapping) can link to
>   it. But which would not link against libcurl and all functions would
>   simply fail.
> 

Yes, I support the creation of a stub library.

Created attachment 12628 [details]
debuginfod: Add --disable-libdebuginfod and --enable-libdebuginfod=dummy.

How about the attached? It splits off disabling of the client by introducing --disable-libdebuginfod and you can specificy --enable-libdebuginfod=dummy to create a dummy libdebuginfod which simply fails all calls.

Comment on attachment 12628 [details]
debuginfod: Add --disable-libdebuginfod and --enable-libdebuginfod=dummy.

Looks workable.

I don't know if we need a new --enable-libdebuginfod=dummy
option, as opposed to it being simply the equivalent of
--disable-libdebuginfod, or even --disable-debuginfod
(to have a common configury option for both client & server,
as now).  But that's just a style nit.

(In reply to Frank Ch. Eigler from comment #10)
> Comment on attachment 12628 [details]
> debuginfod: Add --disable-libdebuginfod and --enable-libdebuginfod=dummy.
> 
> Looks workable.
> 
> I don't know if we need a new --enable-libdebuginfod=dummy
> option, as opposed to it being simply the equivalent of
> --disable-libdebuginfod, or even --disable-debuginfod
> (to have a common configury option for both client & server,
> as now).  But that's just a style nit.

Yeah, you are right that it is more complexity than is strictly necessary. But it allowed me to easily test the dummy libdebuginfod by still building debuginfod and debuginfod-find against it and see they still "work".

Martin, could you comment on the setup? Does this help your case? I will probably not actually use it myself, so don't want to add it unless it really makes someone happy to have it as an option.

(In reply to Mark Wielaard from comment #11)
> Martin, could you comment on the setup? Does this help your case? I will
> probably not actually use it myself, so don't want to add it unless it
> really makes someone happy to have it as an option.

Eli, same question for you. Is this setup useful for what you want to do in Arch Linux?

I think we'd prefer to have the ability to build libdebuginfod without the server. Ambivalent about a special option to force the libdebuginfod dummy to be created, but...

If it's just for testing purposes, maybe you could always build the dummy, but only install it if the real libdebuginfod was not enabled? The testsuite could verify the dummy works by using LD_LIBRARY_PATH/LD_PRELOAD (depending on build directory layout) to run some tests explicitly using the dummy.

It's not just for testing purposes.  It's to aid bootstrapping new OS versions, by reducing the transitive requirements of elfutils in the buildroot.

Wouldn't that be covered by "libdebuginfod disabled equals dummy enabled"?

(yup, misinterpreted what the "this" was you meant, sorry!)

Just to be clear, the current setup is:

--enable-debuginfod or --enable-debuginfod=yes: builds all debuginfod server/client artifacts (requires libcurl)
--disable-debuginfod or --enable-debuginfod=no: builds none of the debuginfod server/client artifacts.

The proposed patch introduces:

--enable-libdebuginfod or --enable-libdebuginfod=yes: builds the debuginfod client artifacts
--disable-libdebuginfod or --enable-libdebuginfod=no: don't build the debuginfod client artifacts
--enable-libdebuginfod=dummy: builds all debuginfod clients artifacts, but the libdebuginfod.so is just a stub (does not depend on libcurl).
--enable-debuginfod or --enable-debuginfod=yes: builds all debuginfod server artifacts (requires either --enable-libdebuginfod=yes or --enable-libdebuginfod=dummy and sqlite3, microhttpd, libarchive)
--disable-debuginfod or --enable-debuginfod=no: build none of the debuginfod server artifacts.

I am hoping that helps both the Suse use case which would like a bootstrap/dummy libdebuginfod.so (--enable-libdebuginfod=dummy --disable-debuginfod) and the Arch use case which is to only have the client library, but not the debuginfod server (--enable-libdebuginfod --disable-debuginfod).

I admit that having the combination --enable-libdebuginfod=dummy and --enable-libdebuginfod is somewhat redundant/non-sensical, but it helps with (build time) testing. Other testing matrix would imho be as complicated (you'll get extra install flags or need to setup compile time or runtime environment variables).

> I am hoping that helps both the Suse use case which would like a
> bootstrap/dummy libdebuginfod.so (--enable-libdebuginfod=dummy
> --disable-debuginfod)

Yes, I can confirm the suggested scenario will work for us.
Thanks for working on that!

(In reply to Eli Schwartz from comment #13)
> I think we'd prefer to have the ability to build libdebuginfod without the
> server.

I believe you get that with --enable-libdebuginfod --disable-debuginfod, but maybe I don't understand your use case correctly. I tried to explain more fully in comment #17. Maybe you can try out the patch and let me know if it helps your use case or, if not, what would make things easier for you?

That should be fine, I think.

> At least for Arch Linux, curl is part of the base system while libmicrohttpd is not.

Actually truth be told, by now we moved libmicrohttpd to core, which solves our immediate problem there. But having --enable-libdebuginfod --disable-debuginfod would let us remove it again, which might be beneficial. So it's still IMHO a good-to-have. :)

(In reply to Mark Wielaard from comment #17)
> I admit that having the combination --enable-libdebuginfod=dummy and
> --enable-libdebuginfod is somewhat redundant/non-sensical, but it helps with
> (build time) testing. Other testing matrix would imho be as complicated
> (you'll get extra install flags or need to setup compile time or runtime
> environment variables).

Yeah, I just had a thought that you could avoid doing one configure/build/test with --enable-libdebuginfod=dummy and another with --enable-libdebuginfod.

If the buildsystem logic to have test-time env vars testing both feels too clumsy and you'd rather just build twice in your test matrix, then of course you are the boss. :D

Thanks for all the comments. Pushed to master now.

commit f7f0cdc59a13780938ae3f578955737a75e60ea9
Author: Mark Wielaard <mark@klomp.org>
Date:   Fri Jun 19 19:41:08 2020 +0200

    debuginfod: Add --disable-libdebuginfod and --enable-libdebuginfod=dummy.
    
    Make it possible to build just the debuginfod client or to create a
    dummy libdebuginfod that doesn't link against libcurl. The dummy library
    can be used for bootstrapping. For testing purposes you can also build
    debuginfod against the dummy libdebuginfod but then the debuginfod
    server will not be able to do delegation.
    
    Signed-off-by: Mark Wielaard <mark@klomp.org>