Bug 25583 - Use libarchive to extract .deb packages?
Summary: Use libarchive to extract .deb packages?
Status: RESOLVED FIXED
Alias: None
Product: elfutils
Classification: Unclassified
Component: debuginfod (show other bugs)
Version: unspecified
: P2 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-02-21 11:08 UTC by Ross Burton
Modified: 2020-03-26 19:52 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Last reconfirmed: 2020-02-22 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ross Burton 2020-02-21 11:08:54 UTC
Using rpm2cpio to extract an RPM isn't ideal because the machine serving the RPMs may not actually have that tool on.

However, libarchive which is already a dependency can extract RPMs directly as demonstrated with the libarchive example tool bsdtar:

$ bsdtar -tvf /home/ross/Yocto/openbmc/build/tmp/deploy/rpm/noarch/os-release-1.0-r0.noarch.rpm
drwxr-xr-x  1 0      0           0 Nov  8 16:41 ./etc
lrwxrwxrwx  1 0      0          21 Nov  8 16:41 ./etc/os-release -> ../usr/lib/os-release
drwxr-xr-x  1 0      0           0 Nov  8 16:41 ./usr
drwxr-xr-x  1 0      0           0 Nov  8 16:41 ./usr/lib
-rw-r--r--  1 0      0         287 Nov  8 16:41 ./usr/lib/os-release

Ditto for debs, dpkg-deb may not be present but libarchive can unpack the outer and give you access to the internals:

$ bsdtar  -tvf /var/cache/apt/archives/gdb_8.2.1-2+b1_amd64.deb
-rw-r--r--  0 0      0           4 Aug 28 16:18 debian-binary
-rw-r--r--  0 0      0        3024 Aug 28 16:18 control.tar.xz
-rw-r--r--  0 0      0     3131068 Aug 28 16:18 data.tar.xz
Comment 1 Frank Ch. Eigler 2020-02-22 00:57:08 UTC
Interesting idea.  OTOH, rpm2cpio and dpkg binaries are not too hard to come by.

One can experiment with the former already with git-master debuginfod with the
"-Z .rpm" option instead of "-R".  Performance seems to be roughly the same.

The latter is less compelling in that it'd require hard-coding the inner data.tar.xz name and its processing ... meh.

Maybe the status quo is good enough?
Comment 2 Mark Wielaard 2020-02-25 12:12:33 UTC
(In reply to Frank Ch. Eigler from comment #1)
> Interesting idea.  OTOH, rpm2cpio and dpkg binaries are not too hard to come
> by.
> 
> One can experiment with the former already with git-master debuginfod with
> the
> "-Z .rpm" option instead of "-R".  Performance seems to be roughly the same.

If this works as well, then I would simply switch -R to:

diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx
index 0acd70e4..c68aafa3 100644
--- a/debuginfod/debuginfod.cxx
+++ b/debuginfod/debuginfod.cxx
@@ -423,7 +423,7 @@ parse_opt (int key, char *arg,
       break;
     case 'F': scan_files = true; break;
     case 'R':
-      scan_archives[".rpm"]="rpm2cpio";
+      scan_archives[".rpm"]="cat";
       break;
     case 'U':
       scan_archives[".deb"]="dpkg-deb --fsys-tarfile";

It seems to just pass the run-debuginfod-find.sh testcase.


> The latter is less compelling in that it'd require hard-coding the inner
> data.tar.xz name and its processing ... meh.
> 
> Maybe the status quo is good enough?

If we can get rid of some extra dependencies with the same performance and no real downsides, except a couple of lines of extra code, I would go with this. Of course, still needs someone to write those couple of extra lines of code.
Comment 3 Frank Ch. Eigler 2020-02-25 15:21:08 UTC
ok, confirmed that relatively old bsdtar (libarchive 3.1.2 rhel7) supports rpm natively too.  Will switch that over as Mark suggests.
Comment 4 Frank Ch. Eigler 2020-02-25 19:31:23 UTC
pushed as obvious

diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx
index be3868bb1e42..7c7e85eb6d14 100644
--- a/debuginfod/debuginfod.cxx
+++ b/debuginfod/debuginfod.cxx
@@ -426,7 +426,7 @@ parse_opt (int key, char *arg,
       break;
     case 'F': scan_files = true; break;
     case 'R':
-      scan_archives[".rpm"]="rpm2cpio";
+      scan_archives[".rpm"]="cat"; // libarchive groks rpm natively
       break;
     case 'U':
       scan_archives[".deb"]="dpkg-deb --fsys-tarfile";
Comment 5 Mark Wielaard 2020-03-03 08:51:46 UTC
Great we have .rpm done now.

Lets keep this open in case someone wants to do the .deb variant too.

BTW. It seems we should tweak elfutils.spec too.
rpm isn't needed anymore, but we should recommend/require dpkg?

diff --git a/config/elfutils.spec.in b/config/elfutils.spec.in
index e992812a..ef7fe31f 100644
--- a/config/elfutils.spec.in
+++ b/config/elfutils.spec.in
@@ -157,8 +157,9 @@ Requires(post):   systemd
 Requires(preun):  systemd
 Requires(postun): systemd
 Requires(pre): shadow-utils
-# For /usr/bin/cpio2rpm
-Requires: rpm
+# To extract .deb files with dpkg-deb --fsys-tarfile
+# Can be Recommends if rpm supports that
+Requires: dpkg
 
 %description debuginfod-client
 The elfutils-debuginfod-client package contains shared libraries
Comment 6 Frank Ch. Eigler 2020-03-03 11:25:15 UTC
> -# For /usr/bin/cpio2rpm
> -Requires: rpm

Heh, that was always moot on an rpm based system (.spec file!).

> +# To extract .deb files with dpkg-deb --fsys-tarfile
> +# Can be Recommends if rpm supports that
> +Requires: dpkg

Sure, as long as one remembers to remove it from rhel distros that have dpkg only in epel.
Comment 7 Frank Ch. Eigler 2020-03-23 15:19:13 UTC
here's the recipe for .deb / .ddeb:

   -Z '.ddeb=(bsdtar -O -x -f - data.tar.xz)<'
Comment 9 Frank Ch. Eigler 2020-03-26 19:52:13 UTC
pushed as debuginfod-internal & uncontroversial