[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: a new tool fedabipkgdiff
Hi,
This is an update version for the new tool fedabipkgdiff. Thanks for Mr.
Dodji's help and patch to enable testing and installation of
fedabipkgdiff. Lots of fixes are done, and now --all-subpackages option
is supported in the 4th use case mentioned, that is all non-noarch RPM
packages within a build could be checked with abipkgdiff. Please review
and any feedback is appreciated.
On 02/17/2016 06:38 PM, Chenxiong Qi wrote:
Hi,
This is a new tool fedabipkgdiff that would be much convenient for
Fedora packagers to check potential ABI/API differences quickly using
abipkgdiff shipped with libabigail. This tool came from a cool idea
from Dodji. Currently, as the first step, it supports following ways,
fedabipkgdiff --from fc23 ./foo-0.1-1.fc23.x86_64.rpm
fedabipkgdiff --from fc23 --to fc24 foo
fedabipkgdiff foo-0.1-1.fc23 foo-0.1-1.fc24
fedabipkgdiff foo-0.1-1.fc23.i686 foo-0.1-1.fc24.i686
For more details, please refer to
https://sourceware.org/bugzilla/show_bug.cgi?id=19428
Next step is to support the 4th use case mentioned in bug 19428.
fedabipkgdiff is being under development, still need to improve.
Welcome any feedback. Thanks.
Happy hacking :)
Regards,
Chenxiong Qi
--
Regards,
Chenxiong Qi
>From 42761dc622218060cd6d7dfdf8774d6189c8788b Mon Sep 17 00:00:00 2001
From: Chenxiong Qi <cqi@redhat.com>
Date: Tue, 9 Feb 2016 18:05:33 +0800
Subject: [PATCH] new tool of fedabipkgdiff
fedabipkgdiff is a convenient way for Fedora packagers to inspect ABI
compatibility issues quickly.
Currently with the first version of fedabipkgdiff, you can invoke it in
following ways.
fedabipkgdiff --from fc23 foo-0.1-1.fc23.x86_64.rpm
fedabipkgdiff --from fc23 --to fc24 foo
fedabipkgdiff foo-0.1-1.fc23 foo-0.1-1.fc24
fedabipkgdiff foo-0.1-1.fc23.i686 foo-0.1-1.fc24.i686
fedabipkgdiff --all-subpackages foo-0.1-1.fc23 foo-0.1-1.fc24
* autoconf-archive/ax_compare_version.m4: New file copied from the
autoconf-archive project.
* autoconf-archive/ax_prog_python_version.m4: Likewise.
* autoconf-archive/ax_python_module.m4: Likewise.
* Makefile.am: Add the new files above to the source distribution.
* configure.ac: Include the new m4 macros from the autoconf
archive. Add a new --enable-fedabipkgdiff option. Update the
report at the end of the configure process to show the status of
the fedabipkgdiff feature. Add check for prerequisite python modules
itertools, shutil, unittest and mock. These are necessary for the
unit test of fedabipkgdiff. Generate tests/runtestfedabipkgdiff.py into the
build directory, from the tests/runtestfedabipkgdiff.py.in input file.
* tools/Makefile.am: Include the fedabipkgdiff to the source
distribution and install it if the "fedabipkgdiff" feature is
enabled.
* tests/Makefile.am: Rename runtestfedabipkgdiff.sh into
runtestfedabipkgdiff.py. Add the new runtestfedabipkgdiff.py.in
autoconf template file in here.
* tests/runtestfedabipkgdiff.py.in: New unit test file.
* tools/fedabipkgdiff: New tool fedabipkgdiff.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
---
.gitignore | 6 +
Makefile.am | 3 +
autoconf-archive/ax_compare_version.m4 | 177 +++++++
autoconf-archive/ax_prog_python_version.m4 | 66 +++
autoconf-archive/ax_python_module.m4 | 56 ++
configure.ac | 86 ++-
tests/Makefile.am | 6 +-
tests/runtestfedabipkgdiff.py.in | 450 ++++++++++++++++
tools/Makefile.am | 6 +
tools/fedabipkgdiff | 805 +++++++++++++++++++++++++++++
10 files changed, 1659 insertions(+), 2 deletions(-)
create mode 100644 autoconf-archive/ax_compare_version.m4
create mode 100644 autoconf-archive/ax_prog_python_version.m4
create mode 100644 autoconf-archive/ax_python_module.m4
create mode 100755 tests/runtestfedabipkgdiff.py.in
create mode 100755 tools/fedabipkgdiff
diff --git a/.gitignore b/.gitignore
index bb7c42a..a60cadb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,6 +6,7 @@ Makefile.in
*.lo
*.o
*~
+*.swp
/aclocal.m4
/autom4te.cache/
@@ -17,3 +18,8 @@ Makefile.in
/include/abg-version.h
/*.pc
+
+.tags
+build/
+TAGS
+fedabipkgdiffc
\ No newline at end of file
diff --git a/Makefile.am b/Makefile.am
index c855cf6..1ae2290 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -12,6 +12,9 @@ pkgconfig_DATA = libabigail.pc
#dist_bashcompletion_DATA =
EXTRA_DIST = \
+autoconf-archive/ax_python_module.m4 \
+autoconf-archive/ax_prog_python_version.m4 \
+autoconf-archive/ax_compare_version.m4 \
NEWS README COPYING ChangeLog \
COPYING-LGPLV2 COPYING-LGPLV3 \
COPYING-GPLV3 gen-changelog.py \
diff --git a/autoconf-archive/ax_compare_version.m4 b/autoconf-archive/ax_compare_version.m4
new file mode 100644
index 0000000..74dc0fd
--- /dev/null
+++ b/autoconf-archive/ax_compare_version.m4
@@ -0,0 +1,177 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_compare_version.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_COMPARE_VERSION(VERSION_A, OP, VERSION_B, [ACTION-IF-TRUE], [ACTION-IF-FALSE])
+#
+# DESCRIPTION
+#
+# This macro compares two version strings. Due to the various number of
+# minor-version numbers that can exist, and the fact that string
+# comparisons are not compatible with numeric comparisons, this is not
+# necessarily trivial to do in a autoconf script. This macro makes doing
+# these comparisons easy.
+#
+# The six basic comparisons are available, as well as checking equality
+# limited to a certain number of minor-version levels.
+#
+# The operator OP determines what type of comparison to do, and can be one
+# of:
+#
+# eq - equal (test A == B)
+# ne - not equal (test A != B)
+# le - less than or equal (test A <= B)
+# ge - greater than or equal (test A >= B)
+# lt - less than (test A < B)
+# gt - greater than (test A > B)
+#
+# Additionally, the eq and ne operator can have a number after it to limit
+# the test to that number of minor versions.
+#
+# eq0 - equal up to the length of the shorter version
+# ne0 - not equal up to the length of the shorter version
+# eqN - equal up to N sub-version levels
+# neN - not equal up to N sub-version levels
+#
+# When the condition is true, shell commands ACTION-IF-TRUE are run,
+# otherwise shell commands ACTION-IF-FALSE are run. The environment
+# variable 'ax_compare_version' is always set to either 'true' or 'false'
+# as well.
+#
+# Examples:
+#
+# AX_COMPARE_VERSION([3.15.7],[lt],[3.15.8])
+# AX_COMPARE_VERSION([3.15],[lt],[3.15.8])
+#
+# would both be true.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq],[3.15.8])
+# AX_COMPARE_VERSION([3.15],[gt],[3.15.8])
+#
+# would both be false.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq2],[3.15.8])
+#
+# would be true because it is only comparing two minor versions.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq0],[3.15])
+#
+# would be true because it is only comparing the lesser number of minor
+# versions of the two values.
+#
+# Note: The characters that separate the version numbers do not matter. An
+# empty string is the same as version 0. OP is evaluated by autoconf, not
+# configure, so must be a string, not a variable.
+#
+# The author would like to acknowledge Guido Draheim whose advice about
+# the m4_case and m4_ifvaln functions make this macro only include the
+# portions necessary to perform the specific comparison specified by the
+# OP argument in the final configure script.
+#
+# LICENSE
+#
+# Copyright (c) 2008 Tim Toolan <toolan@ele.uri.edu>
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 11
+
+dnl #########################################################################
+AC_DEFUN([AX_COMPARE_VERSION], [
+ AC_REQUIRE([AC_PROG_AWK])
+
+ # Used to indicate true or false condition
+ ax_compare_version=false
+
+ # Convert the two version strings to be compared into a format that
+ # allows a simple string comparison. The end result is that a version
+ # string of the form 1.12.5-r617 will be converted to the form
+ # 0001001200050617. In other words, each number is zero padded to four
+ # digits, and non digits are removed.
+ AS_VAR_PUSHDEF([A],[ax_compare_version_A])
+ A=`echo "$1" | sed -e 's/\([[0-9]]*\)/Z\1Z/g' \
+ -e 's/Z\([[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/[[^0-9]]//g'`
+
+ AS_VAR_PUSHDEF([B],[ax_compare_version_B])
+ B=`echo "$3" | sed -e 's/\([[0-9]]*\)/Z\1Z/g' \
+ -e 's/Z\([[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/[[^0-9]]//g'`
+
+ dnl # In the case of le, ge, lt, and gt, the strings are sorted as necessary
+ dnl # then the first line is used to determine if the condition is true.
+ dnl # The sed right after the echo is to remove any indented white space.
+ m4_case(m4_tolower($2),
+ [lt],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort -r | sed "s/x${A}/false/;s/x${B}/true/;1q"`
+ ],
+ [gt],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort | sed "s/x${A}/false/;s/x${B}/true/;1q"`
+ ],
+ [le],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort | sed "s/x${A}/true/;s/x${B}/false/;1q"`
+ ],
+ [ge],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort -r | sed "s/x${A}/true/;s/x${B}/false/;1q"`
+ ],[
+ dnl Split the operator from the subversion count if present.
+ m4_bmatch(m4_substr($2,2),
+ [0],[
+ # A count of zero means use the length of the shorter version.
+ # Determine the number of characters in A and B.
+ ax_compare_version_len_A=`echo "$A" | $AWK '{print(length)}'`
+ ax_compare_version_len_B=`echo "$B" | $AWK '{print(length)}'`
+
+ # Set A to no more than B's length and B to no more than A's length.
+ A=`echo "$A" | sed "s/\(.\{$ax_compare_version_len_B\}\).*/\1/"`
+ B=`echo "$B" | sed "s/\(.\{$ax_compare_version_len_A\}\).*/\1/"`
+ ],
+ [[0-9]+],[
+ # A count greater than zero means use only that many subversions
+ A=`echo "$A" | sed "s/\(\([[0-9]]\{4\}\)\{m4_substr($2,2)\}\).*/\1/"`
+ B=`echo "$B" | sed "s/\(\([[0-9]]\{4\}\)\{m4_substr($2,2)\}\).*/\1/"`
+ ],
+ [.+],[
+ AC_WARNING(
+ [illegal OP numeric parameter: $2])
+ ],[])
+
+ # Pad zeros at end of numbers to make same length.
+ ax_compare_version_tmp_A="$A`echo $B | sed 's/./0/g'`"
+ B="$B`echo $A | sed 's/./0/g'`"
+ A="$ax_compare_version_tmp_A"
+
+ # Check for equality or inequality as necessary.
+ m4_case(m4_tolower(m4_substr($2,0,2)),
+ [eq],[
+ test "x$A" = "x$B" && ax_compare_version=true
+ ],
+ [ne],[
+ test "x$A" != "x$B" && ax_compare_version=true
+ ],[
+ AC_WARNING([illegal OP parameter: $2])
+ ])
+ ])
+
+ AS_VAR_POPDEF([A])dnl
+ AS_VAR_POPDEF([B])dnl
+
+ dnl # Execute ACTION-IF-TRUE / ACTION-IF-FALSE.
+ if test "$ax_compare_version" = "true" ; then
+ m4_ifvaln([$4],[$4],[:])dnl
+ m4_ifvaln([$5],[else $5])dnl
+ fi
+]) dnl AX_COMPARE_VERSION
diff --git a/autoconf-archive/ax_prog_python_version.m4 b/autoconf-archive/ax_prog_python_version.m4
new file mode 100644
index 0000000..628a3e4
--- /dev/null
+++ b/autoconf-archive/ax_prog_python_version.m4
@@ -0,0 +1,66 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_prog_python_version.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_PROG_PYTHON_VERSION([VERSION],[ACTION-IF-TRUE],[ACTION-IF-FALSE])
+#
+# DESCRIPTION
+#
+# Makes sure that python supports the version indicated. If true the shell
+# commands in ACTION-IF-TRUE are executed. If not the shell commands in
+# ACTION-IF-FALSE are run. Note if $PYTHON is not set (for example by
+# running AC_CHECK_PROG or AC_PATH_PROG) the macro will fail.
+#
+# Example:
+#
+# AC_PATH_PROG([PYTHON],[python])
+# AX_PROG_PYTHON_VERSION([2.4.4],[ ... ],[ ... ])
+#
+# This will check to make sure that the python you have supports at least
+# version 2.4.4.
+#
+# NOTE: This macro uses the $PYTHON variable to perform the check.
+# AX_WITH_PYTHON can be used to set that variable prior to running this
+# macro. The $PYTHON_VERSION variable will be valorized with the detected
+# version.
+#
+# LICENSE
+#
+# Copyright (c) 2009 Francesco Salvestrini <salvestrini@users.sourceforge.net>
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 11
+
+AC_DEFUN([AX_PROG_PYTHON_VERSION],[
+ AC_REQUIRE([AC_PROG_SED])
+ AC_REQUIRE([AC_PROG_GREP])
+
+ AS_IF([test -n "$PYTHON"],[
+ ax_python_version="$1"
+
+ AC_MSG_CHECKING([for python version])
+ changequote(<<,>>)
+ python_version=`$PYTHON -V 2>&1 | $GREP "^Python " | $SED -e 's/^.* \([0-9]*\.[0-9]*\.[0-9]*\)/\1/'`
+ changequote([,])
+ AC_MSG_RESULT($python_version)
+
+ AC_SUBST([PYTHON_VERSION],[$python_version])
+
+ AX_COMPARE_VERSION([$ax_python_version],[le],[$python_version],[
+ :
+ $2
+ ],[
+ :
+ $3
+ ])
+ ],[
+ AC_MSG_WARN([could not find the python interpreter])
+ $3
+ ])
+])
diff --git a/autoconf-archive/ax_python_module.m4 b/autoconf-archive/ax_python_module.m4
new file mode 100644
index 0000000..f182c48
--- /dev/null
+++ b/autoconf-archive/ax_python_module.m4
@@ -0,0 +1,56 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_python_module.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_PYTHON_MODULE(modname[, fatal, python])
+#
+# DESCRIPTION
+#
+# Checks for Python module.
+#
+# If fatal is non-empty then absence of a module will trigger an error.
+# The third parameter can either be "python" for Python 2 or "python3" for
+# Python 3; defaults to Python 3.
+#
+# LICENSE
+#
+# Copyright (c) 2008 Andrew Collier
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 8
+
+AU_ALIAS([AC_PYTHON_MODULE], [AX_PYTHON_MODULE])
+AC_DEFUN([AX_PYTHON_MODULE],[
+ if test -z $PYTHON;
+ then
+ if test -z "$3";
+ then
+ PYTHON="python3"
+ else
+ PYTHON="$3"
+ fi
+ fi
+ PYTHON_NAME=`basename $PYTHON`
+ AC_MSG_CHECKING($PYTHON_NAME module: $1)
+ $PYTHON -c "import $1" 2>/dev/null
+ if test $? -eq 0;
+ then
+ AC_MSG_RESULT(yes)
+ eval AS_TR_CPP(HAVE_PYMOD_$1)=yes
+ else
+ AC_MSG_RESULT(no)
+ eval AS_TR_CPP(HAVE_PYMOD_$1)=no
+ #
+ if test -n "$2"
+ then
+ AC_MSG_ERROR(failed to find required module $1)
+ exit 1
+ fi
+ fi
+])
diff --git a/configure.ac b/configure.ac
index b611aca..ae7c11e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -14,6 +14,18 @@ AC_CONFIG_HEADER([config.h])
AC_CONFIG_SRCDIR([README])
AC_CONFIG_MACRO_DIR([m4])
+dnl Include some autoconf macros to check for python modules.
+dnl
+dnl These macros are coming from the autoconf archive at
+dnl http://www.gnu.org/software/autoconf-archive
+
+dnl This one is for the AX_PYTHON_MODULE() macro.
+m4_include([autoconf-archive/ax_python_module.m4])
+
+dnl These two below are for the AX_PROG_PYTHON_VERSION() module.
+m4_include([autoconf-archive/ax_compare_version.m4])
+m4_include([autoconf-archive/ax_prog_python_version.m4])
+
AM_INIT_AUTOMAKE([1.11.1 foreign subdir-objects tar-ustar parallel-tests])
AM_MAINTAINER_MODE([enable])
@@ -76,6 +88,12 @@ AC_ARG_ENABLE([bash-completion],
ENABLE_BASH_COMPLETION=$enableval,
ENABLE_BASH_COMPLETION=auto)
+AC_ARG_ENABLE([fedabipkgdiff],
+ AS_HELP_STRING([--enable-fedabipkgdiff=yes|no|auto],
+ [enable the fedabipkgdiff tool]),
+ ENABLE_FEDABIPKGDIFF=$enableval,
+ ENABLE_FEDABIPKGDIFF=auto)
+
dnl *************************************************
dnl check for dependencies
dnl *************************************************
@@ -219,6 +237,68 @@ fi
AM_CONDITIONAL(ENABLE_BASH_COMPLETION, test x$ENABLE_BASH_COMPLETION = xyes)
+dnl if --enable-fedabipkgdiff has the 'auto' value, then check for the required
+dnl python modules. If they are present, then enable the fedabipkgdiff program.
+dnl If they are not then disable the program.
+dnl
+dnl If --enable-fedabipkgdiff has the 'yes' value, then check for the required
+dnl python modules and whatever dependency fedabipkgdiff needs. If they are
+dnl not present then the configure script will error out.
+
+if test x$ENABLE_FEDABIPKGDIFF = xauto -o x$ENABLE_FEDABIPKGDIFF = xyes; then
+ CHECK_DEPS_FOR_FEDABIPKGDIFF=yes
+else
+ CHECK_DEPS_FOR_FEDABIPKGDIFF=no
+fi
+
+if test x$CHECK_DEPS_FOR_FEDABIPKGDIFF = xyes; then
+ if test x$ENABLE_FEDABIPKGDIFF = xyes; then
+ FATAL=yes
+ fi
+
+ AC_PATH_PROG(WGET, wget, no)
+
+ if test x$WGET = x$no; then
+ AC_MSG_ERROR(could not find the wget program)
+ fi
+
+ # The minimal python version we want to support is 2.6.6 because EL6
+ # distributions have that version installed.
+ MINIMAL_PYTHON_VERSION="2.6.6"
+
+ AC_PATH_PROG(PYTHON, python, no)
+ AX_PROG_PYTHON_VERSION($MINIMAL_PYTHON_VERSION,
+ [MINIMAL_PYTHON_VERSION_FOUND=yes],
+ [MINIMAL_PYTHON_VERSION_FOUND=no])
+
+ if test x$MINIMAL_PYTHON_VERSION_FOUND = xno; then
+ AC_MSG_ERROR([could not find a python program of version at least $MINIMAL_PYTHON_VERSION])
+ fi
+
+ AX_PYTHON_MODULE(argparse, $FATAL, python2)
+ AX_PYTHON_MODULE(glob, $FATAL, python2)
+ AX_PYTHON_MODULE(logging, $FATAL, python2)
+ AX_PYTHON_MODULE(os, $FATAL, python2)
+ AX_PYTHON_MODULE(re, $FATAL, python2)
+ AX_PYTHON_MODULE(shlex, $FATAL, python2)
+ AX_PYTHON_MODULE(subprocess, $FATAL, python2)
+ AX_PYTHON_MODULE(sys, $FATAL, python2)
+ AX_PYTHON_MODULE(itertools, $FATAL, python2)
+ AX_PYTHON_MODULE(urlparse, $FATAL, python2)
+ AX_PYTHON_MODULE(itertools, $FATAL, python2)
+ AX_PYTHON_MODULE(shutil, $FATAL, python2)
+ AX_PYTHON_MODULE(unittest, $FATAL, python2)
+ AX_PYTHON_MODULE(koji, $FATAL, python2)
+ AX_PYTHON_MODULE(mock, $FATAL, python2)
+ ENABLE_FEDABIPKGDIFF=yes
+
+ if test x$ENABLE_FEDABIPKGDIFF != xyes; then
+ ENABLE_FEDABIPKGDIFF=no
+ fi
+fi
+
+AM_CONDITIONAL(ENABLE_FEDABIPKGDIFF, test x$ENABLE_FEDABIPKGDIFF = xyes)
+
dnl Check for dependency: libzip
LIBZIP_VERSION=0.10.1
@@ -361,7 +441,10 @@ libabigail.pc
bash-completion/Makefile])
dnl Some test scripts are generated by autofoo.
-AC_CONFIG_FILES([tests/runtestcanonicalizetypes.sh], [chmod +x tests/runtestcanonicalizetypes.sh])
+AC_CONFIG_FILES([tests/runtestcanonicalizetypes.sh],
+ [chmod +x tests/runtestcanonicalizetypes.sh])
+AC_CONFIG_FILES([tests/runtestfedabipkgdiff.py],
+ [chmod +x tests/runtestfedabipkgdiff.py])
AC_OUTPUT
@@ -384,6 +467,7 @@ AC_MSG_NOTICE([
Enable deb support in abipkgdiff : ${ENABLE_DEB}
Enable GNU tar archive support in abipkgdiff : ${ENABLE_TAR}
Enable bash completion : ${ENABLE_BASH_COMPLETION}
+ Enable fedabipkgdiff : ${ENABLE_FEDABIPKGDIFF}
Generate html apidoc : ${ENABLE_APIDOC}
Generate html manual : ${ENABLE_MANUAL}
])
diff --git a/tests/Makefile.am b/tests/Makefile.am
index caf49e6..953dfef 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -31,9 +31,10 @@ runtestlookupsyms \
runtestaltdwarf \
runtestcorediff \
runtestabidiffexit \
+runtestfedabipkgdiff.py \
$(CXX11_TESTS)
-EXTRA_DIST = runtestcanonicalizetypes.sh.in
+EXTRA_DIST = runtestcanonicalizetypes.sh.in runtestfedabipkgdiff.py.in
CLEANFILES = \
runtestcanonicalizetypes.output.txt \
runtestcanonicalizetypes.output.final.txt
@@ -114,6 +115,9 @@ printdifftree_LDADD = $(top_builddir)/src/libabigail.la
runtestcanonicalizetypes_sh_SOURCES =
runtestcanonicalizetypes.sh$(EXEEXT):
+runtestfedabipkgdiff_py_SOURCES =
+runtestfedabipkgdiff.py$(EXEEXT):
+
AM_CPPFLAGS=-I${abs_top_srcdir}/include \
-I${abs_top_builddir}/include -I${abs_top_srcdir}/tools -fPIC
diff --git a/tests/runtestfedabipkgdiff.py.in b/tests/runtestfedabipkgdiff.py.in
new file mode 100755
index 0000000..3087212
--- /dev/null
+++ b/tests/runtestfedabipkgdiff.py.in
@@ -0,0 +1,450 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+# -*- Mode: Python
+#
+# This file is part of the GNU Application Binary Interface Generic
+# Analysis and Instrumentation Library. This program is free
+# software; you can redistribute it and/or modify it under the terms
+# of the GNU General Public License as published by the Free Software
+# Foundation; either version 3, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Lesser Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this program; see the file COPYING-LGPLV3. If
+# not, see <http://www.gnu.org/licenses/>.
+#
+# Author: Chenxiong Qi
+
+import os
+import itertools
+import shutil
+import unittest
+
+import koji
+
+try:
+ from mock import patch
+except ImportError:
+ print >>sys.stderr, \
+ 'mock is not installed. Please install it before running tests.'
+ sys.exit(1)
+
+import imp
+# Import the fedabipkgdiff program file from the source directory.
+fedabipkgdiff_mod = imp.load_source('fedabidiff',
+ '@top_srcdir@/tools/fedabipkgdiff')
+
+counter = itertools.count(0)
+
+
+class UtilsTest(unittest.TestCase):
+
+ def test_is_fedora_distro(self):
+ distro = 'fc5'
+ self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'f5'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'fc23'
+ self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'fc'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'fc234'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'el7'
+ self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'el7_2'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+
+class RPMTest(unittest.TestCase):
+ """Test case for RPM class"""
+
+ def setUp(self):
+ self.debuginfo_rpm_info = {
+ 'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'
+ }
+ self.rpm_info = {
+ 'arch': 'x86_64',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'
+ }
+
+ def test_attribute_access(self):
+ rpm = fedabipkgdiff_mod.RPM(self.debuginfo_rpm_info)
+ self.assertEquals(self.debuginfo_rpm_info['arch'], rpm.arch)
+ self.assertEquals(self.debuginfo_rpm_info['name'], rpm.name)
+ self.assertEquals(self.debuginfo_rpm_info['release'], rpm.release)
+ self.assertEquals(self.debuginfo_rpm_info['version'], rpm.version)
+
+ def test_raise_error_if_name_not_exist(self):
+ rpm = fedabipkgdiff_mod.RPM({})
+ try:
+ rpm.xxxxx
+ except AttributeError:
+ # Succeed, exit normally
+ return
+ self.fail('AttributeError should be raised, but not.')
+
+ def test_is_debuginfo(self):
+ rpm = fedabipkgdiff_mod.RPM(self.debuginfo_rpm_info)
+ self.assertTrue(rpm.is_debuginfo)
+
+ rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
+ self.assertFalse(rpm.is_debuginfo)
+
+ def test_nvra(self):
+ rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
+ nvra = koji.parse_NVRA(rpm.nvra)
+ self.assertEquals(nvra['name'], rpm.name)
+ self.assertEquals(nvra['version'], rpm.version)
+ self.assertEquals(nvra['release'], rpm.release)
+ self.assertEquals(nvra['arch'], rpm.arch)
+
+ def test_str_representation(self):
+ rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
+ self.assertEquals(str(self.rpm_info), str(rpm))
+
+
+class LocalRPMTest(unittest.TestCase):
+ """Test case for local RPM"""
+
+ def setUp(self):
+ self.filename = 'httpd-2.4.18-1.fc22.x86_64.rpm'
+
+ def test_file_parser(self):
+ rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
+ nvra = koji.parse_NVRA(self.filename)
+ self.assertEquals(nvra['name'], rpm.name)
+ self.assertEquals(nvra['version'], rpm.version)
+ self.assertEquals(nvra['release'], rpm.release)
+ self.assertEquals(nvra['arch'], rpm.arch)
+
+ full_filename = os.path.join('/', 'tmp', self.filename)
+ rpm = fedabipkgdiff_mod.LocalRPM(full_filename)
+ nvra = koji.parse_NVRA(self.filename)
+ self.assertEquals(nvra['name'], rpm.name)
+ self.assertEquals(nvra['version'], rpm.version)
+ self.assertEquals(nvra['release'], rpm.release)
+ self.assertEquals(nvra['arch'], rpm.arch)
+ self.assertEquals(full_filename, rpm.downloaded_file)
+
+ @patch('os.path.exists')
+ def test_find_existent_debuginfo(self, mock_exists):
+ mock_exists.return_value = True
+
+ rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
+ self.assertTrue(isinstance(rpm, fedabipkgdiff_mod.LocalRPM))
+
+ nvra = koji.parse_NVRA(self.filename)
+ expected_debuginfo = fedabipkgdiff_mod.LocalRPM(
+ '%(name)s-debuginfo-%(version)s-%(release)s.%(arch)s.rpm' % nvra)
+ debuginfo = rpm.find_debuginfo()
+ self.assertEquals(expected_debuginfo.name, debuginfo.name)
+ self.assertEquals(expected_debuginfo.version, debuginfo.version)
+ self.assertEquals(expected_debuginfo.release, debuginfo.release)
+
+ def test_find_non_existent_debuginfo(self):
+ rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
+ self.assertEquals(None, rpm.find_debuginfo())
+
+
+class RunAbipkgdiffTest(unittest.TestCase):
+ """Test case for method run_abipkgdiff"""
+
+ def setUp(self):
+ self.pkg1_single_info = {
+ 'i686': [
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ })
+ ],
+ }
+
+ # Whatever the concrete content of pkg2_infos is, so just make a copy
+ # from self.pkg1_infos
+ self.pkg2_single_info = self.pkg1_single_info.copy()
+
+ self.pkg1_infos = {
+ 'i686': [
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ ],
+ 'x86_64': [
+ fedabipkgdiff_mod.RPM({'arch': 'x86_64',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'x86_64',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ ],
+ 'armv7hl': [
+ fedabipkgdiff_mod.RPM({'arch': 'armv7hl',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'armv7hl',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ ],
+ }
+
+ # Whatever the concrete content of pkg2_infos is, so just make a copy
+ # from self.pkg1_infos
+ self.pkg2_infos = self.pkg1_infos.copy()
+
+ @patch('fedabidiff.abipkgdiff')
+ def test_all_success(self, mock_abipkgdiff):
+ mock_abipkgdiff.return_value = 0
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_single_info,
+ self.pkg2_single_info)
+ self.assertEquals(0, result)
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
+ self.pkg2_infos)
+ self.assertEquals(0, result)
+
+ @patch('fedabidiff.abipkgdiff')
+ def test_all_failure(self, mock_abipkgdiff):
+ mock_abipkgdiff.return_value = 4
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_single_info,
+ self.pkg2_single_info)
+ self.assertEquals(4, result)
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
+ self.pkg2_infos)
+ self.assertEquals(4, result)
+
+ @patch('fedabidiff.abipkgdiff', new=lambda param1, param2: counter.next())
+ def test_partial_failure(self):
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
+ self.pkg2_infos)
+ self.assertTrue(result > 0)
+
+
+fake_rpm_file = 'foo-0.1-1.fc24.x86_64.rpm'
+
+
+class MockGlobalConfig(object):
+ koji_server = fedabipkgdiff_mod.DEFAULT_KOJI_SERVER
+
+
+def mock_get_session():
+ return MockKojiClientSession(baseurl=fedabipkgdiff_mod.DEFAULT_KOJI_SERVER)
+
+
+class MockKojiClientSession(object):
+
+ def __init__(self, *args, **kwargs):
+ """Accept arbitrary parameters but do nothing for this mock"""
+ self.args = args
+ self.kwargs = kwargs
+
+ def getPackage(self, *args, **kwargs):
+ return {
+ 'id': 1,
+ 'name': 'whatever a name of a package',
+ }
+
+ def listRPMs(self, *args, **kwargs):
+ return [{'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'nvr': 'httpd-debuginfo-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_session',
+ 'nvr': 'mod_session-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_proxy_html',
+ 'nvr': 'mod_proxy_html-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ldap',
+ 'nvr': 'mod_ldap-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ssl',
+ 'nvr': 'mod_ssl-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'}]
+
+ def listBuilds(self, *args, **kwargs):
+ return [
+ {'build_id': 720222,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-2.fc24',
+ 'release': '2.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 708769,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'build_id': 708711,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc23',
+ 'release': '1.fc23',
+ 'version': '2.4.18'},
+ {'build_id': 705335,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc24',
+ 'release': '1.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 704434,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc24',
+ 'release': '4.fc24',
+ 'version': '2.4.17'},
+ {'build_id': 704433,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc23',
+ 'release': '4.fc23',
+ 'version': '2.4.17'},
+ ]
+
+
+class SelectRpmsFromABuildTest(unittest.TestCase):
+ """Test case for select_rpms_from_a_build"""
+
+ def assert_rpms(self, rpms):
+ for item in rpms:
+ self.assertTrue(item.arch in ['i686', 'x86_64'])
+ self.assertTrue(item.name in ('httpd', 'httpd-debuginfo'))
+
+ @patch('fedabidiff.Brew.listRPMs')
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ def test_select_rpms_from_all_arches(self, mock_listRPMs):
+ mock_listRPMs.return_value = [
+ {'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'x86_64',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'x86_64',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ ]
+
+ session = fedabipkgdiff_mod.get_session()
+ rpms = session.select_rpms_from_a_build(1, 'httpd')
+ self.assert_rpms(rpms)
+
+ @patch('fedabidiff.Brew.listRPMs')
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ def test_select_rpms_from_one_arch(self, mock_listRPMs):
+ mock_listRPMs.return_value = [
+ {'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ ]
+
+ session = fedabipkgdiff_mod.get_session()
+ rpms = session.select_rpms_from_a_build(1, 'httpd')
+ self.assert_rpms(rpms)
+
+
+class GetPackageLatestBuildTest(unittest.TestCase):
+ """Test case for get_package_latest_build"""
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_get_latest_one(self):
+ session = fedabipkgdiff_mod.get_session()
+ build = session.get_package_latest_build('httpd', 'fc23')
+ self.assertEquals('httpd-2.4.18-1.fc23', build['nvr'])
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_cannot_find_a_latest_build_with_invalid_distro(self):
+ session = fedabipkgdiff_mod.get_session()
+ self.assertRaises(fedabipkgdiff_mod.NoCompleteBuilds,
+ session.get_package_latest_build, 'httpd', 'xxxx')
+
+
+class BrewListRPMsTest(unittest.TestCase):
+ """Test case for Brew.listRPMs"""
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('fedabidiff.koji.ClientSession', new=MockKojiClientSession)
+ def test_select_specific_rpms(self):
+ session = fedabipkgdiff_mod.get_session()
+ selector = lambda rpm: rpm['name'].startswith('httpd')
+ rpms = session.listRPMs(buildID=1000, selector=selector)
+ self.assertTrue(
+ len(rpms) > 0,
+ 'More than one rpms should be selected. But, it\'s empty.')
+ for rpm in rpms:
+ self.assertTrue(rpm['name'] in ('httpd', 'httpd-debuginfo'),
+ '{0} should not be selected'.format(rpm['name']))
+
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/tools/Makefile.am b/tools/Makefile.am
index b855f41..0d96215 100644
--- a/tools/Makefile.am
+++ b/tools/Makefile.am
@@ -6,6 +6,12 @@ else
bin_PROGRAMS = abidiff abilint abidw abicompat abipkgdiff
endif
+if ENABLE_FEDABIPKGDIFF
+ bin_SCRIPTS = fedabipkgdiff
+else
+ noinst_SCRIPTS = fedabipkgdiff
+endif
+
noinst_PROGRAMS = abisym abinilint
if ENABLE_ZIP_ARCHIVE
diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
new file mode 100755
index 0000000..4a5ba80
--- /dev/null
+++ b/tools/fedabipkgdiff
@@ -0,0 +1,805 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+# -*- Mode: Python
+#
+# This file is part of the GNU Application Binary Interface Generic
+# Analysis and Instrumentation Library. This program is free
+# software; you can redistribute it and/or modify it under the terms
+# of the GNU General Public License as published by the Free Software
+# Foundation; either version 3, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Lesser Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this program; see the file COPYING-LGPLV3. If
+# not, see <http://www.gnu.org/licenses/>.
+#
+# Author: Chenxiong Qi
+
+import argparse
+import logging
+import os
+import re
+import shlex
+import subprocess
+import sys
+
+from collections import namedtuple
+from itertools import groupby
+
+import koji
+
+
+DEFAULT_KOJI_SERVER = 'http://koji.fedoraproject.org/kojihub'
+DEFAULT_KOJI_TOPDIR = 'https://kojipkgs.fedoraproject.org'
+
+HOME_DIR = os.path.join('/tmp',
+ os.path.splitext(os.path.basename(__file__))[0])
+
+# Used to construct abipkgdiff command line argument, package and associated
+# debuginfo package
+PkgInfo = namedtuple('PkgInfo', 'package debuginfo_package')
+
+
+global_config = None
+pathinfo = None
+session = None
+
+logging.basicConfig(format='[%(levelname)s] %(message)s',
+ level=logging.CRITICAL)
+logger = logging.getLogger(os.path.basename(__file__))
+
+
+class KojiPackageNotFound(Exception):
+ """Package is not found in Koji"""
+
+
+class PackageNotFound(Exception):
+ """Package is not found locally"""
+
+
+class RpmNotFound(Exception):
+ """RPM is not found"""
+
+
+class NoBuildsError(Exception):
+ """No builds returned from a method to select specific builds"""
+
+
+class NoCompleteBuilds(Exception):
+ """No complete builds for a package
+
+ This is a serious problem, nothing can be done if there is no complete
+ builds for a package.
+ """
+
+
+class InvalidDistroError(Exception):
+ """Invalid distro error"""
+
+
+class CannotFindLatestBuildError(Exception):
+ """Cannot find latest build from a package"""
+
+
+def is_distro_valid(distro):
+ """Adjust if a distro is valid
+
+ Currently, check for Fedora and RHEL.
+
+ :param str distro: a string representing a distro value.
+ :return: True if distro is the one specific to Fedora, like fc24, el7.
+ "rtype: bool
+ """
+ return re.match(r'^(fc|el)\d{1,2}$', distro) is not None
+
+
+def log_call(func):
+ def proxy(*args, **kwargs):
+ logger.debug('Call %s, args: %s, kwargs: %s',
+ func.__name__,
+ args if args else '',
+ kwargs if kwargs else '')
+ result = func(*args, **kwargs)
+ logger.debug('Result from %s: %s', func.__name__, result)
+ return result
+ return proxy
+
+
+class RPM(object):
+ """Represeting a RPM"""
+
+ def __init__(self, data):
+ """Initialize a RPM object
+
+ :param dict data: a dict representing a RPM information got from koji
+ API, either listRPMs or getRPM
+ """
+ self.data = data
+
+ def __str__(self):
+ return str(self.data)
+
+ def __getattr__(self, name):
+ if name in self.data:
+ return self.data[name]
+ else:
+ raise AttributeError('No attribute name {0}'.format(name))
+
+ @property
+ def nvra(self):
+ return '%(name)s-%(version)s-%(release)s.%(arch)s' % self.data
+
+ @property
+ def filename(self):
+ return '{0}.rpm'.format(self.nvra)
+
+ @property
+ def is_debuginfo(self):
+ """Check if a RPM is a debuginfo"""
+ return koji.is_debuginfo(self.data['name'])
+
+ @property
+ def download_url(self):
+ """Get the URL from where to download from koji"""
+ build = session.getBuild(self.build_id)
+ return os.path.join(pathinfo.build(build), pathinfo.rpm(self.data))
+
+ @property
+ def downloaded_file(self):
+ """Get a pridictable downloaded file name with absolute path"""
+ # arch should be removed from the result returned from PathInfo.rpm
+ filename = os.path.basename(pathinfo.rpm(self.data))
+ return os.path.join(get_download_dir(), filename)
+
+ @property
+ def is_downloaded(self):
+ return os.path.exists(self.downloaded_file)
+
+
+class LocalRPM(RPM):
+ """Representing a local RPM
+
+ Local RPM means the one that could be already downloaded or built from
+ where I can find it
+ """
+
+ def __init__(self, filename):
+ self.local_filename = filename
+ self.data = koji.parse_NVRA(os.path.basename(filename))
+
+ @property
+ def downloaded_file(self):
+ return self.local_filename
+
+ @property
+ def download_url(self):
+ raise NotImplementedError('LocalRPM has no URL to download')
+
+ @log_call
+ def find_debuginfo(self):
+ """Find debuginfo rpm package from a directory
+
+ :param str rpm_file: the rpm file name
+ :return: the absolute file name of the found debuginfo rpm
+ :rtype: str or None
+ """
+ search_dir = os.path.dirname(os.path.abspath(self.local_filename))
+ filename = \
+ '%(name)s-debuginfo-%(version)s-%(release)s.%(arch)s.rpm' % \
+ self.data
+ filename = os.path.join(search_dir, filename)
+ return LocalRPM(filename) if os.path.exists(filename) else None
+
+
+class Brew(object):
+ """Proxy to kojihub XMLRPC with additional extensions to fedabipkgdiff
+
+ kojihub XMLRPC APIs are well-documented in koji's source code. For more
+ details information, please refer to class RootExports within kojihub.py.
+ """
+
+ def __init__(self, baseurl):
+ """Initialize Brew that is a proxy to koji.ClientSession"""
+ self.session = koji.ClientSession(baseurl)
+
+ @log_call
+ def listRPMs(self, selector=None, **kwargs):
+ """Proxy to kojihub.listRPMs
+
+ :param selector: to adjust if a RPM should be selected
+ :type selector: a callable object
+ :param kwargs: keyword parameters accepted by kojihub.listRPMs
+ :type kwargs: dict
+ :return: a list of RPMs, each of them is a dict object
+ :rtype: list
+ """
+ if selector:
+ assert hasattr(selector, '__call__'), 'selector should be callable'
+ rpms = self.session.listRPMs(**kwargs)
+ if selector:
+ rpms = [rpm for rpm in rpms if selector(rpm)]
+ return rpms
+
+ @log_call
+ def getRPM(self, *args, **kwargs):
+ rpm = self.session.getRPM(*args, **kwargs)
+ if rpm is None:
+ raise RpmNotFound('Cannot find RPM {0}'.format(args[0]))
+ return rpm
+
+ @log_call
+ def listBuilds(self, topone=None, selector=None, order_by=None,
+ reverse=None, **kwargs):
+ """Proxy to kojihub.listBuilds to list completed builds
+
+ Suport additional two keyword parameters:
+
+ :param bool topone: whether to return the top first one
+ :param selector: a callable object used to select specific subset of
+ builds
+ :type selector: callable object
+ :param str order_by: the attribute name by which to order the builds,
+ for example, name, version, or nvr.
+ :param bool reverse: whether to order builds reversely
+ :param dict kwargs: keyword parameters accepted by kojihub.listBuilds
+ :return: a list of builds, even if just return only one build
+ :rtype: list
+ """
+ if 'state' not in kwargs:
+ kwargs['state'] = koji.BUILD_STATES['COMPLETE']
+
+ if selector is not None and not hasattr(selector, '__call__'):
+ raise TypeError(
+ '{0} is not a callable object.'.format(str(selector)))
+
+ if order_by is not None and not isinstance(order_by, basestring):
+ raise TypeError('order_by {0} is invalid.'.format(order_by))
+
+ builds = self.session.listBuilds(**kwargs)
+ if selector is not None:
+ builds = [build for build in builds if selector(build)]
+ if order_by is not None:
+ # FIXME: is it possible to sort builds by using opts parameter of
+ # listBuilds
+ builds = sorted(builds,
+ key=lambda item: item[order_by],
+ reverse=reverse)
+ if topone:
+ builds = builds[0:1]
+
+ return builds
+
+ @log_call
+ def getPackage(self, name):
+ """Proxy to kojihub.getPackage
+
+ :param str name: package name
+ :return: a dict object representing a package
+ :rtype: dict
+ """
+ package = self.session.getPackage(name)
+ if package is None:
+ package = self.session.getPackage(name.rsplit('-', 1)[0])
+ if package is None:
+ raise KojiPackageNotFound(
+ 'Cannot find package {0}.'.format(name))
+ return package
+
+ @log_call
+ def getBuild(self, *args, **kwargs):
+ """Proxy to kojihub.getBuild"""
+ return self.session.getBuild(*args, **kwargs)
+
+ @log_call
+ def get_rpm_build_id(self, name, version, release, arch=None):
+ """Get build ID that contains a rpm with specific nvra
+
+ If arch is omitted, a rpm can be identified easily by its N-V-R-A.
+
+ If arch is omitted, name is used to get associated package, and then
+ to get the build.
+
+ :param str name: name of a rpm
+ :param str version: version of a rpm
+ :param str release: release of a rpm
+ :param arch: arch of a rpm
+ :type arch: str or None
+ :return: the build from where the rpm is built
+ :rtype: dict
+ :raises KojiPackageNotFound: if name is not found from koji when arch
+ is None
+ """
+ if arch is None:
+ package = self.getPackage(name)
+ selector = lambda item: item['version'] == version and \
+ item['release'] == release
+ builds = self.listBuilds(packageID=package['id'],
+ selector=selector)
+ if not builds:
+ raise NoBuildsError(
+ 'No builds are selected from package {0}.'.format(
+ package['name']))
+ return builds[0]['build_id']
+ else:
+ rpm = self.getRPM({'name': name,
+ 'version': version,
+ 'release': release,
+ 'arch': arch,
+ })
+ return rpm['build_id']
+
+ @log_call
+ def get_package_latest_build(self, package_name, distro):
+ """Get latest build from a package
+
+ :param str package_name: from which package to get the latest build
+ :param str distro: which distro the latest build belongs to
+ :return: the found build
+ :rtype: dict or None
+ """
+ package = self.getPackage(package_name)
+ selector = lambda item: item['release'].find(distro) > -1
+
+ builds = self.listBuilds(packageID=package['id'],
+ selector=selector,
+ order_by='nvr',
+ reverse=True)
+ if not builds:
+ raise NoCompleteBuilds(
+ 'No complete builds of package {0}'.format(package_name))
+
+ return builds[0]
+
+ @log_call
+ def select_rpms_from_a_build(self, build_id, package_name, arches=None,
+ select_subpackages=None):
+ """Select specific RPMs within a build
+
+ rpms could be filtered be specific criterias by the parameters.
+
+ :param int build_id: from which build to select rpms.
+ :param str package_name: which rpm to select that matches this name.
+ :param arches: which arches to select. If arches omits, rpms with all
+ arches except noarch and src will be selected.
+ :type arches: list, tuple or None
+ :return: a list of rpms returned from listRPMs
+ :rtype: list
+ """
+ excluded_arches = ('noarch', 'src')
+
+ def rpms_selector(package_name, excluded_arches):
+ return lambda rpm: \
+ rpm['arch'] not in excluded_arches and \
+ (rpm['name'] == package_name or
+ rpm['name'].endswith('-debuginfo'))
+
+ if select_subpackages:
+ selector = lambda rpm: rpm['arch'] not in excluded_arches
+ else:
+ selector = rpms_selector(package_name, excluded_arches)
+ rpm_infos = self.listRPMs(buildID=build_id,
+ arches=arches,
+ selector=selector)
+ return [RPM(rpm_info) for rpm_info in rpm_infos]
+
+ @log_call
+ def get_latest_built_rpms(self, package_name, distro, arches=None):
+ """Get rpms from latest build of a package
+
+ By default, debuginfo rpm is also retrieved.
+
+ :param str package_name: from which package to get the rpms
+ :param str distro: which distro the rpms belong to
+ :param arches: which arches the rpms belong to
+ :type arches: str or None
+ :return: the selected rpms
+ :rtype: list
+ """
+ latest_build = self.get_package_latest_build(package_name, distro)
+ # Get rpm and debuginfo rpm from each arch
+ return self.select_rpms_from_a_build(latest_build['build_id'],
+ package_name,
+ arches=arches)
+
+
+@log_call
+def get_session():
+ return Brew(global_config.koji_server)
+
+
+@log_call
+def get_download_dir():
+ """Return the directory holding all downloaded rpms"""
+ download_dir = os.path.join(HOME_DIR, 'downloads')
+ if not os.path.exists(download_dir):
+ os.makedirs(download_dir)
+ return download_dir
+
+
+@log_call
+def download_rpm(url):
+ """Download a rpm"""
+ # TODO: wget is good, but there maybe a better way to do this
+ cmd = shlex.split('wget -q -P {0} -c {1}'.format(get_download_dir(), url))
+ proc = subprocess.Popen(cmd)
+ s_stdout, s_stderr = proc.communicate()
+ if proc.returncode > 0:
+ logger.error('wget fails. returned code: %d. message: %s',
+ proc.returncode, s_stderr)
+ return False
+ return True
+
+
+@log_call
+def download_rpms(pkg_info):
+ def _download(rpm):
+ if rpm.is_downloaded:
+ logger.debug('Reuse %s', rpm.downloaded_file)
+ else:
+ logger.debug('Download %s', rpm.download_url)
+ download_rpm(rpm.download_url)
+
+ for arch, rpm_infos in pkg_info.iteritems():
+ map(_download, rpm_infos)
+
+
+@log_call
+def abipkgdiff(pkg_info1, pkg_info2):
+ """Run abipkgdiff against found two RPM packages"""
+
+ cmd = 'abipkgdiff --d1 {0} --d2 {1} {2} {3}'.format(
+ pkg_info1.debuginfo_package.downloaded_file,
+ pkg_info2.debuginfo_package.downloaded_file,
+ pkg_info1.package.downloaded_file,
+ pkg_info2.package.downloaded_file)
+
+ if global_config.dry_run:
+ print 'DRY-RUN:', cmd
+ return
+
+ logger.debug('Run: %s', cmd)
+
+ print 'ABI check on {0} and {1}'.format(pkg_info1.package.filename,
+ pkg_info2.package.filename)
+ print
+
+ proc = subprocess.Popen(shlex.split(cmd))
+ return proc.wait()
+
+
+def magic_construct(rpms):
+ """Construct RPMs into a magic structure
+
+ Convert list of
+
+ foo-1.0-1.fc22.i686
+ foo-debuginfo-1.0-1.fc22.i686
+ foo-devel-1.0-1.fc22.i686
+
+ to list of
+
+ (foo-1.0-1.fc22.i686, foo-debuginfo-1.0-1.fc22.i686)
+ (foo-devel-1.0-1.fc22.i686, foo-debuginfo-1.0-1.fc22.i686)
+ """
+ debuginfo = None
+ packages = []
+ for rpm in rpms:
+ if rpm.is_debuginfo:
+ debuginfo = rpm
+ else:
+ packages.append(rpm)
+ return [PkgInfo(package, debuginfo) for package in packages]
+
+
+@log_call
+def run_abipkgdiff(pkg1_infos, pkg2_infos):
+ """Run abipkgdiff
+
+ If one of the executions finds ABI differences, the return code is the
+ return code from abipkgdiff.
+
+ :param dict pkg1_infos: a dict mapping from arch to list of rpms, that is
+ returned from method make_rpms_usable_for_abipkgdiff
+ :return: exit code of the last non-zero returned from underlying abipkgdiff
+ :rtype: number
+ """
+ arches = pkg1_infos.keys()
+ arches.sort()
+
+ return_code = 0
+
+ for arch in arches:
+ pkg_infos = magic_construct(pkg1_infos[arch])
+
+ for pkg_info in pkg_infos:
+ rpms = pkg2_infos[arch]
+
+ package = [rpm for rpm in rpms
+ if rpm.name == pkg_info.package.name][0]
+ debuginfo = [rpm for rpm in rpms
+ if rpm.name == pkg_info.debuginfo_package.name][0]
+
+ ret = abipkgdiff(pkg_info,
+ PkgInfo(package=package,
+ debuginfo_package=debuginfo))
+ if ret > 0:
+ return_code = ret
+
+ return return_code
+
+
+@log_call
+def diff_local_rpm_with_latest_rpm_from_koji():
+ """Diff against local rpm and remove latest rpm
+
+ This operation handles a local rpm and debuginfo rpm and remote ones
+ located in remote Koji server, that has specific distro specificed by
+ argument --from.
+
+ 1/ Suppose the packager has just locally built a package named
+ foo-3.0.fc24.rpm. To compare the ABI of this locally build package with the
+ latest stable package from Fedora 23, one would do:
+
+ fedabipkgdiff --from f23 ./foo-3.0.fc24.rpm
+ """
+
+ from_distro = global_config.from_distro
+ if not is_distro_valid(from_distro):
+ raise InvalidDistroError('Invalid distro {0}'.format(from_distro))
+
+ local_rpm_file = global_config.NVR[0]
+ if not os.path.exists(local_rpm_file):
+ raise ValueError('{0} does not exist.'.format(local_rpm_file))
+
+ local_rpm = LocalRPM(local_rpm_file)
+ local_debuginfo = local_rpm.find_debuginfo()
+ if local_debuginfo is None:
+ raise ValueError(
+ 'debuginfo rpm {0} does not exist.'.format(local_debuginfo))
+
+ rpms = session.get_latest_built_rpms(local_rpm.name,
+ from_distro,
+ arches=local_rpm.arch)
+ pkg_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg_infos)
+
+ rpms = pkg_infos.values()[0]
+ package, debuginfo = sorted(rpms, key=lambda rpm: rpm.name)
+ return abipkgdiff(PkgInfo(package, debuginfo),
+ PkgInfo(local_rpm, local_debuginfo))
+
+
+@log_call
+def make_rpms_usable_for_abipkgdiff(rpms):
+ """
+ Construct result that contains mappings from arch to download url and
+ downloaded rpm filename of rpm and debuginfo rpm
+
+ :return: a mapping from an arch to a list of rpms
+ :rtype: dict
+ """
+ result = {}
+ rpms_iter = groupby(sorted(rpms, key=lambda rpm: rpm.arch),
+ key=lambda item: item.arch)
+ for arch, rpms in rpms_iter:
+ result[arch] = list(rpms)
+ return result
+
+
+@log_call
+def diff_latest_rpms_based_on_distros():
+ """abipkgdiff rpms based on two distros
+
+ 2/ Suppose the packager wants to see how the ABIs of the package foo
+ evolved between fedora 19 and fedora 22. She would thus type the command:
+
+ fedabipkgdiff --from f19 --to f22 foo
+ """
+
+ from_distro = global_config.from_distro
+ to_distro = global_config.to_distro
+
+ if not is_distro_valid(from_distro):
+ raise InvalidDistroError('Invalid distro {0}'.format(from_distro))
+
+ if not is_distro_valid(to_distro):
+ raise InvalidDistroError('Invalid distro {0}'.format(distro))
+
+ package_name = global_config.NVR[0]
+
+ rpms = session.get_latest_built_rpms(package_name,
+ distro=global_config.from_distro)
+ pkg1_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg1_infos)
+
+ rpms = session.get_latest_built_rpms(package_name,
+ distro=global_config.to_distro)
+ pkg2_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg2_infos)
+
+ return run_abipkgdiff(pkg1_infos, pkg2_infos)
+
+
+@log_call
+def diff_rpms_with_nvra(name, version, release, arch=None,
+ all_subpackages=None):
+ build_id = session.get_rpm_build_id(name, version, release, arch)
+ rpms = session.select_rpms_from_a_build(build_id, name, arches=arch,
+ select_subpackages=all_subpackages)
+ return make_rpms_usable_for_abipkgdiff(rpms)
+
+
+@log_call
+def diff_two_nvras_from_koji():
+ """Diff two nvras from koji
+
+ The arch probably omits, that means febabipkgdiff will diff all arches. If
+ specificed, the specific arch will be handled.
+
+ 3/ Suppose the packager wants to compare the ABI of two packages designated
+ by their name and version. She would issue a command like this:
+
+ fedabipkgdiff foo-1.0.fc19 foo-3.0.fc24
+ fedabipkgdiff foo-1.0.fc19.i686 foo-1.0.fc24.i686
+ """
+ left_rpm = koji.parse_NVRA(global_config.NVR[0])
+ right_rpm = koji.parse_NVRA(global_config.NVR[1])
+
+ if is_distro_valid(left_rpm['arch']) and \
+ is_distro_valid(right_rpm['arch']):
+ nvr = koji.parse_NVR(global_config.NVR[0])
+ params1 = (nvr['name'], nvr['version'], nvr['release'])
+
+ nvr = koji.parse_NVR(global_config.NVR[1])
+ params2 = (nvr['name'], nvr['version'], nvr['release'])
+ else:
+ params1 = (left_rpm['name'],
+ left_rpm['version'],
+ left_rpm['release'],
+ left_rpm['arch'])
+ params2 = (right_rpm['name'],
+ right_rpm['version'],
+ right_rpm['release'],
+ right_rpm['arch'])
+
+ pkg1_infos = diff_rpms_with_nvra(
+ *params1, all_subpackages=global_config.check_all_subpackages)
+ download_rpms(pkg1_infos)
+
+ pkg2_infos = diff_rpms_with_nvra(
+ *params2, all_subpackages=global_config.check_all_subpackages)
+ download_rpms(pkg2_infos)
+
+ return run_abipkgdiff(pkg1_infos, pkg2_infos)
+
+
+def build_commandline_args_parser():
+ parser = argparse.ArgumentParser(
+ description='Run abipkgdiff against RPM packages from koji')
+
+ parser.add_argument(
+ 'NVR',
+ nargs='*',
+ help='RPM package N-V-R, N-V-R-A, N, or a local RPM '
+ 'file name with relative or absolute path.')
+ parser.add_argument(
+ '--dry-run',
+ required=False,
+ dest='dry_run',
+ action='store_true',
+ help='Don\'t actually run abipkgdiff. The commands that should be '
+ 'run will be sent to stdout.')
+ parser.add_argument(
+ '--from',
+ required=False,
+ metavar='DISTRO',
+ dest='from_distro',
+ help='baseline Fedora distro, for example, fc23')
+ parser.add_argument(
+ '--to',
+ required=False,
+ metavar='DISTRO',
+ dest='to_distro',
+ help='which Fedora distro to compare, for example, fc24')
+ parser.add_argument(
+ '-a',
+ '--all-subpackages',
+ required=False,
+ action='store_true',
+ dest='check_all_subpackages',
+ help='Check all subpackages instead of only the package specificed in '
+ 'command line.')
+ parser.add_argument(
+ '--debug',
+ required=False,
+ action='store_true',
+ dest='debug',
+ help='show debug output')
+ parser.add_argument(
+ '--traceback',
+ required=False,
+ action='store_true',
+ dest='show_traceback',
+ help='show traceback when there is an exception thrown.')
+ parser.add_argument(
+ '--server',
+ required=False,
+ metavar='URL',
+ dest='koji_server',
+ default=DEFAULT_KOJI_SERVER,
+ help='URL of koji XMLRPC service. Default is {0}'.format(
+ DEFAULT_KOJI_SERVER))
+ parser.add_argument(
+ '--topdir',
+ required=False,
+ metavar='URL',
+ dest='koji_topdir',
+ default=DEFAULT_KOJI_TOPDIR,
+ help='URL for RPM files access')
+
+ return parser
+
+
+def main():
+ parser = build_commandline_args_parser()
+
+ args = parser.parse_args()
+
+ global global_config
+ global_config = args
+
+ global pathinfo
+ pathinfo = koji.PathInfo(topdir=global_config.koji_topdir)
+
+ global session
+ session = get_session()
+
+ if global_config.debug:
+ logger.setLevel(logging.DEBUG)
+
+ logger.debug(args)
+
+ if global_config.from_distro and global_config.to_distro is None and \
+ global_config.NVR:
+ returncode = diff_local_rpm_with_latest_rpm_from_koji()
+
+ elif global_config.from_distro and global_config.to_distro and \
+ global_config.NVR:
+ returncode = diff_latest_rpms_based_on_distros()
+
+ elif global_config.from_distro is None and \
+ global_config.to_distro is None and len(global_config.NVR) > 1:
+ returncode = diff_two_nvras_from_koji()
+
+ else:
+ print >>sys.stderr, 'Unknown arguments. Please refer to -h.'
+ returncode = 1
+
+ return returncode
+
+
+if __name__ == '__main__':
+ try:
+ main()
+ except KeyboardInterrupt:
+ if global_config.debug:
+ logger.debug('Terminate by user')
+ else:
+ print >>sys.stderr, 'Terminate by user'
+ if global_config.show_traceback:
+ raise
+ else:
+ sys.exit(2)
+ except Exception as e:
+ if global_config.debug:
+ logger.debug(str(e))
+ else:
+ print >>sys.stderr, str(e)
+ if global_config.show_traceback:
+ raise
+ else:
+ sys.exit(1)
--
2.5.0