[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] Rewrite integration tests
Hello Chenxiong,
Chenxiong Qi <cqi@redhat.com> a écrit:
> This time integration tests are rewritten totally and other
> unnecessary unit tests are removed at this moment that can be covered
> by the integration tests.
Thanks for looking into this important work. I think the direction of
the patch is good.
The key comment I'd have is about the way test failure detection is
performed in this patch.
As I believe I was saying when he had this dicussion on IRC, I'd prefer
test failure detection to be done by comparing (using GNU diff) the
output of fedabipkgdiff run in the test harness, against a reference
comparison output that is present in tests/data/test-fedabipkgdiff.
This is similar to what the other tests are doing today, and is what is
documented in the CONTRIBUTING file. This helps catching almost all the
issues about things that are represented in the output of the tool in a
simple and yet comprehensive manner. And when the output of
fedabipkgdiff changes, "make check" makes you see directly what the
change is (and assess its important), by showing you a diff. This is
much more pleasant and effective to work with, rather than figuring out
why an assert was violated in a middle of a test case.
Rather than making you work on this again, as I appreciate the time and
effort you spent on this already (thank you for that), I took the
liberty to amend your patch, so please find below what I came up with,
along with a full cover letter explaining what I did.
Tested and committed to master.
Thanks!
------------------------------->8<------------------------------------
>From 1a00cad493ade8b34bfb60b8398225339fa65dd3 Mon Sep 17 00:00:00 2001
From: Chenxiong Qi <cqi@redhat.com>
Date: Wed, 6 Jul 2016 17:51:13 +0800
Subject: [PATCH] Make fedabipkgdiff consistent with Libabigail's other tests
In the libabigail project, almost all the tests for the tools follow a
similar pattern. The test contains a set of input files to compare.
The result of the comparison is thus compared to a set of reference
comparison result.
This approach is already documented in the CONTRIBUTING file.
There are several interesting properties with this scheme. First, it
capture the behaviour of the tools, including what is shown to the
user.
Second, it eases the job of a hacker who wants to add a new test for a
new behaviour of a given tool. The user just has to provide:
1/ A new reference output of the new use case of the tool (that is easily
constructed by using the tool itself and saving its output) and add an
entry to array of entries that describe what to compare
2/ A new set of inputs to the tool
And voila.
Unfortunately, fedabipkgdiff tests don't follow this scheme. That
make them surprising to hackers who read the source code of the
existing tests, at very least. Also, the fedabipkgdiff tests were
only unit tests. They were not testing the tool as used by users in
general.
This patch makes the fedabipkgdiff tests follow the general approach
of the tests of the other Libabigail tools.
The patch first craetes a program names tests/mockfedabipkgdiff. It's
a wrapper around tools/fedabipkgdiff. It overloads the Koji client of
fedabipkgdiff with a fake Koji client that gets the Fedora packages
locally, from tests/data/test-fedabipkgdiff/packages. In other words,
mockfedabipkgdiff is fedabipkgdiff without going to the network.
I believe that in the future, tests/mockfedabipkgdiff should be killed
when fedabipkgdiff is able to cache a local description of a local
partial view of a Koji repository, along with the build packages that
were already downloaded.
That way, calling fedabipkgdiff twice with the same set of option
should make the tool perform correctly without going to the netword on
the second invocation. We should be able to save the local partial
view of the Koji repository under e.g,
tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use
that, instead of using the network. But we are not there yet. So for
now, I am using mockfedabipkgdiff.
Then, tests/runtestfedabipkdiff.py.in has been re-written to use
tests/mockfedabipkgdiff and to look like what is described in
CONTRIBUTING as far as how Libabigail tools' tests are organized:
mockfedabipkgdiff is called to perform a comparison. The result of
the comparison is then compared (using GNU diff) to a reference
comparison result file.
Please note that tests/runtestfedabipkdiff.py is relatively fast for
the moment. But when it contains much more tests and start becoming
slow, then we'll need to change the code to run several comparisons in
parallel, just like we do today in tests/test-diff-filter.cc. At that
point, I believe I'll just re-write this in C++, just like
tests/test-diff-filter.cc because that will allow us to have true
concurrent code based on the abigail:workers API. For now, I am
keeping this in Python also because I think that keeps Chenxiong happy
;-)
To be sure that fedabipkgdiff (and its underlying abipkgdiff) are
really comparing all the packages they are supposed to compare and
also all the binaries in those packages, I have added a new
--show-identical-binaries to abipkgdiff and fedabipkgdiff. That
option shows the name of the binaries (and packages) that are
compared, *even if* the ABI of those binaries are identical. Because
otherwise, today, if two binaries (or packages) have the same ABI,
nothing is displayed.
For instance, here is the result of comparing a package against
itself, using this new --show-identical-binaries options:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
================ changes of 'dbus-binding-tool'===============
No ABI change detected
================ end of changes of 'dbus-binding-tool'===============
================ changes of 'libdbus-glib-1.so.2.1.0'===============
No ABI change detected
================ end of changes of 'libdbus-glib-1.so.2.1.0'===============
dodji@adjoa:patches$
And here is what this command before that patch would do:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
dodji@adjoa:patches$
The rest of the patch is mostly new test inputs material and the
necessary adjustments following all the changes above.
* configure.ac: Do not require Python dependencies itertools,
unittest and StringIO anymore as they are not used anymore.
Require new module tempfile now. Generate new executable script
tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in.
* doc/manuals/abipkgdiff.rst: Add doc for new option
--show-identical-binaries to abipkgdiff
* doc/manuals/fedabipkgdiff.rst: Add doc for new options
--show-identical-binaries to fedabipkgdiff.
* tools/abipkgdiff.cc (options::show_identical_binaries): New data
member.
(options::options): Initialize new data member.
(display_usage): Add a new help string for the new
--show-identical-binaries option.
(parse_command_line): Parse the newq --show-identical-binaries
command line switch.
(pthread_routine_compare): When the comparison of two binaries is
empty, if --show-identical-binaries was provided, then emit some
output saying the comparison did yield the empty set.
* tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path
to abipkgdiff in this new global variable. Naming this default
path is useful because it can then be cleanly overloaded when
using mock.patch.
(build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF
global variable.
(cmd): Parse the new --show-identical-binaries command line
switch.
* tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt:
New reference output.
* tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt:
Likewise.
* tests/mockfedabipkgdiff.in: New uninstalled script template.
* tests/runtestfedabipkgdiff.py.in (counter)
(temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest)
(RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest)
(BrewListRPMsTest, AssertionHelper, MockGlobalConfig)
(BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these
classes, global variables and functions.
(FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR)
(OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables.
(ensure_output_dir_created, run_fedabipkgdiff_tests, main): New
functions.
* tests/test-diff-pkg.cc (in_out_specs): Add
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt
to the set of reference outputs to consider.
* tests/Makefile.am: Add non-installed script mockfedabipkgdiff to
source distribution. Also added
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt,
tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt
and
tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt
to source distribution.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
---
configure.ac | 6 +-
doc/manuals/abipkgdiff.rst | 7 +
doc/manuals/fedabipkgdiff.rst | 7 +
tests/Makefile.am | 3 +-
tests/data/Makefile.am | 8 +-
.../test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt | 8 +
.../test0-from-fc20-to-fc23-dbus-glib-report-0.txt | 63 +
...0-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt | 26 +
...0.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt | 63 +
....i686--dbus-glib-0.106-1.fc23.i686-report-0.txt | 37 +
tests/mockfedabipkgdiff.in | 372 ++++++
tests/runtestfedabipkgdiff.py.in | 1298 ++------------------
tests/test-diff-pkg.cc | 12 +
tools/abipkgdiff.cc | 17 +-
tools/fedabipkgdiff | 11 +-
15 files changed, 737 insertions(+), 1201 deletions(-)
create mode 100644 tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt
create mode 100644 tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt
create mode 100644 tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt
create mode 100644 tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt
create mode 100644 tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt
create mode 100644 tests/mockfedabipkgdiff.in
diff --git a/configure.ac b/configure.ac
index ade4d2d..3328860 100644
--- a/configure.ac
+++ b/configure.ac
@@ -290,8 +290,8 @@ if test x$CHECK_DEPS_FOR_FEDABIPKGDIFF = xyes; then
fi
REQUIRED_PYTHON_MODULES_FOR_FEDABIPKGDIFF="\
- argparse logging os re subprocess sys itertools urlparse \
- unittest xdg koji mock StringIO rpm"
+ argparse logging os re subprocess sys urlparse \
+ xdg koji mock rpm imp tempfile"
if test -x$ENABLE_FEDABIPKGDIFF != xno; then
AX_CHECK_PYTHON_MODULES([$REQUIRED_PYTHON_MODULES_FOR_FEDABIPKGDIFF],
@@ -464,6 +464,8 @@ libabigail.pc
dnl Some test scripts are generated by autofoo.
AC_CONFIG_FILES([tests/runtestcanonicalizetypes.sh],
[chmod +x tests/runtestcanonicalizetypes.sh])
+AC_CONFIG_FILES([tests/mockfedabipkgdiff],
+ [chmod +x tests/mockfedabipkgdiff])
AC_CONFIG_FILES([tests/runtestfedabipkgdiff.py],
[chmod +x tests/runtestfedabipkgdiff.py])
AC_CONFIG_FILES([tests/runtestdefaultsupprs.py],
diff --git a/doc/manuals/abipkgdiff.rst b/doc/manuals/abipkgdiff.rst
index ccc8552..f4b5fa7 100644
--- a/doc/manuals/abipkgdiff.rst
+++ b/doc/manuals/abipkgdiff.rst
@@ -164,6 +164,13 @@ Options
Do not show information about where in the *second shared library*
the respective type was changed.
+ * ``--show-identical-binaries``
+
+ Show the names of the all binaries compared, including the
+ binaries whose ABI compare equal. By default, when this option is
+ not provided, only binaries with ABI changes are mentionned in the
+ output.
+
* ``--fail-no-dbg``
Make the program fail and return a non-zero exit code if couldn't
diff --git a/doc/manuals/fedabipkgdiff.rst b/doc/manuals/fedabipkgdiff.rst
index 21cd8ea..9a5e843 100644
--- a/doc/manuals/fedabipkgdiff.rst
+++ b/doc/manuals/fedabipkgdiff.rst
@@ -131,6 +131,13 @@ Options
defined in public header files available from the packages being
compared.
+ * ``--show-identical-binaries``
+
+ Show the names of the all binaries compared, including the
+ binaries whose ABI compare equal. By default, when this option is
+ not provided, only binaries with ABI changes are mentionned in the
+ output.
+
* ``--abipkgdiff`` <path/to/abipkgdiff>
Specify an alternative abipkgdiff instead of the one installed in system.
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 36e0425..becadaa 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -43,6 +43,7 @@ $(FEDABIPKGDIFF_TEST) $(CXX11_TESTS)
EXTRA_DIST = \
runtestcanonicalizetypes.sh.in \
runtestfedabipkgdiff.py.in \
+mockfedabipkgdiff.in \
test-valgrind-suppressions.supp
CLEANFILES = \
@@ -50,7 +51,7 @@ runtestcanonicalizetypes.output.txt \
runtestcanonicalizetypes.output.final.txt
noinst_PROGRAMS= $(TESTS) testirwalker testdiff2 printdifftree
-
+noinst_SCRIPTS = mockfedabipkgdiff
noinst_LTLIBRARIES = libtestutils.la
libtestutils_la_SOURCES= \
diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
index fc53d88..92321a5 100644
--- a/tests/data/Makefile.am
+++ b/tests/data/Makefile.am
@@ -1162,10 +1162,16 @@ test-diff-pkg/gtk2-immodule-xim-2.24.28-8.el7.i686.rpm \
test-diff-pkg/tarpkg-1-dir1.tar.gz \
test-diff-pkg/tarpkg-1-dir2.tar.gz \
test-diff-pkg/tarpkg-1-report-0.txt \
+test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt \
+\
test-fedabipkgdiff/dbus-glib-0.104-3.fc23.x86_64.rpm \
test-fedabipkgdiff/dbus-glib-debuginfo-0.104-3.fc23.x86_64.rpm \
test-fedabipkgdiff/dbus-glib-0.80-3.fc12.x86_64.rpm \
test-fedabipkgdiff/dbus-glib-debuginfo-0.80-3.fc12.x86_64.rpm \
+test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt \
+test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt \
+test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt \
+test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt \
test-fedabipkgdiff/packages/dbus-glib/0.100.2/2.fc20/i686/dbus-glib-0.100.2-2.fc20.i686.rpm \
test-fedabipkgdiff/packages/dbus-glib/0.100.2/2.fc20/i686/dbus-glib-debuginfo-0.100.2-2.fc20.i686.rpm \
test-fedabipkgdiff/packages/dbus-glib/0.100.2/2.fc20/i686/dbus-glib-devel-0.100.2-2.fc20.i686.rpm \
@@ -1184,6 +1190,7 @@ test-fedabipkgdiff/packages/nss-util/3.12.6/1.fc14/x86_64/nss-util-debuginfo-3.1
test-fedabipkgdiff/packages/nss-util/3.24.0/2.0.fc25/x86_64/nss-util-debuginfo-3.24.0-2.0.fc25.x86_64.rpm \
test-fedabipkgdiff/packages/nss-util/3.24.0/2.0.fc25/x86_64/nss-util-devel-3.24.0-2.0.fc25.x86_64.rpm \
test-fedabipkgdiff/packages/nss-util/3.24.0/2.0.fc25/x86_64/nss-util-3.24.0-2.0.fc25.x86_64.rpm \
+\
test-default-supprs/test0-type-suppr-0.suppr \
test-default-supprs/test0-type-suppr-report-0.txt \
test-default-supprs/test0-type-suppr-v0.o \
@@ -1194,4 +1201,3 @@ test-default-supprs/dirpkg-1-dir1/obj-v0.cc \
test-default-supprs/dirpkg-1-dir2/dir.abignore \
test-default-supprs/dirpkg-1-dir2/libobj-v0.so \
test-default-supprs/dirpkg-1-dir2/obj-v0.cc
-
diff --git a/tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt b/tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt
new file mode 100644
index 0000000..0344d68
--- /dev/null
+++ b/tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt
@@ -0,0 +1,8 @@
+================ changes of 'dbus-binding-tool'===============
+No ABI change detected
+================ end of changes of 'dbus-binding-tool'===============
+
+================ changes of 'libdbus-glib-1.so.2.1.0'===============
+No ABI change detected
+================ end of changes of 'libdbus-glib-1.so.2.1.0'===============
+
diff --git a/tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt b/tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt
new file mode 100644
index 0000000..0aaf4ee
--- /dev/null
+++ b/tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt
@@ -0,0 +1,63 @@
+Comparing the ABI of binaries between dbus-glib-0.100.2-2.fc20.i686.rpm and dbus-glib-0.106-1.fc23.i686.rpm:
+
+================ changes of 'dbus-binding-tool'===============
+ Functions changes summary: 2 Removed, 0 Changed, 1 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+ Function symbols changes summary: 0 Removed, 0 Added function symbol not referenced by debug info
+ Variable symbols changes summary: 2 Removed, 0 Added variable symbols not referenced by debug info
+
+ 2 Removed functions:
+
+ 'function BaseInfo* base_info_ref(BaseInfo*)' {base_info_ref}
+ 'function void base_info_unref(BaseInfo*)' {base_info_unref}
+
+ 1 Added function:
+
+ 'function int main(int, char**)' {main}
+
+
+ 2 Removed variable symbols not referenced by debug info:
+
+ stderr
+ stdout
+
+================ end of changes of 'dbus-binding-tool'===============
+
+================ changes of 'libdbus-glib-1.so.2.2.2'===============
+ Functions changes summary: 0 Removed, 0 Changed, 2 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+
+ 2 Added functions:
+
+ 'function DBusGConnection* dbus_g_connection_open_private(const gchar*, GMainContext*, GError**)' {dbus_g_connection_open_private}
+ 'function DBusGConnection* dbus_g_method_invocation_get_g_connection(DBusGMethodInvocation*)' {dbus_g_method_invocation_get_g_connection}
+
+================ end of changes of 'libdbus-glib-1.so.2.2.2'===============
+
+
+Comparing the ABI of binaries between dbus-glib-0.100.2-2.fc20.x86_64.rpm and dbus-glib-0.106-1.fc23.x86_64.rpm:
+
+================ changes of 'dbus-binding-tool'===============
+ Functions changes summary: 2 Removed, 0 Changed, 0 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+
+ 2 Removed functions:
+
+ 'function BaseInfo* base_info_ref(BaseInfo*)' {base_info_ref}
+ 'function void base_info_unref(BaseInfo*)' {base_info_unref}
+
+
+================ end of changes of 'dbus-binding-tool'===============
+
+================ changes of 'libdbus-glib-1.so.2.2.2'===============
+ Functions changes summary: 0 Removed, 0 Changed, 2 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+
+ 2 Added functions:
+
+ 'function DBusGConnection* dbus_g_connection_open_private(const gchar*, GMainContext*, GError**)' {dbus_g_connection_open_private}
+ 'function DBusGConnection* dbus_g_method_invocation_get_g_connection(DBusGMethodInvocation*)' {dbus_g_method_invocation_get_g_connection}
+
+================ end of changes of 'libdbus-glib-1.so.2.2.2'===============
+
+
diff --git a/tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt b/tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt
new file mode 100644
index 0000000..98998e4
--- /dev/null
+++ b/tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt
@@ -0,0 +1,26 @@
+Comparing the ABI of binaries between dbus-glib-0.100.2-2.fc20.x86_64.rpm and dbus-glib-0.106-1.fc23.x86_64.rpm:
+
+================ changes of 'dbus-binding-tool'===============
+ Functions changes summary: 2 Removed, 0 Changed, 0 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+
+ 2 Removed functions:
+
+ 'function BaseInfo* base_info_ref(BaseInfo*)' {base_info_ref}
+ 'function void base_info_unref(BaseInfo*)' {base_info_unref}
+
+
+================ end of changes of 'dbus-binding-tool'===============
+
+================ changes of 'libdbus-glib-1.so.2.2.2'===============
+ Functions changes summary: 0 Removed, 0 Changed, 2 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+
+ 2 Added functions:
+
+ 'function DBusGConnection* dbus_g_connection_open_private(const gchar*, GMainContext*, GError**)' {dbus_g_connection_open_private}
+ 'function DBusGConnection* dbus_g_method_invocation_get_g_connection(DBusGMethodInvocation*)' {dbus_g_method_invocation_get_g_connection}
+
+================ end of changes of 'libdbus-glib-1.so.2.2.2'===============
+
+
diff --git a/tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt b/tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt
new file mode 100644
index 0000000..0aaf4ee
--- /dev/null
+++ b/tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt
@@ -0,0 +1,63 @@
+Comparing the ABI of binaries between dbus-glib-0.100.2-2.fc20.i686.rpm and dbus-glib-0.106-1.fc23.i686.rpm:
+
+================ changes of 'dbus-binding-tool'===============
+ Functions changes summary: 2 Removed, 0 Changed, 1 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+ Function symbols changes summary: 0 Removed, 0 Added function symbol not referenced by debug info
+ Variable symbols changes summary: 2 Removed, 0 Added variable symbols not referenced by debug info
+
+ 2 Removed functions:
+
+ 'function BaseInfo* base_info_ref(BaseInfo*)' {base_info_ref}
+ 'function void base_info_unref(BaseInfo*)' {base_info_unref}
+
+ 1 Added function:
+
+ 'function int main(int, char**)' {main}
+
+
+ 2 Removed variable symbols not referenced by debug info:
+
+ stderr
+ stdout
+
+================ end of changes of 'dbus-binding-tool'===============
+
+================ changes of 'libdbus-glib-1.so.2.2.2'===============
+ Functions changes summary: 0 Removed, 0 Changed, 2 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+
+ 2 Added functions:
+
+ 'function DBusGConnection* dbus_g_connection_open_private(const gchar*, GMainContext*, GError**)' {dbus_g_connection_open_private}
+ 'function DBusGConnection* dbus_g_method_invocation_get_g_connection(DBusGMethodInvocation*)' {dbus_g_method_invocation_get_g_connection}
+
+================ end of changes of 'libdbus-glib-1.so.2.2.2'===============
+
+
+Comparing the ABI of binaries between dbus-glib-0.100.2-2.fc20.x86_64.rpm and dbus-glib-0.106-1.fc23.x86_64.rpm:
+
+================ changes of 'dbus-binding-tool'===============
+ Functions changes summary: 2 Removed, 0 Changed, 0 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+
+ 2 Removed functions:
+
+ 'function BaseInfo* base_info_ref(BaseInfo*)' {base_info_ref}
+ 'function void base_info_unref(BaseInfo*)' {base_info_unref}
+
+
+================ end of changes of 'dbus-binding-tool'===============
+
+================ changes of 'libdbus-glib-1.so.2.2.2'===============
+ Functions changes summary: 0 Removed, 0 Changed, 2 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+
+ 2 Added functions:
+
+ 'function DBusGConnection* dbus_g_connection_open_private(const gchar*, GMainContext*, GError**)' {dbus_g_connection_open_private}
+ 'function DBusGConnection* dbus_g_method_invocation_get_g_connection(DBusGMethodInvocation*)' {dbus_g_method_invocation_get_g_connection}
+
+================ end of changes of 'libdbus-glib-1.so.2.2.2'===============
+
+
diff --git a/tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt b/tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt
new file mode 100644
index 0000000..c860ff0
--- /dev/null
+++ b/tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt
@@ -0,0 +1,37 @@
+Comparing the ABI of binaries between dbus-glib-0.100.2-2.fc20.i686.rpm and dbus-glib-0.106-1.fc23.i686.rpm:
+
+================ changes of 'dbus-binding-tool'===============
+ Functions changes summary: 2 Removed, 0 Changed, 1 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+ Function symbols changes summary: 0 Removed, 0 Added function symbol not referenced by debug info
+ Variable symbols changes summary: 2 Removed, 0 Added variable symbols not referenced by debug info
+
+ 2 Removed functions:
+
+ 'function BaseInfo* base_info_ref(BaseInfo*)' {base_info_ref}
+ 'function void base_info_unref(BaseInfo*)' {base_info_unref}
+
+ 1 Added function:
+
+ 'function int main(int, char**)' {main}
+
+
+ 2 Removed variable symbols not referenced by debug info:
+
+ stderr
+ stdout
+
+================ end of changes of 'dbus-binding-tool'===============
+
+================ changes of 'libdbus-glib-1.so.2.2.2'===============
+ Functions changes summary: 0 Removed, 0 Changed, 2 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+
+ 2 Added functions:
+
+ 'function DBusGConnection* dbus_g_connection_open_private(const gchar*, GMainContext*, GError**)' {dbus_g_connection_open_private}
+ 'function DBusGConnection* dbus_g_method_invocation_get_g_connection(DBusGMethodInvocation*)' {dbus_g_method_invocation_get_g_connection}
+
+================ end of changes of 'libdbus-glib-1.so.2.2.2'===============
+
+
diff --git a/tests/mockfedabipkgdiff.in b/tests/mockfedabipkgdiff.in
new file mode 100644
index 0000000..2141983
--- /dev/null
+++ b/tests/mockfedabipkgdiff.in
@@ -0,0 +1,372 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+# -*- Mode: Python
+#
+# This file is part of the GNU Application Binary Interface Generic
+# Analysis and Instrumentation Library (libabigail). This library is
+# free software; you can redistribute it and/or modify it under the
+# terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 3, or (at your option) any
+# later version.
+#
+# This library is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public
+# License along with this program; see the file COPYING-GPLV3. If
+# not, see <http:#www.gnu.org/licenses/>.
+#
+# Author: Dodji Seketeli
+#
+# Based on some preliminary work from Chenxiong Qi, posted to
+# https://sourceware.org/ml/libabigail/2016-q3/msg00031.html.
+
+
+'''A local-only version of the fedabipkgdiff program
+
+This program behaves exactly like the fedabipkgdiff tool, except
+that it doesn't use the network (via the Koji client) to get Fedora
+packages. The Koji client is the interface that is used to access
+the Fedora build system to get Fedora binary packages.
+
+This program thus loads the fedabipkgdiff tool and overloads
+(replaces) its Koji Client session with a fake (also known as a
+mock) Koji client named MockClientSession that gets the packages
+locally. Packages are stored under the directory
+tests/data/test-fedabipkgdiff/packages. This program then invokes
+the fedabipkgdiff program just as if the user invoked it from the
+command line.
+
+This program is useful for tests that can be called from "make
+check" because those must be able to perform in environments where
+no network is avaible.
+
+Please note that the descriptions of the builds of each package are
+stored in at least three global variables below: packages, builds
+and rpms.
+
+Whenever a user wants to add a new build to the local set of builds
+locally available via MockClientSession, she must update these three
+variables.
+'''
+
+import os
+import tempfile
+import imp
+
+try:
+ from mock import patch
+except ImportError:
+ import sys
+ print >>sys.stderr, \
+ 'mock is required to run tests. Please install before running tests.'
+ sys.exit(1)
+
+ABIPKGDIFF = '@abs_top_builddir@/tools/abipkgdiff'
+FEDABIPKGDIFF = '@abs_top_srcdir@/tools/fedabipkgdiff'
+INPUT_DIR = '@abs_top_srcdir@/tests/data/test-fedabipkgdiff'
+OUTPUT_DIR = '@abs_top_builddir@/tests/output/test-fedabipkgdiff'
+DOWNLOAD_CACHE_DIR_PREFIX=os.path.join(OUTPUT_DIR, 'download-cache')
+TEST_TOPDIR = 'file://{0}'.format(INPUT_DIR)
+
+def get_download_dir():
+
+ if not os.path.exists(DOWNLOAD_CACHE_DIR_PREFIX):
+ try:
+ os.makedirs(DOWNLOAD_CACHE_DIR_PREFIX)
+ except:
+ pass
+ if not os.path.exists(DOWNLOAD_CACHE_DIR_PREFIX):
+ sys.exit(1);
+
+ return tempfile.mkdtemp(dir=DOWNLOAD_CACHE_DIR_PREFIX)
+
+DOWNLOAD_CACHE_DIR = get_download_dir()
+
+def mock_get_download_dir():
+ return DOWNLOAD_CACHE_DIR
+
+
+# Import the fedabipkgdiff program file from the source directory.
+fedabipkgdiff_mod = imp.load_source('fedabipkgdiff', FEDABIPKGDIFF)
+
+
+# ----------------- Koji resource storage begins ------------------
+#
+# List all necessary Koji resources for running tests within this test
+# module. Currently, packages, builds, and rpms are listed here, and their
+# relationship is maintained well. At the same time, all information including
+# the ID for each package, build and rpm is real and can be queried from Koji,
+# that is for convenience once developers need this.
+#
+# When additional packages, builds and rpms are required for test cases, here
+# is the right place to add them. Just think them as a super simple in-memory
+# database, and methods of MockClientSession knows well how to read them.
+
+packages = [
+ {'id': 286, 'name': 'gnutls'},
+ {'id': 612, 'name': 'dbus-glib'},
+ {'id': 9041, 'name': 'nss-util'},
+ ]
+
+builds = [
+ {'build_id': 428835, 'nvr': 'dbus-glib-0.100-4.fc20',
+ 'name': 'dbus-glib', 'release': '4.fc20', 'version': '0.100',
+ 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
+ },
+ {'build_id': 430720, 'nvr': 'dbus-glib-0.100.2-1.fc20',
+ 'name': 'dbus-glib', 'release': '1.fc20', 'version': '0.100.2',
+ 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
+ },
+ {'build_id': 442366, 'nvr': 'dbus-glib-0.100.2-2.fc20',
+ 'name': 'dbus-glib', 'release': '2.fc20', 'version': '0.100.2',
+ 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
+ },
+ {'build_id': 715478, 'nvr': 'dbus-glib-0.106-1.fc23',
+ 'name': 'dbus-glib', 'release': '1.fc23', 'version': '0.106',
+ 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
+ },
+ {'build_id': 648058, 'nvr': 'dbus-glib-0.104-3.fc23',
+ 'name': 'dbus-glib', 'release': '3.fc23', 'version': '0.104',
+ 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
+ },
+ {'build_id': 613769, 'nvr': 'dbus-glib-0.104-2.fc23',
+ 'name': 'dbus-glib', 'release': '2.fc23', 'version': '0.104',
+ 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
+ },
+
+ {'build_id': 160295, 'nvr': 'nss-util-3.12.6-1.fc14',
+ 'name': 'nss-util', 'version': '3.12.6', 'release': '1.fc14',
+ 'package_id': 9041, 'package_name': 'nss-util', 'state': 1,
+ },
+ {'build_id': 767978, 'nvr': 'nss-util-3.24.0-2.0.fc25',
+ 'name': 'nss-util', 'version': '3.24.0', 'release': '2.0.fc25',
+ 'package_id': 9041, 'package_name': 'nss-util', 'state': 1,
+ },
+
+ # builds of package gnutls
+ {'build_id': 767306, 'nvr': 'gnutls-3.4.12-1.fc23',
+ 'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.12',
+ 'package_id': 286, 'package_name': 'gnutls', 'state': 1,
+ },
+ {'build_id': 770965, 'nvr': 'gnutls-3.4.13-1.fc23',
+ 'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.13',
+ 'package_id': 286, 'package_name': 'gnutls', 'state': 1,
+ },
+ {'build_id': 649701, 'nvr': 'gnutls-3.4.2-1.fc23',
+ 'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.2',
+ 'package_id': 286, 'package_name': 'gnutls', 'state': 1,
+ },
+ ]
+
+rpms = [
+ {'build_id': 442366,
+ 'name': 'dbus-glib', 'release': '2.fc20', 'version': '0.100.2',
+ 'arch': 'x86_64', 'nvr': 'dbus-glib-0.100.2-2.fc20',
+ },
+ {'build_id': 442366,
+ 'name': 'dbus-glib-devel', 'release': '2.fc20', 'version': '0.100.2',
+ 'arch': 'x86_64', 'nvr': 'dbus-glib-devel-0.100.2-2.fc20',
+ },
+ {'build_id': 442366,
+ 'name': 'dbus-glib-debuginfo', 'release': '2.fc20', 'version': '0.100.2',
+ 'arch': 'x86_64', 'nvr': 'dbus-glib-debuginfo-0.100.2-2.fc20',
+ },
+ {'build_id': 442366,
+ 'name': 'dbus-glib-devel', 'release': '2.fc20', 'version': '0.100.2',
+ 'arch': 'i686', 'nvr': 'dbus-glib-devel-0.100.2-2.fc20',
+ },
+ {'build_id': 442366,
+ 'name': 'dbus-glib-debuginfo', 'release': '2.fc20', 'version': '0.100.2',
+ 'arch': 'i686', 'nvr': 'dbus-glib-debuginfo-0.100.2-2.fc20',
+ },
+ {'build_id': 442366,
+ 'name': 'dbus-glib', 'version': '0.100.2', 'release': '2.fc20',
+ 'arch': 'i686', 'nvr': 'dbus-glib-0.100.2-2.fc20',
+ },
+
+ {'build_id': 715478,
+ 'name': 'dbus-glib-debuginfo', 'version': '0.106', 'release': '1.fc23',
+ 'arch': 'i686', 'nvr': 'dbus-glib-debuginfo-0.106-1.fc23',
+ },
+ {'build_id': 715478,
+ 'name': 'dbus-glib', 'version': '0.106', 'release': '1.fc23',
+ 'arch': 'i686', 'nvr': 'dbus-glib-0.106-1.fc23',
+ },
+ {'build_id': 715478,
+ 'name': 'dbus-glib-devel', 'version': '0.106', 'release': '1.fc23',
+ 'arch': 'i686', 'nvr': 'dbus-glib-devel-0.106-1.fc23',
+ },
+ {'build_id': 715478,
+ 'name': 'dbus-glib', 'version': '0.106', 'release': '1.fc23',
+ 'arch': 'x86_64', 'nvr': 'dbus-glib-0.106-1.fc23',
+ },
+ {'build_id': 715478,
+ 'name': 'dbus-glib-debuginfo', 'version': '0.106', 'release': '1.fc23',
+ 'arch': 'x86_64', 'nvr': 'dbus-glib-debuginfo-0.106-1.fc23',
+ },
+ {'build_id': 715478,
+ 'name': 'dbus-glib-devel', 'release': '1.fc23', 'version': '0.106',
+ 'arch': 'x86_64', 'nvr': 'dbus-glib-devel-0.106-1.fc23',
+ },
+
+ # RPMs of build nss-util-3.12.6-1.fc14
+ {'build_id': 160295,
+ 'name': 'nss-util', 'release': '1.fc14', 'version': '3.12.6',
+ 'arch': 'x86_64', 'nvr': 'nss-util-3.12.6-1.fc14',
+ },
+ {'build_id': 160295,
+ 'name': 'nss-util-devel', 'release': '1.fc14', 'version': '3.12.6',
+ 'arch': 'x86_64', 'nvr': 'nss-util-devel-3.12.6-1.fc14',
+ },
+ {'build_id': 160295,
+ 'name': 'nss-util-debuginfo', 'release': '1.fc14', 'version': '3.12.6',
+ 'arch': 'x86_64', 'nvr': 'nss-util-debuginfo-3.12.6-1.fc14',
+ },
+
+ # RPMs of build nss-util-3.24.0-2.0.fc25
+ {'build_id': 767978,
+ 'name': 'nss-util-debuginfo', 'release': '2.0.fc25', 'version': '3.24.0',
+ 'arch': 'x86_64', 'nvr': 'nss-util-debuginfo-3.24.0-2.0.fc25',
+ },
+ {'build_id': 767978,
+ 'name': 'nss-util', 'release': '2.0.fc25', 'version': '3.24.0',
+ 'arch': 'x86_64', 'nvr': 'nss-util-3.24.0-2.0.fc25',
+ },
+ {'build_id': 767978,
+ 'name': 'nss-util-devel', 'release': '2.0.fc25', 'version': '3.24.0',
+ 'arch': 'x86_64', 'nvr': 'nss-util-devel-3.24.0-2.0.fc25',
+ },
+ ]
+
+# ----------------- End of Koji resource storage ------------------
+
+
+class MockClientSession(object):
+ """Mock koji.ClientSession
+
+ This mock ClientSession aims to avoid touching a real Koji instance to
+ interact with XMLRPC APIs required by fedabipkgdiff.
+
+ For the tests within this module, methods do not necessarily to return
+ complete RPM and build information. So, if you need more additional
+ information, here is the right place to add them.
+ """
+
+ def __init__(self, baseurl):
+ """Initialize a mock ClientSession
+
+ :param str baseurl: the URL to remote kojihub service. As of writing
+ this mock class, `baseurl` is not used at all, just keep it here if
+ it's useful in the future.
+
+ All mock methods have same signature as corresponding kojihub.*
+ methods, and type of parameters may be different and only satify
+ fedabipkgdiff requirement.
+ """
+ self.baseurl = baseurl
+
+ def getPackage(self, name):
+ """Mock kojihub.getPackage
+
+ :param str name: name of package to find and return
+ :return: the found package
+ :rtype: dict
+ """
+ assert isinstance(name, basestring)
+
+ def selector(package):
+ return package['name'] == name
+
+ return filter(selector, packages)[0]
+
+ def getBuild(self, build_id):
+ """Mock kojihub.getBuild
+
+ :param int build_id: ID of build to find and return
+ :return: the found build
+ :rtype: dict
+ """
+ assert isinstance(build_id, int)
+
+ def selector(build):
+ return build['build_id'] == build_id
+
+ return filter(selector, builds)[0]
+
+ def listBuilds(self, packageID, state=None):
+ """Mock kojihub.listBuilds
+
+ :param int packageID: ID of package whose builds is found and returned
+ :param state: build state. If state is omitted, all builds of a package
+ are returned
+ :type state: int or None
+ """
+ assert isinstance(packageID, int)
+ if state is not None:
+ assert isinstance(state, int)
+
+ def selector(build):
+ selected = build['package_id'] == packageID
+ if state is not None:
+ selected = selected and build['state'] == state
+ return selected
+
+ return filter(selector, builds)
+
+ def getRPM(self, rpminfo):
+ """Mock kojihub.getRPM
+
+ :param dict rpminfo: a mapping containing rpm information, at least,
+ it contains name, version, release, and arch.
+ """
+ assert isinstance(rpminfo, dict)
+
+ def selector(rpm):
+ return rpm['name'] == rpminfo['name'] and \
+ rpm['version'] == rpminfo['version'] and \
+ rpm['release'] == rpminfo['release'] and \
+ rpm['arch'] == rpminfo['arch']
+
+ return filter(selector, rpms)[0]
+
+ def listRPMs(self, buildID, arches=None):
+ """Mock kojihub.listRPMs
+
+ :param int buildID: ID of build from which to list rpms
+ :param arches: to list rpms built for specific arches. If arches is
+ omitted, rpms of all arches will be listed.
+ :type arches: list, tuple, str, or None
+ :return: list of rpms
+ :rtype: list
+ """
+ assert isinstance(buildID, int)
+ if arches is not None:
+ assert isinstance(arches, (tuple, list, basestring))
+
+ if arches is not None and isinstance(arches, basestring):
+ arches = [arches]
+
+ def selector(rpm):
+ selected = rpm['build_id'] == buildID
+ if arches is not None:
+ selected = selected and rpm['arch'] in arches
+ return selected
+
+ return filter(selector, rpms)
+
+
+@patch('koji.ClientSession', new=MockClientSession)
+@patch('fedabipkgdiff.DEFAULT_KOJI_TOPDIR', new=TEST_TOPDIR)
+@patch('fedabipkgdiff.DEFAULT_ABIPKGDIFF', new=ABIPKGDIFF)
+@patch('fedabipkgdiff.get_download_dir', side_effect=mock_get_download_dir)
+def run_fedabipkgdiff(get_download_dir):
+ return fedabipkgdiff_mod.main()
+
+def do_main():
+ run_fedabipkgdiff()
+
+if __name__ == '__main__':
+ do_main()
diff --git a/tests/runtestfedabipkgdiff.py.in b/tests/runtestfedabipkgdiff.py.in
index 76b1dd4..c477e64 100755
--- a/tests/runtestfedabipkgdiff.py.in
+++ b/tests/runtestfedabipkgdiff.py.in
@@ -20,1216 +20,126 @@
#
# Author: Chenxiong Qi
-import re
-import os
-import itertools
-import unittest
-import tempfile
-from StringIO import StringIO
-
-import koji
-
-"""
-This test harness tests various global methods and classes within
-tools/fedabipkgdiff.
-"""
-
-try:
- from mock import patch
-except ImportError:
- import sys
- print >>sys.stderr, \
- 'mock is required to run tests. Please install before running tests.'
- sys.exit(1)
-
-import imp
-# Import the fedabipkgdiff program file from the source directory.
-fedabipkgdiff_mod = imp.load_source('fedabipkgdiff',
- '@top_srcdir@/tools/fedabipkgdiff')
-
-# Used to generate integer values (greater or equal to zero) in
-# RunAbipkgdiffTest.test_partial_failure, those values simulate return code
-# from run_abipkgdiff. To represent partial failure, counter must start from 0.
-counter = itertools.count(0)
-
-# prefix for creating a temporary file or directory. The name would be
-# fedabipkgdiff-test-slkw3ksox
-temp_file_or_dir_prefix = 'fedabipkgdiff-test-'
+'''Runs tests for the fedabipkgdiff program
-# Directory holding data used by following tests
-TEST_DATA_DIR = os.path.abspath(
- os.path.join('@top_srcdir@', 'tests', 'data', 'test-fedabipkgdiff'))
+This program runs the fedabipkgdiff tool (via the mockfedabipkgdiff
+tool) repeatedly and compares its output against a set of reference
+output files.
-# topdir for tests from where to find and be treated as remote rpms to
-# download.
-TEST_TOPDIR = 'file://{0}'.format(TEST_DATA_DIR)
+'''
-# download cache directory for tests to avoid touch the real xdg_cache_home and
-# break any normal usage with fedabipkgdiff.
-# Note, as `tempfile.mkdtemp` is called, each time to run tests from this
-# module, this cache directory will be created at once.
-TEST_DOWNLOAD_CACHE_DIR = tempfile.mkdtemp(
- prefix='libabigail-test-fedabipkgdiff-download-cache-')
-
-# Reference to built abipkgdiff command
-BUILT_ABIPKGDIFF = '@top_builddir@/tools/abipkgdiff'
+import sys
+import os
+import subprocess
+FEDABIPKGDIFF = '@abs_top_builddir@/tests/mockfedabipkgdiff'
+TEST_SRC_DIR = '@abs_top_srcdir@/tests'
+TEST_BUILD_DIR = '@abs_top_builddir@/tests'
+INPUT_DIR = '@abs_top_srcdir@/tests/data/test-fedabipkgdiff'
+OUTPUT_DIR = '@abs_top_builddir@/tests/output/test-fedabipkgdiff'
-# ----------------- Koji resource storage begins ------------------
+# This variable is a list of 3-uples. Here is the meaning of
+# the elements of each 3-uples:
#
-# List all necessary Koji resources for running tests within this test
-# module. Currently, packages, builds, and rpms are listed here, and their
-# relationship is maintained well. At the same time, all information including
-# the ID for each package, build and rpm is real and can be queried from Koji,
-# that is for convenience once developers need this.
+# (arguments to pass to fedabipkgdiff,
+# reference output file,
+# where to store the result of the comparison)
#
-# When additional packages, builds and rpms are required for test cases, here
-# is the right place to add them. Just think them as a super simple in-memory
-# database, and methods of MockClientSession knows well how to read them.
-
-packages = [
- {'id': 286, 'name': 'gnutls'},
- {'id': 612, 'name': 'dbus-glib'},
- {'id': 9041, 'name': 'nss-util'},
- ]
-
-builds = [
- {'build_id': 428835, 'nvr': 'dbus-glib-0.100-4.fc20',
- 'name': 'dbus-glib', 'release': '4.fc20', 'version': '0.100',
- 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
- },
- {'build_id': 430720, 'nvr': 'dbus-glib-0.100.2-1.fc20',
- 'name': 'dbus-glib', 'release': '1.fc20', 'version': '0.100.2',
- 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
- },
- {'build_id': 442366, 'nvr': 'dbus-glib-0.100.2-2.fc20',
- 'name': 'dbus-glib', 'release': '2.fc20', 'version': '0.100.2',
- 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
- },
- {'build_id': 715478, 'nvr': 'dbus-glib-0.106-1.fc23',
- 'name': 'dbus-glib', 'release': '1.fc23', 'version': '0.106',
- 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
- },
- {'build_id': 648058, 'nvr': 'dbus-glib-0.104-3.fc23',
- 'name': 'dbus-glib', 'release': '3.fc23', 'version': '0.104',
- 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
- },
- {'build_id': 613769, 'nvr': 'dbus-glib-0.104-2.fc23',
- 'name': 'dbus-glib', 'release': '2.fc23', 'version': '0.104',
- 'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
- },
-
- {'build_id': 160295, 'nvr': 'nss-util-3.12.6-1.fc14',
- 'name': 'nss-util', 'version': '3.12.6', 'release': '1.fc14',
- 'package_id': 9041, 'package_name': 'nss-util', 'state': 1,
- },
- {'build_id': 767978, 'nvr': 'nss-util-3.24.0-2.0.fc25',
- 'name': 'nss-util', 'version': '3.24.0', 'release': '2.0.fc25',
- 'package_id': 9041, 'package_name': 'nss-util', 'state': 1,
- },
-
- # builds of package gnutls
- {'build_id': 767306, 'nvr': 'gnutls-3.4.12-1.fc23',
- 'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.12',
- 'package_id': 286, 'package_name': 'gnutls', 'state': 1,
- },
- {'build_id': 770965, 'nvr': 'gnutls-3.4.13-1.fc23',
- 'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.13',
- 'package_id': 286, 'package_name': 'gnutls', 'state': 1,
- },
- {'build_id': 649701, 'nvr': 'gnutls-3.4.2-1.fc23',
- 'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.2',
- 'package_id': 286, 'package_name': 'gnutls', 'state': 1,
- },
- ]
-
-rpms = [
- {'build_id': 442366,
- 'name': 'dbus-glib', 'release': '2.fc20', 'version': '0.100.2',
- 'arch': 'x86_64', 'nvr': 'dbus-glib-0.100.2-2.fc20',
- },
- {'build_id': 442366,
- 'name': 'dbus-glib-devel', 'release': '2.fc20', 'version': '0.100.2',
- 'arch': 'x86_64', 'nvr': 'dbus-glib-devel-0.100.2-2.fc20',
- },
- {'build_id': 442366,
- 'name': 'dbus-glib-debuginfo', 'release': '2.fc20', 'version': '0.100.2',
- 'arch': 'x86_64', 'nvr': 'dbus-glib-debuginfo-0.100.2-2.fc20',
- },
- {'build_id': 442366,
- 'name': 'dbus-glib-devel', 'release': '2.fc20', 'version': '0.100.2',
- 'arch': 'i686', 'nvr': 'dbus-glib-devel-0.100.2-2.fc20',
- },
- {'build_id': 442366,
- 'name': 'dbus-glib-debuginfo', 'release': '2.fc20', 'version': '0.100.2',
- 'arch': 'i686', 'nvr': 'dbus-glib-debuginfo-0.100.2-2.fc20',
- },
- {'build_id': 442366,
- 'name': 'dbus-glib', 'version': '0.100.2', 'release': '2.fc20',
- 'arch': 'i686', 'nvr': 'dbus-glib-0.100.2-2.fc20',
- },
-
- {'build_id': 715478,
- 'name': 'dbus-glib-debuginfo', 'version': '0.106', 'release': '1.fc23',
- 'arch': 'i686', 'nvr': 'dbus-glib-debuginfo-0.106-1.fc23',
- },
- {'build_id': 715478,
- 'name': 'dbus-glib', 'version': '0.106', 'release': '1.fc23',
- 'arch': 'i686', 'nvr': 'dbus-glib-0.106-1.fc23',
- },
- {'build_id': 715478,
- 'name': 'dbus-glib-devel', 'version': '0.106', 'release': '1.fc23',
- 'arch': 'i686', 'nvr': 'dbus-glib-devel-0.106-1.fc23',
- },
- {'build_id': 715478,
- 'name': 'dbus-glib', 'version': '0.106', 'release': '1.fc23',
- 'arch': 'x86_64', 'nvr': 'dbus-glib-0.106-1.fc23',
- },
- {'build_id': 715478,
- 'name': 'dbus-glib-debuginfo', 'version': '0.106', 'release': '1.fc23',
- 'arch': 'x86_64', 'nvr': 'dbus-glib-debuginfo-0.106-1.fc23',
- },
- {'build_id': 715478,
- 'name': 'dbus-glib-devel', 'release': '1.fc23', 'version': '0.106',
- 'arch': 'x86_64', 'nvr': 'dbus-glib-devel-0.106-1.fc23',
- },
-
- # RPMs of build nss-util-3.12.6-1.fc14
- {'build_id': 160295,
- 'name': 'nss-util', 'release': '1.fc14', 'version': '3.12.6',
- 'arch': 'x86_64', 'nvr': 'nss-util-3.12.6-1.fc14',
- },
- {'build_id': 160295,
- 'name': 'nss-util-devel', 'release': '1.fc14', 'version': '3.12.6',
- 'arch': 'x86_64', 'nvr': 'nss-util-devel-3.12.6-1.fc14',
- },
- {'build_id': 160295,
- 'name': 'nss-util-debuginfo', 'release': '1.fc14', 'version': '3.12.6',
- 'arch': 'x86_64', 'nvr': 'nss-util-debuginfo-3.12.6-1.fc14',
- },
-
- # RPMs of build nss-util-3.24.0-2.0.fc25
- {'build_id': 767978,
- 'name': 'nss-util-debuginfo', 'release': '2.0.fc25', 'version': '3.24.0',
- 'arch': 'x86_64', 'nvr': 'nss-util-debuginfo-3.24.0-2.0.fc25',
- },
- {'build_id': 767978,
- 'name': 'nss-util', 'release': '2.0.fc25', 'version': '3.24.0',
- 'arch': 'x86_64', 'nvr': 'nss-util-3.24.0-2.0.fc25',
- },
- {'build_id': 767978,
- 'name': 'nss-util-devel', 'release': '2.0.fc25', 'version': '3.24.0',
- 'arch': 'x86_64', 'nvr': 'nss-util-devel-3.24.0-2.0.fc25',
- },
- ]
-
-# ----------------- End of Koji resource storage ------------------
-
-
-class AssertionHelper(object):
- """A helper class providing methods to assert output of abipkgdiff"""
-
- def assert_compared_binaries(self, output, expected_binaries):
- """Assert specific binaries are already compared as expected
-
- Caller doesn't need to care about the order of elements of
- expected_binaries, it will be sorted by `list.sort` before assertion.
-
- :param str output: output from abipkgdiff that is used to parse and
- assert whether it contains expected binaries.
- :param list expected_binaries: which binaries are expected to be
- compared. Each of the binaries is a rpm filename, for example, foo.so.
- """
- assert isinstance(expected_binaries, (tuple, list))
-
- binaries = re.findall(r"=+\s?changes of '(.+)'\s*=+",
- output,
- re.MULTILINE)
- binaries = list(set(binaries))
- binaries.sort()
- expected_binaries.sort()
- self.assertEquals(expected_binaries, binaries)
-
- def assert_compared_rpms(self, output, expected_rpms):
- """Assert specific rpms are already compared as expected
-
- Caller doesn't need to care about the order of elements of
- expected_rpms, but the order of each element, that is ('foo-0.1.rpm',
- 'foo-0.2.rpm') is different from ('foo-0.2.rpm', 'foo-0.1.rpm').
- expected_rpms will be sorted properly.
-
- :param str output: output from abipkgdiff that is used to parse and
- assert whether it contains expected rpms.
- :param list expected_rpms: which rpms are expected to be compared.
- """
- assert isinstance(expected_rpms, (tuple, list))
-
- compared_rpms = re.findall(
- r'Comparing the ABI of binaries between ([^\s]+) and ([^\s]+):',
- output,
- re.MULTILINE)
-
- sort_key_func = lambda rpm_pair: ' '.join(rpm_pair)
- compared_rpms = sorted(compared_rpms, key=sort_key_func)
- expected_rpms = sorted(expected_rpms, key=sort_key_func)
- self.assertEquals(expected_rpms, compared_rpms)
-
- def assert_functions_changes_summary(self, output,
- expected_func_change_summary):
- """Assert functions changes summary is the expected result
-
- :param str output: output from abipkgdiff that is used to parse and
- assert whether "Functions changes summary:" contains expected changes.
- :param dict expected_func_change_summary: a mapping containing
- expected functions changes summary information, where keys must
- contain removed, changed, changed_filtered_out, functions_added.
- """
- assert isinstance(expected_func_change_summary, dict)
-
- pattern = r'Functions changes summary: (\d+) Removed, (\d+) Changed \((\d+) filtered out\), (\d+) Added functions' # noqa
- match = re.search(pattern, output)
- if match is None:
- raise ValueError(
- 'ABI changes does not contain functions changes summary.')
- else:
- groups = match.groups()
- removed = int(groups[0])
- changed = int(groups[1])
- changed_filtered_out = int(groups[2])
- functions_added = int(groups[3])
-
- self.assertEquals(expected_func_change_summary['removed'], removed)
- self.assertEquals(expected_func_change_summary['changed'], changed)
- self.assertEquals(
- expected_func_change_summary['changed_filtered_out'],
- changed_filtered_out)
- self.assertEquals(expected_func_change_summary['functions_added'],
- functions_added)
-
- def assert_abi_comparison_result(self,
- output,
- expected_binaries=None,
- expected_rpms=None,
- expected_func_change_summary=None):
- """A convenient way to assert expected values
-
- Parameters will be passed to corresponding methods. Each assertio
- method called within this method could be invoked individually. This
- method exists as a convenient way to assert all expected values within
- only one method call.
- """
- if expected_rpms:
- self.assert_compared_rpms(output, expected_rpms)
- if expected_binaries:
- self.assert_compared_binaries(output, expected_binaries)
- if expected_func_change_summary:
- self.assert_functions_changes_summary(output,
- expected_func_change_summary)
-
-
-class MockClientSession(object):
- """Mock koji.ClientSession
-
- This mock ClientSession aims to avoid touching a real Koji instance to
- interact with XMLRPC APIs required by fedabipkgdiff.
-
- For the tests within this module, methods do not necessarily to return
- complete RPM and build information. So, if you need more additional
- information, here is the right place to add them.
- """
-
- def __init__(self, baseurl):
- """Initialize a mock ClientSession
-
- :param str baseurl: the URL to remote kojihub service. As of writing
- this mock class, `baseurl` is not used at all, just keep it here if
- it's useful in the future.
-
- All mock methods have same signature as corresponding kojihub.*
- methods, and type of parameters may be different and only satify
- fedabipkgdiff requirement.
- """
- self.baseurl = baseurl
-
- def getPackage(self, name):
- """Mock kojihub.getPackage
-
- :param str name: name of package to find and return
- :return: the found package
- :rtype: dict
- """
- assert isinstance(name, basestring)
-
- def selector(package):
- return package['name'] == name
-
- return filter(selector, packages)[0]
-
- def getBuild(self, build_id):
- """Mock kojihub.getBuild
-
- :param int build_id: ID of build to find and return
- :return: the found build
- :rtype: dict
- """
- assert isinstance(build_id, int)
-
- def selector(build):
- return build['build_id'] == build_id
-
- return filter(selector, builds)[0]
-
- def listBuilds(self, packageID, state=None):
- """Mock kojihub.listBuilds
-
- :param int packageID: ID of package whose builds is found and returned
- :param state: build state. If state is omitted, all builds of a package
- are returned
- :type state: int or None
- """
- assert isinstance(packageID, int)
- if state is not None:
- assert isinstance(state, int)
-
- def selector(build):
- selected = build['package_id'] == packageID
- if state is not None:
- selected = selected and build['state'] == state
- return selected
-
- return filter(selector, builds)
-
- def getRPM(self, rpminfo):
- """Mock kojihub.getRPM
-
- :param dict rpminfo: a mapping containing rpm information, at least,
- it contains name, version, release, and arch.
- """
- assert isinstance(rpminfo, dict)
-
- def selector(rpm):
- return rpm['name'] == rpminfo['name'] and \
- rpm['version'] == rpminfo['version'] and \
- rpm['release'] == rpminfo['release'] and \
- rpm['arch'] == rpminfo['arch']
-
- return filter(selector, rpms)[0]
-
- def listRPMs(self, buildID, arches=None):
- """Mock kojihub.listRPMs
-
- :param int buildID: ID of build from which to list rpms
- :param arches: to list rpms built for specific arches. If arches is
- omitted, rpms of all arches will be listed.
- :type arches: list, tuple, str, or None
- :return: list of rpms
- :rtype: list
- """
- assert isinstance(buildID, int)
- if arches is not None:
- assert isinstance(arches, (tuple, list, basestring))
-
- if arches is not None and isinstance(arches, basestring):
- arches = [arches]
-
- def selector(rpm):
- selected = rpm['build_id'] == buildID
- if arches is not None:
- selected = selected and rpm['arch'] in arches
- return selected
-
- return filter(selector, rpms)
-
-
-class MockGlobalConfig(object):
- """Used to mock global_config
-
- Since tests do not parse options from command line, so this class is
- helpful for tests to contain all potential parsed (simulated)
- options.
-
- Currently, only koji_server and dry_run are required for running
- tests. If any new test cases need others, please add them add as
- class attribute directly.
- """
- koji_server = fedabipkgdiff_mod.DEFAULT_KOJI_SERVER
- koji_topdir = fedabipkgdiff_mod.DEFAULT_KOJI_TOPDIR
- dry_run = False
- dso_only = True
- abipkgdiff = BUILT_ABIPKGDIFF
- no_default_suppr = True
- no_devel_pkg = None
- check_all_subpackages = None
-
-
-# Test Cases go here
-
-
-class UtilsTest(unittest.TestCase):
-
- def test_is_distro_valid(self):
- """Test is_fedora_distro method
-
- is_fedora_distro aims to test if a string is a valid Fedora distro. I
- don't see there is a general rule or format definition for such a
- Fedora distro. I refer to second part of %{dist} splited by dot as the
- reference. Generally, fc4, fc19, fc23 are valid ones, and el6, el7 are
- also valid one currently.
- """
- distro = 'fc5'
- self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
-
- distro = 'f5'
- self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
-
- distro = 'fc23'
- self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
-
- distro = 'fc'
- self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
-
- distro = 'fc234'
- self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
-
- distro = 'el7'
- self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
-
- distro = 'el7_2'
- self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
-
-
-class RPMTest(unittest.TestCase):
- """Test case for RPM class
-
- RPM class is a class wrapping a underlying dict object represeting a RPM
- information, that is returned from Koji XMLRPC APIs.
-
- This test aims to test the class to see if RPM attributes is accessible in
- Python class attribute way, and if a RPM is specific type of RPM, for
- example, if it's a debuginfo.
- """
-
- def setUp(self):
- """Setup test data for testing RPM class
-
- According to the tests, it's unnecessary to contruct a complete dict
- containing full RPM information. So, only part of of them is
- enough. This test case only need name, version, release, and arch.
-
- In case Koji changes the name of name, version, release or arch in the
- future to express same meaning individually, (I don't think it could
- happen), please update there also.
- """
-
- # Argument passed to RPM.__init__ to construct a RPM class object, that
- # represents a debuginfo RPM.
- self.debuginfo_rpm_info = {
- 'arch': 'i686',
- 'name': 'httpd-debuginfo',
- 'release': '1.fc22',
- 'version': '2.4.18'
- }
-
- # Argument passed to RPM.__init__ to construct a RPM class object, that
- # represents a RPM.
- self.rpm_info = {
- 'arch': 'x86_64',
- 'name': 'httpd',
- 'release': '1.fc22',
- 'version': '2.4.18'
- }
-
- self.devel_rpm_info = {
- 'arch': 'x86_64',
- 'name': 'httpd-devel',
- 'release': '1.fc22',
- 'version': '2.4.18'
- }
-
- def test_attribute_access(self):
- """Ensure wrapped RPM information is accessible via attribute"""
- rpm = fedabipkgdiff_mod.RPM(self.debuginfo_rpm_info)
- self.assertEquals(self.debuginfo_rpm_info['arch'], rpm.arch)
- self.assertEquals(self.debuginfo_rpm_info['name'], rpm.name)
- self.assertEquals(self.debuginfo_rpm_info['release'], rpm.release)
- self.assertEquals(self.debuginfo_rpm_info['version'], rpm.version)
-
- def test_raise_error_if_name_not_exist(self):
- """
- Ensure AttributeError should be raised when accessing a non-existent
- attribute
- """
- rpm = fedabipkgdiff_mod.RPM({})
- try:
- rpm.xxxxx
- except AttributeError:
- # Succeed, exit normally
- return
- self.fail('AttributeError should be raised, but not.')
-
- def test_is_debuginfo(self):
- """Ensure to return True if a RPM's name contains -debuginfo"""
- rpm = fedabipkgdiff_mod.RPM(self.debuginfo_rpm_info)
- self.assertTrue(rpm.is_debuginfo)
-
- rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
- self.assertFalse(rpm.is_debuginfo)
-
- def test_is_devel(self):
- """Ensure return True if a package is a development package"""
- rpm = fedabipkgdiff_mod.RPM(self.debuginfo_rpm_info)
- self.assertFalse(rpm.is_devel)
-
- rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
- self.assertFalse(rpm.is_devel)
-
- rpm = fedabipkgdiff_mod.RPM(self.devel_rpm_info)
- self.assertTrue(rpm.is_devel)
-
- def test_nvra(self):
- """
- Ensure value from RPM.nvra is parsable and contains correct value from
- underlying RPM information
- """
- rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
- nvra = koji.parse_NVRA(rpm.nvra)
- self.assertEquals(nvra['name'], rpm.name)
- self.assertEquals(nvra['version'], rpm.version)
- self.assertEquals(nvra['release'], rpm.release)
- self.assertEquals(nvra['arch'], rpm.arch)
-
- def test_str_representation(self):
- """
- Enforce a RPM object has same string represetation as underlying
- wrapped rpm information that is a dict object.
- """
- rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
- self.assertEquals(str(self.rpm_info), str(rpm))
-
-
-class LocalRPMTest(unittest.TestCase):
- """Test case for LocalRPM class
-
- Because LocalRPM inherits from RPM, all tests against RPM class are also
- applied to LocalRPM, so I don't repeat them again here. This test case
- mainly focus on the abilities against files on the local disk.
- """
-
- def setUp(self):
- # A RPM filename that simulates a RPM file that is stored somewhere on
- # the disk.
- # This is the only argument passed to LocalRPM.__init__ to initialize
- # an object.
- self.filename = 'httpd-2.4.18-1.fc22.x86_64.rpm'
-
- def test_file_parser_without_path(self):
- """Ensure LocalRPM can get RPM information from a filename
-
- LocalRPM gets name, version, release, and arch of a RPM by parsing the
- passed filename to __init__ method. Then, all these information is
- accessible via LocalRPM name, version, release, and arch attribute.
-
- A filename either with an absolute path, relative path, or without a
- path, LocalRPM should be able to find these files and get correct
- information by removing the potential present path. For example, by
- giving following filenames,
-
- - httpd-2.4.18-1.fc22.x86_64.rpm
- - artifacts/httpd-2.4.18-1.fc22.x86_64.rpm
- - /mnt/koji/packages/httpd/2.4.18/1.fc22/httpd-2.4.18-1.fc22.x86_64.rpm
-
- LocalRPM has to determine the necessary RPM information from
- httpd-2.4.18-1.fc22.x86_64.rpm
-
- Without specifying path in the filename, it usually means LocalRPM
- should find the RPM file relative to current working directory. So, no
- need of additional test against a filename with a relative path.
- """
- rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
- nvra = koji.parse_NVRA(self.filename)
- self.assertEquals(nvra['name'], rpm.name)
- self.assertEquals(nvra['version'], rpm.version)
- self.assertEquals(nvra['release'], rpm.release)
- self.assertEquals(nvra['arch'], rpm.arch)
-
- full_filename = os.path.join('/', 'tmp', self.filename)
- rpm = fedabipkgdiff_mod.LocalRPM(full_filename)
- nvra = koji.parse_NVRA(self.filename)
- self.assertEquals(nvra['name'], rpm.name)
- self.assertEquals(nvra['version'], rpm.version)
- self.assertEquals(nvra['release'], rpm.release)
- self.assertEquals(nvra['arch'], rpm.arch)
- self.assertEquals(full_filename, rpm.downloaded_file)
-
- @patch('os.path.exists')
- def test_find_existent_debuginfo(self, mock_exists):
- """Ensure LocalRPM can find an associated existent debuginfo RPM
-
- Currently, find_debuginfo is only able to find associated debuginfo RPM
- from the directory where local RPM resides. This test works for this
- case at this moment. If there is a requirement to allow find debuginfo
- RPM from somewhere else, any level of subdirectory for instance, add
- new test case for that, and update these words you are reading :)
- """
- mock_exists.return_value = True
-
- rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
- self.assertTrue(isinstance(rpm, fedabipkgdiff_mod.LocalRPM))
-
- nvra = koji.parse_NVRA(self.filename)
- expected_debuginfo = fedabipkgdiff_mod.LocalRPM(
- '%(name)s-debuginfo-%(version)s-%(release)s.%(arch)s.rpm' % nvra)
- debuginfo = rpm.find_debuginfo()
- self.assertEquals(expected_debuginfo.name, debuginfo.name)
- self.assertEquals(expected_debuginfo.version, debuginfo.version)
- self.assertEquals(expected_debuginfo.release, debuginfo.release)
-
- def test_find_non_existent_debuginfo(self):
- """Ensure to return None if cannot find associated debuginfo RPM
-
- os.path.exists is not mocked, that is because the associated debuginfo
- RPM of httpd-2.4.18-1.fc22.x86_64.rpm given in setUp must be
- non-existed during this test's run.
- """
- rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
- self.assertEquals(None, rpm.find_debuginfo())
-
-
-class RunAbipkgdiffTest(unittest.TestCase):
- """Test case for method run_abipkgdiff
-
- Method run_abipkgdiff accepts package informations and passes them to and
- run abipkgdiff command line utility. Since run_abipkgdiff does not catch
- output to either standard output or standard error, and only returns the
- return code that is returned from underlying abipkgdiff, these various test
- cases test whether run_abipkgdiff is able to return the return code
- correctly.
- """
-
- def setUp(self):
- """Define packages information for calling run_abipkgdiff method
-
- Due to the tests just care about the return code from underlying
- abipkgdiff, only partial attributes of a RPM is required. That means,
- it's unnecessary to give a full dict representing a complete RPM, just
- build_id, name, version, release, and arch.
-
- Full RPM information is not required. For this test case, only partial
- information arch, build_id, name, release, and version are enough.
- """
-
- # Used for testing the case of running abipkgdiff against one RPM
- self.pkg1_single_info = {
- 'i686': [
- fedabipkgdiff_mod.RPM({'arch': 'i686',
- 'build_id': 720222,
- 'name': 'httpd',
- 'release': '2.fc24',
- 'version': '2.4.18',
- }),
- fedabipkgdiff_mod.RPM({'arch': 'i686',
- 'build_id': 720222,
- 'name': 'httpd-debuginfo',
- 'release': '2.fc24',
- 'version': '2.4.18',
- }),
- fedabipkgdiff_mod.RPM({'arch': 'i686',
- 'build_id': 720222,
- 'name': 'httpd-devel',
- 'release': '2.fc24',
- 'version': '2.4.18',
- })
- ],
- }
-
- # Whatever the concrete content of pkg2_infos is, so just make a copy
- # from self.pkg1_infos
- self.pkg2_single_info = self.pkg1_single_info.copy()
-
- # Used for testing the case of running abipkgdiff against multiple RPMs
- self.pkg1_infos = {
- 'i686': [
- fedabipkgdiff_mod.RPM({'arch': 'i686',
- 'build_id': 720222,
- 'name': 'httpd',
- 'release': '2.fc24',
- 'version': '2.4.18',
- }),
- fedabipkgdiff_mod.RPM({'arch': 'i686',
- 'build_id': 720222,
- 'name': 'httpd-debuginfo',
- 'release': '2.fc24',
- 'version': '2.4.18',
- }),
- fedabipkgdiff_mod.RPM({'arch': 'i686',
- 'build_id': 720222,
- 'name': 'httpd-devel',
- 'release': '2.fc24',
- 'version': '2.4.18',
- }),
- ],
- 'x86_64': [
- fedabipkgdiff_mod.RPM({'arch': 'x86_64',
- 'build_id': 720222,
- 'name': 'httpd',
- 'release': '2.fc24',
- 'version': '2.4.18',
- }),
- fedabipkgdiff_mod.RPM({'arch': 'x86_64',
- 'build_id': 720222,
- 'name': 'httpd-debuginfo',
- 'release': '2.fc24',
- 'version': '2.4.18',
- }),
- fedabipkgdiff_mod.RPM({'arch': 'x86_64',
- 'build_id': 720222,
- 'name': 'httpd-devel',
- 'release': '2.fc24',
- 'version': '2.4.18',
- }),
- ],
- 'armv7hl': [
- fedabipkgdiff_mod.RPM({'arch': 'armv7hl',
- 'build_id': 720222,
- 'name': 'httpd',
- 'release': '2.fc24',
- 'version': '2.4.18',
- }),
- fedabipkgdiff_mod.RPM({'arch': 'armv7hl',
- 'build_id': 720222,
- 'name': 'httpd-debuginfo',
- 'release': '2.fc24',
- 'version': '2.4.18',
- }),
- fedabipkgdiff_mod.RPM({'arch': 'armv7hl',
- 'build_id': 720222,
- 'name': 'httpd-devel',
- 'release': '2.fc24',
- 'version': '2.4.18',
- }),
- ],
- }
-
- # Whatever the concrete content of pkg2_infos is, so just make a copy
- # from self.pkg1_infos
- self.pkg2_infos = self.pkg1_infos.copy()
-
- @patch('fedabipkgdiff.abipkgdiff')
- @patch('fedabipkgdiff.global_config', new=MockGlobalConfig)
- def test_all_success(self, mock_abipkgdiff):
- """
- Ensure run_abipkgdiff returns 0 when it succeeds to run against one or
- more packages.
- """
- mock_abipkgdiff.return_value = 0
-
- result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_single_info,
- self.pkg2_single_info)
- self.assertEquals(0, result)
-
- result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
- self.pkg2_infos)
- self.assertEquals(0, result)
-
- @patch('fedabipkgdiff.abipkgdiff')
- @patch('fedabipkgdiff.global_config', new=MockGlobalConfig)
- def test_all_failure(self, mock_abipkgdiff):
- """
- Ensure run_abipkgdiff returns the return code from underlying
- abipkgdiff when all calls to abipkgdiff fails against one or more
- packages.
- """
- mock_abipkgdiff.return_value = 4
-
- result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_single_info,
- self.pkg2_single_info)
- self.assertEquals(4, result)
-
- result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
- self.pkg2_infos)
- self.assertEquals(4, result)
-
- @patch('fedabipkgdiff.abipkgdiff',
- new=lambda param1, param2: counter.next())
- @patch('fedabipkgdiff.global_config', new=MockGlobalConfig)
- def test_partial_failure(self):
- """
- Ensure run_abipkgdiff returns non-zero when partial calls to
- run_abipkgdiff succeed
-
- abipkgdiff is mocked in order to simulte the partial success
- calls. Why? That is because, counter starts from 0. So, it will
- generate 0, 1, 2, ...
- """
- result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
- self.pkg2_infos)
- self.assertTrue(result > 0)
-
-
-class GetPackageLatestBuildTest(unittest.TestCase):
- """Test case for get_package_latest_build"""
-
- @patch('fedabipkgdiff.global_config', new=MockGlobalConfig)
- @patch('koji.ClientSession', new=MockClientSession)
- def test_get_latest_one(self):
- """Ensure to get latest build of a package"""
- session = fedabipkgdiff_mod.get_session()
- build = session.get_package_latest_build('gnutls', 'fc23')
- self.assertEquals('gnutls-3.4.13-1.fc23', build['nvr'])
-
- @patch('fedabipkgdiff.global_config', new=MockGlobalConfig)
- @patch('koji.ClientSession', new=MockClientSession)
- def test_cannot_find_a_latest_build_with_invalid_distro(self):
- """
- Ensure NoCompleteBuilds is raised when trying to find a latest build of
- a package for unknown Fedora distribution.
- """
- session = fedabipkgdiff_mod.get_session()
- self.assertRaises(fedabipkgdiff_mod.NoCompleteBuilds,
- session.get_package_latest_build,
- 'gnutls', 'xxxx')
-
-
-class DownloadRPMTest(unittest.TestCase):
- """Test case for download_rpm
-
- Download a remote file, which is a local file simulating a remote file with
- scheme file://, for example file:///tmp/a.txt, to download directory.
- """
-
- def setUp(self):
- # Create a remote file for testing download of this file
- self.fd, self.remote_filename = tempfile.mkstemp(
- prefix=temp_file_or_dir_prefix)
- # Whatever the content is, this case does not care about. Close it
- # immediately.
- os.close(self.fd)
-
- def tearDown(self):
- os.remove(self.remote_filename)
-
- def make_remote_file_url(self):
- """Make URL of remote file that is used for downloading this file"""
- return 'file://{0}'.format(self.remote_filename)
-
- def make_nonexistent_remote_file_url(self):
- """Return URL to a non-existent remote file"""
- return os.path.join(self.make_remote_file_url(), 'nonexistent-file')
-
- @patch('fedabipkgdiff.global_config', new=MockGlobalConfig)
- @patch('fedabipkgdiff.get_download_dir',
- return_value=TEST_DOWNLOAD_CACHE_DIR)
- def test_succeed_to_download_a_rpm(self, mock_get_download_dir):
- """Enusre True is returned if curl succeeds to download remote file
-
- Download remote file to a fake download directory. Ensure everything is
- okay, and return value from download_rpm should be truth.
- """
- url = self.make_remote_file_url()
- ret = fedabipkgdiff_mod.download_rpm(url)
- self.assertTrue(ret)
-
- @patch('fedabipkgdiff.global_config', new=MockGlobalConfig)
- @patch('fedabipkgdiff.get_download_dir',
- return_value=TEST_DOWNLOAD_CACHE_DIR)
- def test_failed_to_download_a_rpm(self, mock_get_download_dir):
- """Ensure False is returned if curl fails to download remote file
-
- Download remote file to a fake download directory. But, making
- something wrong to cause download_rpm returns false.
- """
- url = self.make_nonexistent_remote_file_url()
- ret = fedabipkgdiff_mod.download_rpm(url)
- self.assertFalse(ret)
-
-
-class BrewListRPMsTest(unittest.TestCase):
- """Test case for Brew.listRPMs"""
-
- @patch('fedabipkgdiff.global_config', new=MockGlobalConfig)
- @patch('koji.ClientSession', new=MockClientSession)
- def test_select_specific_rpms(self):
- """Ensure Brew.listRPMs can select RPMs by a specific selector
-
- This test will select RPMs whose name starts with httpd, that is only
- httpd and httpd-debuginfo RPMs are selected and returned.
- """
- def selector(rpm):
- return rpm['name'] == 'dbus-glib-debuginfo' and \
- rpm['arch'] == 'x86_64' and \
- rpm['release'].endswith('fc20')
-
- session = fedabipkgdiff_mod.get_session()
- rpms = session.listRPMs(buildID=442366, selector=selector)
- expected_rpms = [{
- 'build_id': 442366,
- 'name': 'dbus-glib-debuginfo',
- 'release': '2.fc20',
- 'version': '0.100.2',
- 'arch': 'x86_64',
- 'nvr': 'dbus-glib-debuginfo-0.100.2-2.fc20',
- }]
- self.assertEquals(expected_rpms, rpms)
-
-
-# Integration tests
+# Note that after fedabipkgdiff is run (using the arguments that are
+# the first element of the tuple) the result of the comparison is
+# going to be compared against the reference output file, and both
+# must be equal.
+#
+# If a user wants to add a new test, she must add a new tuple to the
+# variable below, and also add a new reference output to the source
+# distribution.
#
-# Following integration tests aim to test execution of fedabipkgdiff from
-# command line options to the expected ABI comparison result.
-
-
-class CompareABIFromCommandLineTest(AssertionHelper, unittest.TestCase):
- """Test case for testing various use cases
-
- All these tests aim to test fedabipkgdiff by executing against command line
- options, that is to execute underlying abipkgdiff command. Output of
- fedabipkgdiff is checked to see if it contains proper compared rpms and
- binaries.
-
- Following are mocked when run each test
-
- * koji.ClientSession: avoid calling to the real Koji instance.
-
- * --topdir option: let fedabipkgdiff download rpms from fedabipkgdiff's
- test data directory instead of the remote Koji server.
-
- * global method get_download_dir: use the fake directory to hold downloaded
- rpm packages instead of .cache/fedabipkgdiff to break anything.
-
- * sys.argv: to provide fedabipkgdiff command line options for each test.
+FEDABIPKGDIFF_TEST_SPECS = [
+ (['--from', 'fc20', '--to', 'fc23', 'dbus-glib'],
+ 'data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt',
+ 'output/test-fedabipkgdiff/test0-type-suppr-report-0.txt'),
+
+ (['--from', 'fc20', os.path.join(INPUT_DIR,
+ 'packages/dbus-glib/0.106/1.fc23/x86_64/'
+ 'dbus-glib-0.106-1.fc23.x86_64.rpm')],
+ 'data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt',
+ 'output/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt'),
+
+ (['dbus-glib-0.100.2-2.fc20', 'dbus-glib-0.106-1.fc23'],
+ 'data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt',
+ 'output/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt'),
+
+ (['dbus-glib-0.100.2-2.fc20.i686', 'dbus-glib-0.106-1.fc23.i686'],
+ 'data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt',
+ 'output/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt'),
+]
+
+def ensure_output_dir_created():
+ '''Create output dir if it's not already created.'''
+
+ try:
+ os.makedirs(OUTPUT_DIR)
+ except:
+ pass
+
+ if not os.path.exists(OUTPUT_DIR):
+ sys.exit(1);
+
+def run_fedabipkgdiff_tests():
+ """Run the fedabipkgdiff tests
+
+ Loop through the test inputs in the FEDABIPKGDIFF_TEST_SPECS global
+ variable and for each of the test input, launch a comparison using
+ mockfedabipkgdiff, which calls fedabipkgdiff by making sure it
+ doesn't use the network.
+
+ The result of the comparison is stored in an output file, and that
+ output file is compared (using GNU diff) against the reference
+ output file specified in the FEDABIPKGDIFF_TEST_SPECS global
+ variable.
+
+ This function returns True iff all comparison specified by
+ FEDABIPKGDIFF_TEST_SPECS succeed.
"""
- @patch('koji.ClientSession', new=MockClientSession)
- @patch('sys.stdout', new_callable=StringIO)
- @patch('fedabipkgdiff.get_download_dir',
- return_value=TEST_DOWNLOAD_CACHE_DIR)
- def run_abipkgdiff(self, mock_get_download_dir, mock_stdout):
- fedabipkgdiff_mod.main()
- return mock_stdout.getvalue()
-
- @patch('sys.argv', new=['fedabipkgdiff', '--topdir', TEST_TOPDIR,
- '--abipkgdiff', BUILT_ABIPKGDIFF,
- '--from', 'fc20', '--to', 'fc23', 'dbus-glib'])
- def test_compare_against_latest_build_of_package(self):
- """Test compare all arches of a package's latest build
-
- Use case to test::
-
- fedabipkgdiff --from fc20 --to fc23 dbus-glib
- """
- output = self.run_abipkgdiff()
-
- expected_rpms = [
- ('dbus-glib-0.100.2-2.fc20.i686.rpm',
- 'dbus-glib-0.106-1.fc23.i686.rpm'),
- ('dbus-glib-0.100.2-2.fc20.x86_64.rpm',
- 'dbus-glib-0.106-1.fc23.x86_64.rpm'),
- ]
- expected_binaries = ['dbus-binding-tool', 'libdbus-glib-1.so.2.2.2']
-
- self.assert_abi_comparison_result(output,
- expected_binaries=expected_binaries,
- expected_rpms=expected_rpms)
-
- @patch('sys.argv',
- new=['fedabipkgdiff', '--topdir', TEST_TOPDIR,
- '--abipkgdiff', BUILT_ABIPKGDIFF,
- '--from', 'fc20',
- os.path.join(TEST_DATA_DIR,
- 'packages/dbus-glib/0.106/1.fc23/x86_64/'
- 'dbus-glib-0.106-1.fc23.x86_64.rpm')])
- def test_compare_local_rpms_with_koji_rpms(self):
- """Test compare local rpm with remote associated one from Koji
-
- Use case to test::
-
- fedabipkgdiff --from fc20 path/to/local/rpm
- """
- output = self.run_abipkgdiff()
-
- expected_rpms = [
- ('dbus-glib-0.100.2-2.fc20.x86_64.rpm',
- 'dbus-glib-0.106-1.fc23.x86_64.rpm'),
- ]
- expected_binaries = ['dbus-binding-tool', 'libdbus-glib-1.so.2.2.2']
+ result = True;
+ for args, reference_report_path, output_path in FEDABIPKGDIFF_TEST_SPECS:
+ reference_report_path = os.path.join(TEST_SRC_DIR, reference_report_path)
+ output_path = os.path.join(TEST_BUILD_DIR, output_path)
+ cmd = [FEDABIPKGDIFF,
+ '--no-default-suppression',
+ '--show-identical-binaries'] + args
- self.assert_abi_comparison_result(output,
- expected_binaries=expected_binaries,
- expected_rpms=expected_rpms)
+ with open(output_path, 'w') as out_file:
+ subprocess.call(cmd, stdout=out_file)
- @patch('sys.argv', new=['fedabipkgdiff', '--topdir', TEST_TOPDIR,
- '--abipkgdiff', BUILT_ABIPKGDIFF,
- 'dbus-glib-0.100.2-2.fc20',
- 'dbus-glib-0.106-1.fc23'])
- def test_compare_rpm_packages(self):
- """Test compare rpms of packages specified by N-V-R
+ if not os.path.exists(reference_report_path):
+ sys.stderr.write('file not found: ' + reference_report_path)
+ return_code = -1
+ else:
+ diffcmd = ['diff', '-u', reference_report_path, output_path]
+ return_code = subprocess.call(diffcmd)
+ if return_code:
+ sys.stderr.write('fedabipkgdiff test failed for ' +
+ reference_report_path + '\n')
+ result &= not return_code
- Use case to test::
+ return result;
- fedabipkgdiff dbus-glib-0.100.2-2.fc20 dbus-glib-0.106-1.fc23
- """
- output = self.run_abipkgdiff()
+def main():
+ """The main entry point of this program.
- expected_rpms = [
- ('dbus-glib-0.100.2-2.fc20.i686.rpm',
- 'dbus-glib-0.106-1.fc23.i686.rpm'),
- ('dbus-glib-0.100.2-2.fc20.x86_64.rpm',
- 'dbus-glib-0.106-1.fc23.x86_64.rpm'),
- ]
- expected_binaries = ['dbus-binding-tool', 'libdbus-glib-1.so.2.2.2']
+ This creates the output directory and launches the tests for the
+ fedabipkgdiff program.
- self.assert_abi_comparison_result(output,
- expected_binaries=expected_binaries,
- expected_rpms=expected_rpms)
-
- @patch('sys.argv', new=['fedabipkgdiff', '--topdir', TEST_TOPDIR,
- '--abipkgdiff', BUILT_ABIPKGDIFF,
- 'dbus-glib-0.100.2-2.fc20.i686',
- 'dbus-glib-0.106-1.fc23.i686'])
- def test_compare_rpm_packages_with_specific_arch(self):
- """Test compare rpms of packages specified by N-V-R-A
-
- Use case to test::
-
- fedabipkgdiff \
- dbus-glib-0.100.2-2.fc20.i686 dbus-glib-0.106-1.fc23.i686
- """
- output = self.run_abipkgdiff()
-
- expected_rpms = [
- ('dbus-glib-0.100.2-2.fc20.i686.rpm',
- 'dbus-glib-0.106-1.fc23.i686.rpm'),
- ]
- expected_binaries = ['dbus-binding-tool', 'libdbus-glib-1.so.2.2.2']
-
- self.assert_abi_comparison_result(output,
- expected_binaries=expected_binaries,
- expected_rpms=expected_rpms)
-
- @patch('sys.argv', new=['fedabipkgdiff', '--topdir', TEST_TOPDIR,
- '--abipkgdiff', BUILT_ABIPKGDIFF,
- '--all-subpackages',
- 'dbus-glib-0.100.2-2.fc20',
- 'dbus-glib-0.106-1.fc23'])
- def test_compare_rpm_packages_all_subpackages(self):
- """Test compare all rpms instead of the main rpm only
-
- Use case to test:
-
- fedabipkgdiff --all-subpackages \
- dbus-glib-0.100.2-2.fc20.i686 dbus-glib-0.106-1.fc23.i686
- """
- output = self.run_abipkgdiff()
-
- expected_rpms = [
- ('dbus-glib-devel-0.100.2-2.fc20.i686.rpm',
- 'dbus-glib-devel-0.106-1.fc23.i686.rpm'),
- ('dbus-glib-0.100.2-2.fc20.i686.rpm',
- 'dbus-glib-0.106-1.fc23.i686.rpm'),
- ('dbus-glib-0.100.2-2.fc20.x86_64.rpm',
- 'dbus-glib-0.106-1.fc23.x86_64.rpm'),
- ('dbus-glib-devel-0.100.2-2.fc20.x86_64.rpm',
- 'dbus-glib-devel-0.106-1.fc23.x86_64.rpm')
- ]
- expected_binaries = [
- 'dbus-bash-completion-helper',
- 'dbus-binding-tool',
- 'libdbus-glib-1.so.2.2.2',
- ]
-
- self.assert_abi_comparison_result(output,
- expected_binaries=expected_binaries,
- expected_rpms=expected_rpms)
-
- @patch('sys.argv', new=['fedabipkgdiff', '--topdir', TEST_TOPDIR,
- '--abipkgdiff', BUILT_ABIPKGDIFF,
- '--dso-only',
- '--from', 'fc20', '--to', 'fc23', 'dbus-glib'])
- def test_compare_dso_only(self):
- """Test only compare shared libraries with --dso-only
-
- Use case to test:
-
- fedabipkgdiff --dso-only --from fc20 --to fc23 dbus-glib
- """
- output = self.run_abipkgdiff()
-
- expected_rpms = [
- ('dbus-glib-0.100.2-2.fc20.i686.rpm',
- 'dbus-glib-0.106-1.fc23.i686.rpm'),
- ('dbus-glib-0.100.2-2.fc20.x86_64.rpm',
- 'dbus-glib-0.106-1.fc23.x86_64.rpm'),
- ]
- expected_binaries = ['libdbus-glib-1.so.2.2.2']
-
- self.assert_abi_comparison_result(output,
- expected_binaries=expected_binaries,
- expected_rpms=expected_rpms)
-
- def test_compare_with_no_devel_pkg(self):
- """Test compare ABI with and without --no-devel-pkg
-
- Use case to test::
-
- fedabipkgdiff --no-devel-pkg \
- nss-util-3.12.6-1.fc14 nss-util-3.24.0-2.0.fc25
-
- and
-
- fedabipkgdiff nss-util-3.12.6-1.fc14 nss-util-3.24.0-2.0.fc25
- """
-
- compared_rpms = [('nss-util-3.12.6-1.fc14.x86_64.rpm',
- 'nss-util-3.24.0-2.0.fc25.x86_64.rpm')]
- compared_binaries = ['libnssutil3.so']
-
- # Without --no-devel-pkg, abipkgdiff is run with development packages
- # by default. In this case, expected changed will be 5 and expected
- # changed_filtered_out will be 17. Otherwise, with --no-devel-pkg,
- # changed will be 6 and changed_filtered_out will be 16.
- abipkgdiff_cmd_results = (
- (['fedabipkgdiff', '--topdir', TEST_TOPDIR,
- '--abipkgdiff', BUILT_ABIPKGDIFF,
- 'nss-util-3.12.6-1.fc14.x86_64',
- 'nss-util-3.24.0-2.0.fc25.x86_64',
- ],
- {'compared_rpms': compared_rpms,
- 'compared_binaries': compared_binaries,
- 'result': {
- 'removed': 0,
- 'changed': 5,
- 'changed_filtered_out': 17,
- 'functions_added': 37,
- }
- },
- ),
- (['fedabipkgdiff', '--topdir', TEST_TOPDIR,
- '--abipkgdiff', BUILT_ABIPKGDIFF,
- '--no-devel-pkg',
- 'nss-util-3.12.6-1.fc14.x86_64',
- 'nss-util-3.24.0-2.0.fc25.x86_64',
- ],
- {'compared_rpms': compared_rpms,
- 'compared_binaries': compared_binaries,
- 'result': {
- 'removed': 0,
- 'changed': 6,
- 'changed_filtered_out': 16,
- 'functions_added': 37,
- }
- })
- )
-
- for abipkgdiff_cmd, compare_result in abipkgdiff_cmd_results:
- with patch('sys.argv', new=abipkgdiff_cmd):
- output = self.run_abipkgdiff()
-
- self.assert_abi_comparison_result(
- output,
- expected_rpms=compare_result['compared_rpms'],
- expected_binaries=compare_result['compared_binaries'],
- expected_func_change_summary=compare_result['result'])
+ """
+ ensure_output_dir_created()
+ result = 0;
+ result = run_fedabipkgdiff_tests()
+ if not result:
+ return result;
if __name__ == '__main__':
- unittest.main()
+ exit_code = main()
+ sys.exit(exit_code)
diff --git a/tests/test-diff-pkg.cc b/tests/test-diff-pkg.cc
index f772379..902a5cc 100644
--- a/tests/test-diff-pkg.cc
+++ b/tests/test-diff-pkg.cc
@@ -390,6 +390,18 @@ static InOutSpec in_out_specs[] =
"data/test-diff-pkg/gtk2-immodule-xim-2.24.22-5.el7.i686--gtk2-immodule-xim-2.24.28-8.el7.i686-report-0.txt",
"output/test-diff-pkg/gtk2-immodule-xim-2.24.22-5.el7.i686--gtk2-immodule-xim-2.24.28-8.el7.i686-report-0.txt"
},
+ {
+ "data/test-diff-pkg/dbus-glib-0.80-3.fc12.x86_64.rpm",
+ "data/test-diff-pkg/dbus-glib-0.80-3.fc12.x86_64.rpm",
+ "--no-default-suppression --show-identical-binaries",
+ "",
+ "data/test-diff-pkg/dbus-glib-debuginfo-0.80-3.fc12.x86_64.rpm",
+ "data/test-diff-pkg/dbus-glib-debuginfo-0.80-3.fc12.x86_64.rpm",
+ "",
+ "",
+ "data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt",
+ "output/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt"
+ },
#endif //WITH_RPM
#ifdef WITH_DEB
diff --git a/tools/abipkgdiff.cc b/tools/abipkgdiff.cc
index d08af56..076f519 100644
--- a/tools/abipkgdiff.cc
+++ b/tools/abipkgdiff.cc
@@ -177,6 +177,7 @@ public:
bool show_added_syms;
bool show_added_binaries;
bool fail_if_no_debug_info;
+ bool show_identical_binaries;
vector<string> suppression_paths;
options(const string& program_name)
@@ -194,7 +195,8 @@ public:
show_locs(true),
show_added_syms(true),
show_added_binaries(true),
- fail_if_no_debug_info()
+ fail_if_no_debug_info(),
+ show_identical_binaries()
{}
};
@@ -609,6 +611,7 @@ display_usage(const string& prog_name, ostream& out)
<< " --no-abignore do not look for *.abignore files\n"
<< " --no-parallel do not execute in parallel\n"
<< " --fail-no-dbg fail if no debug info was found\n"
+ << " --show-identical-binaries show the names of identical binaries\n"
<< " --verbose emit verbose progress messages\n"
<< " --help|-h display this help message\n"
<< " --version|-v display program version information"
@@ -1224,7 +1227,15 @@ pthread_routine_compare(vector<compare_args_sptr> *args)
else
{
pthread_mutex_lock(&map_lock);
- reports_map[key] = shared_ptr<ostringstream>();
+ if (a->opts.show_identical_binaries)
+ {
+ shared_ptr<ostringstream> out(new ostringstream);
+ *out << "No ABI change detected\n";
+ reports_map[key] = out;
+ env_map[diff] = env;
+ }
+ else
+ reports_map[key] = shared_ptr<ostringstream>();
pthread_mutex_unlock(&map_lock);
}
}
@@ -1898,6 +1909,8 @@ parse_command_line(int argc, char* argv[], options& opts)
opts.abignore = false;
else if (!strcmp(argv[i], "--no-parallel"))
opts.parallel = false;
+ else if (!strcmp(argv[i], "--show-identical-binaries"))
+ opts.show_identical_binaries = true;
else if (!strcmp(argv[i], "--suppressions")
|| !strcmp(argv[i], "--suppr"))
{
diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
index f53613f..b463caa 100755
--- a/tools/fedabipkgdiff
+++ b/tools/fedabipkgdiff
@@ -63,6 +63,8 @@ DEFAULT_KOJI_TOPDIR = 'https://kojipkgs.fedoraproject.org'
HOME_DIR = os.path.join(xdg.BaseDirectory.xdg_cache_home,
os.path.splitext(os.path.basename(__file__))[0])
+DEFAULT_ABIPKGDIFF = 'abipkgdiff'
+
# Mask for determining if underlying fedabipkgdiff succeeds or not.
# This is for when the compared ABIs are equal
ABIDIFF_OK = 0
@@ -716,7 +718,7 @@ def build_path_to_abipkgdiff():
"""
if global_config.abipkgdiff:
return global_config.abipkgdiff
- return 'abipkgdiff'
+ return DEFAULT_ABIPKGDIFF
@log_call
@@ -748,6 +750,7 @@ def abipkgdiff(pkg_info1, pkg_info2):
cmd = [
abipkgdiff_tool,
+ '--show-identical-binaries' if global_config.show_identical_binaries else '',
'--no-default-suppression' if global_config.no_default_suppr else '',
'--dso-only' if global_config.dso_only else '',
'--d1', pkg_info1.debuginfo_package.downloaded_file,
@@ -1133,6 +1136,12 @@ def build_commandline_args_parser():
action='store_true',
dest='no_devel_pkg',
help='Do not compare ABI with development package')
+ parser.add_argument(
+ '--show-identical-binaries',
+ required=False,
+ action='store_true',
+ dest='show_identical_binaries',
+ help='Show information about binaries whose ABI are identical')
return parser
--
1.8.3.1
------------------------------->8<------------------------------------
--
Dodji