Some questions about ImageMagick and cygwin

[Charles Wilson <cygwin at cwilson dot fastmail dot fm>]

(for more background, see 
http://www.cygwin.com/ml/cygwin-apps/2003-12/msg00072.html )

Bob ---

It looks like we have a sucker^W volunteer to contribute (and maintain) 
an ImageMagick package for the cygwin platform.  However, a few 
packaging questions have arisen.  I raise a bunch of issues below, but I 
do have a solution -- and a question (or three).

Because IM provides two shared libraries that can be used from other 
third party applications, the DLLs need to 'live' in their own, 
versioned, package -- like we do for ncurses, for instance:

  libncurses7   (contains cygncurses7.dll, cygform7.dll, etc)
  libncurses-devel (contains the .a's, .dll.a's, .h's, and API docs)
  ncurses       (contains the utility progs, etc)

That way, if rxvt is linked against cygncurses7.dll, but the next 
version of ncurses causes an API change, we simply bump the DLL version 
number, and create a new matching package to distribute the new DLL: 
libncurses8-X.Y-Z.tar.bz2   Thus, the user can install both 
libncurses7(-A.B-C.tar.bz2) and libncurses8(-X.Y-Z.tar.bz2), and get the 
new dll but keep the old dll so that rxvt keeps working without 
requiring a recompile or relink.  Note tho, that the -devel and (main) 
packages get replaced -- so any newly linked 3rd party apps after this 
would use cygncurses8.dll; and of course the ncurses apps shipped with 
ncurses itself would all use the new dll.

In this case, IM creates 'cygMagick-6.dll' (and cygMagick++-6.dll).  So, 
naturally those go into

  libMagick6
     (with release versioning info tacked on after the actual
      "package name":  libMagick6-5.5.7-1.tar.bz2)
  libMagick-devel
  ImageMagick

So that it can coexist with a later 'libMagick7' package which contains 
cygMagick-7.dll if there is ever an API breakage.

Now, let us assume that in the fullness of time, package "FOO" is 
released which uses libMagick.  So, it's linked against cygMagick-6.dll. 
 Then, there's a new release of ImageMagick, version 5.9.2 or whatever, 
which has an API breakage, leading to cygMagick-7.dll.  Fine and dandy.

But IM shared libs "know" about their modules directories; they load 
their modules -- which can be "dropped in" at any time of course -- 
specifically from the **compiled in** path: in this case, 
/usr/lib/ImageMagick-5.5.7/*.  (let's call this "MODULEDIR")  Now, if 
the files in MODULEDIR are part of the 'ImageMagick' package, then when 
I upgrade to the ImageMagick-5.9.2-1 package, it will replace 
(uninstall) my old IM-5.5.7 package.  So

   /usr/lib/ImageMagick-5.5.7/* goes away, to be replaced by
   /usr/lib/ImageMagick-5.9.2/*

But now my poor cygMagick-6.dll (which is still needed by the FOO 
package, remember) can't find its modules (and it can't use the 5.9.2 
modules, of course).

Okay, so it seems that the files in MODULEDIR need to follow the same 
scheme as the DLL's: keep 'em around in a separate package, using the 
DLLVER numbering scheme:

  libMagick6      -- contains the (main) DLLs
  ImageMagick6    -- contains the modules (also dlls)
  libMagick-devel -- implibs, statlibs, headers, man
  ImageMagick     -- progs, man, docs

This way, 5.9.2 can provide ImageMagick7 and libMagick7, which can 
coexist with ImageMagick6 and libMagick6 -- so FOO still works.  (of 
course, I replace my 5.5.7 -devel and (main) package with the 5.9.2 
version, so that I get to use the latest-n-greatest IM applications)

But now, there are possibly two other problems: (MARKER ONE), suppose 
that instead of 5.9.2 being released, we release 5.5.8 -- which does NOT 
change the main DLL's API.  So, the DLLVER stays at 6, so we have

  libMagick6-5.5.8-1 contains a new version of cygMagick-6.dll et.al.
  ImageMagick6-5.5.8-1 now contains /usr/lib/ImageMagick-5.5.8/* (and 
because the package name is the same, this replaces -- e.g. uninstalls 
-- my old ImageMagick6-5.5.7-1 package, so /usr/lib/ImageMagick-5.5.7/* 
is gone.

But this is probably okay, because FOO was dynamically linked against 
cygMagick-6.dll, so it'll dynamically use the NEW cygMagick-6.dll, which 
knows to look in /usr/lib/ImageMagick-5.5.8, not -5.5.7.

(I'm assuming that if FOO was statically linked against libMagick.a, it 
would then ALSO statically link against all of the modules, and use 
ltdl's fake "dlopen-from-a-statlib" stuff to access module 
functionality.  If that is NOT the case [e.g. if you could link 
statically against the old libMagick.a, but still access the DLLs of the 
modules from /usr/lib/ImageMagick-5.5.7/ using "real" dlopen() -- which 
is, I believe, the case] then we've got another big problem, because 
/usr/lib/ImageMagick-5.5.7 is gone and with it, those module dlls -- but 
that problem is also solved by my "solution" below.)

(MARKER TWO) Now, suppose that after the cygwin version of IM-5.5.7 hits 
the streets, something happens which changes the API -- but only on 
cygwin, and BEFORE the IM-5.5.8 release.  Say, for instance, we have to 
recompile all of the packages to enable 64bit file access with 
cygwin-1.5.0 -- which thus breaks backwards compatibility with 
cygwin-1.3.22-based stuff.  Yes, true story.  All the DLL version 
numbers get bumped, but the "5.5.7" part stays the same.

Oops.  Now we have

  libMagick6 (from 5.5.7-1)
  ImageMagick6 (from 5.5.7-1, contains /usr/lib/ImageMagick-5.5.7/*)
  libMagick7 (from 5.5.7-2, post cygwin-specific API change)
  ImageMagick7 (from 5.5.7-2, contains /usr/lib/ImageMagick-5.5.7/*)

Blammo!  ImageMagick6 and ImageMagick7 collide.

-----------------------------------

So here's my solution which solves (almost) all of these problems (but 
has two minor drawbacks of its own).

Version the main DLLs using -release ${MAJ}.${MIN}.${MICRO}

I know it's not libtool-blessed to do that, but the coordination between 
the main dlls, the modules and third-party users of the IM software 
suite -- e.g. package "FOO" or, if you like, GraphicsMagick :-) -- 
causes all kinds of management and upgrade issues as detailed above, at 
least on Cygwin/Windows.  I haven't thought thru the problem on "normal" 
unix so maybe it's not a big deal there; but my gut tells me that it IS 
an issue even on normal unix.

So, you'd end up with
  libMagick5.5.7 package  which contains
      cygMagick-5-5-7.dll
      cygMagick++-5-5-7-dll
      (actually named "libMagick5.5.7-5.5.7-1.tar.bz2, so
       PKG="libMagick5.5.7" VER="5.5.7" and REL="1")
  ImageMagick5.5.7 which contains
      /usr/lib/ImageMagick-5.5.7/*
      (tarball named "ImageMagick5.5.7-5.5.7-1.tar.bz2")
  libMagick-devel
      contains implibs, statlibs, and headers for "the most recent"
      cygwin release of ImageMagick.
      (tarball named "libMagick-devel-5.5.7-1.tar.bz2")
  ImageMagick
      contains the application binaries and man pages for "the most
      recent" cygwin release of ImageMagick.  Each new release would
      require: the libMagick* package and the ImageMagick* package whose
      name equals "my" version.  So the tarball named
      "ImageMagick-5.5.7-1.tar.bz2" would require:
         libMagick5.5.7 (-5.5.7-1.tar.bz2) and
         ImageMagick5.5.7 (-5.5.7-1.tar.bz2)

This does not solve the "MARKER TWO" problem above, but that merely 
means "cygwin cannot change the API in cygwin-specific ways, except when 
coupled with a new upstream release -- even if that upstream release 
does not itself change the API"

The other minor drawback with my solution is that it would lead to users 
-- even users who do not install the FOO package -- having lots and lots 
of cygMagick-X-Y-Z.dll's and /usr/lib/ImageMagick-X.Y.Z/ directories, 
unless they manually, every once in a while, uninstall the old 
"libMagickX.Y.Z" and "ImageMagickX.Y.Z" packages.

So here's my (first) question:

Back in the 5.5.2 days (around early November of 2002) the CVS version 
of IM **did** in fact, use -release to version the main dlls.  That 
changed somewhere along the way to -versioninfo.  However, I can find no 
record of when, why, or who, as the simplesystems CVS is gone and the 
imagemagick.org cvs only has records going back to late March of 2003. 
It appears that all cvs history prior to that is gone.

So: why was this change made, and can it (should it) be reverted back to 
-release?  If not for every platform, what is the advisability of our 
cygwin maintainer making that change for his cygwin releases?  Is there 
a way to do "#if CYGWIN then libMagick_LDFLAGS = -release ..... else 
libMagick_LDFLAGS = -version-info ...." ?

------------------

Two more questions:

2.
ImageMagick-5.5.7/guide/ contains two files whose names differ only in 
case.  This is bad -- one or the other should be renamed:
  montage.tex
  Montage.tex
(you can check for this -- even on windows/cygwin -- using:
  tar tvzf ImageMagick-5.5.7.tar.gz | awk '{print $6}' | \
    grep -v "$/" | sort -f | uniq -i -d
)

------------------

3.
Harold's current packaging script, attempting to follow cygwin's 
convention for the location of text documentation, does this:
  mv /usr/share/ImageMagick-5.5.7/* /usr/share/doc/ImageMagick-5.5.7/
e.g. text documentation goes in a {$PKG}-${VER} directory under 
/usr/share/doc. (**A**)

But the stuff in /usr/share/ImageMagick-5.5.7/ is not ALL text 
documentation, is it?  For instance, .../image/patterns/* are actually 
used your "patterns" module, right?  So moving those would break things? 
But oops.  If the "patterns" module from ImageMagick5.5.7-5.5.7-1 
"knows" to look in /usr/share/ImageMagick-5.5.7/ -- and that tree is 
supplied by the "main" package -- then we're right back where we 
started, with version mismatch if the "main" IM package is upgraded.  IF 
these patterns are in fact used by the patterns module, shouldn't they 
be installed in, say, /usr/share/ImageMagick/image/patterns/ (no 
version) -- since they are just data and are thus strictly compatible 
with ANY version of the "patterns" module...or live in 
/usr/lib/ImageMagick-X.Y.Z/ with the module that uses them? (**B**)

OTOH, the stuff in ..../image/* (excluding patterns/) are used by the 
HTML docu in ..../www ? so THAT would be fair game for Harolds manual, 
cygwin-specific, relocating into /usr/share/doc/ImageMagick-5.5.7/image/ 
and .../www/, correct?  Along with the (actual text documentation and 
index.html) stuff in /usr/share/ImageMagick-5.5.7/ itself? (**A**)

(**A**) these manipulations are post-install phase; they should not 
reflect any desire on our part to change what IM 'make install' does by 
default on ANY other platform, NOR to change what IM 'make install' does 
by default even on OUR platform.

(**B**) This suggestion about the patterns/*.png's is applicable to all 
platforms, and indicates a neccessary change in IM's build/install 
process.  It is not cygwin specific, and the change should be effected 
by modifications to IM's Makefile.am's and configury.

--
Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/