This is the mail archive of the
mailing list for the Cygwin project.
Re: [CFT] libtool on nix->cygwin cross, with wine
What's the [CFT] stand for?
Call For Test?
Are you requesting test volunteers?
On Tue, 24 Feb 2009 00:01:19 -0500
Charles Wilson wrote:
> The most recent release of libtool (2.2.7a-1 for cygwin-1.5, and
> 2.2.7a-10 for cygwin-1.7) ought to support cross builds at least as well
> as libtool-1.5 did. Note that in *ordinary* cross builds (SomeBUILD ->
> SomeHOST) you can't run the $host executables on the $build machine --
> but you can still *build* them. That kind of thing has (hopefully)
> always worked, and still works, for ->cygwin crosses. However, under
> certain conditions, it USED to be possible to run the $host (cygwin)
> executables on the $build machine, provided $build's CPU was x86, and
> $build had wine installed, and $build was running linux with the
> 'binfmt' kernel extension, and there was a cygwin "installation" in the
> wine "arena", etc, etc.
> /That/ has been broken for about a year, due to a change in how
> "wrapper" executables are handled by libtool in libtool-2.2.2 and above.
> ============ 1 ==============
> See, in the ANCIENT days, there was a wrapper script. When you built an
> executable for $host=cygwin, libtool would create
> myprog (a wrapper script)
> .libs/myprog.exe (the actual exe)
> The wrapper script would set $PATH and various other environment
> variables so that the EXE would be able to "find" its (as yet
> uninstalled) DLLs, and then it would launch the EXE. Obviously, scripts
> are not bound to any specific platform, so $build has no problem running
> the script. So as long as $build could execute EXE (via wine, etc), the
> the wrapper/EXE combo worked fine.
> Why'd we change it? Well, there IS one little problem with that scheme:
> the Makefile rule for building the EXE looks like this:
> ./my_prog$EXEEXT: my_prog.o <stuff>
> ... some libtool commands ...
> But libtool doesn't create ./my_prog$EXEEXT -- it creates a wrapper
> named ./my_prog with NO $EXEEXT. So make always believes that the
> executable must be rebuilt. 'make' -> go link the exe. 'make check' ->
> 'go link the exe again'. 'make install' -> go link the exe. Very annoying.
> ============ 2 ==============
> So, in the OLD days, we had a wrapper executable AND a wrapper script:
> my_prog.exe (not the real exe; just a wrapper exe)
> my_prog (the wrapper script)
> .libs/my_prog.exe (the real exe)
> The way this scheme worked was: (a) the wrapper exe would launch the
> wrapper script, (b) the wrapper script would set $PATH and such, and
> then (c) launch the real exe. This worked pretty well -- and was fairly
> transparent to cross builders: they'd try to run 'my_prog' (not
> my_prog.exe, because who types "blahblah.EXE" on unix?). But, my_prog is
> a shell script, and it Just Works(tm) like it did in the ANCIENT days.
> So cross-builders were happy -- they just ignored that wrapper exe (and,
> incidentally, never tested it...)
> So, what was wrong with this? Did we "fix" something that wasn't broken?
> Well, not exactly. About three years ago, cygwin added a new feature:
> you could set the 'transparent_exe' option in the CYGWIN variable. Then,
> you could pretend you were even more "unixy": files which end in .exe to
> be used by appropriate functions when an input filename is specified
> with no extension. That is, you say spawn("foo") and if foo.exe exists,
> then cygwin will turn that in to spawn("foo.exe").
> So...what if I have foo.exe AND foo, and a really really want to
> spawn("foo") -- NOT spawn("foo.exe"). Say, for instance:
> Uh...don't do that.
> See, transparent_exe caused all KINDS of pain for libtool -- cygwin (and
> libtool) got really confused by the situation. However, as long as
> transparent_exe was just an option, we were ok though: you just had to
> follow some rules:
> 1) if transparent_exe, then do not put files that differ only in
> .exe-extension in the same directory
> 2) since libtool does this, do not use libtool and transparent_exe
> Not the greatest situation, but we lived with it.
> But then, along comes cygwin-1.7...in which 'transparent_exe' will be
> the default behavior. Oops.
> "Don't use libtool and cygwin-1.7" is not a rule we can live with.
> So, (and this is the part that broke the cross-builders), we changed
> libtool's behavior...
> ============ 3 ==============
> About two years ago, for $host cygwin/mingw, libtool was modified to no
> longer put a wrapper *script* in the build dir. Instead, it put an
> uglify-named version of it in .libs:
> my_prog.exe (not the real exe; just a wrapper exe)
> .libs/my_prog_ltshwrapper (the wrapper script #1)
> .libs/my_prog_ltshwrapperTMP (the wrapper script #2)
> .libs/my_prog.exe (the real exe)
> That way, no more 'transparent_exe' clashes. BUT, now the poor
> cross-builders have no wrapper script to execute. They have to find the
> uglified version in .libs, OR run the wrapper executable (which was
> borked anyway in cross-build situations. See below).
> Now, why two copies of the wrapper script? Well, libtool actually uses
> the wrapper script for two different purposes. #1, it "saves"
> information between different executions of libtool -- say, from the
> link phase to the install phase. So, sometimes libtool actually
> 'sources' the wrapper script back in, to read certain variable values.
> That's copy #1. Copy #2 is ever-so-slightly different...
> The way this scheme worked was: (a) the wrapper exe would emit the #2
> copy of wrapper script in .libs/* ON THE FLY, (b) and execute that
> wrapper script. Then, the wrapper script would set $PATH, etc, and
> launch the real exe.
> That way, the #2 copy is ALWAYS up-to-date with the wrapper exe -- and,
> because it is generated by the wrapper exe, it can be modified slightly
> from copy #1 (which is used for communicating variable values from
> different executions of libtool).
> This mostly worked, but was a bit awkward on cross-compile setups. For
> mingw, say, you had to set the environment variable TARGETSHELL to the
> *dos* path to your *unix $build* shell, so that via the magic of wine,
> CreateProcess("C:/some/magic/path/sh /path/to/wrapper/script args"...)
> would cause the $build machine's /bin/sh to run the desired script.
> Also, there were problems with atomicity: if you're building with make
> -jN, you might launch the same wrapper exe multiple times
> simultaneously. They all clobber each other's "temporary" wrapper scripts.
> ============ 4 ==============
> So, the obvious solution is to have the wrapper executable itself do all
> the work of setting $PATH, and then launching the real executable.
> That's what happened in libtool-2.2.2 (about a year ago). But, the
> initial implementation worked great for native builds (cygwin, mingw) --
> but completely "broke" cross-builds. That is, you could still *build*
> stuff in a nix->cygwin or nix->mingw environment, but the wrapper system
> was completely fubared if you wanted to actually *run* those programs
> via their wrappers, in a cross environment. Even if you had wine installed.
> The problem boils down to this (it's easier to explain by talking about
> unix->mingw crosses, but similar arguments apply for unix->cygwin):
> libtool itself is executing in $build (unix). It knows about
> $build-style paths. But the wrapper executable is a mingw executable,
> and will use the win32 _spawnv("C:\I\Want\A\DOS\Path") function to
> launch the real executable. libtool knows the unix path to the real
> executable -- when creating the "C" source code for the wrapper
> executable, libtool needs to convert the unix-style (that is, $build)
> path into the DOS-style (that is, $host) format.
> So, recently (libtool-2.2.4ish) libtool learned how to do that -- for
> unix->mingw. It doesn't yet -- officially -- know how to do that for
> unix->cygwin, because that is more complicated:
> 1) you have to use winepath to convert the $build (unix) path to "win32"
> 2) THEN, you have to use cygpath -- running under wine -- to convert
> that "win32" path to cygwin format "inside" wine.
> It gets even more complicated if your wine "arena" has, for instance,
> TWO different cygwin installations: a cygwin-1.5 one and a cygwin-1.7
> one. Which cygpath do you use -- the cygwin-1.5 cygpath which use the
> "registry" in wine to find the cygwin mount table, and will convert the
> "win32" path using that mount mapping? Or the cygwin-1.7 cygpath, which
> will find itself, locate its ../etc/fstab file, and use THAT for
> converting the "win32" path to cygwin format.
> ======== CALL FOR TEST ========
> This recent cygwin release of libtool has a mechanism for supporting
> these sorts of cross builds. BUT: you have to set an environment
> variable in $build (that is, your unix environment) to tell libtool
> which cygpath program you want to use:
> export LT_CYGPATH=/unix/path/to/cygpath
> Something like:
> export LT_CYGPATH=/opt/winestuff/cygwin-1.7/bin/cygpath.exe
> You also have to be running linux -- not any other unix -- with the
> binfmt kernel extension turned on. It usually is, so you probably don't
> need to worry about that one. (I don't THINK other unices support
> anything like binfmt...)
> Then, you can try to autoreconf (to get the new libtool stuff) and build
> your favorite package using your unix->cygwin cross tools.
> Finally: try to actually *execute* a wrapper executable -- and see if it
> I'm not set up to test that. If you are, please try to exercise this
> new capability and let me know if it works. The easiest way to do so is:
> 1) download libtool-2.2.7a-20090223.tar.lzma from here
> (you COULD get libtool-2.2.7a-1-src.tar.bz2 from the cygwin mirrors,
> BUT: it's a git checkout, with patches, so it's a bit of work. You have
> to apply the patches in order, then run the bootstrap script. The
> .tar.lzma file above has already had all that done)
> 2) unpack, and create separate build directory
> $ ls
> 3) export LT_CYGPATH=/unix/path/to/cygpath.exe
> 4) cd build/
> 5) ../libtool-2.2.7a-20090223/configure \
> --srcdir=../libtool-2.2.7a-20090223 \
> 6) make
> Assuming that all goes well, then try:
> 7) make check TESTSUITEFLAGS="-V" TESTS='tests/mdemo-shared.test
> tests/mdemo-make.test tests/mdemo-exec.test' VERBOSE=t
> and see what happens. If it works, GREAT. Tell me.
> If not, then the interesting things to look at:
> a) did tests/mdemo-shared.test pass?
> b) did tests/mdemo-make.test pass?
> c) so, if tests/mdemo-exec.test failed, then what does
> build/tests/mdemo/.libs/lt-mdemo.c look like?
> are the following "C" variables correct:
> const char * LIB_PATH_VALUE
> const char * EXE_PATH_VALUE
> d) try rebuilding the wrapper with debugging:
> i) cd build/tests/mdemo
> ii) rm -f mdemo.exe # the wrapper
> iii) make CFLAGS="-DLT_DEBUGWRAPPER" mdemo.exe
> then run the wrapper manually, and see what it prints out.
> Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
> Problem reports: http://cygwin.com/problems.html
> Documentation: http://cygwin.com/docs.html
> FAQ: http://cygwin.com/faq/
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html