RFC: cygport cross-compiling APIs
Charles Wilson
cygwin@cwilson.fastmail.fm
Mon Jul 12 18:41:00 GMT 2010
On 7/12/2010 6:19 AM, Yaakov (Cygwin/X) wrote:
> I think I'm getting close to nailing down cross-compiling support in
> cygport.
Great! (Is your current state checked in to git master or some other
branch?)
> Throughout, the prefix=/usr assumption has been removed; *-*-mingw*
> hosts use /mingw, everything else is /usr for now.
Are we sure about this, given mingw64's express desire to avoid this
$prefix -- specifically because "the other mingw" has historically used
it, and "the other mingw" has had its preferences enshrined in the
upstream gcc source code?
Do the guys at #mingw64 have any comment on the "proper" prefix that
cross compilers should use underneath a sysroot?
> (Anyone know any
> specific systems where that's not the case?) Cross-compiled packages
> are properly strip(1)ed with the correct binutils. Libtool fixup puts
> non-native DLLs into $CROSS_HOST/prefix/bin
I ask the following question for clarification only:
So by DLL you mean, explicitly, PE/COFF Dynamic Libraries for win32, and
NOT the generic "dynamically loaded library" category that can, loosely
speaking, include also ELF .so's.
I assume so, because that's the only interpretation that doesn't
conflict with this statement:
> if not already there, and
> doesn't move non-PE libs
Now, for the next bit:
> but still removes static modules.
You mean that if a (shared) library -- whether ".dll" or ".so" -- is
built using libtool's -module flag (e.g. shouldnotlink=yes in the .la
file), then the .a's (and .dll.a's) are removed?
I suppose the "linktime symbolic links" often used with ELF shared
libraries aren't even created by 'libtool --mode=install' for modules,
so that's a non-issue.
I ask the preceding question for clarification only.
> Also,
> package dependencies for mingw* and linux* cross-compiles are supported.
> While I was at it, some initial work was done to make adapting to a
> possible future x86_64-pc-cygwin easier as well.
Neato.
> Here's what the API looks like so far:
>
> toolchain.cygclass: for building binutils/gcc/gdb (cross-)compilers
> * TOOLCHAIN_TARGET: the target triplet (defined in .cygport BEFORE
> "inherit toolchain")
> * toolchain_compile(): runs cygconf with --build/host/target and
> --with-sysroot, followed by cygmake
> * src_compile(): just toolchain_compile; no cygautoreconf (these
> packages don't autoreconf easily)
Sounds good.
> cross.cygclass: for cross-compiling
> * CROSS_HOST: the host triplet (defined in .cygport BEFORE "inherit
> cross")
> * CROSS_SYSROOT: the host sys-root (== /usr/${CROSS_HOST}/sys-root)
> * CC/CXX/F77/FC are $host-prefixed
> * PKG_CONFIG_LIBDIR and PKG_CONFIG_SYSROOT_DIR are set for pkg-config
> * cygconf automatically adds --build/host/target
What about $prefix? Is this handled automatically by cygport (e.g.
always /usr, unless $host=mingw* then /mingw?) and can't be overridden?
I'd really like to be able to override $prefix in general, because it
would make the gcc-tools-* packages a lot easier.
Also, particularly with respect to the mingw* cross compilers, I'm
unsure that all three should all use sys-root/mingw/. Again, what do
the #mingw64 folks recommend in this situation?
> I implemented the cygconf hook instead of a separate cross_compile() so
> that other cygconf-based cygclasses would benefit. For example, a
> mingw64-x86_64-glib2.0 would "inherit cross gnome2", where
> gnome2_compile calls cygconf.
I see.
> Right now my dilemma is with the cross.cygclass install functions. I'm
> not sure whether cyginstall/do*/new* should just flip personality to
> install under $CROSS_SYSROOT (much like cygconf does), or whether to
> implement separate docross{bin,include,lib,...} functions. My reason
> for considering the former is that it would reduce duplicate code and
> avoid easy mistakes (e.g. installing cross-compiled stuff into native
> directories). It would also make it easier to take a native .cygport,
> rename it, define CROSS_HOST and inherit cross, and build.
Another argument is similar to the one above: what about other
.cygclasses that use these functions? If I'm required to use
docrossbin, I can easily do that in my own .cygport -- but not if I
inherit another .cygclass in addition to 'cross', and THAT .cygclass
uses the (original) dobin.
As far as the package .cygport calling dodoc...I think we'll have to
take that as it goes. The NOCROSS idea below might help here.
> OTOH I'm not
> sure whether the more manually-controlled dodir, exe/insinto, doexe/ins,
> and newexe/ins should also imply $CROSS_SYSROOT or not:
Well, you know me: I'm always in favor of giving the end user of cygport
more power, and less authoritarian control by cygport itself.
Boat-on-water with channel markers, rather than car-on-road with
concrete guardrails.
However, in THIS case...see below.
> IOW, is there
> any case where a cross-compiled package would legitimately install
> anything outside of $CROSS_SYSROOT (besides the standard docs). I also
> wonder if that would itself be confusing.
Well, there's the mingw-runtime stuff, which is still used/needed by
gcc3's -mno-cygwin. Plus there's the particular version of w32api that
is used by cygwin's own gcc (and gcc3's -mno-cygwin) and is installed
into /usr/include/w32api + /usr/lib/w32api.
I assume this version would NOT be compiled using the i686-pc-mingw32
toolchain, but instead by the cygwin one (e.g. its cygport -- if it is
actually built USING cygport -- wouldn't inherit cross), so that's
probably not an issue.
Other than that, the only "mingw-ish" stuff we actually ship are, IIRC:
setup.exe
-- special case
the mingw-{zlib,bzip2,xz,libgcrypt,libgpg-error} stuff
-- should be handled by the new cross cygclass with no problems
the tools in the cygwin package, like strace.exe and cygcheck.exe
-- no changes; handled by cygwin gcc with -nostdlib? and not
built using cygport anyway.
So...there aren't that many cross-compiled packages that are part of the
cygwin distribution -- and few would use any additional cygclasses,
other than 'cross', so...manual docross* is not that difficult.
However, cygport is supposed to make "doing it right" easy. So...
I think, on balance, it'd be better to have the existing do* scripts
change their behavior when 'cross' has been inherited, rather than
requiring the use of a separate set of functions.
The best of all worlds would be to have a local override, that can be
set per-command:
inherit cross gtk
src_install()
{
cyginstall
# goes in /usr/$target/sys-root/$prefix/something
dodir something
# goes in /usr/$target/sys-root/$prefix/bin
dobin cross_app1
# goes in /$other_prefix/bin
# e.g. $other_prefix might be /usr, while the
# 'cross.cygclass'-enforced "$prefix" is actually
# /mingw
NOCROSS=$other_prefix dobin special_app
# goes in /usr/$target/sys-root/$prefix/bin
# NOCROSS gets unset by previous command?
dobin cross_app2
}
This scheme can't address the issues related to OTHER inherited
cygclasses which use the do* commands. But then, we probably don't care
about that very much. .cygclasses are supposed to be black boxes that
'just work'(tm); their internal operation shouldn't be second guessed.
> As for the package naming scheme -- which is not being enforced by
> cygport itself -- my thoughts are OS[-CPU]-PACKAGE, IOW:
>
> mingw64-i686-binutils
> mingw64-i686-gcc
> mingw64-i686-pthreads
> mingw64-i686-runtime
> mingw64-i686-zlib
>
> mingw64-x86_64-binutils
> mingw64-x86_64-gcc
> mingw64-x86_64-pthreads
> mingw64-x86_64-runtime
> mingw64-x86_64-zlib
Remember that in mingw64-speak, the "mingw64-crt-headers" is a
completely separate package than "mingw64-crt-lib" -- there's no
"top-level" configure typing them together, AFAIR; so it'd be rather
difficult to munge them together into a single package named
"mingw64-*-runtime".
-headers contains all of the .h's for both what we call "mingw-runtime"
as well as "w32api". -lib contains all of the static, import, and DLLs
for both, as well -- but no headers.
So, rather than mingw64-$CPU-runtime, I'd suggest sticking with
mingw64's own nomenclature, and using:
mingw64-$CPU-crt-headers
mingw64-$CPU-crt-lib
Also, you've made no provision here for allowing to install the runtime
DLLs (from mingw64-$CPU-crt-lib and from mingw64-$CPU-gcc itself) as
separate packages. I know, you say this is a deliberate decision; see
below.
Or, for that matter, splitting mingw64-$CPU-gcc into separate languages.
I *really* don't want to have to download Fortran, ObjC, (and perhaps
Ada and java?) when all I really want is C++.
Finally, you appear to be deliberately precluding any POSSIBILITY of
providing a multilib gcc. I know cygport can't yet support it, but we
might find it necessary or desirable down the road to add that support.
However, this naming scheme won't EVER support that -- unless (e.g.)
this hypothetical, future multilib version of
mingw64-$CPU-crt-lib
ACTUALLY included the $CPU stuff as well as the $OTHER-CPU (or
$OTHER-ABI) stuff in */lib32 or */lib64 (or: */o64 or */n64). Maybe
that's the right thing to do, but somehow it just seems odd.
<aside>
Aside, and thinking-out-loud:
This is another reason for all three of our mingw-ish cross compilers to
use a $prefix OTHER than /mingw underneath sys-root: that way, the
following would -- I think -- work (see HERE:)
[root]
+- usr
|
+- bin - Links to cross compiler toolchain
|
+- i686-pc-mingw32 (mingw32.org stuff)
| |
| +- sys-root - root for cross compiled binaries
| |
| +- mingw32
| |
| +- bin - cross-compiled binaries & runtime DLL
| +- doc - documentation
| +- include - include files for cross compiled libs
| +- lib - cross-compiled static and link libraries
| | |
| | +- pkgconfig
| |
| +- share
| |
| +- man
|
+- i686-w64-mingw32 (mingw64 32-bit stuff)
| |
| +- sys-root - root for cross compiled binaries
| |
| +- mingw64-32
| | |
| | +- bin - cross-compiled binaries & runtime DLL
| | +- doc - documentation
| | +- include - include files for cross compiled libs
| | +- lib - cross-compiled static and link libraries
| | | |
| | | +- pkgconfig
| | |
| | +- share
| | |
| | +- man
| |
HERE: | +- mingw64 -> symlink to
| /usr/x86_64-w64-mingw32/sys-root/mingw64
|
+- x86_64-w64-mingw32 (mingw64 64bit stuff)
|
+- sys-root - root for cross compiled binaries
|
+- mingw64
| |
| +- bin - cross-compiled binaries & runtime DLL
| +- doc - documentation
| +- include - include files for cross compiled libs
| +- lib - cross-compiled static and link libraries
| | |
| | +- pkgconfig
| |
| +- share
| |
| +- man
|
HERE: +- mingw64-32 -> symlink to
/usr/i686-w64-mingw32/sys-root/mingw64-32
Since nobody is actually using the hardcoded /mingw "$prefix" there'd be
no conflict in this scenario between the -m32 and -m64 mode of a
hypothetical future multilib compiler.
I don't know if the $(MULTILIBDIR) stuff would cause any problems in
this scenario; right now, libtool doesn't support that (everything goes
into $prefix/lib, regardless) although there have been a few patches
posted to libtool-patches in the last week.
If libtool were better at MULTILIB, then perhaps the symlinking would
need to occur at a "deeper" level:
+- i686-w64-mingw32 (mingw64 32-bit stuff)
| |
| +- sys-root - root for cross compiled binaries
| |
| +- mingw64-32
| | |
| | +- bin - cross-compiled binaries & runtime DLL
| | |
| | +- bin64 - symlink to
| | /usr/x86_64-w64-mingw32/sys-root/mingw64/bin
| |
| | +- doc - documentation
| | +- include - include files for cross compiled libs
| | +- lib - cross-compiled static and link libraries
| | | |
| | | +- pkgconfig
| | |
| | +- lib64 - symlink to
/usr/x86_64-w64-mingw32/sys-root/mingw64/lib
In *that* case, (a) the "conflicting /mingw $prefix" problem goes away,
so all three cross compilers could use the same sys-root $prefix, but
(b) there are, as you've discovered, additional complications. Like
setting PKGCONFIG_DIR appropriately, and invoking the correct
bin[32|64]/foo-config scripts during configure...These questions all get
more complicated in this "deep linking" scenario.
Maybe THAT is yet another reason to defer multilib until later (if
ever); apparently libtool still needs some help with the concept.
Which is why I say, *defer* this until later -- don't try to solve these
multilib problems right now. But also, don't deliberately preclude
various possible solutions, by package nomenclature whose that isn't
expressive enough, or poorly chosen sys-root $prefix values.
Again, any idea how the #mingw64 guys recommend approaching sys-root
$prefixes, as distinct from the install prefix of native compilers and
compiled libs/apps?
</aside>
That [leave room for multilib in the nomenclature, even if current
toolchains are single-lib] was the thinking behind the original
nomenclature JonY and I were trying to develop, here:
http://cygwin.com/ml/cygwin-apps/2010-06/msg00208.html
Whether this specific "-tc64-/-tc32-" + "-m64-/-m32-" nomenclature is
used, or a some different one sufficiently expressive to allow for
multilib, doesn't matter to me.
I just wonder if we shouldn't leave some place in the nomenclature for
distinguishing between the "normal" $CPU of the toolchain, and all the
stuff related to that, and stuff related to any other -mNN mode (e.g.
"sufficiently expressive") Not just for mingw*, but also perhaps for
other cross toolchains *down the road*, if/when we decide that cygport
and *our packaging/naming policies* should allow multilib.
Say, a cygwin-$hosted solaris-$target cross compiler with 32/64 bit
multilib support. Or a cygwin-$hosted IRIX-$target one with o64 and n64
ABI support.
Now, as I said above, maybe the correct answer is "don't try to split it
up: the multilib "mingw64-crt-lib" should include whatever is necessary
in both sys-root/prefix/lib and sys-root/prefix/lib$MULTI; ditto
sys-root/prefix/bin and ...bin$MULTI, all in one happy package. Ditto
for the other "core" components of a toolchain.
If you do that, then mingw64-$CPU-$PKG would work fine (but might be a
little confusing, since it could contain $OTHER-CPU stuff).
But this is all "cygwin policy" (e.g. "how should I name my packages
that I ITP'd") and not cygport-specific. So package *naming* doesn't
directly impinge on the design or implementation of cygport(1) -- only
on how JonY's ultimate cygport(5) scripts are written.
> linux-i686-binutils
> linux-i686-gcc
> linux-i686-glibc (not built yet; borrowed from Fedora for bootstrapping)
> linux-i686-kernel-headers (ditto)
> linux-i686-zlib
>
> (none of these are built yet)
> mingw-binutils
> mingw-bzip2
> mingw-gcc
> mingw-runtime
Right, well, I'll obviously handle the stuff I currently maintain: the
mingw-{zlib,bzip2,xz,libgcrypt,libgpg-error} stuff. And I assume DaveK
would do the mingw-{binutils,gcc} ones, since he's already volunteered
for that (unless he's changed his mind?).
And Chris S. would probably handle the mingw-runtime [1] (and
mingw-w32api [2], plus (cygwin) w32api [3]) as needed -- since those are
"his" packages right now.
Naturally Chris S. and DaveK would need to coordinate; after they are
done I can do "mine".
[1] where the "new" mingw-runtime (derived from winsup/mingw)
would go in
/usr/i686-pc-mingw32/sys-root/mingw*/{include,lib,bin}
and not
/usr/bin, /usr/include/mingw, /usr/lib/mingw
[2] where mingw-w32api (derived from winsup/w32api) would go in
/usr/i686-pc-mingw32/sys-root/mingw*/{include,lib,runtime}
and not
/usr/include/w32api, /usr/lib/w32api
[3] where this w32api (derived from winsup/w32api) would go in
/usr/include/w32api and /usr/lib/w32api
just like our current version.
> And so on. For simplicity, none of these are multilib, and the
> conflicting binutils/gcc locales/info/etc. are removed, for reasons
> stated in earlier discussions.
Ack. For now, and maybe forever wrt mingw.org $target and mingw64
$target(s). Probably, tho, the no-multilib "policy" will be hard to
maintain if cygwin starts shipping cross compilers targeting other
$hosts like Solaris, IRIX, etc. Not that I have any burning desire to
ship or maintain those myself -- but somebody out there might volunteer
in the future.
I just don't like deliberately hardcoding a possibly-subject-to-change
policy discussion into a difficult-to-reverse nomenclature.
> Note that I'm *not* breaking out separate lib/devel packages, because if
> you think about the reasons for these splits with native packages,
> within the context of cross-compiling they doesn't make sense.
No, I disagree with this. mingw-bzip2 provides, in toto, the DLL,
headers, static/link libs, and utility apps.
Nobody who ONLY wants to build setup.exe using the new cross compiler
cares about the DLLs (because setup.exe uses -static) nor do they care
about the apps. OTOH, those elements ARE part of the cross compiled
"sys-root" that, if someone were to 'repackage' for independent
deployment selected contents of sysroot/$prefix/bin, they would need.
Since we ship a mingw-bzip2 complex of packages, why should (a) the
first group have to download and install everything, if they only need
the headers and static libs but everything were bundled into one
package, or (b) the second group have to recompile mingw-bzip2
themselves if we strip the "unnecessary" parts from a single mingw-bzip2
package?
The same argument holds with regards to the gcc language runtimes.
Maybe somebody is interested not in cross compiling anything themselves,
but using (from a private, or corporate) setup.exe repo some native
win32 tools that are meant to be installed into a cygwin-hosted mingw
sysroot. Shouldn't they be able to get our libgcc_s-1-sjlj.dll without
installing the entire cross compiler?
> Comments welcome, but let's not bikeshed this.
Well, nomenclature and installation locations are probably the ultimate
bikeshed topic; but getting it RIGHT is important, if we don't want to
have to revisit this topic again and again every year or two.
--
Chuck
More information about the Cygwin-apps
mailing list