This is the mail archive of the crossgcc@sourceware.org mailing list for the crossgcc project.
See the CrossGCC FAQ for lots more information.
Index Nav: | [Date Index] [Subject Index] [Author Index] [Thread Index] | |
---|---|---|
Message Nav: | [Date Prev] [Date Next] | [Thread Prev] [Thread Next] |
Other format: | [Raw text] |
Hello all! Recently, I've been challenged about the design of crosstool-NG. This post is to present the overall design of crosstool-NG, how and why I came up with it, and to eventually serve as a base for an open discussion on the matter. Hopefully some will jump in to offer their views on the subject, and offer sugestions as what should be done to improve the situation, should the need arise. The mail is structured that way: 1) Genesis of crosstool-NG 2) The way to fullfill the requirements 2.a) Ease maintenance 2.b) Ease configuration of the toolchain 2.c) Support newer versions of components 2.d) Add new features 2.e) Add alternatives where available 3) crosstool-NG installation 3.a) Setting up crosstool-NG: why using a ./configure? 3.b) Installing crosstool-NG: why is it required? 3.c) Runing crosstool-NG: why can't I run make menuconfig? 4) crosstool-NG internals 4.a) Programming languages used in crosstool-NG 4.b) Internal API 5) Conclusion In advance, I do apology for the really, really long post, and for the limited subset of the English language I use. ======================================= 1) Genesis of crosstool-NG First, a little introduction to put things straight. About four years ago, I needed to generate cross-compilers for ARM and MIPS. One of the requirements was to be able to use various versions of the components (gcc, glibc, binutils...), and a second was to be able to switch between glibc and uClibc. Of the different tools I tested, crosstool was the one most closely matching the requirements, so I ended up using that for the following 1.5 years. But crosstool was not easy to configure, and the available versions of the components were most of the time lagging behind. It was glibc-centric, and I had to add uClibc support, which was not accepted mainstream. In the end, maintaining my own tree became problematic, and I decided to give a try at enhancing crosstool with the following main goals in mind, in this approximative order of importance: a- ease overall maintenance b- ease configuration of the toolchain c- support newer versions of components d- add new features e- add alternatives where it was available I mostly saw my changes as a experimental branch of crosstool, which would ultimately pick interesting features as they mature, while dumping the uninteresting ones. So crosstool would be the stable branch, while my work would serve as a kind of testbed. Hence the name: crosstool-NG, "NG" for "Next Generation". Never, at any one time, did I intend this stuff to replace crosstool. What happened is that, around the time I was working on this, Dan KEGEL became less and less responsive, and changes sent to the list (by any one, not just me) took ages to get applied, if they even get applied at all. So that was how crosstool-NG was born to the world... ======================================= 2) The way to fullfill the requirements The first move I made was to first start from scratch. That way, it sounded to me it would be easier to come up with a good layout of things. 2.a) Ease maintenance At the heart of crosstool was a single script. In there was all the build procedures for all the components, from the installing the kernel headers up to building gdb. The first step was to split up this script into smaller ones, each dedicated to building a single component. This way, I hoped that it would be easier to maintain each build procedure on its own. 2.b) Ease configuration of the toolchain In the state, configuring crosstool required editing a file containing shell variables assignements. There was no proper documentation at what variables were used, and no clear explanations about each variables meaning. The need for a proper way to configure a toolchain arose, and I quite instinctively turned to the configuration scheme used by the Linux kernel. This kconfig language is easy to write. The frontends that then present the resulting menuconfig have limitations in some corner cases, but they are maintained by the kernel folks. Again, of with the build scripts, above, I decided to split each components configuration into separate files, with an almost 1-to-1 mapping. Of course, there are configuration sections that do not apply to a specific component, but to the overall toolchain: the place to install it, the target and its options (BE/LE, CPU variants...). And some options tell crosstool-NG how to behave: the place to find source tarballs, log verbosity, and so on... 2.c) Support newer versions of components Adding support for newer versions merely required building an appropriate patchset for that version. As I didn't have time to dig in every individual mailing lists, I shamelessly vampirised patches from different sources: - the original crosstool patchset - the Gentoo patchset - maybe even the LFS and CLFS patchests - the buildroot patchset (later, when integrating uClibc) - others, floating around, or made up by myself Of course I kept appropriate attribution (or at least I tried to, I may have forgotten some, apologies) on every patches I so vampirised. 2.d) Add new features Ah! Big sentence that the one reading "new feature"! So, by new features, I mean, for example, new configuration options for: - NPTL-enabled glibc - using Linux exported headers - ARM EABI - ... That implied adding aconfig kno in a config file, and updating the code in the corresponding build script, to set the appropriate ./configure option(s), to create the appropriate config file... 2.e) Add alternatives where available The most obvious example of an alternative component is the C library to use: glibc vs uClibc. Then came eglibc. It was almost as simple as adding a new config file, a new build script and a choice in the menuconfig. Sometimes, it required tweaking other components so they recognised the new alternative; gcc was one that needed a few tricks and patches to recognise uClibc. But all in all, it went quite smoothly. ======================================= 3) crosstool-NG installation 3.a) Setting up crosstool-NG: why using a ./configure? crosstool-NG itself requires a few tools to be present so it can run. Usualy, packages do that via a script called ./configure. So it goes for crosstool-NG. crosstool-NG, for its own usage, requires the following: - grep that accepts -E - sed that accepts -i and -e - bash-3 (or bash-4) - cut, xargs, install, awk - gcc, to build the kconfig parsers - curl (or wget), tar, gzip, bzip2, patch, to fetch, extract and patch the tarballs - make Alongside those tools, there are a few that are absolutely required to build the components: - bison - flex, bison - makeinfo (yeah, we migh not want the doc, but some components build in inconditionally) - automake - libtool crosstool-NG does not use them, but better check them now, than fail later. Now, once crosstool-NG has been configured, it's time to "build" it. Merely, this is a matter of putting the right paths in a file that the build scripts do include, and to replace the #!@@bash@@ occurences with the bash we found (or were told to use) in a few scripts. 3.b) Installing crosstool-NG: why is it required? OK, now why would it be needed to _install_ crosstool-NG? Why can't it simply run from the source dir? The first answer is: yes, you can! Just pass --local to ./configure and crosstool-NG will be able to run from its source directory. But then, why on earth is it needed to run make install at all? Well, the install step sets the execute bit on the frontend, ct-ng, and I find it more convenient, in the point of view of the prgorammer I am, to have a single code path in the Makefile. That maybe is not what you would expect, but that's the way I did it. I agree that it can seem ugly, but that's a minor issue, IMHO. Now, why is crosstool-NG intended to be installed at all? Think of crosstool-NG as a package like any other: you configure it, you install it, then you use it. For example, would it make sense that you build your own program in the gcc source directory? I don't think you do that, but that you really install gcc in some place, add that to the PATH, and then cd into your programs source dir and run gcc from there. To parallel between crosstool-NG to gcc: gcc: crosstool-NG: vi foo.c ct-ng menuconfig gcc -o foo foo.c ct-ng build ./foo target-tuple-gcc Of course, this is overly simplistic, but you get the idea... 3.c) Runing crosstool-NG: why can't I run make menuconfig? As said below, the frontend, ct-ng, is written as a Makefile script. That does *not* mean that is is a _Makefile_: it is an executable that happens to be written in the same language that Makefiles are. Now, what if it were written in Python? Would you expect to run it with "python ct-ng", or would you simply expect to run it with "ct-ng"? The same applies if it were written as shell script: you don't call shell scripts with "sh my_script", do you?. And worse, what if it were written in C ? Surely you'd expect to run it with "ct-ng", no? Well, the fact that ct-ng is written in the Makefile language might not be the wisest decision I made on crosstool-NG, but that does *not* change the fact that you don't want to run make (or any other interpreter) manually to run it, right? ======================================= 4) crosstool-NG internals In this section, we'll discuss the internals of crosstool-NG. Most notably, we'll speak about the API that acts as a frontier between the various parts of crosstool-NG: the config entries, the main script, the specific build scripts. Note: although probably the most important section, it is also the shortest. I can't and don't want to simply quote what is already available in the docs/ directory in the sources. 4.a) Programming languages used in crosstool-NG The frontend, called ct-ng out of laziness, is written as a GNU Makefile script. Yes. It's a Makefile that can get run. It uses the sha-bang #!/where/ever/is/make -rf The core of crosstool-NG is written as a set of bash-3 shell scripts. I do insist on bash-3, because I use bashisms, and thus the scripts are *not* POSIX compliant. So, they do not use the traditional sha-bang #!/bin/sh, but they do explictly use the sha-bang #!/where/ever/is/bash The config files are written with the kconfig syntax. The parsers were initially copied and updated from the one in toybox, then updated with the one from the Linux kernel 2.6.28. It has had a few changes: - the CONFIG_ prefix has been replaced with CT_ - a leading | in prompts is skipped, and subsequent leading spaces are not trimmed - otherwise-leading spaces in prompts are silently trimmed 4.b) Internal API That API is 2-fold: - variables set by the main script, and available to build scripts - entry-points that build scripts must provide, and called by the main script The file in docs/overview.txt tries to document this API. It is almost complete wrt the "arch" and "kernel" API. Others are inexistant. This is due to a lack of time to properly document the API, because it has been much fluctuating until not so long ago. In this respect, I did not do much better than the original crosstool. :-( I won't discuss it here, you are encouraged to read that file for more on the subject. ======================================= 5) Conclusion So far, I hope that I've explained enough so people can jump in and comment. All critiscism, whether positive or negative, is very welcome. I don't claim to be either omniscient or omnipotent. I will gladly look at your coments! Of course, should you find a point on which you disagree, I'd appreciate an alternate solution to the problem. Again, I do apologise for the long mail. But that seemed to be really needed. Regards, Yann E. MORIN. PS. My partner is currently badly sick, and I need to take care of her. In advance, sorry for the delay that situation may impose to my answers. YEM. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +0/33 662376056 | Software Designer | \ / CAMPAIGN | ___ | | --==< ^_^ >==-- `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | `------------------------------^-------^------------------^--------------------' -- For unsubscribe information see http://sourceware.org/lists.html#faq
Index Nav: | [Date Index] [Subject Index] [Author Index] [Thread Index] | |
---|---|---|
Message Nav: | [Date Prev] [Date Next] | [Thread Prev] [Thread Next] |