[crosstool-NG] Design discussion

[Yann E. MORIN]

Rob,
All,

On Monday 06 April 2009 07:45:49 Rob Landley wrote:
> Ok, I think I left off around here:

OK, I think I'm coping with the back-log ;-)

> > 2.b) Ease configuration of the toolchain
> >
> > In the state, configuring crosstool required editing a file containing
> > shell variables assignements. There was no proper documentation at what
> > variables were used, and no clear explanations about each variables
> > meaning.
> My response to this problem was to write documentation.

Sure. But documentation is not all. Fixing and enhancing both the code
and the configuration scheme is also a way to achieve a better end-user
experience (Ahaha! I've wandered too much with the marketting dept lately).

> While I've used kconfig myself, there's an old saying: "If all you have is a 
> hammer, everything looks like a nail".

Hehe! Show me a better and simpler way.

> The failure mode of kconfig is having so much granularity

So what? Should I restrict what the end-user is allowed to do, based solely
on my own experience? If the possibility exists in the tools, then why
prevent the user from using (or not using) that? Configuring a compiler
has soooo many options, it drives you mad. Quite a lot can be inferred
from higher-level options, such as the arch name and the CPU variant, or
the ABI...

> Ironically, kconfig is only really worth using when you have enough config 
> options to bother with it.

But there *are* a lot of options! There isn't even all that could be
available!

> When you have small numbers of config options  
> that are usually going to be off, I prefer environment variables (with a 
> config file in which you can set those in a persistent manner) or command 
> line options.  Since you can set an environment variable on the command line, 
> ala:
> 
>   FORK=1 ./buildall.sh
> 
> I lean towards those.  Possibly a matter of personal taste...

I think so. To configure stuff, I prefer having a GUI that is not vi, but
that is still simple. and the kconfig language and its mconf interpreter
seems quite fitted, even if it is not the best. But here is no alternative
that I'm aware of. Plus, it is quite well known thanks to the Linux kernel
using it.

> Getting back to menuconfig, since it _is_ so central to your design, let's 
> look at the menuconfig entries.  I still have 1.3.2 installed here, which 
> starts with nine sub-menus, let's go into the first, "paths and misc 
> options":
> 
> The first three options in the first menu aren't immediately useful to a 
> newbie like me:
> 
>   [ ] Use obsolete features
>   [ ] Try features marked as EXPERIMENTAL (NEW)
>   [ ] Debug crosstool-NG (NEW)
> I dunno what your obsolete versions are, I don't know what your experimental 
> options are, and I dunno what debugging crosstool-ng does.  I am not 
> currently qualified to make any decisions about them, because I don't know 
> what they actually control.
> Looking at the help... the "obsolete features" thing seems useless?  We've 
> already got menus to select kernel and gcc versions, this just hides some of 
> those versions?  Why?  (Shouldn't it default to the newest stable version?  
> If it doesn't, shouldn't it be _obvious_ that the newest stable version is 
> probably what you want?)

OK, obsolete means that I can't afford ensuring that they still build.

> Marking old versions "deprecated" might make a certain mount of sense.  
> marking them obsolete and hiding them, but still having them available, less 
> so.

OK, deprecated is much meaningful, I admit to it. Let's set for
s/OBSOLETE/DEPRECATED/, then.

> Similarly, the "experimental" one seems useless because when you enable it the 
> experimental versions already say "EXPERIMENTAL" in their descriptions 
> (wandered around until I found the binutils version choice menu and looked at 
> it to be sure).  They're marked anyway, so why is an option to hide them an 
> improvement?

EXPERIMENTAL in the prompt is just a string. A config knob makes the user
really aware that he/she's trying something that might break.

Plus, it marks the resulting config file as containing EXPERIMENTAL
features/versions/... and is easier to process.

> As for the third, wasn't there a debug menu?  Why is "Debug crostool-NG" in 
> the paths menu?  (Rummage, rummage... Ah, I see, the debug menu is a list of 
> packages you might want to build and add to the toolchain.  Ok, sort of makes 
> sense.  Still, the third thing a newbie sees going through in order as 
> a "very very expert" option.  Moving on...)

That's why the former in titled "Debug crosstol-NG", while the latter
is titled "debug facilities". Again, maybe the wording is wrong.

>   ()  Local tarballs directory (NEW)
>   (${CT_TOP_DIR}/targets) Working directory (NEW)
>   (${HOME}/x-tools/${CT_TARGET}) Prefix directory (NEW)
> 
> Most users aren't going to care where the local tarballs directory is, or the 
> working directory.

Most. Not all. And the help entries are here to tel the user whether
it wise to change.

> The "prefix directory" is presumably different from where  
> we just installed with --prefix.

The help roughly says: the path were the toolchain is expected to run from.
Unfortunately, there is yet no support for DESTDIR, the place where the
toolchain will be installed, to allow installing out-of-tree. For the time
being, the DESTDIR is plainly / that is, the toolchain is expected to run
on the system it is built on. But that should eventually be fixed.

> I suppose it's nice that you can override  
> the defaults, but having it be one of the first questions a user's posed with 
> when going through the options in order trying to configure the thing isn't 
> really very helpful.  It's not my problem, just _work_.

Where should I install the toolchain? In the user's home directory?
This is indeed the default, but you are pestering against it!

If not in ${HOME}, where should I install the toolchain? In /opt ?
In /usr/local ? Bah, most useres don't have right access there.
Except root. But building as root is asking for problems!

> (I also don't know  
> what CT_TOP_DIR and CT_TARGET are, I'd have to go look them up.)

docs/overview.txt is advertised in the top-level README.

> For comparison, my system creates a tarball from the resulting cross compiler, 
> and leaves an extracted copy as "build/cross-compiler-$ARCH".  You can put 
> them wherever you like, it's not my problem.  They're fully relocatable.

Toolchains built with crosstool-NG are also fully relocatable.
Having the user tell before hand where to install the stuff is also
another good option.

>   [*] Remove documentation (NEW)
> Nice, and possibly the first question someone who _isn't_ a cross compiler 
> toolchain developer (but just wants to build and use the thing) might 
> actually be interested in.

:-)

> Your ./configure still requires you to install makeinfo no matter what this is 
> set to.  You have to install the package so this can delete its output?

Unfortunately, gcc/glibc/... build and install their documentation by
default. I haven't seen any ./configure option that would prevent them
from doing so... :-(

> Wouldn't it be better to group this with a "strip the resulting binaries" 
> option, and any other space saving switches?  (I'm just assuming you _have_ 
> them, somewhere...)

Nope. But that's a good idea. :-)

>   [*] Render the toolchain read-only (NEW)
> This is something the end user can do fairly easily for themselves, and I'm 
> not quite sure what the advantage of doing it is supposed to be anyway.  In 
> any case it's an install option, and should probably go with other install 
> options, but I personally wouldn't have bothered having this option at all.

In fact, it's here and ON by default, and why this is so is clearly explained
both in docs/overview.txt and in the help of this option. Well, that might not
be so obvious, after all. :-(

>   [ ] Force downloads (NEW)
> I noticed your build doesn't detect whether or not the tarballs downloaded 
> properly.
> Forcing re-downloads every build puts unnecessary strain on the mirrors, and 
> seems a bit impolite.  (Plus your re-download can time out halfway through if 
> the net hiccups.)  But the alternative you've got is your infrastructure 
> won't notice corrupted tarballs other than by dying.

Yeah. Sad. Will work on it. until now, it was a minor problem, as there were
more important ones. Now that most of the stuff is functional, it's time to
polish things...

>   [ ] Use a proxy (NEW)  --->
> Wow, are these still used in 2009?  Ok?  (It just never came up for me...)

Yes! Big and not-so-big companies have proxies to connect you to the internet
and they use that as a filter against, well, preventing you from doing pr0n at
work, or going on other unlawful sites, such as hacking and stuff...

>   [ ] Use LAN mirror (NEW)  --->
> In the sub-menu this options, why do you have individual selections instead of 
> just having 'em provide a URL prefix pointing to the directory in which to 
> find the packages in question?  You already know the name of each package 
> you're looking for...

Hmmm... There must have been a good idea behind that... Can't think of it
any more... :-(

>   (10) connection timeout (NEW)
> This is an implementation detail.  Users should hardly ever care.

No. I have a case were the network is sooo slow that connections are
established well after the 10s default timeout (17s if I remember well).

> My system uses wget instead of curl (because wget is in busybox and curl 
> isn't).

If you don't have curl, crosstool-NG falls back to using wget. That's just
a matter of taste, here. And no so many people are using busybox-based
workstations. ;-)

> As a higher level design issue, It would have been easier for me to implement 
> my build system in python than in bash, but the point of doing it in bash is 
> it's the exact same set of commands you'd run on the command line, in the 
> order you'd run them, to do this yourself by hand.  So to an extent the shell 
> scripts act as documentation and a tutorial on how to build cross compilers.  
> (And I added a lot of #comments to help out there, because I _expect_ people 
> to read the scripts if they care about much more than just grabbing prebuilt 
> binary tarballs and using them to cross compile stuff.)

That paragraph also applies to crosstool-NG. Word for word, except for
the python stuff, that I don't grok.

>   [ ] Stop after downloading tarballs (NEW)
> This seems like it should be a command line option.

Granted, same answer as for "Force downloads"

>   [ ] Force extractions (NEW)
> Ah, you cache the results of tarball extraction too.  I hadn't noticed.  (I 
> hadn't bothered to mention that mine's doing it because it's just an 
> implementation detail.)
> 
> This is one of the things my setupfor function does: it extracts source into 
> build/sources, in a subdirectory with the same name as the package.

Unfortunately not all package are good boys. Some have a hyphen in the
package name and a dash in the corresponding directory.

> Again, I detect "good/stale" cached data via sha1sums.

I'm missing this. It's on my TODO as well, but low priority...

>   [*] Override config.{guess,sub} (NEW)
> I can sort of see this, but it's one of those "you really, really, really need 
> to know what you're doing, and you might be better off patching or upgrading 
> the package in question instead".

Yep. Some packages still don't know about *-*-linux-ublibc* tuples...
Sigh...

>   [ ] Stop after extracting tarballs (NEW)

Ditto as for "Stop after downloading tarballs".

>   (1) Number of parallel jobs (NEW)
> My sources/includes.sh autodetects the number of processors and sets CPUS.  
> You can override it by setting CPUS on the command line.  (I often 
> do "CPUS=1 ./build.sh x86_64" when something breaks so I get more 
> understandable error messages.)
> 
> In general, I try never to ask the user for information I can autodetect sane 
> defaults for, I just let them override the defaults if they want to.

At work, we have a build farm whose purpose is to build the firmwares for our
targets. The machines are quad-CPUs. Deploying a new toolchain needs not be
done in the snap of fingers, and building the firmwares have the priority.
So I use that to restrain the number of jobs to run in // .

>   (0) Maximum allowed load (NEW)
> Ooh, that's nice, and something mine doesn't have.

Yeah! One good point! :-)

> Personally I've never had  
> a clear enough idea of what loadavg's units were to figure out how it equated 
> to slowing down my desktop, and I've actually found that my laptop's 
> interactivity going down the drain is almost never due to loadavg, it's due 
> to running out of memory and the thing going swap happy with the disk pegged 
> as constantaly active.  (The CPU scheduler is way the heck better than the 
> I/O scheduler, and virtual memory is conceptually horrible and quite possibly 
> _never_ going to be properly fixed at the theoretical level.  You have to 
> accurately predict the future in order to do it right, that's slightly 
> _worse_ than solving the halting problem...)
> 
>   (0) Nice level (NEW)
> Again, trivial to do from the command line:

I already have # of // jobs, and loadavg. Why not having nice as well?...

>   [*] Use -pipe (NEW)
> Why would you ever bother the user with this?  It's a gcc implementation 
> detail, and these days with a modern 2.6 kernel dentry and page caches you 
> probably can't even tell the difference in benchmarks because the data never 
> actually hits the disk anyway.
> Have you actually benchmarked the difference?

10% gain on my machine (Dual AMD64, 1GiB RAM) at the time of testing.
Much less now I have a Quad-core with 4GiB RAM.

>   [ ] Use 'ash' as CONFIG_SHELL (NEW)
> A) I haven't got /bin/ash installed.  Presumablly you need to install it since 
> the help says it's calling it from an absolute path?

crosstool-NG does not build and install it. If the user wants that, he/she's
responsible for installing it, yes. Maybe I should build my own...

> B) If your scripts are so slow that you need a faster shell to run them, 
> possibly the problem is with the scripts rather than with the shell?

My scripts are not so slow. ./configure scripts and Makefiles are.
Again, using dash, the build went 10%/15% faster on my quad-core.

> I admit that one of the potential weaknesses of my current system is that it 
> calls #!/bin/bash instead of #!/bin/sh.  I agonized over that one a bit.  But 
> I stayed with bash because A) dash is seriously broken, B) bash has been the 
> default shell of Linux since before the 0.0.1 release.

I do explicitly call bash as well, because I use bashisms in my scripts.
./configure and Makefiles should be POSIX compliant. I am not.

>   Maximum log level to see: (INFO)  --->
> I don't have a decent idea of what you get with each of these.  (Yes, I've 
> read the help.)

OK, that may require a litle bit more explanations in the help.
I don't care about the components build log. I just want to know whether the
build was successful, or failed. In crosstool-NG the messages are sorted
in order of importance:
ERROR   : crosstool-NG detected an error: failed download/extract/patch/...
          incorrect settings, internal error... ERRORs are fatal
WARNING : non-fatal condition that crosstool-NG knows how to work around,
          but if you better give it correct input, rather than letting it
          guess.
INFO    : informs the user of the overall process going on. Very terse.
          Tells the current high-level step being done: doanloading,
          extracting, building a component...
EXTRA   : informs the user with a finer level of what's going on. For
          each steps listed above, prints sub-sequences: package being
          downloaded/exxtracted/patched, sub-step in building a component:
          ./configure-ing, make-ing, installing...
DEBUG   : messages aimed at debugging crosstool-NG's behavior.
ALL     : print everything: ./configure output, make output...

> Oh, I pull one other dirty trick, at the end of setupfor:
>   # Change window title bar to package now
>   echo -en "\033]2;$ARCH_NAME $STAGE_NAME $PACKAGE\007" 
> So you can see in the window title bar what architecture, stage, and package 
> it's currently building.

Hmm... Nice!

>   [ ] Warnings from the tools' builds (NEW)
> Again, filtering the output of the build I leave to the user.  They're better 
> at it,

>From an end-user perspective (yes, I *am* using crosstool-NG ;-) ), I don't
care what commands are being executed to build such or such package, just
that it's doing it, and that it's not stuck.

> and 90% of the time they just want to know that it's still going, or  
> that it succeeded, or what error it died with.

Yep. Exactly.

> But I can only _guess_ what they want, so I don't.

Not wrong per se.

> In general, I try not to  
> assume they're not going to want to do some insane crazy thing I never 
> thought of, because usually I'm the one doing the insane crazy things the 
> people who wrote the stuff I'm using never thought of, so I sympathize.

;-)

>   [*] Progress bar (NEW)
> I have the "dotprogress" function I use for extracting tarballs, prints a 
> period every 25 lines of input.

Mine rotates the bar every tenlines. Which is the better? ;-)
> I used to change the color of the output so you could see at a glance what 
> stage it was, but people complained

Right. crosstool-NG's output used to be colored, but that just plainly sucks.
And, believe it or not, some people are still using non-color-capable
terminals...

>   [*] Log to a file (NEW)
> Again, "./build.sh 2>&1 | tee out.txt".  Pretty much programming 101 these 
> days, if you haven't learned that for building all the other source packages 
> out there, cross compiling probably isn't something you're ready for.

Don't believe that. I have seen many a newbie asked to build embedded stuff
on an exotic board that barely had support for in Linux, said newby just
getting out of school, totaly amazed at the fact that you could actually
run something else than Windows on a PC, let alone that something else
than PCs even existed...

(Well that newbie has grown since, and is now quite a capable Linux guru).

> And I'm at the end of this menu, so I'll pause here for now.  (And you were 
> apologizing for writing a long message... :)

I knew you would surpass me in this respect! ;-P

( Woohoo! I caught up with your mails! :-) )

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +0/33 662376056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| --==< ^_^ >==-- `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
`------------------------------^-------^------------------^--------------------'


--
For unsubscribe information see http://sourceware.org/lists.html#faq