Re: [RFC] Splitting kernel headers and deprecating __KERNEL__

[Alexandre Oliva <aoliva at redhat dot com>]

On Nov 29, 2004, Linus Torvalds <torvalds@osdl.org> wrote:

> On Mon, 29 Nov 2004, Alexandre Oliva wrote:

>> - Linux gets to define the ABI between kernel and userland, and
>> userland must duplicate the contents of headers in which the kernel
>> defines the kernel<->userland ABI, tracking changes in them in the
>> hope that nothing falls through the cracks

> This is unquestionably true. The kernel obviously _does_ define the
> ABI, and userland just lives with it.

I don't see it as obvious at all.  The need for an agreement between
two parties on an ABI doesn't imply that one party gets to define it
and the other gets to follow it.  An ABI has to be set, and then
kernel and userland get to follow it, but I don't see why it should be
the kernel that determines it.  It might very well be userland that
sets the ABI, and the kernel is implemented so as to comply.  Stating
it the way you did is what I alluded to as being an arrogant attitude
in my previous e-mail.  It sounds as if the kernel was the center of
the universe, and everything else gravitated around it.

Just like userland needs a kernel to operate, a kernel without
any userland programs to run is pretty much useless.  Yes, I know one
can point out examples of exceptions to both statements, but those are
just that, exceptions.

> At some point you have to track things, just because new features
> etc just can't be sanely handled any other way.

This applies to both sides of the bridge.

>> - we move the kernel<->userland to a separate package, where it's
>> maintained such that it can be used by both kernel and userland, and
>> Linux will only build when given a pointer to the location of this
>> package.

> No. Quite frankly, I absolutely hate projects that do that.

I'm glad to see we agree on this.  I happen to find the current
proposition of duplicating contents of kernel headers in some other
package just because `kernel headers are just that' just as
disgusting.  If you want to get to set the ABI, which you appear to
take as a given (and, in spite of my argumentation above, I'm fine
with that), you should do so in a way in which others can sanely
comply with the ABI.  Duplicating header contents is not a sane way to
do it.  Just consider that the problem of tracking the dependencies
needed to build Linux, should the ukabi headers be moved out of it,
affects any project that purports to maintain this set of headers for
userland use outside the kernel, just in a more subtle way: you need
not only to make sure that you have the package that offers the
interfaces you want to use, but also you have to make sure you have a
specific version of that package that matches the kernel baseline your
userland is intended to work with.

Although I can see that, for established ports, the rate of change in
the ukabi is quite small (addition of syscalls, mostly), during the
development of a new port, the rate of change is very high, and that's
the point in which you need the most to make sure that the ABIs are in
sync, and maintaining them separately is exactly the right way to make
sure they aren't.

> Now, on trying to make the tracking _easier_, I would not mind at all to 
> move (well-defined) things around a bit to make it clearer what is 
> actually exported to user space. But on the other hand, I don't think it's 
> actively wrong either to just "mark" them in the headers some way, and 
> have tools to extract it automatically _without_ having to separate them 
> into some magic location.

As long as the marks are not ones that actually require C
preprocessing, which would render them useless to extract bits in a
form that makes them usable for a multi-ABI userland targets, I happen
to like this better as well.  #ifdef __KERNEL__ can be easily
machine-extracted (I know dwmw2 has a script for that, dunno whether
he wrote it himself or got it from someone else), and the extraction
of bits for userland could even be done as part of say a make
install-ukabi-headers.

There are problems, though: sometimes, someone will make a change such
as adding one #include to a header file that is needed because of
kernel-only bits, and if this include is nor protected by the same
guards and a possibly empty header file with that name is not
installed as part of the ukabi, userland builds fail.

So, again, we agree that the marking should be more explicit than
implicit export.  The advantage of __KERNEL__, however, is that it
makes the very same header files (presumably) usable for userland and
the kernel, which could save some space and speed up the installation
of such headers.  I'm not sure this advantage is worth the trouble.


Another important problem about sharing headers between kernel and
userland, that I don't recall anyone having brought up before, is that
of licensing.  If the kernel headers are licensed under the GNU GPL,
and say a libc that is GNU LGPL in the source form uses them, it might
inherit the full GPL protection and start imposing the GPL
requirements on binaries linked with it.  Since the intent of the
ukabi headers is to set the interface between kernel and userland and
AFAIK you copyright protection doesn't apply to interfaces (IANAL so
this is probably quite inaccurate), one might get away with that, but
since there's often more than just interfaces in headers that are used
by both kernel and userland, the argument might fail.  Some (myself
included) would probably consider this trap against proprietary
software a good thing, but others might consider it a huge problem.

If not for any other reason, at least for the sake of openness, it
would probably be a good thing to separate the ukabi contents from
headers that define other kernel-internal macros, inline functions,
data structures, etc, and perhaps use different licensing terms in
these split-out headers, explaining to what extent using them limits
the licensing choices available to programs that include them.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}