More thoughts about ioctl and kernel ABIs in general

[H. Peter Anvin]

On July 11, 2025 7:18:24 AM PDT, Adhemerval Zanella Netto <adhemerval.zanella@linaro.org> wrote:
>
>
>On 10/07/25 19:37, H. Peter Anvin wrote:
>> So I have spent some more thinking about all of this.
>> 
>> First of all, let's face it, in *most* cases the attempts at hyper-abstracting the low-level kernel interfaces in glibc have been less than successful.  There are some definite exceptions, of course, and higher level interfaces have been very successfully implemented.
>> 
>> Notably the powerpc-unique user space emulation of the TCGETS/TCSETS* ioctls -- and only those ioctls -- was a notoriously weird platform dependence.
>> 
>> The termios interfaces are kind of unique in that way, in that they theoretically define a higher-level interface, but that due to namespace overlap with termio the separation was never complete, and so a system wanting to implement termio effectively had to make termios a slightly extended termio interface, using the same low-level constants.
>> 
>> The versioning interface in glibc, however, is an extremely powerful get-out-of-jail card *when* it can be applied, as are the Linux tagging of ioctls by length.
>> 
>> That is reason I really don't like the idea of having ioctl definitions without using the _IO* constants, except of course for the legacy ioctls that don't have them.  They function as the kernel equivalent of glibc symbol versions in that sense.  The _IO* constants being generated from the compile-time size of the structure communicate to the kernel what structure was expected at compile time, and can be further enhanced to enforce that match at compile time so that the wrong information isn't communicated to the kernel. Hard-coding that information bypasses the checking regardless of if it is done by doing something like _IOR(.... long[x]) or by just hard-coding the resulting constant.
>> 
>> I would like to work on remedying this by continue the work already done in the Linux kernel of partitioning out the uapi headers. Linus is still somewhat allergic to the idea, mostly because his idea of how this should work is that libc should include a *snapshot* of the kernel uapi headers with appropriate tweaks for that particular libc (such as changing "struct termios" to "struct __kernel_termios" if desired).
>> 
>> For the Linus model, tools like coccinelle, perl, and git with resolution rules can of course be used without requiring the user to manually make the changes every time. This also avoids the problem of "old kernel headers" at some additional cost of additional dependencies for out-of-tree kernel work defining new interfaces -- but not really any more than for user space libraries.
>> 
>> I am also working on some more tools to transform the uapi headers. In that sense it would be good to know if glibc would prefer the direct or the snapshot model in the long run, because it affects how low-level the tools need to be.
>> 
>> The downside may be that more frequent point releases may be required to update the headers, but that also encourages keeping wrappers for system calls up to date, discouraging the use of syscall(), which is even worse than ioctl() in terms of exposing kernel interfaces to user space.
>> 
>> Regardless of if glibc wants to adopt the imported or snapshotted uapi model, I'm sure there are things we can do on both the glibc and the kernel side to make the situation better, and avoid another 17-year feature enabling gap.
>
>I think we don't have much consensus of which model to follow; although we 
>tend to favor the former (imported) recently. And we already do some header 
>sync on each kernel release, and recently also added some scripts to check 
>if glibc and kernel header are out-of-sync.
>
>The glibc still requires a toolchain with an installed kernel to build, which
>is a model that other libc are not following anymore (bionic uses pre-processed
>header [1], while musl has its own definition which it syncs from time to
>time).
>
>The current model will first try to include the kernel definition and then
>resort the default one if the kernel include is not available; this allows 
>some Linux API to work regardless of the installed kernel version. This also
>works better for APIs where kernel aims to update in future releases, and for
>distros that constantly update the kernel, even for major releases;
>since there is no need to constant update the glibc (which tends to be more
>troublesome).
>
>It also requires less work on glibc, since with an updated kernel users will
>readily get any new definition.  We will still need to eventually sync to
>fix some regression tests; but this is not really an urgent problem.
>
>The snapshotted model will allow to simplify building glibc and get rid of
>the kernel header version check.  I personally prefer this because it also 
>simplify cross-compiling and bootstrapping.  What I would like is to
>have the glibc *build* to not depend neither use any kernel header from
>the installed toolchain (helping on reproducible builds), and having
>the __has_include tricks only on *installed* headers (so user will keep
>using any update definition from system).
>
>I don't expect the snapshot sync to be troublesome, so having someone
>done it though a script run on each kernel release should be doable.
>
>[1] https://android.googlesource.com/platform/bionic/+/master/libc/kernel/
>
>> 
>> If I can be useful in help bridging this gap, I would like to try. This isn't just limited to header files or interface descriptions, but also missing features in the kernel. The lack of adequate randomization of mmaps have already been mentioned, and I flagged quite a while ago to the kernel team that the kernel ought to handle alignment requests for memory mappings; I also believe the mremap() interface was misdesigned
>> 
>> 
>>  I am familiar enough with POSIX (I was on the committee for a while in 1999-2000) and libc (having written klibc) that I at least have a vague notion of the peculiarity of libc in the system.
>

After I spent a good chuck of yesterday hacking on this, I think there is a *lot* to be said for the snapshot model, for the simple reason that it allows for more powerful preprocessing tools – notably Coccinelle, which in turns depends on Ocaml – to be used than one can realistically expect to require as build dependencies, and of course it allows inspection of the resulting output, minimizing the risk of external breakage.

If there is a perception it would be too much churn then it is also, of course, possible to make glibc-kheaders a separate package with more frequent distribution.

I have already written up a set of scripts to automate the creation of a panarchitectural set of kernel uapi headers produced by running headers_install for each architecture and merging identical files into a common directory hierarchy. The cocci and Perl scripts I posted this morning feel like a very reasonable starting point for the requisite header mangling, too, even though I ran into some annoying limitations in Coccinelle which at least based on my limited knowledge of cocci's 
SpML language required me to add some redundant rules, but it is not a huge problem either. 

I will set up an experimental git tree for these scripts later today.