Hi.
As this problem has some historic weight to it, here's a short recap:
Unless compiling for a specific host, which ships it's own
<sys/dirent.h> (which is included by <dirent.h>), newlib's default
behaviour since 2005 has been to simply #error, which causes tests to
work as expected (to fail). Since the very beginning (early 1990s?), it
used to be #include_next, but as <sys/dirent.h> is included by
<dirent.h>, #include_next <dirent.h> at <sys/dirent.h> will include
again <dirent.h>, thus producing no easily detectable error -- fair
enough [1].
At the time the fix was implemented, in order to provide a proper
<dirent.h> the way was to simply provide an include path where the file
resides using e.g. -isystem, so that it appears before newlib's
include/sys. It stills kinda works as of today, but not really, so
here's why I'm reluctant to use it:
Currently, I'm using newlib for a tiny RTOS that runs on a certain ARM
based processors. Under the hood, I've implemented dirent.h
functionality and everything works as expected. To keep the toolchain
side as clean and tidy as possible, I'm pretty much limiting all the
customizations to simply running any configuration scripts with the
options I need. So, e.g. my newlib doesn't really know that it will be
running hosted on an OS that does dirent.h stuff -- and it really
doesn't have to. While configure.host is not that involved, doing a
custom job there would require me to maintain that file and any
corresponding files under libc/machine/. While it would be nice, I
simply don't have time for this -- every moment spent building the
toolchain is taken away from maintaining the actual RTOS and everything
that sits atop of that (it's not like newlib is the only link in the
chain).
Now, the majority of application code which runs on the OS is written
using modern C++ (17 and onwards). For the compiler I'm using GCC, but I
guess any decent compiler would do it. Again, when (cross-)compiling the
GCC I'm limiting myself to any options, that are settable via the
autoconf's configure script because my time is limited and kludges have
tendency to break. Here's where the things have have changed, not
perhaps dramatically, but changed nevertheless since the 2005 patch was
made.
Evolved from Boost.Filesystem, STL has had an (experimental)
std::(experimental::)filesystem since 2015. I would like to use, as I
don't have any necessity to reinvent those features. When using GCC,
this will be compiled as libstdc++fs, as part of the libstdc++. So, when
the cross-compiling GCC is built, the proper dirent.h must be available
as filesystem is not a header-only library. To configure native system
headers for the GCC, as of 2011 --with-native-system-header-dir options
has been available. It sets up a builtin include location, from which
the system headers are supposed to be pulled -- in reality, it is of
course just an additional builtin include path. As an unfortunate
effect, the native-system-header-dir appears after, not before, the
include path set up by --with-sysroot prefix (it will appear as
-isystem, before any CXXFLAGS, set up by autoconf).
So, the default <sys/dirent.h> gets included and the libstdc++fs does
not get built (the presence of <dirent.h> is determined by autoconf).
One could do a dirty copy-over kludge here, but I just don't see that as
a proper thing to do as it'll always pile up. The same goes with simply
removing the <sys/dirent.h> -- while either would would fix the problem
for me, it wouldn't so much do it for the others (e.g. the official ARM
GNU toolchain). Also, hammering the default <sys/dirent.h> with GCC's
fixincludes isn't any better, as it has no external hooks so one would
have to manually maintain yet another configuration file.
I presume the source of this problem, and the reason why the 2005 patch
was written, is the way #include_next works. If you have just one
<dirent.h>, the #include_next will function just like #include. However,
things have evolved and nowadays at least both GCC and LLVM (clang)
support __has_include_next. Unfortunately, only __has_include is part of
the C++17, but I guess this makes an excellent use case for
__has_include_next: i) should you not have any next include file, (a
properly implemented) __has_include_next will return 0, ii) should you
not have __has_include_next functionality, simply define
__has_include_next as zero or check, if it's defined (as compilers
should really define __has_include as __builtin_has_include etc., or
whatever internal name the particular vendor chooses).
So, instead of simply doing an #error, I would very much like to see
something like:
#if defined(__has_include_next) && __has_include_next(<dirent.h>)
# include_next <dirent.h>
#else
# error "<dirent.h> not supported"
#endif
and it would just work (given that #if would have to be likely split
into #ifdef/#if etc.): If there's no additional dirent.h, or your
compiler is of previous generation, you'll get the old error message.
Should you have both, it'll be included next -- just like the original
did (or wished to do) back in the day. This way one could also pull a
standard, uncustomized prebuilt newlib installation and not have to
worry about <dirent.h> include order so much, would you like to
implement one yourself. Like I said, of course `the proper' way to do
this would be customize newlib, but the way configure scripts are
currently implemented it involves so much work that it's not simply
worth it -- patching, copying, checking, working this all up as an yet
another messy script, for every release etc. takes so much time while
you simply wanted `not to fail so early'.
What do you think? Worth doing, at least looking into, or is there
something this sort of approach would hinder? Obviously, this would need
some testing, so that it's not suddenly 2005 all over again.
-- Pekka
[1] https://sourceware.org/ml/newlib/2005/msg00608.html