Bug 25214

Summary: ctype.h performance (and potentially behavior) depends on include order
Product: glibc Reporter: Travis Downs <travis.downs>
Component: libcAssignee: Not yet assigned to anyone <unassigned>
Status: UNCONFIRMED ---    
Severity: normal CC: andysem, drepper.fsp, fweimer
Priority: P2 Flags: fweimer: security-
Version: 2.27   
Target Milestone: ---   
Host: Target:
Build: Last reconfirmed:

Description Travis Downs 2019-11-22 01:21:45 UTC
At least one some versions of libc, whether you include <stdlib.> before or after <ctype.h> changes the way the toupper and tolower functions are defined.

See for example here:

https://godbolt.org/z/4a7htx

If you include the <ctype.h> header first, the bodies of extern inline functions are exposed in the header, and are inlined (note the loop contains no toupper call). If you swap the order, and include <stdlib.h> first, then __NO_CTYPE gets defined somewhere in the header chain, and when <ctype.h> is included the inline bodies are not available, and so inlining doesn't happen.

See the block guarded by __NO_CTYPE in ctype.h at:

 https://sourceware.org/git/?p=glibc.git;a=blob;f=ctype/ctype.h;h=d17f727cf0dc2a0f6c62fa50aff799b175dcb426;hb=2a764c6ee848dfe92cb2921ed3b14085f15d9e79#l205 

At least this causes a large performance variation (I measure the non-inlined version as 3 to 4 times slower), but if you pick through that conditionally defined block I bet you can find functional differences too.

I would expect __NO_CTYPE to defined consistently regardless of header include order.
Comment 1 joseph@codesourcery.com 2019-11-22 01:43:40 UTC
Are you compiling as C++?  Some libstdc++ headers define __NO_CTYPE 
(because C++ prohibits headers from defining standard functions as 
macros), but I don't see anything in the glibc headers defining that 
macro.
Comment 2 Travis Downs 2019-11-22 01:46:58 UTC
Yes, I'm compiling as C++, and I should have made that clear from the start.

In C, the flow is totally different and toupper/tolower are macros, not functions (as you pointed out).

Although I'm compiling as C++, the only headers I'm including are <stdlib.h> and <ctype.h>. I'm not sure exactly where __NO_CTYPE is getting defined in this case, but in another case I looked at (this time including C++ headers like <algorithm>, the __NO_CTYPE definition came from <bits/os_defines.h>).
Comment 3 Travis Downs 2019-11-22 01:49:39 UTC
Here's an example that shows that <stdlib.h> defines __NO_CTYPE when compiled in C++:

https://godbolt.org/z/y6Quxs

In C it is not defined.
Comment 4 andysem 2021-04-05 13:18:59 UTC
(In reply to Travis Downs from comment #3)
> Here's an example that shows that <stdlib.h> defines __NO_CTYPE when
> compiled in C++:
> 
> https://godbolt.org/z/y6Quxs
> 
> In C it is not defined.

gcc ships C library header overrides, including stdlib.h, so when you're including stdlib.h from a C++ program you're getting its C++ version. Eventually it includes the C stdlib.h, but before that it may define necessary config macros and do other stuff necessary for compliance with C++.

You can see all includes, with paths, in preprocessed output by adding -P to gcc command line. On my system with gcc 10 your test program shows these includes, in that order, among the first lines of the preprocessed output:

/usr/include/c++/10/stdlib.h
/usr/include/c++/10/cstdlib
/usr/include/x86_64-linux-gnu/c++/10/bits/c++config.h
/usr/include/x86_64-linux-gnu/c++/10/bits/os_defines.h

and os_defines.h defines __NO_CTYPE.

I would think that the macro should be either defined or not in C++ regardless of the header you include first. Otherwise there is potential for ODR violations, which result in nasty bugs. If this isn't the case, you should probably report a libstdc++ bug, describing the sequence of includes and the effects.

Separately, I think it is worth requesting libstdc++ to implement a similar optimization that glibc does so that the locale functions are inlined, when possible. This could even reuse glibc implementation by juggling pragma push_macro/pop_macro (https://gcc.gnu.org/onlinedocs/gcc/Push_002fPop-Macro-Pragmas.html).
Comment 5 andysem 2021-04-05 13:27:38 UTC
Sorry, the command line argument is -E.
Comment 6 Travis Downs 2021-04-17 00:42:27 UTC
Thanks Andrey, I've filed this bug against libstdc++:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100128