The GNU C Library Manual - Authoritative or not?

Fri May 22 00:22:21 GMT 2020

On 5/21/20 4:11 PM, Carlos O'Donell wrote:
> On 5/21/20 1:46 PM, Martin Sebor wrote:
>> They do.  Unfortunately, the answers are wrong or at least unhelpful.
> 
> Which answer is wrong? I'd be happy to correct anything that is wrong.

There's a difference between saying "we want the Glibc manual to be
the authoritative reference on the implementation" versus "the manual
is the authoritative reference."

Clearly, on the details I was interested in (the readlink and mbstowcs
examples), the manual is not aa good a reference as the Linux man pages
because it doesn't cover them.  (I would call that authoritative but
I don't insist on using that word.)

> 
> As a contributor to the GNU Toolchain I encourage you to help us add
> to the manual where we think we need to document such interfaces.
> 
> I also suggest a tactical approach, add only the interface you're
> interested in improving.

Thanks, but no.  I wasn't looking for a project, just for an answer
to what I thought was a simple question :)  It would be a considerable
effort to bring the manual up to par with the Linux man pages.  I see
little point in investing it into duplicating what already exists
elsewhere and what according to documentation.html many of you are
already contributing to.  My suggestion instead is to declare
the Linux man pages the reference and treat the manual as a user
guide.

> 
> This is not a duplication of effort IMO. The manual and the linux man
> pages solve different needs. The manual is task-oriented, covering
> sections of the standard APIs and how they could and should be used
> together, while the the linux man pages are API references only
> (in isolation to the larger set of APIs).
> 
> One should be able to `info libc mbstowcs` and get accurate information.
> 
> Given that you raised the issue and I wanted to be helpful, I tried to
> add the docs and a test case (because we have zero test cases for this):
> 
> tst-mbstowcs.c: In function ‘do_test’:
> tst-mbstowcs.c:44:12: error: argument 1 is null but the corresponding size argument 3 value is 24 [-Werror=nonnull]
>     44 |   result = mbstowcs (NULL, string, len);
>        |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
> In file included from ../include/stdlib.h:15,
>                   from tst-mbstowcs.c:19:
> ../stdlib/stdlib.h:933:15: note: in a call to function ‘mbstowcs’ declared with attribute ‘write_only (1, 3)’
>    933 | extern size_t mbstowcs (wchar_t *__restrict  __pwcs,
>        |               ^~~~~~~~
> cc1: all warnings being treated as errors
> 
> Which is expected given the markup. Because of the ability for pwcs to
> be NULL, what can we do here? Remove the markup for the first argument?

Yes, it needs to be removed for now.

A report of the warning above is what prompted my question.  I had
checked the C and in some cases also POSIX standard as well as
the Glibc manual when adding the attribute.  C doesn't have this
extension, Glibc doesn't document it, and I missed it in POSIX (or
more likely didn't think to look there in this case).  What I was
looking for with the question is an acknowledgment of what I had
suspected, namely that the Linux pages can be trusted to accurately
document this and other Glibc extensions.

> 
>> The two examples that prompted the questions show the Glibc manual
>> is less complete and less accurate than the Linux man pages (or than
>> POSIX that the man pages are derived from).
> 
> Yes, for mbstowcs the manual doesn't say anything about the first
> argument.
> 
> All of the following functions behave the same way:
> * mbstowcs (XSI extension)
> * mbsrtowcs
> * mbsnrtowcs
> * wcsrtombs
> * wcsnrtombs

The Glibc mbsnrtowcs and wcsnrtombs documentation is also missing
the detail about the first pointer being null.  The Linux man pages,
on the other hand, do mention it.

> 
> The last 4 functions are all worded in ISO C11 to operate correctly with
> a NULL dst, but not mbstowcs, which is what I assume this discussion is
> really about, that glibc doesn't quite meet what ISO C11 requires? All of
> the "if dst is not null" language is missing from mbstowcs in C11.
> 
> In mbsrtowcs we have this note from 2001 by Ulrich:
> 
> We have 0 tests for mbstowcs that use NULL as the first argument in
> the implementation. So we have no regression test for breaking this
> behaviour AFAIK.
> 
> In 1996 when Roland implemented mbsrtowcs (the pre-cursor for the _l
> variant and the backend to this function) we had this code:
> 
> 30de3b18a52 (Roland McGrath 1996-04-02 13:27:17 +0000 32)   size_t result = 0;
> ...
> 30de3b18a52 (Roland McGrath 1996-04-02 13:27:17 +0000 45)       if (dst != NULL)
> 30de3b18a52 (Roland McGrath 1996-04-02 13:27:17 +0000 46)           dst[result] = (wchar_t) **src;
> 30de3b18a52 (Roland McGrath 1996-04-02 13:27:17 +0000 47)       ++result;
> ...
> 30de3b18a52 (Roland McGrath 1996-04-02 13:27:17 +0000 62)   return result;
> 
> So the very first implementation already supported this behaviour.
> 
> Following XSI conformance we set _XOPEN_UNIX 1 and _XOPEN_VERSION 700 so
> we do indicate this behaviour to applications looking to check for such
> semantics.
> 
>> So in reality it cannot
>> realistically be taken as authoritative (at least in these two cases).
> 
> I don't see how that follows. Could you expand on this please?

Because it doesn't cover the implementation-defined details as
completely or accurately as the Linux man pages cover it.

> 
> The fact that the manual says nothing means there is no authoritative
> answer for this. You can rely on external documentation to help.

That's not how it's supposed to work.  For one, standards require
implementations to document the choices they let them make.
The documentation is (obviously) expected to be provided with
the implementation, not by some unknown third party or parties.

But from a simple usability point of view, it's unhelpful to tell
people to consider the union of the Glibc manual and all external
documentation (or some subset of it).  Not just because it's
impractical to read everything, but also because not everything
is correct or up to date.  How are they/we supposed to resolve conflicts?

> 
> My preference would be for us to improve the project documentation.
> 
> You can say that the linux man pages represent general user expectations
> and you would be right, and that is one factor we take into consideration
> when making changes to the API and ABI.

That's not all I'm saying: from what I've seen they document
the implementation in more detail, more accurately, and more
completely than the Glibc manual does.  And that's okay.

>> My impression is that this will hold in general.  There are functions
>> the manual doesn't even mention (e.g., readlink's cousin readlinkat).
> 
> Correct, please see manual/filesys.text:

That doesn't help your argument.

> 
> 3580 @c FIXME these are undocumented:
> 3581 @c faccessat
> 3582 @c fchmodat
> 3583 @c fchownat
> 3584 @c futimesat
> 3585 @c fstatat (there's a commented-out safety assessment for this one)
> 3586 @c statx
> 3587 @c mkdirat
> 3588 @c mkfifoat
> 3589 @c name_to_handle_at
> 3590 @c openat
> 3591 @c open_by_handle_at
> 3592 @c readlinkat
> 3593 @c renameat
> 3594 @c renameat2
> 3595 @c scandirat
> 3596 @c symlinkat
> 3597 @c unlinkat
> 3598 @c utimensat
> 3599 @c mknodat
> 
> I just reviewed Florian's patches for several *at functions:
> 
> https://patchwork.sourceware.org/project/glibc/patch/87d07p9v73.fsf@mid.deneb.enyo.de/
> 
> I think we're at v2 for that patch.
> 
>> I realize these are just omissions that could be fixed.  But until
>> they are, the manual cannot very well be taken as authoritative.
> 
> I look forward to the day when all the APIs are covered.
> 
> However, until that day, it does not follow that because the manual is
> incomplete that it is not the authoritative reference.
> 
> Missing information does not make it less authoritative, just incomplete.

The Cambridge Dictionary defines authoritative as:

   containing complete and accurate information, and therefore respected

But that's just semantics.  I'm sorry if my questioning the authority
of the Glibc manual struck a nerve.  All I wanted to know is where I'm
more likely to find complete and accurate documentation of Glibc
implementation details.  I've got my answer.

Thanks
Martin

> 
> You can come to the project and ask for clarification, and that act in
> and of itself helps us prioritize what to work on next.
> 