[PATCH v5 0/3] manual: Header & Standards Cleanup

[Rical Jasan <ricaljasan at pacific dot net>]

This is a multi-part message in MIME format.

There exists a convention for annotating which headers and standards
functions, variables, etc., provided by the glibc come from,
guaranteeing their automatic inclusion in the Summary of Library
Facilities, where they are indexed along with their headers and
standards with references into the manual.  The convention is based
upon expectations present in summary.awk, though that script does not
do any enforcing, merely indexing what it can find.  It is roughly:

  @comment HEADER(S)
  @comment STANDARD(S)
  @(def|item|vindex)

It would be nice to use something other than ad-hoc @comments for such
annotations, and also provide a framework for ensuring annotations
exist and are correctly formatted.  To that end, a new set of macros
is proposed: @standards and @standardsx.

Examples of intended use are:

  @item FOO
  @standards{STD, HDR}

  @defvar BAR
  @standards{STD1, HDR1}
  @standards{STD2, HDR2}

  @deftypefun foo
  @deftypefunx fool
  @standards{STD, HDR}

  @item bar
  @itemx barf
  @itemx barl
  @standardsx{bar, STD, HDR}
  @standardsx{barf, STD2, HDR2}
  @standardsx{barl, STD2, HDR2}

(Note the annotation for bar may also be @standards.)

A one-off script, supplied separately for review, was used to convert
existing @comment-based annotations to @standards.  In this patchset,
the @standards* macros are first added to macros.texi, then the
resultant patch from the conversion is applied, and lastly,
summary.awk is replaced by summary.pl, along with miscellaneous fixes
required to complete the transition.

summary.pl contains all the machinery to perform syntax-checking and
report errors, but since the manual is still largely incomplete with
regards to header and standards annotations, it is not turned on.  The
script has been written so that once the manual is completely
annotated, a simple two-line patch will begin enforcing @standards.

In order to provide insight on the differences between summary.awk and
summary.pl, and how @standards will differ from the @comment-based
annotations, some analysis of the immediate and expected long-term
differences follows.

Analysis of the generated summary.texis is not straightforward because
summary.awk respected the user's locale, and while summary.pl does
too, the C locale is now imposed in the Makefile, significantly
reordering the Summary.  This change will result in consistent
ordering across builds, however.  Further, summary.pl has improved
detection of the names of annotated elements, which affects sorting;
specifically, extraneous syntax is removed from function pointers (3),
functions that return pointers where the "*" character is attached to
the element name (2), and structs, which no longer include "struct"
for purposes of indexing and sorting (47).

Overall, nine new entries are present, due to either partial header
and standard annotations being completed (5; albeit with "???", since
the conversion can't know what should go there) or seeming
idiosyncrasies in summary.awk (4).

Thirty-four entries are no longer present.  While this number seems
high, there are a variety of reasons, a few of which are actually more
correct, and most of which will be addressed by future clean up
requiring more than the basic mechanical changes in this patchset.

The absent count is based on unique entries, so isn't entirely
representative of the changes.  In particular, due to the difference
in name detection, some entries are duplicates now (7).  One example
is mallinfo, for which summary.awk used "mallinfo" (the function) and
"struct mallinfo" (the data type).  summary.pl sorts both by
"mallinfo", so they are now adjacent.  The entries are immediately
distinguishable, however, as the Summary shows the prototypes, not
merely the names of the elements.

One entry is present in both the absent (1) and new counts:
error_print_progname.  summary.awk detected it as "(void)", while
summary.pl extracts "error_print_progname".

The legitimate absences (5) are all due to the fact summary.pl ignores
everything in @ignore context, whereas summary.awk did not.

The remaining absences will be addressed in future patches.  There is
an annotated @item in a @table (1), which is recognized as an error
because that is not a generally annotatable context.  There is also a
block of annotated @vindex commands (20), which summary.awk picked up
but summary.pl does not because @standards are expected to be rendered
at some point, and @vindex commands are not rendered; instead, such a
situation provokes an error.

Overall, there is a net loss of 21 entries, to be addressed in 2
follow-up patches.

The conversion of existing @comment-based annotations to @standards is
the immediate goal of this patchset, expected to be followed by
another patchset completing annotations and making summary.pl begin
enforcing the presence and syntax of @standards, helping establish a
minimum and mandatory baseline for complete documentation.  The
follow-up patchset is expected to require a greater level of review,
due to both volume and the fact new @standards will need a higher
level of scrutiny.

In the long term, @standards are expected to be rendered in place,
similar to @safety.  Some discussion has already taken place around
this, but by defining @standards* to be empty macros, time and care
may be taken about how that is done, allowing it to be switched on
everywhere when ready.  A related discussion which has also begun is
the canonicalization of standards names, which may also influence the
completion patchset.  summary.pl has been written with extending the
syntax of @standards in mind, and it should be straightforward to
introduce additional checks such as validating the names of standards
used in annotations.

So, without further ado, here is my @standards proposal.

Thank you,
Rical Jasan

---
Changes since v4:

  * Annotating @*x lists is loosened to accept an optional, single,
    "default" @standards at the beginning of the @standardsx
    annotations.  This will save much rendered space and results in a
    more compact possible annotation.  @standardsx are now optional.
  * "(optional)" headers no longer include that string in the @file
    macro.
  * Users of @standards are directed to ask summary.pl for help in
    macros.texi.
  * Improved error detection and reporting:
    * The 14 Summary entries previously lost due to partial annotation
      errors are present.
    * Line numbers are provided for @standards errors without any
      other context (i.e., when not related to a specific element).
    * Additional check for spurious @standardsx introduced.

Changes since v3:

  * Patches 3/7 and 5/7 were committed.
  * The errno.texi preparatory patch has been reworked to use a new
    macro, @errno, and scripts depending on the previous @comment
    framework have been adjusted accordingly.

---
Rical Jasan (3):
  manual: Create empty placeholder macros for @standards.
  manual: Convert header and standards @comments to @standards.
  manual: Replace summary.awk with summary.pl.

 manual/Makefile      |   7 +-
 manual/argp.texi     | 126 ++++------
 manual/arith.texi    | 679 ++++++++++++---------------------------------------
 manual/charset.texi  |  96 +++-----
 manual/conf.texi     | 651 ++++++++++++++++--------------------------------
 manual/creature.texi |  44 ++--
 manual/crypt.texi    |  65 ++---
 manual/ctype.texi    | 116 +++------
 manual/debug.texi    |   9 +-
 manual/errno.texi    | 504 +++++++++++++-------------------------
 manual/filesys.texi  | 389 ++++++++++-------------------
 manual/getopt.texi   |  24 +-
 manual/header.texi   |   2 +-
 manual/job.texi      |  33 +--
 manual/lang.texi     | 190 ++++----------
 manual/llio.texi     | 333 +++++++++----------------
 manual/locale.texi   |  39 +--
 manual/macros.texi   |   8 +
 manual/math.texi     | 524 ++++++++-------------------------------
 manual/memory.texi   | 152 +++++-------
 manual/message.texi  |  30 +--
 manual/pattern.texi  | 219 ++++++-----------
 manual/pipe.texi     |  16 +-
 manual/process.texi  |  69 ++----
 manual/resource.texi | 169 +++++--------
 manual/search.texi   |  45 ++--
 manual/setjmp.texi   |  33 +--
 manual/signal.texi   | 257 +++++++------------
 manual/socket.texi   | 348 +++++++++-----------------
 manual/startup.texi  |  52 ++--
 manual/stdio.texi    | 495 +++++++++++++------------------------
 manual/string.texi   | 301 ++++++++---------------
 manual/summary.awk   | 133 ----------
 manual/summary.pl    | 403 ++++++++++++++++++++++++++++++
 manual/sysinfo.texi  |  77 ++----
 manual/syslog.texi   |  15 +-
 manual/terminal.texi | 303 ++++++++---------------
 manual/threads.texi  |  18 +-
 manual/time.texi     | 151 ++++--------
 manual/users.texi    | 281 ++++++++-------------
 40 files changed, 2563 insertions(+), 4843 deletions(-)
 delete mode 100644 manual/summary.awk
 create mode 100755 manual/summary.pl

-- 
2.12.2