This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Use gen-libm-test.py to generate ulps table for manual
- From: Joseph Myers <joseph at codesourcery dot com>
- To: <libc-alpha at sourceware dot org>
- Date: Thu, 9 Aug 2018 20:26:31 +0000
- Subject: Use gen-libm-test.py to generate ulps table for manual
This patch, relative to a tree with
<https://sourceware.org/ml/libc-alpha/2018-08/msg00133.html> (pending
review) applied, extends gen-libm-test.py to generate the ulps table
for the manual, so meaning there is only a single ulps file parser
needed and another Perl script is eliminated. As with the
introduction of gen-libm-test.py, this is designed to generate exactly
the same libm-err.texi as libm-err-tab.pl did. (gen-libm-test.py is
still shorter in lines than the old gen-libm-test.pl even after this
patch.) Note that this introduces a Python dependency for building
the manual, which is thus noted in install.texi and NEWS.
Tested building html / info / pdf versions of the manual.
2018-08-09 Joseph Myers <joseph@codesourcery.com>
* math/gen-libm-test.py: Import os.
(ALL_FLOATS_MANUAL): New constant.
(ALL_FLOATS_SUFFIX): Likewise.
(Ulps.all_functions): New function.
(real_all_ulps): Likewise.
(generate_err_table_sub): Likewise.
(generate_err_table): Likewise.
(main): Handle -s and -m options.
* manual/libm-err-tab.pl: Remove.
* manual/Makefile ($(objpfx)stamp-libm-err): Use gen-libm-test.py
instead of libm-err-tab.pl.
[$(PERL) != no]: Change condition to [$(if $(PYTHON),$(PERL),no)
!= no].
* manual/install.texi (Tools for Compilation): Document
requirement for Python to build manual.
* INSTALL: Regenerated.
diff --git a/INSTALL b/INSTALL
index 603b23b..518ed58 100644
--- a/INSTALL
+++ b/INSTALL
@@ -514,8 +514,9 @@ build the GNU C Library:
* Python 2.7/3.4 or later
- Python is required to run some tests. As of release time, Python
- 3.6 is the newest verified to work for testing the GNU C Library.
+ Python is required to build the GNU C Library manual and to run
+ some tests. As of release time, Python 3.6 is the newest verified
+ to work for testing the GNU C Library.
* PExpect 4.0
diff --git a/NEWS b/NEWS
index 7868c84..b208404 100644
--- a/NEWS
+++ b/NEWS
@@ -23,7 +23,7 @@ Deprecated and removed features, and other changes affecting compatibility:
Changes to build and runtime requirements:
- [Add changes to build and runtime requirements here]
+* Python is required to build the GNU C Library manual.
Security related changes:
diff --git a/manual/Makefile b/manual/Makefile
index c275664..9c35c9d 100644
--- a/manual/Makefile
+++ b/manual/Makefile
@@ -99,10 +99,11 @@ $(objpfx)dir-add.texi: xtract-typefun.awk $(texis-path)
# The table with the math errors is generated.
$(objpfx)libm-err.texi: $(objpfx)stamp-libm-err
-$(objpfx)stamp-libm-err: libm-err-tab.pl $(wildcard $(foreach dir,$(sysdirs),\
- $(dir)/libm-test-ulps))
+$(objpfx)stamp-libm-err: $(..)math/gen-libm-test.py \
+ $(wildcard $(foreach dir,$(sysdirs),\
+ $(dir)/libm-test-ulps))
pwd=`pwd`; \
- $(PERL) $< $$pwd/.. > $(objpfx)libm-err-tmp
+ $(PYTHON) $< -s $$pwd/.. -m $(objpfx)libm-err-tmp
$(move-if-change) $(objpfx)libm-err-tmp $(objpfx)libm-err.texi
touch $@
@@ -169,8 +170,9 @@ include ../Rules
.PHONY: install subdir_install install-data
install-data subdir_install: install
-# Generated files requiring perl: libm-err.texi, summary.texi
-ifneq ($(PERL),no)
+# Generated files requiring python: libm-err.texi
+# Generated files requiring perl: summary.texi
+ifneq ($(if $(PYTHON),$(PERL),no),no)
ifneq ($(strip $(MAKEINFO)),:)
install: $(inst_infodir)/libc.info
@if $(SHELL) -c '$(INSTALL_INFO) --version' >/dev/null 2>&1; then \
diff --git a/manual/install.texi b/manual/install.texi
index df32b39..5372bc4 100644
--- a/manual/install.texi
+++ b/manual/install.texi
@@ -560,8 +560,9 @@ with any version of @code{sed}. As of release time, @code{sed} version
@item
Python 2.7/3.4 or later
-Python is required to run some tests. As of release time, Python 3.6
-is the newest verified to work for testing @theglibc{}.
+Python is required to build the @glibcadj{} manual and to run some
+tests. As of release time, Python 3.6 is the newest verified to work
+for testing @theglibc{}.
@item PExpect 4.0
diff --git a/manual/libm-err-tab.pl b/manual/libm-err-tab.pl
deleted file mode 100755
index 768d2da..0000000
--- a/manual/libm-err-tab.pl
+++ /dev/null
@@ -1,206 +0,0 @@
-#!/usr/bin/perl -w
-# Copyright (C) 1999-2018 Free Software Foundation, Inc.
-# This file is part of the GNU C Library.
-# Contributed by Andreas Jaeger <aj@suse.de>, 1999.
-
-# The GNU C Library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-
-# The GNU C Library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-
-# You should have received a copy of the GNU Lesser General Public
-# License along with the GNU C Library; if not, see
-# <http://www.gnu.org/licenses/>.
-
-# Information about tests are stored in: %results
-# $results{$test}{"type"} is the result type, e.g. normal or complex.
-# In the following description $platform, $type and $float are:
-# - $platform is the used platform
-# - $type is either "normal", "real" (for the real part of a complex number)
-# or "imag" (for the imaginary part # of a complex number).
-# - $float is either of float, ifloat, double, idouble, ldouble, ildouble;
-# It represents the underlying floating point type (float, double or long
-# double) and if inline functions (the leading i stands for inline)
-# are used.
-# $results{$test}{$platform}{$type}{$float} is defined and has a delta
-# or 'fail' as value.
-
-use File::Find;
-
-use strict;
-
-use vars qw ($sources @platforms %pplatforms);
-use vars qw (%results @all_floats %suffices %all_functions);
-
-
-# all_floats is in output order and contains all recognised float types that
-# we're going to output
-@all_floats = ('float', 'double', 'ldouble', 'float128');
-%suffices =
- ( 'float' => 'f',
- 'double' => '',
- 'ldouble' => 'l',
- 'float128' => 'f128'
- );
-
-# Pretty description of platform
-%pplatforms = ();
-
-%all_functions = ();
-
-if ($#ARGV == 0) {
- $sources = $ARGV[0];
-} else {
- $sources = '/usr/src/cvs/libc';
-}
-
-find (\&find_files, $sources);
-
-@platforms = sort by_platforms @platforms;
-
-&print_all;
-
-sub find_files {
- if ($_ eq 'libm-test-ulps') {
- # print "Parsing $File::Find::name\n";
- push @platforms, $File::Find::dir;
- my ($file, $name);
- $file = "${File::Find::name}-name";
- open NAME, $file or die ("Can't open $file: $!");
- $name = <NAME>;
- chomp $name;
- close NAME;
- $pplatforms{$File::Find::dir} = $name;
- &parse_ulps ($File::Find::name, $File::Find::dir);
- }
-}
-
-# Parse ulps file
-sub parse_ulps {
- my ($file, $platform) = @_;
- my ($test, $type, $float, $eps, $ignore_fn);
-
- # $type has the following values:
- # "normal": No complex variable
- # "real": Real part of complex result
- # "imag": Imaginary part of complex result
- open ULP, $file or die ("Can't open $file: $!");
- while (<ULP>) {
- chop;
- # ignore comments and empty lines
- next if /^#/;
- next if /^\s*$/;
- if (/^Function: /) {
- if (/Real part of/) {
- s/Real part of //;
- $type = 'real';
- } elsif (/Imaginary part of/) {
- s/Imaginary part of //;
- $type = 'imag';
- } else {
- $type = 'normal';
- }
- ($test) = ($_ =~ /^Function:\s*\"([a-zA-Z0-9_]+)\"/);
- next;
- }
- if ($test =~ /_(downward|towardzero|upward|vlen)/) {
- $ignore_fn = 1;
- } else {
- $ignore_fn = 0;
- $all_functions{$test} = 1;
- }
- if (/^i?(float|double|ldouble|float128):/) {
- ($float, $eps) = split /\s*:\s*/,$_,2;
- if ($ignore_fn) {
- next;
- } elsif ($eps eq 'fail') {
- $results{$test}{$platform}{$type}{$float} = 'fail';
- } elsif ($eps eq "0") {
- # ignore
- next;
- } elsif (!exists $results{$test}{$platform}{$type}{$float}
- || $results{$test}{$platform}{$type}{$float} ne 'fail') {
- $results{$test}{$platform}{$type}{$float} = $eps;
- }
- if ($type =~ /^real|imag$/) {
- $results{$test}{'type'} = 'complex';
- } elsif ($type eq 'normal') {
- $results{$test}{'type'} = 'normal';
- }
- next;
- }
- print "Skipping unknown entry: `$_'\n";
- }
- close ULP;
-}
-
-sub get_value {
- my ($fct, $platform, $type, $float) = @_;
-
- return (exists $results{$fct}{$platform}{$type}{$float}
- ? $results{$fct}{$platform}{$type}{$float} : "0");
-}
-
-sub print_platforms {
- my (@p) = @_;
- my ($fct, $platform, $float, $first, $i, $platform_no, $platform_total);
-
- print '@multitable {nexttowardf} ';
- foreach (@p) {
- print ' {1000 + i 1000}';
- }
- print "\n";
-
- print '@item Function ';
- foreach (@p) {
- print ' @tab ';
- print $pplatforms{$_};
- }
- print "\n";
-
-
- foreach $fct (sort keys %all_functions) {
- foreach $float (@all_floats) {
- print "\@item $fct$suffices{$float} ";
- foreach $platform (@p) {
- print ' @tab ';
- if (exists $results{$fct}{$platform}{'normal'}{$float}
- || exists $results{$fct}{$platform}{'real'}{$float}
- || exists $results{$fct}{$platform}{'imag'}{$float}) {
- if ($results{$fct}{'type'} eq 'complex') {
- print &get_value ($fct, $platform, 'real', $float),
- ' + i ', &get_value ($fct, $platform, 'imag', $float);
- } else {
- print $results{$fct}{$platform}{'normal'}{$float};
- }
- } else {
- print '-';
- }
- }
- print "\n";
- }
- }
-
- print "\@end multitable\n";
-}
-
-sub print_all {
- my ($i, $max);
-
- my ($columns) = 5;
-
- # Print only 5 platforms at a time.
- for ($i=0; $i < $#platforms; $i+=$columns) {
- $max = $i+$columns-1 > $#platforms ? $#platforms : $i+$columns-1;
- print_platforms (@platforms[$i .. $max]);
- }
-}
-
-sub by_platforms {
- return $pplatforms{$a} cmp $pplatforms{$b};
-}
diff --git a/math/gen-libm-test.py b/math/gen-libm-test.py
index 8d156ea..0b25a90 100755
--- a/math/gen-libm-test.py
+++ b/math/gen-libm-test.py
@@ -19,6 +19,7 @@
import argparse
from collections import defaultdict
+import os
import re
@@ -32,6 +33,16 @@ ALL_FLOATS_PFX = {'double': 'DBL',
'float': 'FLT',
'float128': 'FLT128'}
+# Float types in the order used in the generated ulps tables in the
+# manual.
+ALL_FLOATS_MANUAL = ('float', 'double', 'ldouble', 'float128')
+
+# Map float types in ulps files to C function suffix.
+ALL_FLOATS_SUFFIX = {'double': '',
+ 'ldouble': 'l',
+ 'float': 'f',
+ 'float128': 'f128'}
+
# Number of arguments in structure (as opposed to arguments that are
# pointers to return values) for an argument descriptor.
DESCR_NUM_ARGS = {'f': 1, 'a': 1, 'j': 1, 'i': 1, 'u': 1, 'l': 1, 'L': 1,
@@ -161,6 +172,17 @@ class Ulps(object):
ulps_dict[ulps_fn][line_first],
ulps_val)
+ def all_functions(self):
+ """Return the set of functions with ulps and whether they are
+ complex."""
+ funcs = set()
+ complex = {}
+ for k_prefix, k_dict in self.ulps_kinds:
+ for f in k_dict:
+ funcs.add(f)
+ complex[f] = True if k_prefix else False
+ return funcs, complex
+
def write(self, ulps_file):
"""Write ulps back out as a sorted ulps file."""
# Output is sorted first by function name, then by (real,
@@ -231,6 +253,18 @@ class Ulps(object):
f.write(header_text)
+def read_all_ulps(srcdir):
+ """Read all platforms' libm-test-ulps files."""
+ all_ulps = {}
+ for dirpath, dirnames, filenames in os.walk(srcdir):
+ if 'libm-test-ulps' in filenames:
+ with open(os.path.join(dirpath, 'libm-test-ulps-name')) as f:
+ name = f.read().rstrip()
+ all_ulps[name] = Ulps()
+ all_ulps[name].read(os.path.join(dirpath, 'libm-test-ulps'))
+ return all_ulps
+
+
def read_auto_tests(test_file):
"""Read tests from auto-libm-test-out-<function> (possibly None)."""
auto_tests = defaultdict(lambda: defaultdict(dict))
@@ -571,6 +605,60 @@ def generate_testfile(inc_input, auto_tests, c_output):
f.write(''.join(test_list))
+def generate_err_table_sub(all_ulps, all_functions, fns_complex, platforms):
+ """Generate a single table within the overall ulps table section."""
+ plat_width = [' {1000 + i 1000}' for p in platforms]
+ plat_header = [' @tab %s' % p for p in platforms]
+ table_list = ['@multitable {nexttowardf} %s\n' % ''.join(plat_width),
+ '@item Function %s\n' % ''.join(plat_header)]
+ for func in all_functions:
+ for flt in ALL_FLOATS_MANUAL:
+ func_ulps = []
+ for p in platforms:
+ p_ulps = all_ulps[p]
+ if fns_complex[func]:
+ ulp_real = p_ulps.real[func][flt]
+ ulp_imag = p_ulps.imag[func][flt]
+ ulp_str = '%d + i %d' % (ulp_real, ulp_imag)
+ ulp_str = ulp_str if ulp_real or ulp_imag else '-'
+ else:
+ ulp = p_ulps.normal[func][flt]
+ ulp_str = str(ulp) if ulp else '-'
+ func_ulps.append(ulp_str)
+ table_list.append('@item %s%s @tab %s\n'
+ % (func, ALL_FLOATS_SUFFIX[flt],
+ ' @tab '.join(func_ulps)))
+ table_list.append('@end multitable\n')
+ return ''.join(table_list)
+
+
+def generate_err_table(all_ulps, err_table):
+ """Generate ulps table for manual."""
+ all_platforms = sorted(all_ulps.keys())
+ functions_set = set()
+ functions_complex = {}
+ for p in all_platforms:
+ p_functions, p_complex = all_ulps[p].all_functions()
+ functions_set.update(p_functions)
+ functions_complex.update(p_complex)
+ all_functions = sorted([f for f in functions_set
+ if ('_downward' not in f
+ and '_towardzero' not in f
+ and '_upward' not in f
+ and '_vlen' not in f)])
+ err_table_list = []
+ # Print five platforms at a time.
+ num_platforms = len(all_platforms)
+ for i in range((num_platforms + 4) // 5):
+ start = i * 5
+ end = i * 5 + 5 if num_platforms >= i * 5 + 5 else num_platforms
+ err_table_list.append(generate_err_table_sub(all_ulps, all_functions,
+ functions_complex,
+ all_platforms[start:end]))
+ with open(err_table, 'w') as f:
+ f.write(''.join(err_table_list))
+
+
def main():
"""The main entry point."""
parser = argparse.ArgumentParser(description='Generate libm tests.')
@@ -580,23 +668,31 @@ def main():
help='input file .inc file with tests')
parser.add_argument('-u', dest='ulps_file', metavar='FILE',
help='input file with ulps')
+ parser.add_argument('-s', dest='srcdir', metavar='DIR',
+ help='input source directory with all ulps')
parser.add_argument('-n', dest='ulps_output', metavar='FILE',
help='generate sorted ulps file FILE')
parser.add_argument('-C', dest='c_output', metavar='FILE',
help='generate output C file FILE from .inc file')
parser.add_argument('-H', dest='ulps_header', metavar='FILE',
help='generate output ulps header FILE')
+ parser.add_argument('-m', dest='err_table', metavar='FILE',
+ help='generate output ulps table for manual FILE')
args = parser.parse_args()
ulps = Ulps()
if args.ulps_file is not None:
ulps.read(args.ulps_file)
auto_tests = read_auto_tests(args.auto_input)
+ if args.srcdir is not None:
+ all_ulps = read_all_ulps(args.srcdir)
if args.ulps_output is not None:
ulps.write(args.ulps_output)
if args.ulps_header is not None:
ulps.write_header(args.ulps_header)
if args.c_output is not None:
generate_testfile(args.inc_input, auto_tests, args.c_output)
+ if args.err_table is not None:
+ generate_err_table(all_ulps, args.err_table)
if __name__ == '__main__':
--
Joseph S. Myers
joseph@codesourcery.com