Differences between revisions 79 and 80
Revision 79 as of 2013-11-06 23:15:27
Size: 48584
Editor: JosephMyers
Comment: Update on cross-localedef, EGLIBC patches status.
Revision 80 as of 2013-11-06 23:20:24
Size: 49012
Editor: JosephMyers
Comment: Update IEEE 754-2008 bindings references.
Deletions are marked like this. Additions are marked like this.
Line 394: Line 394:
 * Support for the draft ISO C bindings to IEEE 754-2008 (WG14 N1664 or any more recent version if available).  * Support for the first part of draft ISO C bindings to IEEE 754-2008 (WG14 N1774 or any more recent version if available).

 * Integrate libdfp into the glibc repository and update it to N1775 or any more recent version of the proposed updated bindings.

 * Support the third part of the IEEE 754-2008 bindings (N1758 or any more recent version), so superseding GCC's libquadmath (make the ldbl-128 functions usable both for {{{long double}}}, and for {{{_Float128}}} as a type with different representation from {{{long double}}}).

Master Todo List

This is the master TODO list for the project all other TODO lists reference this list.

Smaller self-contained projects, where it is readily possible to assess whether they are completed and where it is uncontroversial what should be changed, should generally go in Bugzilla. Projects here may be less precisely defined and may not always have consensus on whether there is a genuine problem and how it should be fixed.

Documentation

It's always good to start by writing documentation. It helps you learn the API and understand what problems real users face day-to-day when working with the interface.

  • The README file in git needs some cleanup to update the lists of supported systems in libc and ports
  • Clean up the FAQ.

  • move the PROJECTS file from git to wiki - and remove obsolete entries.
  • WUR-REPORT is similarly a projects list that should be merged into this page.
  • Likewise, the nptl/TODO* files.
  • Some functions that are documented in the manual as "GNU" are now in newer versions of ISO C or POSIX; the documentation should be updated accordingly.
  • The installation documentation needs further review and updating (especially as regards the minimum versions of some tools documented as supported).

POSIX Threads

At present the glibc manual has *no* pthread documentation, and we need to fix this. The complete project is large, but it should be broken up into chunks, each chunk doable in a quick sit-down hacking session.

  • Documenting the glibc pthread API.
    • Add a new section to the manual for pthreads.
    • Document one function from pthreads in the new pthreads section in the manual.
    • Add an example to one pthreads function in the manual.

Please note that you must not copy from other sources and for copyright purposes this must be your own work.

We should coordinate with linux man pages project.

Missing Functions

There are some functions that were added to glibc but not documented. We need to find them.

Again this work is incremental, find a function, add it to the manual.

  • Find a public function that was added and not described in the manual.
  • Document the public function in the manual.
  • Add a man page to the linux man-pages project if it doesn't have one already.

Please note that you must not copy from other sources and for copyright purposes this must be your own work.

  • scripts/documented.sh can be used to find undocumented functions. First, it needs to be updated to reflect the current set of libraries built by glibc (the script currently refers to Linuxthreads, for example). It should also not include an individual email address in the output. Then, the output could be made available somewhere (linked from the wiki), or used to file bugs.
  • Note that documented.sh only covers functions, and it would be good to have such an automatic search for undocumented public macros as well.
  • If a list of known undocumented interfaces were in the sources, it might be possible to make the addition of any new undocumented interfaces result in a build or testsuite failure.

Internals documentation

  • Document how to debug your application from dynamic loader all the way to the application main.
    • Start by debugging the dynamic linker.
    • Progress to loading the symbol file for your application at the correct offset.
  • Update wiki:
    • Include "What do I need to file a locale change?"
  • Create a glibc internals manual seperate from the user manual
    • Internals manual will cover makefile targets and porting.
    • Move appending porting info from user manual to internals manual.

Bugs

There are many bugs in the bug database for product=glibc. We need help managing these bugs.

  • move the BUGS file from git to bugzilla (one bugreport for each entry) - and check that the bugs are valid
  • Systematically review open bugs to ensure they are valid, still present in current glibc and that all the Bugzilla fields are set correctly (including removing default assignments). This was done by Joseph for open "ports", "linuxthreads", "math" and "soft-fp" bugs already.

Reproducing

We need help reproducing bugs. We need to know if they are real and if they apply to trunk.

  • Pick a bug that is in the NEW or UNCONFIRMED or REOPENED state, or a SUSPENDED or ASSIGNED bug that hasn't been touched recently.
  • Try to reproduce the bug.
  • If you succeed at reproducing the bug you should:
    • Describe the steps to reproduce it if they are different from the submitters instructions.
    • Update the "Last Reconfirmed" date with the date on which you reproduced the bug.
    • See Bug_Triage and Bugzilla Procedures for more details. You don't need to do everything listed there; any bit of work on reproducing a bug is useful.

Testing

Infrastructure

  • Support for generating a log with PASS or FAIL results for individual tests.
    • Tests not supported (e.g. requiring a function or other feature not implemented on the particular architecture) should show as UNSUPPORTED in the logs.
    • There should be a standard way of handling tests that are only partly supported, e.g. that can test some but not all things if floating-point exceptions or rounding modes are not supported. This should make it readily visible that parts of the test have not been run, in case the unsupported feature *should* have been supported on the given architecture.
    • Tests should show as UNRESOLVED in cases such as insufficient memory.
    • Compiling / linking a test should produce results separately from running it.
    • Tests that are expected to fail should show as XFAIL (or XPASS, if they pass).
    • See http://sourceware.org/ml/libc-alpha/2012-09/msg00439.html for more regarding test statuses.

  • Support for testing an installed C library without needing its build tree to be available.
    • For this, some tests such as tst-xmmymm.sh or check-local-headers.sh that fundamentally depend on the build tree would need to be run with a separate makefile target from the build tree, since they could not be run at "make check" time.
    • Tests with lesser build-tree dependence might require an option to install some files from the build tree that are only actually needed for testing.
  • Describe more features of tests through annotations in "tests" or a per-test variable listing test properties, as suggested in http://sourceware.org/ml/libc-alpha/2012-09/msg00276.html

Untested interfaces.

  • All public interfaces should be tested by the testsuite as far as possible (some may require root access or otherwise be problematic to test).
  • It would be good to have an automatic search for untested interfaces, possibly like scripts/documented.sh (see above) for undocumented interfaces.
  • Addition of tests for any untested interface is a useful contribution. ldiv, lldiv and imaxdiv are examples of untested interfaces - tests would have shown up imaxdiv being broken on alpha for a long time.
  • See http://sourceware.org/ml/libc-alpha/2013-07/msg00386.html for a list of untested interfaces and some automation for finding them.

Test IFUNC implementations.

  • Extend framework of testing IFUNC implementations on target to math functions, similar to string function tests:
    • Extract IFUNC test framework from string/test-string.h to test-ifunc.h.
    • Use FOR_EACH_IMPL and CALL to test each IFUNC implementation of math functions.
  • Track and report untested IFUNC implementations (see commit 11dd4af and this this email for details).

We need help testing releases.

  • Go to the release page.

  • Pick a release from the "Current Branches" list.
  • Checkout the release, build it, and run the testsuite.
  • Under the releases page create a section titled "User-submitted Testsuite Failures"
  • Describe your configuration and the failures, it should look like this in markup:

== User-submitted Testsuite Failures ==

User: Carlos O'Donell <carlos@systemhalted.org>
Build system: Ubuntu 11.04, x86, GCC 4.5.2, Binutils 2.21.0, Kernel 2.6.38-8-generic (32-bit).
{{{
make[2]: *** [/home/carlos/build/glibc-2.15/localedata/tst-xlocale1.out] Error 1
make[1]: *** [localedata/tests] Error 2
make[2]: [/home/carlos/build/glibc-2.15/posix/annexc.out] Error 1 (ignored)
make[2]: *** [/home/carlos/build/glibc-2.15/rt/tst-cpuclock2.out] Error 1
make[1]: *** [rt/tests] Error 2
make[2]: *** [/home/carlos/build/glibc-2.15/elf/tst-unique3lib.os] Error 1
make[2]: *** [/home/carlos/build/glibc-2.15/elf/tst-unique3lib2.os] Error 1
make[2]: *** [/home/carlos/build/glibc-2.15/elf/tst-unique3.o] Error 1
make[2]: *** [/home/carlos/build/glibc-2.15/elf/tst-unique4.o] Error 1
make[2]: *** [/home/carlos/build/glibc-2.15/elf/check-localplt.out] Error 1
}}}

Use test-skeleton.c

  • Migrate all tests to use the top-level test-skeleton.c
    • Add a TEST_DIRECT so that tests which test fork are not forked by test-skeleton.c.
  • Fix test-skeleton.c to support tst-initializers1 tests and similar which fail with:

In file included from ../malloc/malloc.h:24:0,
                  from ../include/malloc.h:2,
                  from ../test-skeleton.c:22,
                  from tst-initializers2.c:54:
../include/stdio.h:111:15: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__getwc_unlocked’
cc1: warnings being treated as errors

conformtest improvements

  • Merge stdlib/isomac and posix/annexc into conformtest and then remove the separate tests, and with them $(native-compile). The first needs ISO C90 Amendment 1 support in conformtest; the second is more confused about what it is testing, but review of the existing "POSIX" (POSIX.1, 1996 edition) support, together with new support for POSIX.1-1990, would cover it.
  • In general the expectations need reviewing against relevant standards; this has only been done so far for the ISO C standards supported.

Miscellaneous testsuite issues

  • The c++-types tests should cover more types from more headers (preferably, every typedef glibc provides as a public interface, since any such type could be used in C++ code and needs a stable ABI there).
  • There should be some form of automatic testing for cases when a function in standard X calls another function, whose name is in the user namespace when using standard X, as such cases are not namespace-clean for static linking. The conform/ data might be a source of information about what is in what standard, if reviewed and expanded to cover more standards.
  • As noted in http://sourceware.org/ml/libc-alpha/2012-10/msg00894.html the elf/tst-array* tests should use the .out files as the actual final output, not an intermediate file.

  • Each installed header should be tested with each feature test macro; either a #include of the header should compile OK, or it should produce a #error error, but not some other kind of error. Also test with -pedantic -Wsystem-headers and different -std= options; as far as possible, no warnings should be produced by the headers in any case.

  • It's not ideal that for cross-testing it's necessary to pass arbitrary environment variables through to the test host (hence the code in cross-test-ssh.sh to do so). It would be better if, whenever environment settings are needed for testcases, they are passed explicitly through $(test-wrapper-env), so that cross-test-ssh.sh no longer needs to pass through the build system environment. In some cases environment settings may be needed on both the build and host systems (e.g. LC_ALL=C set in test .sh scripts), in which case they should be explicitly passed to the host as well as set on the build system.

Benchmarks

There are currently benchmarks for a variety of libm functions and string functions in the benchtests directory.

  • A wider array of benchmarks is needed to make sure performance regressions are not introduced. For example, more math functions, more string functions, crypo, atoi, rand, qsort, malloc.
  • The existing string benchmarks can be improved in several ways:
    • The alignments tested are not very useful. Generally the interesting cases are 1, 2, 4, 8, 16, 32, 64 and anything in between is a variation in practical terms.
    • The number of function calls per benchmark should be adjusted to provide reasonably low variance and reasonable running time. At the moment there are probably too few calls (too high variance).
    • The performance of memcpy in particular can be affected by physical page placement when using a physically indexed cache (reads and writes contending for the same cache set) which should be controlled for.
    • The benchmarks currently only output the mean execution time but do not give any measure of variance or min/max.
    • There is no mechanism for showing the output of the benchmarks graphically.

Cross build / test improvements

There are still at least the following issues with getting full feature parity of native and cross builds of glibc:

  • localedef can only be built as part of a glibc build. "make localedata/install-locales" uses the newly built localedef program and can only be used when building natively. It is possible to set up build infrastructure for building localedef outside of glibc, so that it could be built and used in much the same way as a cross-compiler, although it does make fairly heavy use of glibc features and is not that easy to make build on a wide range of systems.
  • The conform/ tests are only enabled for native building. They build programs using new headers but not linked with the new C library (linked with whatever one the compiler uses by default - which may of course not exist / be installed in the case of a cross compiler), and then run such programs. They would probably need changing to link properly with the new library, and run programs the way programs built with it should be run, so using $(run-program-prefix) - ideally supporting doing just compilation and not execution tests if $(run-built-tests) is "no".
  • iconvdata/Makefile and sysdeps/s390/s390-64/Makefile try to update an existing gconv-modules.cache file, for native builds, using the newly built iconvconfig. Some form of cross-iconvconfig would be needed to generate / update such a file in general for cross builds.
  • The toplevel Makefile runs ldconfig for native builds. Cross-ldconfig is, like cross-localedef, something potentially of use when cross-building a system's root filesystem image. A version was once implemented at CodeSourcery for an old glibc version; the target dependencies are substantially more complicated than for ldconfig, so it may not readily be possible to make an ldconfig binary work generically for all architectures, and as with localedef the code has a lot of glibc dependencies so portability to a range of build systems may also be hard.

  • malloc/Makefile disables memusagestat when cross-compiling: "Unless we get a test for the availability of libgd which also works for cross-compiling we disable the memusagestat generation in this situation.". It should be possible to implement such a test - it wouldn't work in bootstrap situations, but could be used if a glibc build has been installed, then libgd built against that installed glibc, and now glibc is being built again. But really this is a program that should be split out into a separate package (glibc-memusage, or glibc-utils) - released alongside glibc, maybe, but built on its own after glibc is built and installed.

Distribution merging

It is desirable to merge in changes from the various distributions of glibc or files based on glibc and to reduce divergence between those versions.

Code Quality

Cleanup

  • Add and use macros to do align-up and align-down a value or contents of a buffer: http://sourceware.org/ml/libc-alpha/2012-04/msg00006.html

    • This does not imply that all of the code be changed at once to use such macros. The idea here is to introduce these macros with a change and then mandate their use in future changes.
  • Further cleanup of the Linux specific <bits/fcntl.h> headers so that struct flock does not need to be declared for each and every architecture. It should be part of the generic bits/fcntl-linux.h.

  • Several other Linux bits/*.h headers should be refactored similarly to how much of bits/fcntl.h went into fcntl-linux.h, to reduce the amount of content in architecture-specific files. This includes at least: ipc.h mman.h msq.h sem.h shm.h siginfo.h.
  • Look at refactoring bits/mathdef.h as far as possible as suggested in http://sourceware.org/ml/libc-alpha/2012-02/msg00003.html

  • If all targets include a generic x86 version of the file, move that file into generic (since it is) and delete all target versions of the file.
  • The abstractions involved in _G_config.h are no longer relevant now libio is only used in glibc not libstdc++; it may be possible to eliminate that header completely, but in any case it involves unnecessary levels of indirection.
  • Enable -Werror by default (with appropriate -Wno- or -Wno-error= options if any warnings cannot reasonably be cleaned up).
  • bug 14 (closed) identifies some latent bugs in getdents regarding nbytes, which should be fixed or filed as separate bugs in Bugzilla.

  • The Makeconfig file contains many variables in the "These are the configuration variables." section that are not in fact variables users should ever configure. That section should contain only variables that it is genuinely appropriate for users to configure. The other variables there should be separated into those that sysdeps makefiles may override but users should not, and those that are not intended to be overridden at all.
  • Review cases where code in glibc (including the various programs installed with glibc) uses non-large-files interfaces such as fopen or __open internally, as such uses may mean large files are not supported on 32-bit systems in some cases.

  • libc architectures should be more like ports architectures regarding putting configuration information in sysdeps files instead of architecture-independent files. Specifically:
    • Architecture cases in configure scripts.
    • Architecture cases in elf/sotruss-lib.c.
    • shlib-versions (it may be necessary to keep e.g. powerpc32 and powerpc64 cases in the same file to build gnu/lib-names.h correctly, but beyond such sharing it should be possible to move most entries to sysdeps shlib-versions files).
    • Architecture cases in the generic Linux kernel-features.h.
    • Split up config.h.in (bug 14068).

  • Develop a consistent policy for handling assertions in glibc code (maybe enable them by default but disable with --disable-assert).
  • Unify the code for copyright and bug-reporting messages in miscellaneous argp-using programs installed by glibc, as suggested in http://sourceware.org/ml/libc-alpha/2012-11/msg00261.html

  • Existing code shared by various miscellaneous programs installed by glibc (xmalloc and xstrdup) comes from locale/programs, even when the programs have nothing to do with locales. A separate programs toplevel directory for such code might be better.

  • Go through all files of ten lines or more without FSF copyright and license notices, and add such notices if the files are in fact FSF copyright or if there are significant FSF-copyright changes to a file that has some other copyright and license notice.
  • A 2013-02-19 checkout of the master branch has about 6500 files using License along with the GNU C Library; if not, see, and about 1000 using License along with the GNU C Library.  If not, see. Probably these should be unified, using the standard GNU wording (to be determined).

  • Stop using wildcards in symbol names in Versions files (stdlib/Versions and wcsmbs/Versions), instead naming all relevant symbols explicitly.
  • Where makefiles use -fexceptions, -fasynchronous-unwind-tables or $(exceptions), make them use makefile variables that relate to the logical reason those options are needed (e.g. "may be cancelled"), as described in http://sourceware.org/ml/libc-alpha/2012-11/msg00147.html - and more generally, develop a way to check for consistency issues where a function using callbacks isn't marked as such, or where a function may be cancelled or uses callbacks but a caller of that function isn't marked accordingly (in some cases that may be OK, however).

  • Where glibc headers include each other with __need_* macros defined, replace that construct with smaller bits/*.h headers with exactly the relevant subset of definitions.

  • The minimum (documented) supported Linux kernel version is 2.6.16 final release, so some bits of compatibility only relevant for 2.6.16-rcN releases but not for 2.6.16-final could be removed.
  • Remove #if conditionals in glibc relating to __NR_* macros for system calls being defined, where those macros will always be defined (for all architectures using the relevant file) for headers from Linux 2.6.19.1 and later. We could also considering increasing the minimum kernel header version to something rather more recent, and them eliminating more such conditionals.

  • Review the public ABI for symbols starting with '_' and make them only compatibility symbols, not available for new links, unless there is a good reason they need to be in the public ABI for new links (e.g. use from macros or inline functions in current headers), as described at http://sourceware.org/ml/libc-alpha/2012-12/msg00232.html - this includes the unwinder symbols exported at version GCC_3.0 for old binaries, as well as symbols exported at GLIBC_* versions.

  • Consider deprecating strfry and memfrob (making them only compatibility symbols) as not useful in practice.
  • Convert old-style function definitions to ISO C prototype definitions. It should be possible to script most of such a conversion using heuristics and regular expressions, then add -Wold-style-definition to the default warning options to catch any other cases.

  • Cleanup feature macros: The original intent of all the __USE_* macros was that they would always be used singly. That's why we have __USE_MISC at all instead of just using "#if defined __USE_BSD || defined __USE_SVID" everywhere. That is, all the headers would only ever have simple "#ifdef __USE_FOO" conditionals (never nested), specifically so that features.h is the one and only place where all the complex interrelationships have to be understood. A cleanup should consider removing old macros like _BSD_SOURCE (see http://sourceware.org/ml/libc-alpha/2013-04/msg00355.html for more details)

  • Use <intprops.h> within glibc for integer overflow checks (see http://sourceware.org/ml/libc-alpha/2013-05/msg00549.html)

  • Consolidate the makefile variables elfobjdir and elf-objpfx into a single variable. Use that variable where $(common-objpfx)elf is used at present.

  • Remove --enable-oldest-abi (bitrotten, see bug 6652).

Enhancing malloc

Split to separate page

Support initialization of non-POD thread-local objects

See: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=27557

We need to coordinate glibc support.

Enhance LD_DEBUG to report IFUNC bindings

IRC, freenode, #glibc, 2013-06-20:

<ryanarn> So does anyone have any ideas for a runtime test that can be
  performed to verify that multi-lib/IFUNC support is selecting the proper
  code paths on particular hardware?  Think regression testing, not debug
  testing.
<f11> on the hardware? if it's using some specific instructions, you may be
  able to use perf or similar to count them
<f11> or just script gdb
<ryanarn> f11: I thought of the gdb method.. I suppose if I give the
  testers some scripts.
<ryanarn> actually I like the perf method too.
<ryanarn> f11: thanks
<pasky> ryanarn: can't you simply call the ifunc and check the pointer it
  returns?
<ryanarn> pasky: yes, but I imagine that'll require gdb as well to get the
  start addresses of the libraries whereby I can then determine the address
  of the ifunc selected functions..
<ryanarn> hrm... of the binary itself might be able to do some
  introspection on it's own /proc/pid/maps
<AJaeger> I was just checking whether there'S some LD_DEBUG output for this
  - but couldn't find any.
<AJaeger> Perhaps we need to enhance LD_DEBUG to report ifunc bindings?
<tschwinge> AJaeger: That makes sense to me, yes.
<pasky> ryanarn: well you could just export the symbol of the ifunc as a
  normal function too
<ryanarn> pasky: this is for regression testing of libraries that I don't
  build personally... so doing that would have to be acceptable to do
  upstream.
<ryanarn> but it'd certain be ideal to be able to do something like that
<pasky> ah i see.. hmm, do you at least have debugging symbols?
<ryanarn> pasky: probably in a debug info library.

Support auditing IFUNC resolver functions

See: http://sourceware.org/bugzilla/show_bug.cgi?id=14843

Explore using STT_GNU_IFUNC to support data-dependent function resolvers

The use of STT_GNU_IFUNC is currently limited to a single global decision at application startup when the set of IRELATIVE functions are processed. Even if this were lazy, once processed, the GOT entry referenced by the PLT (at least in the x86_64 implementation) would be set with no easy way to change it. There is an interesting use case where the resolver's decision is data-dependent e.g. signal processing FFT of 4096 points vs. FFT of 8 points. It would be interesting to investigate how we could use the STT_GNU_IFUNC functionality to support redoing the resolver and IRELATIVE relocation based on new information available to the resolver. There is no easy way to support this except with an API directly into the dynamic loader that allows you to re-run the IRELATIVE relocation handling for symbol in question. The alternatives to STT_GNU_IFUNC are: large switch statement with inlined copies, and function pointers; each of which have their own problems.

Locale Voting

Government documents often allow multiple characters to be used for things like thousands separators or decimal points. It would be interesting to try the following:

  • Use the glibc locales to create a set of webpages.
  • Wire the webpages up to a voting system that allows users to vote on alternatives and correctness of each part of the locale.
  • Gather data from voting.
  • Incorporate feedback into the locales.

Thus we have crowd-sourced the review of the locales.

Strict and relaxed locales

As soon as you allow crowd-sourcing new locales you need to have a way for governments and schools to select either the strict conforming locale or the more liberal one. We should support some notion of strictness within the locale data and expose a knob to set the locale behaviour e.g. strict or not-strict. In strict mode the locale adheres to government norms, in relaxed mode it adheres to the crowd-sourced data.

Locale archive alignment

Locale archive generation in localedef does page size (4096) alignment as an optimization as explained at https://sourceware.org/ml/libc-alpha/2013-09/msg00764.html - but it is not clear if this is actually beneficial; the effects of this alignment should be analyzed for different choices of the page size used for the writer and the reader.

libm projects

libm itself

  • On 32-bit x86, float and double functions should not return values with excess range or precision (which generally means storing return values to memory then loading before return to avoid such excess range and precision). The approaches for fixing this may be different for .c and .S implementations, and there should also be some form of testcase.
  • Throughout glibc, use classification / comparison macros such as isnan and isfinite instead of the internal functions such as __isnan or __finite that they call, unless there is a specific reason the internal functions need to be used directly.

  • A file k_rem_pio2l.c is automatically built in libm; some systems use the math/ version, with a stub warning, and others use an empty version, but no system actually needs this file. An empty version should be used everywhere, to save the libm space taken up by the stub math/ version on systems using that version.

  • pow(+-0, -inf) should ideally not raise the divide-by-zero exception. Current code in glibc may be inconsistent between different implementations; libm-test.inc treats this exception as permitted but optional in this case, following C11, but as per WG14 N1515 proper IEEE semantics indicate the exception should not be raised, so the code and testsuite should be made to follow that.

  • Various rint and nearbyint implementations were simplified and cleaned up in early 2012 for bug 2547 and bug 2548. The same cleanups should be applied to the ldbl-96 versions of these functions, or, if in fact those functions are unused because all ldbl-96 systems have their own versions, they should be removed.

  • Where x86 and x86_64 .S versions of libm functions are similar, it may be useful to share the source files with appropriate conditionals in them.

  • It may make sense for x86 and x86_64 expm1l to handle large arguments and setting errno directly, instead of via calling __expl.

  • The C implementations of fmod functions appear suboptimal: they could use __builtin_clz operations for subnormals instead of a loop, where the processor has efficient clz support, and the repeated-subtraction loop may be less efficient than approaches using integer mod (where there is hardware support for that) and repeated squaring, where the exponents are widely different. (Before doing much on this, check what processors would actually benefit.)

  • There are various problems with the errno setting for overflow and underflow through wrappers. Where the wrappers check the result using __finite, this does not detect overflows in directed rounding modes where the result of overflow is the largest finite value of appropriate sign, and does not detect underflows at all. In some cases, it may be the case that the largest finite value can only arise through overflow, so that could be checked for as well, and it may be possible to determine when a subnormal or zero result indicates underflow (as opposed to an exact result). Setting errno through __kernel_standard in cases of subnormal results will require passing the original result to __kernel_standard so it can be properly preserved; this may also help with preserving overflowing results for directed rounding.

  • Various routines "optimize" access to constants by storing them in static const variables. Creating an explicit C object like this disables a GCC optimization to store literal constants into .rodata.cstn mergable sections, so that common constants (e.g. pi/2) can be shared between compilation units.

libm itself (major new features)

The following would be huge projects.

  • Support for ISO 24747 special mathematical functions.
  • Support for the first part of draft ISO C bindings to IEEE 754-2008 (WG14 N1774 or any more recent version if available).
  • Integrate libdfp into the glibc repository and update it to N1775 or any more recent version of the proposed updated bindings.
  • Support the third part of the IEEE 754-2008 bindings (N1758 or any more recent version), so superseding GCC's libquadmath (make the ldbl-128 functions usable both for long double, and for _Float128 as a type with different representation from long double).

libm testsuite

  • Go through the bugs closed as duplicates of bug 14759 for functions not being correctly rounded, and make sure the testcases in question, or other testcases showing ulps at least as big, are in libm-test.inc so the known errors get automatically documented in the manual.

  • Review libm-test.inc cases permitting but not requiring exceptions. EXCEPTIONS_OK should no longer be needed.

  • libm-test.inc should test INEXACT exceptions (both that they are present when required, and that they are absent when not permitted) for fma and cases where input and output are quiet NaNs (other cases are already tested).

  • For functions such as fma, sqrt and rint with fully defined IEEE semantics, libm-test.inc should have a way to specify that functions should have no ulps at all, so ulps don't get accidentally added to libm-test-ulps files.

  • The tests in pow_test_data in libm-test.inc should be sorted into a more logical order; there may be duplicates to remove.

  • libm-test.inc should test nearbyint with all the inputs used to test rint. Both should have tests for bug 13848 if not already present.

  • libm-test.inc should test nexttoward and nextafter more thoroughly, including overflow, underflow and either argument being infinity.

  • Review "XXX" comments in libm-test.inc.

  • Replace conditions in libm-test.inc on such macros as TEST_FLOAT and TEST_LDOUBLE with testing the relevant features from macros such as MANT_DIG or MAX_EXP instead.

  • Ideally, errno tests in libm-test.inc should be implied by the exception tests rather than needing to be given explicitly at all, although lots of bugs about not setting errno need to be fixed first.

  • libm-test.inc tests should also be run for -lieee, except for tests of errno setting. (There is no need to test the combination of -lieee and inlines.) More generally, run tests for all error-handling modes.

  • libm-test.inc tests with finite inputs and results should be run for -ffinite-math-only, to test the __*_finite functions.

  • For most cases where libm-test.inc tests specific finite inputs to functions, and associated results, it would make sense for those tests to be generated automatically with MPFR and MPC, with a source file just specifying the inputs, and a program using MPFR and MPC automatically determining the expected results, exceptions and errno values for each of the five supported floating-point formats. This could also be done for all rounding modes automatically, facilitating running all tests in all rounding modes; where inputs are not exactly representable for all floating-point formats, tests could be generated using all the rounded versions of those inputs. (The output generated using MPFR and MPC would probably still be checked in, so that sufficiently recent versions of those libraries are only needed for modifying the tests, not just for testing glibc.) Until errno setting is more reliable, the test source might need to contain indications of which functions should be expected to set errno in which cases.

  • Systematically review coverage in libm-test.inc for the following cases for each function (some of which may result in bugs that need to be filed and fixed before the relevant tests can be added):

    • Large and small inputs of either sign, including subnormals (and expected underflow exceptions for inexact subnormal results).
    • When overflow is possible, inputs (for each floating-point format) just on either side of overflow (inputs that don't quite overflow, and inputs that just overflow).
    • Likewise, for underflow (both values either side of producing subnormal results, and values either side of the rounded result being zero).
    • Also systematically test overflow and underflow cases for directed rounding, which should produce, as appropriate for the rounding direction, either infinity or the largest finite value with the appropriate sign, and either zero or the least subnormal with the appropriate sign.
  • It would be useful to have random test generation for all libm functions to search for cases of inaccurate results (more than about 10ulps, or any ulps for fully-defined functions that should always be correctly rounded) or exceptions or errno settings, for all functions (both real and complex) and all floating-point formats (float, double, long double), in all four rounding modes, by comparing with expected results from MPFR and MPC. This would probably be an external project, used to find and report bugs in glibc where not already filed in Bugzilla, rather than a direct part of the glibc testsuite. The random tests would need running on a range of systems, both because of multiarch variants of libm functions and because all three long double formats (ldbl-96, ldbl-128, ldbl-128ibm) should be tested for bugs in this way, as well as architecture-specific versions of various functions.

NPTL

  • Some improvements (such as the use of FUTEX_WAIT_BITSET, FUTEX_*REQUEUE_PI) have been added to x86 code only. Need to make analogous changes to the generic code so that architectures that use them can benefit from these improvements. While this is partly done already, there's still more to go.

New functions and features

  • Implement aswprintf, vaswprintf, getwdelim, getwline (ISO C TR 24731-2).
  • Implement scanf hooks mechanism for use by libraries such as libdfp to extend scanf.
  • Implement C11 threads (bug 14092).

  • Provide DWARF with proper types for parameters of syscall stubs. Current

Breakpoint 1, open64 () at ../sysdeps/unix/syscall-template.S:82
  • should display:

Breakpoint 1, open64 (pathname=0x3dcb21b7d4 "/etc/ld.so.cache", flags=524288) at ../sysdeps/unix/syscall-template.S:82
  • Use different libpthreadlibthread_db ABI than the "_thread_db_*" .symtab ELF symbols. Distros/people commonly strip all shared libraries and libthread_db no longer works for them.

Architecture-specific

* Update architectures that are out of date or out of sync with other architectures. See http://sourceware.org/ml/libc-alpha/2012-12/msg00319.html for a list of issues.

Power

Fixup dl-trampoline.S

Save VRS in fpu dl-trampoline.S

  • Need to save non-volatile VRS in:
    • sysdeps/powerpc/powerpc32/dl-trampoline.S
    • sysdeps/powerpc/powerpc64/dl-trampoline.S

Move dl-trampoline.S

  • Currently in:
    • sysdeps/powerpc/powerpc32/dl-trampoline.S
    • sysdeps/powerpc/powerpc64/dl-trampoline.S
  • Todo: Move to:
    • sysdeps/powerpc/powerpc32/fpu/dl-trampoline.S
    • sysdeps/powerpc/powerpc64/fpu/dl-trampoline.S

Add nofpu version of dl-trampoline.S

  • Current versions of powerpc dl-trampoline.S use FP regs and are picked up by soft-float build.
  • Currently only a problem when building a profile build.
  • Todo: Clone and modify for nofpu which doesn't use FP regs in:
    • sysdeps/powerpc/powerpc32/nofpu/dl-trampoline.S
    • sysdeps/powerpc/powerpc64/nofpu/dl-trampoline.S

Use Cache-Line Size Querying String Routines in ld.so

  • When building a toolchain for the Power Architecture it is standard to build with -mcpu=power4 (or later) for the base. This populates lib[64]/ with power4 tuned versions of ld[64].so and libc.so, et al as the 'base' libraries.

  • Following this, optimized versions of libc.so, libm.so, et al are put into lib[64]/power-foo/ but not an optimized version of ld[64].so.

  • The 'base' ld[64].so is used always used. It loads the optimized lib[64]/power-foo/ libraries at application runtime.

  • This can be problematic when 'base' is built with -mcpu=power4 which has a 128-byte cache-line size but ld[64].so is running on power-foo which has a 64-byte cache-line size.

  • In this case the base ld[64].so using the power4 optimized string routines makes an incorrect assumption of the cache-line size of power-foo.

  • To solve this the dynamic link loader shouldn't make an assumption of cache-line size in the string routines it uses. The dynamic-linker should always be built using the cache-line-size querying string routines. Then the optimized versions of libc.so can use the hard-coded cache-line size.

  • Currently building with a base of -mcpu=power4 will select:
    • sysdeps/powerpc/powerpc32/power4/memcpy.S
    • sysdeps/powerpc/powerpc64/power4/memcpy.S
  • This is okay for libc.so but ld[64].so needs to have the file selection hard coded to use:
    • string/memcpy.c

Make Malloc Return Type Alignment Honor ISO C-spec

  • This is required by the PowerPC ABIs as well for the Long Double 128, _Decimal128, and Vector Scalar data-types. Unfortunately the impacts are into common code.
  • See bug 6527. The changes to generic code are present, but the correct alignment is enabled only for x32, not powerpc32.

    • Rejection Rationale: Ulrich Drepper says this breaks EMACS.
  • Suggested approach:
    • Add version wrappers around the old versions of the function.
    • Create new versions of the function which follow the ISO C Spec and psABIs where the sizeof the malloc return type is aligned with the size of the largest supported data-type.
    • Make sure EMACS explicitly uses the old versions.

Improve Time Performance

  • unix/time.c calls gettimeofday the linux/time.c uses the time syscall
  • gettimeofday will use the vdso
  • for vdso platforms the unix/time.c is better then the linux/time.c
  • copy sysdeps/unix/sysv/linux/sparc/sparc64/time.c to sysdeps/unix/sysv/linux/powerpc/powerpc64/time.c
  • for extra credit create a linux/powerpc/time.c that calls vdso directly.
  • vdso availability is based on a kernel feature. The gettimeofday implementations call INLINE_VSYSCALL(gettimeofday). The INLINE_VSYSCALL macro checks for vdso availability. Lacking that is calls the slower syscall.

Improve rand/random Performance

  • For 'random' calls of TYPE_0 the sequence:
      if (buf->rand_type == TYPE_0)
        {
          int32_t val = state[0];
          val = ((state[0] * 1103515245) + 12345) & 0x7fffffff;
          state[0] = val;
          *result = val;
        }
    • doesn't need to be protected by a heavy weight lock and unlock when a compare and swap will suffice.
  • Todo: In stdlib/random.c:
      __libc_lock_lock (lock);
      (void) __srandom_r (x, &unsafe_state);
      __libc_lock_unlock (lock);
  • Add a condition check for TYPE_0:
       if (x->rand_type == TYPE_0)
        {
          /* Compare and Exchange and set '&unsafe_state'  */
        } else {
          __libc_lock_lock (lock);
          (void) __srandom_r (x, &unsafe_state);
          __libc_lock_unlock (lock);
        }

Platform specific implementations that use hardware acceleration when it is faster(AES instruction..)

Optimize Wrappers For Libm

For P5 and P6 inline the isnan, isinf etc tests into the wrapper functions (w_pow.c, w_exp.c, w_log.c, w_sin.c ....). For P7 inline the ftdiv instruction, especially for w_pow as it needs to test 2 operands for isnan(), ifinite(), sign,....

Optimize itoa_word using DFP Instructions for P6 and P7

  • Converting into to DPD then DPD to packed then converting to string will be faster than the multiply/divide by 10.

Optimize math functions for P7: __ieee754_exp, ___ieee754_pow, __dubsin

  • Verify where we can gain some performance in these functions using P7 instructions.

Thread Priority

  • Per ISA 2.06: Section 3.1 (page 671) titled "Program Priority Registers", The 'normal' process priority level has changed from "001 medium" to "100 medium low".
  • Reference "Program Priority Registers per ISA 2.06".

  • Leveraging this change allows two levels of performance improvement.
    1. Kernel Change - Make priority changes permanent. Without the kernel patch the priority will return to the old normal on a context switch.
    2. GLIBC change - Optimize locking loops to increase priority when lock is taken, decrease when lock is tried but missed, and return priority when lock is released.
      • If there is no accompanying kernel change the GLIBC patch is still beneficial. If a thread is in a critical section and the critical section is long enough that a context switch takes place the critical section is probably too long anyway so the return to the old-normal priority is OK.

Build Wide Character Strings Functions With -O3 Optimization

Memcpy Optimizations for 32-bit and 64-bit Cell

Fix Excessive Implies Files

  • Removing excessive Implies files from the powerpc sysdep directories should return the search order to some semblance of hierarchical sanity.

Remove no-fpu Context Routine Save and Restore of fprs

  • Currently the context routines reside in powerpc/powerpc[32|64]/ and they contain saving and restoring of fprs and the fpscr.

  • This breaks soft-float. This should be fixed so that the fpr and fpscr save and restore are moved into the sysdeps powerpc/powerpc[32|64]/fpu directory.

MIPS

  • Review all sysdeps/unix/sysv/linux/wordsize-64 files for cases where n64 (which doesn't use sysdeps/unix/sysv/linux/wordsize-64) needs something fixed.
  • n64 should use sysdeps/ieee754/dbl-64/wordsize-64/; n32 should use it except for lround / llround.
  • Add localplt baselines or otherwise adapt or skip the localplt tests for MIPS.
  • Add CFI information to assembly sources without it - see http://sourceware.org/ml/libc-ports/2013-02/msg00006.html for a list.

i386

  • We no longer support i386 and therefore i486 should become the default selection. In order to make i486 the default we will need to merge the i486 sysdep directory down one into the generic i386 directory. Unfortunately the i386 directory will no longer be i386, which is why there is another todo entry to rename from i386 to ix86 or x86/ia32.

  • We no longer support i386, but still all of our configuration systems and naming for 32-bit builds is called i386. This should be changed to ix86 or x86. The problem with changing to x86 is that x86-64 and x32 use x86 as a common shared directory. The linux kernel uses x86/ia32 for 32-bit x86 code and we could follow suit to allow kernel developers and other developers familiar with kernel layout to more easily follow our tree layout. During the name transition the i486 implementation should be collapsed out of the i486 directory and moved into the generic ia32 directory, thus we would have i486 default with i586, i686 and i786 options.

None: Development_Todo/Master (last edited 2014-06-30 11:57:24 by FlorianWeimer)