Master Todo List

This is the master TODO list for the project all other TODO lists reference this list.

Smaller self-contained projects, where it is readily possible to assess whether they are completed and where it is uncontroversial what should be changed, should generally go in Bugzilla. Projects here may be less precisely defined and may not always have consensus on whether there is a genuine problem and how it should be fixed.

Contents

  1. Master Todo List
  2. Documentation
    1. POSIX Threads
    2. Missing Functions
    3. Internals documentation
  3. Bugs
    1. Reproducing
  4. Testing
    1. Infrastructure
    2. Untested interfaces.
    3. Test IFUNC implementations.
    4. We need help testing releases.
    5. Use test-skeleton.c
    6. conformtest improvements
    7. Miscellaneous testsuite issues
  5. Benchmarks
  6. Cross build / test improvements
  7. Date and Time API
    1. MT-Safe and TZ-aware functions
  8. Distribution merging
  9. Code Quality
  10. Security
  11. Cleanup
  12. Enhancing malloc
  13. Enhance LD_DEBUG to report IFUNC bindings
  14. Support auditing IFUNC resolver functions
  15. Explore using STT_GNU_IFUNC to support data-dependent function resolvers
  16. Locale Voting
  17. Strict and relaxed locales
  18. Locale archive alignment
  19. libm projects
    1. libm itself
    2. libm itself (major new features)
    3. libm testsuite
  20. NPTL
  21. New functions and features
  22. Configure based on compiler configuration
  23. Standards conformance
  24. Architecture-specific
    1. Power
      1. Fixup dl-trampoline.S
        1. Save VRS in fpu dl-trampoline.S
        2. Move dl-trampoline.S
        3. Add nofpu version of dl-trampoline.S
      2. Use Cache-Line Size Querying String Routines in ld.so
      3. Make Malloc Return Type Alignment Honor ISO C-spec
      4. Improve Time Performance
      5. Improve rand/random Performance
      6. Optimize Wrappers For Libm
      7. Optimize itoa_word using DFP Instructions for P6 and P7
      8. Optimize math functions for P7: __ieee754_exp, ___ieee754_pow, __dubsin
      9. Thread Priority
      10. Build Wide Character Strings Functions With -O3 Optimization
      11. Memcpy Optimizations for 32-bit and 64-bit Cell
      12. Fix Excessive Implies Files
      13. Remove no-fpu Context Routine Save and Restore of fprs
    2. MIPS
    3. i386
  25. Linux Kernel
    1. Improvements to GLibc to better support sandboxing
    2. setXid issues
    3. Synchronizing Headers
  26. nscd projects

Documentation

It's always good to start by writing documentation. It helps you learn the API and understand what problems real users face day-to-day when working with the interface.

POSIX Threads

At present the glibc manual has *no* pthread documentation, and we need to fix this. The complete project is large, but it should be broken up into chunks, each chunk doable in a quick sit-down hacking session.

Please note that you must not copy from other sources and for copyright purposes this must be your own work.

We should coordinate with linux man pages project.

Missing Functions

There are some functions that were added to glibc but not documented. We need to find them.

Again this work is incremental, find a function, add it to the manual.

Please note that you must not copy from other sources and for copyright purposes this must be your own work.

Internals documentation

Bugs

There are many bugs in the bug database for product=glibc. We need help managing these bugs.

Reproducing

We need help reproducing bugs. We need to know if they are real and if they apply to trunk.

Testing

Infrastructure

Untested interfaces.

Test IFUNC implementations.

We need help testing releases.

== User-submitted Testsuite Failures ==

User: Carlos O'Donell <carlos@systemhalted.org>
Build system: Ubuntu 11.04, x86, GCC 4.5.2, Binutils 2.21.0, Kernel 2.6.38-8-generic (32-bit).
{{{
make[2]: *** [/home/carlos/build/glibc-2.15/localedata/tst-xlocale1.out] Error 1
make[1]: *** [localedata/tests] Error 2
make[2]: [/home/carlos/build/glibc-2.15/posix/annexc.out] Error 1 (ignored)
make[2]: *** [/home/carlos/build/glibc-2.15/rt/tst-cpuclock2.out] Error 1
make[1]: *** [rt/tests] Error 2
make[2]: *** [/home/carlos/build/glibc-2.15/elf/tst-unique3lib.os] Error 1
make[2]: *** [/home/carlos/build/glibc-2.15/elf/tst-unique3lib2.os] Error 1
make[2]: *** [/home/carlos/build/glibc-2.15/elf/tst-unique3.o] Error 1
make[2]: *** [/home/carlos/build/glibc-2.15/elf/tst-unique4.o] Error 1
make[2]: *** [/home/carlos/build/glibc-2.15/elf/check-localplt.out] Error 1
}}}

Use test-skeleton.c

In file included from ../malloc/malloc.h:24:0,
                  from ../include/malloc.h:2,
                  from ../test-skeleton.c:22,
                  from tst-initializers2.c:54:
../include/stdio.h:111:15: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__getwc_unlocked’
cc1: warnings being treated as errors

conformtest improvements

Miscellaneous testsuite issues

Benchmarks

There are currently benchmarks for a variety of libm functions and string functions in the benchtests directory.

Cross build / test improvements

There are still at least the following issues with getting full feature parity of native and cross builds of glibc:

Date and Time API

MT-Safe and TZ-aware functions

The problem is that you're running a multithreaded server and you want to output a time stamp in the client's time zone. With glibc, that's a pain, since the TZ setting is process-wide, and if one of your threads sets TZ='Europe/Moscow' (say), then all your threads are affected. NetBSD addresses this by having functions like localtime_rz, which is like localtime_r except with an extra timezone_t argument. There are extra functions like tzalloc to create timezone_t objects. It is not known if there are any other systems with these primitives other than Minix 3, which uses this part of NetBSD userland. See: http://netbsd.gw.com/cgi-bin/man-cgi?localtime_r++NetBSD-current

It has been proposed to add these functions to the public-domain timezone code hosted by IANA, but nobody's gotten around to doing that yet.

Distribution merging

It is desirable to merge in changes from the various distributions of glibc or files based on glibc and to reduce divergence between those versions.

Code Quality

Security

Cleanup

Enhancing malloc

Split to separate page

Enhance LD_DEBUG to report IFUNC bindings

IRC, freenode, #glibc, 2013-06-20:

<ryanarn> So does anyone have any ideas for a runtime test that can be
  performed to verify that multi-lib/IFUNC support is selecting the proper
  code paths on particular hardware?  Think regression testing, not debug
  testing.
<f11> on the hardware? if it's using some specific instructions, you may be
  able to use perf or similar to count them
<f11> or just script gdb
<ryanarn> f11: I thought of the gdb method.. I suppose if I give the
  testers some scripts.
<ryanarn> actually I like the perf method too.
<ryanarn> f11: thanks
<pasky> ryanarn: can't you simply call the ifunc and check the pointer it
  returns?
<ryanarn> pasky: yes, but I imagine that'll require gdb as well to get the
  start addresses of the libraries whereby I can then determine the address
  of the ifunc selected functions..
<ryanarn> hrm... of the binary itself might be able to do some
  introspection on it's own /proc/pid/maps
<AJaeger> I was just checking whether there'S some LD_DEBUG output for this
  - but couldn't find any.
<AJaeger> Perhaps we need to enhance LD_DEBUG to report ifunc bindings?
<tschwinge> AJaeger: That makes sense to me, yes.
<pasky> ryanarn: well you could just export the symbol of the ifunc as a
  normal function too
<ryanarn> pasky: this is for regression testing of libraries that I don't
  build personally... so doing that would have to be acceptable to do
  upstream.
<ryanarn> but it'd certain be ideal to be able to do something like that
<pasky> ah i see.. hmm, do you at least have debugging symbols?
<ryanarn> pasky: probably in a debug info library.

Support auditing IFUNC resolver functions

See: http://sourceware.org/bugzilla/show_bug.cgi?id=14843

Explore using STT_GNU_IFUNC to support data-dependent function resolvers

The use of STT_GNU_IFUNC is currently limited to a single global decision at application startup when the set of IRELATIVE functions are processed. Even if this were lazy, once processed, the GOT entry referenced by the PLT (at least in the x86_64 implementation) would be set with no easy way to change it. There is an interesting use case where the resolver's decision is data-dependent e.g. signal processing FFT of 4096 points vs. FFT of 8 points. It would be interesting to investigate how we could use the STT_GNU_IFUNC functionality to support redoing the resolver and IRELATIVE relocation based on new information available to the resolver. There is no easy way to support this except with an API directly into the dynamic loader that allows you to re-run the IRELATIVE relocation handling for symbol in question. The alternatives to STT_GNU_IFUNC are: large switch statement with inlined copies, and function pointers; each of which have their own problems.

Locale Voting

Government documents often allow multiple characters to be used for things like thousands separators or decimal points. It would be interesting to try the following:

Thus we have crowd-sourced the review of the locales.

Strict and relaxed locales

As soon as you allow crowd-sourcing new locales you need to have a way for governments and schools to select either the strict conforming locale or the more liberal one. We should support some notion of strictness within the locale data and expose a knob to set the locale behaviour e.g. strict or not-strict. In strict mode the locale adheres to government norms, in relaxed mode it adheres to the crowd-sourced data.

Locale archive alignment

Locale archive generation in localedef does page size (4096) alignment as an optimization as explained at https://sourceware.org/ml/libc-alpha/2013-09/msg00764.html - but it is not clear if this is actually beneficial; the effects of this alignment should be analyzed for different choices of the page size used for the writer and the reader.

libm projects

libm itself

libm itself (major new features)

The following would be huge projects.

libm testsuite

NPTL

New functions and features

Breakpoint 1, open64 () at ../sysdeps/unix/syscall-template.S:82

Breakpoint 1, open64 (pathname=0x3dcb21b7d4 "/etc/ld.so.cache", flags=524288) at ../sysdeps/unix/syscall-template.S:82

Configure based on compiler configuration

How glibc configures itself, regarding ABI choice and CPU optimizations, should be based more on how the compiler ($CC $CFLAGS $CPPFLAGS) generates code, and less on the configured host triplet or configure options for selecting particular variants. See sysdeps/arm/preconfigure.ac for an example of the preferred approach. See https://sourceware.org/ml/libc-alpha/2013-04/msg00357.html and https://sourceware.org/ml/libc-alpha/2013-04/msg00393.html for discussion. Specific points include:

Standards conformance

Architecture-specific

* Update architectures that are out of date or out of sync with other architectures. See PortStatus for a list of issues.

Power

Fixup dl-trampoline.S

Save VRS in fpu dl-trampoline.S

Move dl-trampoline.S

Add nofpu version of dl-trampoline.S

Use Cache-Line Size Querying String Routines in ld.so

Make Malloc Return Type Alignment Honor ISO C-spec

Improve Time Performance

Improve rand/random Performance

Platform specific implementations that use hardware acceleration when it is faster(AES instruction..)

Optimize Wrappers For Libm

For P5 and P6 inline the isnan, isinf etc tests into the wrapper functions (w_pow.c, w_exp.c, w_log.c, w_sin.c ....). For P7 inline the ftdiv instruction, especially for w_pow as it needs to test 2 operands for isnan(), ifinite(), sign,....

Optimize itoa_word using DFP Instructions for P6 and P7

Optimize math functions for P7: __ieee754_exp, ___ieee754_pow, __dubsin

Thread Priority

Build Wide Character Strings Functions With -O3 Optimization

Memcpy Optimizations for 32-bit and 64-bit Cell

Fix Excessive Implies Files

Remove no-fpu Context Routine Save and Restore of fprs

MIPS

i386

Linux Kernel

This section documents issues where there are deficiencies in the kernel/userspace interface.

Improvements to GLibc to better support sandboxing

After a fork in a multithreaded program a number of things are messed up and so only AC-safe (async-signal-safe) functions can be relied upon with certainty. Furthermore, Linux drops capabilities after an execve to another program. To better support sandboxing features, GLibc needs to clarify and possible improve the async-signal-safety of a number of functions.

GLibc needs to document pthread_sigmask, prctl and unshare and note them as AS-safe. They are simple system call wrappers and are most likely AS-safe.

Privilege setting system calls such as setuid, and setgroups are important to sandbox and restrict the privileges of a child process. However, these system calls have to do some complicated setxid synchronization. As such, they can cause deadlocks in programs that use them after a fork in a multithreaded program. This problem has already affected at least one project and has probably affected more (see https://www.redhat.com/archives/libvir-list/2013-November/msg00577.html). This part of having GLibc better support sandboxing is basically another part of the setXid issues mentioned below.

It would be desirable to make functionality like initgroups that read configuration information to reset user state async-signal-safe but as they can potentially call into plugins that can do anything it is probably impossible to reasonably accomplish this and it is sort of out of scope.

setXid issues

POSIX requires that the segXid e.g. setgid functions set the id for the process as a whole. This means that any threads also need their ids changed. At present the kernel provides no way to do this atomically. In glibc this is implemented by iterating over all threads making them call signal handlers to call setXid themselves. This is error prone and failure could cause any number of threads to be left with the old gid. The security implications of this are stagering, but in 2004 it seemed that the kernel was unwilling to accept code to implement this (See https://sourceware.org/ml/libc-hacker/2004-09/msg00068.html).

We need to put together a set of patches to implement setXid in the kernel as an atomic operation to change the id of all threads under a given process.

Synchronizing Headers

Synchronize various kernel UAPI headers and their equivalent libc header so either one can be used and both can be included in any order. See the Synchronizing Headers wiki page.

nscd projects

None: Development_Todo/Master (last edited 2014-08-31 03:09:37 by StevenStewartGallus)