Re: Thread-, Signal- and Cancellation-safety documentation

[Michael Kerrisk <mtk dot manpages at gmail dot com>]

[CC += Peng Haitao; Peng see
http://thread.gmane.org/gmane.comp.lib.glibc.alpha/30287 ]

Alexandre,


On Tue, Mar 26, 2013 at 5:49 AM, Alexandre Oliva <aoliva@redhat.com> wrote:
>
> I've (not so) recently started working on documenting these safety
> properties of glibc functions.  I believe now have enough of a patchset
> to get the discussion on form and content started.
>
> I introduced a few macros to document these properties.  I envision that
> they might expand differently in info and pdf docs, but currently
> they're the same.  Some of the *unsafe macros expand to nothing, because
> unsafe is the default, but I wanted to document that the property was
> actually assessed and what makes the function unsafe (in comments in the
> manual sources, but not in the manual formatted output).

Peng Haitao has been making an effort to document similar properties
in the man-pages set. (Unfortunately, I was very slow to follow up on
his work, though we have started making some progress now.) And in
some cases I believe he has also been writing patches to fix parts of
glibc behavior in this area. I wonder whether you might have helpful
information for one another.

In any case, I would be happy if you would CC me on your glibc
patches, so I can track them. I suspect Peng might also be CCed, but I
suppose that he will speak up if that is the case.

Thanks,

Michael


> One of the stumbling blocks I ran into was that setlocale (unlike
> uselocale) may change the global locale object and, even though the
> modification of this object is guarded by a rwlock, most uses are not.
> As a result, many functions were marked as unsafe, with “glocale” as the
> reason.  More recently, talking to Carlos about the consequences of this
> and debating how best to approach the need for some functions to be
> MT-Safe in spite of setlocale, we decided to document setlocale itself
> as unsafe, so that calling it after starting threads was documented as
> problematic.  Then, we could document the fast-path functions as MT-Safe
> even though they don't bother with locking.
>
> A similar “solution” was used for fsetlocking: since it may disable
> internal locking, it would render several functions MT-Unsafe (and also
> cause them to release locks they didn't take, or take locks and never
> release them).  By declaring fsetlocking itself MT-Unsafe, we avoid
> these problems (since then calling it after starting threads invokes
> undefined behavior), without making every function that optionally takes
> care of internal locking MT-Unsafe.  It would be possible, however, to
> make fsetlocking MT-Safer :-) by changing it hold the stream's recursive
> lock regardless of internal-locking status (to make sure all threads
> that are using the stream complete the operation underway, be it guarded
> by internal locking, be it explicit external locking--use without either
> is undefined).  This wouldn't make it AS-Safe, which would require all
> functions that optionally take the lock to load the status only once
> into an automatic variable, and use it to decide whether to take and
> release the lock, or neither, rather than re-reading the setting that
> may have been modified in a signal handler.  This is fixable but
> cumbersome and possibly not worth it, so I didn't implement this
> change.
>
> Another issue was how to enable threads and signal handlers to safely
> call fclose, given that it releases the memory area that contains the
> lock object that other threads might need for synchronization.
> Comparing it with e.g. freopen, that closes a stream and then opens it
> again without releasing the storage, and therefore concurrent and
> subsequent uses of the stream are acceptable, whereas for fclose any use
> of the stream after the call, even on other threads, invoke undefined
> behavior.  Since fclose takes the lock, it will likely be nice and yield
> to other threads that hold the stream's lock, or asked for it before,
> but even concurrent uses that happen to be unordered (in the virtual
> clock happens-before sense) except for the locking can be ruled as
> invoking undefined behavior, and therefore fclose could be documented as
> MT-Safe and AS-Safe (except for other details), with the understanding
> that only fclosing streams that are certainly not going to be used ever
> again (e.g., streams opened within the signal handler itself, or no
> longer usable by other threads) avoids undefined behavior and is
> therefore well-defined and safe.  In the end, fclose turned out to be
> AS-Unsafe for other reasons (a non-recursive gconv mutex that could
> self-deadlock), but the rationale, if agreed by the community as sound,
> may be reused in other settings.
>
> As for AC safety, a number of functions take locks in ways that suffice
> to get the locks released upon synchronous cancellation.  Unfortunately,
> that's not enough to ensure AC-safety of the locking: it is possible for
> the cancellation to take place before the lock is taken, and then the
> cleanup would attempt to released the lock anyway.  I referred to this
> problem in the safety notes as lockleak.  Other leaks, such as memory
> and file descriptor leaks, were not regarded as safety issues, but they
> were noted as well.
>
> Many functions could also leave streams in inconsistent or
> partially-updated states upon cancellation, which made me document them
> as AC-unsafe, and likewise they might encounter streams in such states
> if called from signal handlers that happened to interrupt functions that
> manipulated the same stream, which got marked as AS-unsafe.  These were
> justified with “badstream”.
>
> Another issue that came up was using/changing the floating point
> control/status word.  It seems that all ports that support floating
> point exceptions, rounding modes, etc, use hardware floating point, with
> thread-specific FP registers holding the control&status info, and having
> these registers preserved by the signal machinery just like all other
> registers.  Unfortunately, this is not the case of powerpc fpu
> emulation: it uses a global variable (which makes a number of functions
> mt-unsafe) that's not restored when signal handlers return control to
> the interrupted function (which makes them as-unsafe), but I documented
> them as safe anyway, because I regard the current behavior as a bug that
> ought to be fixed.
>
> Yet another issue of concern is the ability to extend printf with
> register_printf_function.  Besides any thread-safety issues that might
> be exposed by the registered functions themselves, the use of the arrays
> in which the extensions are stored is not thread safe.  We have thus
> documented register_printf_function as unsafe, even though it is
> properly guarded itself: it is not safe to call when other threads might
> be running printf.
>
>
> So far, I've documented all functions in arith.texi and stdio.texi
> (although some that I'd marked as “glocale” unsafe need revisiting to
> tell whether they're otherwise thread-safe), and a couple of
> dependencies elsewhere.
>
> I've uploaded my patchset to branch lxoliva/thread-safety-docs, and I
> intend to update the branch weekly or so, posting updated stgit patches
> here.
>
> Here's the initial patchset, with all the changes to the manual so far,
> plus comments that document the need for thread-specific and signal-safe
> emulated FP status/control words for ppc-nofpu.
>
> I'd appreciate feedback on the decisions described above, on the
> formatting of the notes in the manual, and on the patches themselves.  I
> hope to keep the ChangeLog entries short, going over one texi file at a
> time or so, rather than listing all the functions that get documentation
> notes in a patch.  Let the discussion begin! :-)
>
> Thanks,
>
>
>
>
> --
> Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
> You must be the change you wish to see in the world. -- Gandhi
> Be Free! -- http://FSFLA.org/   FSF Latin America board member
> Free Software Evangelist      Red Hat Brazil Compiler Engineer
>