Re: Consensus on MT-, AS- and AC-Safety docs:

["Carlos O'Donell" <carlos at redhat dot com>]

On 11/30/2013 10:29 AM, Torvald Riegel wrote:
> On Fri, 2013-11-29 at 17:59 -0500, Carlos O'Donell wrote:
>> (1) Preliminary documentation.
>>
>> We need to make it clear that we are documenting a preliminary
>> set of notes on all the functions and their safety.
>>
>> To that end the consensus is to add "Preliminary:" to the
>> safety documentation that is going with each function.
> 
> [snip]

I should note that there are only 4 points requiring consensus.

I should also clarify that consensus is not unanimous agreement,
only that we have rough agreement and no forceful objections.

>> (5) MT-Safe definition is driven from the standards down.
>>
>> Consensus, despite some dissenting, is that the more accurate
>> definition of MT-safe must comes down from the standards
>> first before we re-review our code and documentation.
> 
> I agree that the definitive definition of MT-safe should come from the
> standards body.  Nonetheless, this is a separate question from whether
> we are precise when we document glibc's current behavior now.  We do the
> latter because we don't have the former.

I agree they are orthogonal topics, but we chose not to treat it
way. We are documenting based on the loose POSIX definition.

>> Torvald I am recording for the record that you have raised
>> your objection again to the use of MT-Safe without providing
>> a formal definition of safe that can be used to reason about
>> larger problems.
> 
> I'd like to clarify my intention (for the record :) ): I'd like to have
> a sufficiently precise definition to ensure that (a) users are unlikely
> to interpret a vague definition to mean more than we actually guarantee
> and (b) more conservative users don't doubt whether we actually
> guarantee something and thus will not make use of the guarantees we
> actually give.  (a) is an immediate risk, (b) is more of a long-term
> problem.
> 
> I don't think we necessarily need a literally formal definition; a good,
> verbose explaining full text is probably more accessible, provided that
> it gives users the same detail and, most importantly, *clarity* as a
> proper formal definition.
> 
> Besides this external perspective, we also have an internal reason for
> having sufficiently precise definitions: maintenance.  If I would have
> to continue Alex' work, I wouldn't really know what he had in his mind
> when documenting the functions.  A similar case would arise when I or
> somebody else would add a new function or modify an existing
> implementation: would it still qualify as MT-Safe (although one could go
> the safe route and conservatively tag it as MT-Unsafe always).  And this
> could hit Alex' as well because he can't rely on my understanding of
> MT-Safe being the same as his, so maintenance becomes more difficult for
> him too.

I agree that this is a problem with loosely defined MT-safety like the
definition used in POSIX.

>> This is the second time you have raised this concern, the
>> first being when we started the project. I acknowledge that
>> this is a serious issue, but that it will not be addressed
>> by this work.
> 
> If we can't clarify at this stage, can we at least let users know that
> there is a serious issue, and potential risk of misunderstandings?

We cannot clarify at this stage.

We chose the definition to match POSIX for this initial review.

> The "Preliminary: " prefix on the individual functions' annotations goes
> a long way towards that, so I definitely like that.  Could we also add
> something similar to our definitions?  For example, when we put in the
> disclaimer that this documents the current state and not an API
> guarantee, could we add:
> 
> "These are preliminary definitions and annotations, and certain aspects
> of the definitions are still under discussion and might be subject to
> clarification or change."

I have no objection to this.

> With something like that in place, we tell people to be cautious, but
> still it doesn't sound as if we're not going to guarantee anything
> useful in the future.

OK.

>> At present POSIX has no memory model, and no strict definition
>> of safe. Therefore this documentation project will not be 
>> adopting any more complex definition of safe other than what 
>> is in POSIX or roughly:
>> ~~~
>> MT-Safe functions are safe to call in the presence of 
>> other threads, i.e., that will behave as documented 
>> regardless of what other threads are doing, as long as 
>> they all refrain from invoking undefined behavior.
>> ~~~
>>
>> We previously agreed that the definition of MT-safe must
>> come down through the standards. POSIX must have a logical
>> definition of MT-safe before glibc adopts that logical
>> definition.
> 
> I agreed that the definition we finally want to adopt will have to come
> down from POSIX, including the form this definition takes.  This might
> be subtly different from what you seem to have in mind, though.
> 
>> That doesn't mean we can't start talking about it though,
>> and I've started a document here:
>> https://sourceware.org/glibc/wiki/Multi-threaded%20safety
> 
> Thanks.
> 
>> Please start a distinct thread to discuss this document
>> and it's refinement to allow glibc to eventually express
>> a full definition of MT-Safe. That work should be presently
>> orthogonal to the work of documenting against the current
>> loose POSIX definition.
>>
>> I know you don't like this, but it is the way that it is.
>> We need to drive this from the standards down, and for this
>> first documentation pass it is not feasible to apply a more
>> complex definition of MT-Safe.
> 
> I agree that it's orthogonal, which leaves us with the situation that we
> are documenting against the current *loose* POSIX definition.  While I
> would have hoped that we can get some non-loose (ie, precise but
> potentially conservative) definition (see above for the reasons), I also
> see that it might not be feasible to achieve this at this first
> documentation pass.

It is not feasible.

> But, can we then at least tell people that the POSIX definition is
> loose?  IOW, put the essence of this summary of yours into the docs, so
> that they don't have to find out about what we know by reading
> libc-alpha? ;)
> 
> Are there any objections to this safety measure?

I don't object to an expanded introduction saying that the definition
itself is up for revision.

Cheers,
Carlos.