Linuxthreads memory barrier bugs.

[Kaz Kylheku]

Alexander <TEREKHOV@de.ibm.com> just wrote me an e-mail about what looks
like a bug in pthread_once().  There is a READ_MEMORY_BARRIER()
missing in the case when the function bails when
it sees (*once == DONE).

Also Alexander pointed out that there is no memory barrier after
assignment of the DONE value to *once.   Now I did a code inspection of
pthread_once a long time ago specifically with regard to this point,
and verified that there was a memory barrier performed in
__pthread_lock(). However, Alexander's e-mail prompted me to redo
this inspection, and it turns out that __pthread_lock()
can now bail out without executing a memory barrier, if the caller
grabs the lock in the spin loop. (Another bug I introduced).

I think that the correct fix is to add the READ_MEMORY_BARRIER() and
WRITE_MEMORY_BARRIER() in the right places in pthread_once, and to add
a READ_MEMORY_BARRIER() before the return statement in the
__pthread_lock spin loop.

The rationale is that a read barrier is sufficient on entry to a
critical region, and pthread_once should take care of its own memory
consistency assumptions rather than assume that pthread_mutex_lock
provides a full write barrier.

I'm going to send a concrete patch in a subsequent e-mail.