Re: Question about sparse arrays/atomic operations in systemtap

[Josh Stone <jistone at redhat dot com>]

On 06/03/2014 06:34 AM, Alan Conway wrote:
> I have a question about systemtap. I have a script (attached) for
> measuring mutex contention (not the futex trick, I am trying to measure
> directly from pthread_mutex_* calls). Aside from the merits of the
> script, it has a problem which seems like it would be quite common, I'm
> wondering if there is a well known solution.
> 
> To summarize: I am keeping count of the number of threads that are
> attempting to lock a mutex at any given time but there is a race:
> 
> probe process("/lib64/libpthread.so.0").function("pthread_mutex_lock") {
>     if (++mutex_locks[$mutex] > 1) // Contended
>        ...
> probe process("/lib64/libpthread.so.0").function("pthread_mutex_unlock")
> {
>     // TODO: There is a race here:
>     // - this thread does --mutex_locks[$mutex] and sees 0
>     // - another thread hits the lock probe and does
> ++mutex_locks[$mutex]
>     // - this thread does delete mutex_locks[$mutex] and nukes a non-0
> value
>     // Consequence: we could miss contentions. However we can't drop
>     // the delete or we overrun the mutex_locks array.
>     if (--mutex_locks[$mutex] <= 0)
>       delete mutex_locks[$mutex];

All accessed globals are locked for the duration of the probe, so the
entire handler is effectively atomic.  I want to think more about what
might be your root cause, but there's shouldn't be any race.

> I think what I want is an atomic (decrement and delete if 0) operation
> on an array.
> 
> Another way to solve it (which might have more general use) would be to
> have a new array type: a 'sparse array' which automatically deletes
> values when they are set to the array default (0, "", empty statistic
> etc) Such an array type would be very useful for arrays that are
> potentially huge (e.g. an entry for every mutex in a process) but are
> mostly 0.
> 
> Is there already a solution to this problem in systemtap, or would the
> sparse array be a useful addition?

This might still be a useful feature for brevity and performance.  I'll
have to check, but ISTR the map runtime even has a flag to do this
already, so we'd just have to express that desire somehow.