Problem with multiprocessing module from Python
Jean-Pierre Flori
jpflori@gmail.com
Thu Oct 31 20:55:00 GMT 2013
Le Thu, 31 Oct 2013 14:41:14 -0400, Christopher Faylor a écrit :
> On Thu, Oct 31, 2013 at 06:27:39PM +0000, Jean-Pierre Flori wrote:
>>Le Wed, 30 Oct 2013 10:33:13 +0100, Corinna Vinschen a ??crit??:
>>
>>> On Oct 29 21:22, Jean-Pierre Flori wrote:
>>>> Le Tue, 29 Oct 2013 21:19:14 +0000, Jean-Pierre Flori a ??crit??:
>>>>
>>>> > Le Tue, 29 Oct 2013 16:59:35 -0400, Christopher Faylor a ??crit??:
>>>> >> If you want this fixed, the easiest way to get that to happen is
>>>> >> to post a simple test case which reproduces the problem. That is
>>>> >> not the code snippet that you sent. A real working example would
>>>> >> be required.
>>>> > Sorry about that.
>>>> >
>>>> > Here you go:
>>>> > """
>>>> > from multiprocessing import Pool
>>>> >
>>>> > def f(x): return x
>>>> >
>>>> > p = pool(2)
>>>> >
>>>> > p.map(f, [1, 2])
>>>> > """
>>>> And I managed to introduce a typo. The third line should read Pool,
>>>> so it is:
>>>> """
>>>> from multiprocessing import Pool
>>>>
>>>> def f(x): return x
>>>>
>>>> p = Pool(2)
>>>>
>>>> p.map(f, [1, 2])
>>>> """
>>>
>>> Works for me. I guess. At least, if I run the script, nothing
>>> happens:
>>>
>>> $ python x.py $
>>>
>>> Same on 32 and 64 bit Cygwin.
>>>
>>>
>>> Corinna
>>
>>I think I got to the bottom of this.
>>It seems the new implem of sem_getvalue in cgwin1.dll is the cause, see:
>>http://cygwin.com/ml/cygwin-patches/2013-q3/msg00006.html It may also
>>explain the random reproducibility if sval stays uninitialized or
>>something like that (I did not check it is the case though).
>
> I doubt that was the problem. More likely it is something related to
> the changes in thread.cc that followed that change.
>
> cgf
I must admit that was just a guess.
The thread.cc code changed whereas python code in Modules/
_multiprocessing/semaphore.c did not.
The python multiprocessing module (file semaphore.c) contains:
"""
if (sem_getvalue(self->handle, &sval) < 0) {
return PyErr_SetFromErrno(PyExc_OSError);
} else if (sval >= self->maxvalue) {
PyErr_SetString(PyExc_ValueError, "semaphore or lock "
"released too many times");
return NULL;
}
"""
Changing this to
"""
sval = sem_getvalue(self->handle, &sval);
if (sem_getvalue(self->handle, &sval) < 0) {
return PyErr_SetFromErrno(PyExc_OSError);
} else if (sval >= self->maxvalue) {
PyErr_SetString(PyExc_ValueError, "semaphore or lock "
"released too many times");
return NULL;
}
"""
to emulate the previous behavior of sem_getvalue seems to solve my problem
(and was easier than rebuilding cygwin1.dll).
Commit http://cygwin.com/cgi-bin/cvsweb.cgi/src/winsup/cygwin/thread.cc?
rev=1.286&content-type=text/x-cvsweb-markup&cvsroot=src
would be exactly what I need.
I'll just be waiting for 1.7.26.
Hopefully this will get sorted out.
Best,
JP
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
More information about the Cygwin
mailing list