Created attachment 5676 [details] Demonstration of file descriptor leak due to problem 1 The current approach to implementing pthread cancellation points is to enable asynchronous cancellation prior to making the syscall, and restore the previous cancellation type once the syscall returns. I've asked around and heard conflicting answers as to whether this violates the requirements in POSIX (I believe it does), but either way, from a quality of implementation standpoint this approach is very undesirable due to at least 2 problems, the latter of which is very serious: 1. Cancellation can act after the syscall has returned from kernelspace, but before userspace saves the return value. This results in a resource leak if the syscall allocated a resource, and there is no way to patch over it with cancellation handlers. Even if the syscall did not allocate a resource, it may have had an effect (like consuming data from a socket/pipe/terminal buffer) which the application will never see. 2. If a signal is handled while the thread is blocked at a cancellable syscall, the entire signal handler runs with asynchronous cancellation enabled. This could be extremely dangerous, since the signal handler may call functions which are async-signal-safe but not async-cancel-safe. Even worse, the signal handler may call functions which are not even async-signal-safe (like stdio) if it knows the interrupted code could only be using async-signal-safe functions, and having a thread asynchronously terminated while modifying such functions' internal data structures could lead to serious program malfunction. I am attaching simple programs which demonstrate both issues. The solution to problem 2 is making the thread's current execution context (e.g. stack pointer) at syscall time part of the cancellability state, so that cancellation requests received while the cancellation point is interrupted by a signal handler can identify that the thread is not presently in the cancellable context. The solution to problem 1 is making successful return from kernelspace and exiting the cancellable state an atomic operation. While at first this seems impossible without kernel support, I have a working implementation in musl (http://www.etalabs.net/musl) which solves both problems.
Created attachment 5677 [details] Demonstration of problem 2 This program should hang, or possibly print x=0 if scheduling is really wacky. If it exits printing a nonzero value of the volatile variable x, this means the signal handler wrongly executed under asynchronous cancellation.
It's been 5 months since I filed this bug and there's been no response. I believe this issue it important enough to at least deserve a response. From my perspective, it makes NPTL's pthread_cancel essentially unusable. I've even included a proposed solution (albeit not a patch). Getting a confirmation that you acknowledge the issue exists and are open to a solution would open the door for somebody to start the work to integrate the solution with glibc/NPTL and eventually get it fixed.
Ping. Now that there's some will to revisit bugs that have been long-ignored, is anyone willing to look into and confirm the problem I've reported here? I believe problem 2 is extremely serious and could lead to timing-based attacks that corrupt memory and result in deadlocks or worse. Problem 1 is also serious for long-lived processes with high reliability requirements that use thread cancellation, as rare but undetectable resource leaks are nearly inevitable and will accumulate over time.
I just added a detailed analysis of this bug on my blog at http://ewontfix.com/2/
There is interest in fixing this issue. It's just that it's complicated :-)
Let us see if I can't get resources to fix this for 2.19. We've seen some tst-cancel17 problems specifically around this issue where cancellation is delivered between syscall return and error storage and that causes problems.
Glad to hear that. Have you taken a look at musl's cancellation implementation? The same mechanism could be used in glibc, or I think it could be modified somewhat to use DWARF2 CFI instead of the asm labels. The basic approach is that the cancellation signal handler examines the saved program counter register and determines whether it's in the critical range starting just before the pre-syscall check of the cancellation flag and the syscall instruction (based on asm labels for these two endpoints). The kernel then handles the atomicity of side effects for us: if the signal interrupts the syscall, the kernel must either complete what it's doing and return (positioning the program counter just past the address range that would allow cancellation to be acted upon), or reset the program counter to just before the syscall instruction and setup the register contents for restarting after the signal handler (in which case cancellation can be acted upon).
(In reply to Rich Felker from comment #7) > Glad to hear that. Have you taken a look at musl's cancellation > implementation? The same mechanism could be used in glibc, or I think it > could be modified somewhat to use DWARF2 CFI instead of the asm labels. The > basic approach is that the cancellation signal handler examines the saved > program counter register and determines whether it's in the critical range > starting just before the pre-syscall check of the cancellation flag and the > syscall instruction (based on asm labels for these two endpoints). The > kernel then handles the atomicity of side effects for us: if the signal > interrupts the syscall, the kernel must either complete what it's doing and > return (positioning the program counter just past the address range that > would allow cancellation to be acted upon), or reset the program counter to > just before the syscall instruction and setup the register contents for > restarting after the signal handler (in which case cancellation can be acted > upon). I have not looked at musl's cancellation implementation. I assume you are parsing the rt_sigframe set down by the Linux kernel and extracting information from that?
> I have not looked at musl's cancellation implementation. > > I assume you are parsing the rt_sigframe set down by the Linux kernel and > extracting information from that? We are using the ucontext_t received via the third argument to the SA_SIGINFO type signal handler.
(In reply to Rich Felker from comment #9) > > I have not looked at musl's cancellation implementation. > > > > I assume you are parsing the rt_sigframe set down by the Linux kernel and > > extracting information from that? > > We are using the ucontext_t received via the third argument to the > SA_SIGINFO type signal handler. Same thing. Thanks.
Alex Oliva and I talked about this particular issue today. We believe that an entirely userspace solution is possible without assistance from the kernel, but it requires signal wrappers. Signal wrappers are code that execute before and after a signal handler and does things like save and restore errno (the one use we have for them currently). The signal wrappers would assist in handling deferred cancellation. The proposed solution would look like this: * Stop enabling/disabling asynchronous cancellation around syscalls. * When a blocking library function who is also a cancellation point is entered a word in the thread's TCB (call it IN_SYSCALL) is set to the value of the stack pointer (we assume no further stack adjustments are made before the function exits). The value of IN_SYSCALL is cleared just before the function returns. Deferred cancellation is still checked before and after the syscall. * Add a signal wrapper to all signals that checks to see if IN_SYSCALL == SP stored in the ucontext_t and if it does it immediately cancels the thread. The check is done upon entry and exit of the wrapper to reduce cancellation latency. Just before unwinding the IN_SYCALL value is cleared. * When a thread starts we install a SIGCANCEL (SIGRTMIN) handler like we did before, but this handler checks to see if the thread's IN_SYSCALL matches the SP stored in ucontext_t, indicating that cancellation was requested while executing in the cancellation region of a blocking syscall (and no other signal handler executing). In that case the signal handler cancels the thread immediately. If IN_SYSCALL != SP then another signal handler is running and we defer the cancellation to the signal wrapper or syscall wrapper. The SIGCANCEL handler operates as it previously did when asynchronous cancellation was enabled. Resolved use cases: - Cancellation delivered between first instruction of function and IN_SYSCALL set: Syscall wrapper code will check for cancellation and act upon it. - Cancellation delivered between IN_SYSCALL set and syscall: The SIGCANCEL handler will immediately cancel the thread. - Cancellation delivered between syscall and clearing IN_SYSCALL: The SIGCANCEL handler will immediately cancel the thread. - Cancellation delivered between clearing of IN_SYSCALL and function return: The next cancellation point will act upon the cancellation (still meets POSIX requirement given escape clause of "The thread is suspended at a cancellation point and the event for which it is waiting occurs"). - Cancellation delivered and thread stopped at syscall is executing multiple nested signal handlers and the first signal handler has not checked IN_SYSCALL yet: Only the first signal delivered will have IN_SYSCALL == SP be true. The SIGCANCEL handler will do nothing. The first signal handler's wrapper will detect the cancellation is active and act upon it as it exits (only after all the other signal handlers have completed). - Cancellation delivered and thread stopped at syscall is executing multiple nested signal handlers and the first signal handler is exiting and has already checked IN_SYSCALL: The syscall will be interrupted and return. The syscall wrapper will act upon the cancellation request. The goal here is to have the signal handlers finish executing without interruption. Unresolved use cases: - Related to bug 14147 -- Cancellation delivered while thread is blocked on an async-safe function (in fact it's only executing async-safe functions during the time a signal can be delivered for this to be valid) and executing a signal handler that longjmp's out of the function. In this case IN_SYSCALL is still set to SP and not cleared. If by luck SP ends up the same, and another thread delivers a cancellation request the SIGCANCEL handler will immediately cancel the thread even though it was not in a cancellation region. - What if you are executing fork and someone tries to cancel you? A potential resolution to the first unresolved use case is to use a cleanup handler to reset IN_SYSCALL since such a handler is run when longjmp unwinds the frames. However we then need to consider cancellation during the execution of the cleanup. I haven't fully thought through what to do with the forking a multithreaded program case, but we should try to see if we can make it work. Note: Setting IN_SYSCALL must be atomic.
Your proposed solution is a lot more complex and invasive than mine; it's actually almost equivalent to the first-generation solution I used in musl for the problem, which turned out to be a bad idea, and thus got scrapped. Most importantly, aside from being complex and ugly, it does not actually solve the worst problem, because this case is wrong: "Cancellation delivered between syscall and clearing IN_SYSCALL: The SIGCANCEL handler will immediately cancel the thread." In this case, unless the syscall failed with EINTR, you must not act on the cancellation request. Doing so is non-conforming to the requirement that the side effects upon cancellation match the side effects on EINTR (which is just a fancy way of saying, approximately, that cancellation can only take place if the syscall has already done its job, e.g. closing a fd, transferring some bytes, etc.). In addition, I suspect your solution has further flaws like what happens when you longjmp out of a signal handler that interrupted an AS-safe syscall which is a cancellation point. These issues can be solved with more complexity (extra work in longjmp), but the solution I've proposed is much simpler and has no corner cases that are difficult to handle.
(In reply to Rich Felker from comment #12) > Your proposed solution is a lot more complex and invasive than mine; it's > actually almost equivalent to the first-generation solution I used in musl > for the problem, which turned out to be a bad idea, and thus got scrapped. Experience is knowing what not to do :-) > Most importantly, aside from being complex and ugly, it does not actually > solve the worst problem, because this case is wrong: I like it when we can talk concretely about use cases. > "Cancellation delivered between syscall and clearing IN_SYSCALL: The > SIGCANCEL handler will immediately cancel the thread." > > In this case, unless the syscall failed with EINTR, you must not act on the > cancellation request. Doing so is non-conforming to the requirement that the > side effects upon cancellation match the side effects on EINTR (which is > just a fancy way of saying, approximately, that cancellation can only take > place if the syscall has already done its job, e.g. closing a fd, > transferring some bytes, etc.). I was not aware of this requirement. Is this written in POSIX or did this come about from discussion with the Austin group around the problems with close() being cancelled? Can you provide a reference to this? > In addition, I suspect your solution has further flaws like what happens > when you longjmp out of a signal handler that interrupted an AS-safe syscall > which is a cancellation point. These issues can be solved with more > complexity (extra work in longjmp), but the solution I've proposed is much > simpler and has no corner cases that are difficult to handle. Could you propose your design as a glibc wiki page so that we can look at it and critique it? I'd be happy to adopt your solution, but I Want to review it and put it through the same kind of use-cases as we discussed here.
From XSH 2.9.5 Thread Cancellation: "The side-effects of acting upon a cancellation request while suspended during a call of a function are the same as the side-effects that may be seen in a single-threaded program when a call to a function is interrupted by a signal and the given function returns [EINTR]. Any such side-effects occur before any cancellation cleanup handlers are called." This is the important paragraph. By requiring that the side-effects on cancellation match the side effects on EINTR, the standard requires that cancellation cannot be acted upon if other irreversible side effects have already taken place. For example, if a file descriptor has been closed, data transferred, etc. then cancellation can't happen. The following paragraph explains further: "Whenever a thread has cancelability enabled and a cancellation request has been made with that thread as the target, and the thread then calls any function that is a cancellation point (such as pthread_testcancel() or read()), the cancellation request shall be acted upon before the function returns." This is simple. If cancellation is already pending when a cancellation point is called, it must be acted upon. The next part is less clear: "If a thread has cancelability enabled and a cancellation request is made with the thread as a target while the thread is suspended at a cancellation point, the thread shall be awakened and the cancellation request shall be acted upon. It is unspecified whether the cancellation request is acted upon or whether the cancellation request remains pending and the thread resumes normal execution if: * The thread is suspended at a cancellation point and the event for which it is waiting occurs * A specified timeout expired before the cancellation request is acted upon." This is covering the case of blocking syscalls. If a cancellation request arrives during a blocking syscall, it's normally acted upon, but there's one race condition being described: it's possible that the "event being waited for" arrives just before the cancellation request arrives, but before the target thread unblocks. In this case, it's implementation-defined whether cancellation is acted upon (in which case, by the first paragraph, the event remains pending) or the event is acted upon (in which case the cancellation request remains pending). The reason for there being two bullet points above is that some blocking syscalls wait for either an event or a timeout (think of sem_timedwait or recv with a timeout set by setsockopt), and in that case, the timeout can also be 'consumed' and cause the cancellation to remain pending. Anyway, this race condition is the whole matter at hand here. The two possibilities allowed (again, due to the limitations imposed by the first paragrah) are acting on the event while leaving cancellation pending, or acting on cancellation while leaving the event pending. But glibc also has a race window where it can act on both the event, producing side effects (because the kernel already has), and act on cancellation. This makes it non-conforming and makes it impossible to use cancellation safely.
(In reply to Rich Felker from comment #14) Thanks for the recap. > Anyway, this race condition is the whole matter at hand here. The two > possibilities allowed (again, due to the limitations imposed by the first > paragrah) are acting on the event while leaving cancellation pending, or > acting on cancellation while leaving the event pending. But glibc also has a > race window where it can act on both the event, producing side effects > (because the kernel already has), and act on cancellation. This makes it > non-conforming and makes it impossible to use cancellation safely. So does this imply that the cancellation *must* happen at some point after errno is known? Thus if a cancellation arrives and we're already in the syscall there is nothing to do but let the syscall return and let the syscall wrapper handle the cancellation. That seems reasonable to me.
There are several points at which the cancellation signal could arrive: 1. Before the final "testcancel" before the syscall is made. 2. Between the "testcancel" and the syscall. 3. While the syscall is blocked and no side effects have yet taken place. 4. While the syscall is blocked but with some side effects already having taken place (e.g. a partial read or write). 5. After the syscall has returned. You want to act on cancellation in cases 1-3 but not in case 4 or 5. Handling case 1 is of course trivial, since you're about to do a conditional branch based on whether the thread has received a cancellation request; nothing needs to be done in the signal handler (but it also wouldn't hurt to handle it from the signal handler). Case 2 can be caught by the signal handler determining that the saved program counter (from the ucontext_t) is in some address range beginning just before the "testcancel" and ending with the syscall instruction. The rest of the cases are the "tricky" part but it turns out they too are easy: Case 3: In this case, except for certain syscalls that ALWAYS fail with EINTR even for non-interrupting signals, the kernel will reset the program counter to point at the syscall instruction during signal handling, so that the syscall is restarted when the signal handler returns. So, from the signal handler's standpoint, this looks the same as case 2, and thus it's taken care of. Case 4: In this case, the kernel cannot restart the syscall; when it's interrupted by a signal, the kernel must cause the syscall to return with whatever partial result it obtained (e.g. partial read or write). In this case, the saved program counter points just after the syscall instruction, so the signal handler won't act on cancellation. Case 5: OK, I lied. This one is trivial too since the program counter is past the syscall instruction already. What about syscalls that fail with EINTR even when the signal handler is non-interrupting? In this case, the syscall wrapper code can just check the cancellation flag when the errno result is EINTR, and act on cancellation if it's set. Note that an exception needs to be made for close(), where EINTR should be treated as EINPROGRESS and thus not permit cancellation to take place. BTW, I should justify why the signal handler should be non-interrupting (SA_RESTART): if it weren't, you would risk causing spurious EINTR in programs not written to handle it, e.g. if the user incorrectly send signal 32/33 to the process or if pthread_cancel were called while cancellation is disabled in the target thread. The kernel folks have spent a great deal of effort getting rid of spurious EINTRs (which cause all sorts of ugly bugs) and it would be a shame to reintroduce them. Also it doesn't buy you anything moving the cancellation action to the EINTR check after the syscall returns; the same check in the signal handler that handles case 2 above also handles the case of restartable syscalls correctly, for free.
(In reply to Rich Felker from comment #16) > There are several points at which the cancellation signal could arrive: > > 1. Before the final "testcancel" before the syscall is made. > 2. Between the "testcancel" and the syscall. > 3. While the syscall is blocked and no side effects have yet taken place. > 4. While the syscall is blocked but with some side effects already having > taken place (e.g. a partial read or write). > 5. After the syscall has returned. > > You want to act on cancellation in cases 1-3 but not in case 4 or 5. > Handling case 1 is of course trivial, since you're about to do a conditional > branch based on whether the thread has received a cancellation request; > nothing needs to be done in the signal handler (but it also wouldn't hurt to > handle it from the signal handler). Case 2 can be caught by the signal > handler determining that the saved program counter (from the ucontext_t) is > in some address range beginning just before the "testcancel" and ending with > the syscall instruction. > > The rest of the cases are the "tricky" part but it turns out they too are > easy: > > Case 3: In this case, except for certain syscalls that ALWAYS fail with > EINTR even for non-interrupting signals, the kernel will reset the program > counter to point at the syscall instruction during signal handling, so that > the syscall is restarted when the signal handler returns. So, from the > signal handler's standpoint, this looks the same as case 2, and thus it's > taken care of. > > Case 4: In this case, the kernel cannot restart the syscall; when it's > interrupted by a signal, the kernel must cause the syscall to return with > whatever partial result it obtained (e.g. partial read or write). In this > case, the saved program counter points just after the syscall instruction, > so the signal handler won't act on cancellation. > > Case 5: OK, I lied. This one is trivial too since the program counter is > past the syscall instruction already. Excellent. I like your idea then. It seems like a list of PC's using either markers or dwarf2 is the way to go here. > What about syscalls that fail with EINTR even when the signal handler is > non-interrupting? In this case, the syscall wrapper code can just check the > cancellation flag when the errno result is EINTR, and act on cancellation if > it's set. Note that an exception needs to be made for close(), where EINTR > should be treated as EINPROGRESS and thus not permit cancellation to take > place. We'll need a big disclaimer about close and a detailed comment. I know some of the details there, specifically that although EINTR has been returned the close will complete. > BTW, I should justify why the signal handler should be non-interrupting > (SA_RESTART): if it weren't, you would risk causing spurious EINTR in > programs not written to handle it, e.g. if the user incorrectly send signal > 32/33 to the process or if pthread_cancel were called while cancellation is > disabled in the target thread. The kernel folks have spent a great deal of > effort getting rid of spurious EINTRs (which cause all sorts of ugly bugs) > and it would be a shame to reintroduce them. Also it doesn't buy you > anything moving the cancellation action to the EINTR check after the syscall > returns; the same check in the signal handler that handles case 2 above also > handles the case of restartable syscalls correctly, for free. That makes sense.
*** Bug 260998 has been marked as a duplicate of this bug. *** Seen from the domain http://volichat.com Page where seen: http://volichat.com/adult-chat-rooms Marked for reference. Resolved as fixed @bugzilla.
I am confused but if the proposed fix for this bug is implemented than that means that my bug at https://sourceware.org/bugzilla/show_bug.cgi?id=17168 where I can't cancel FUTEX_WAITs would be automatically fixed right? I would have to do no extra effort to let my blocking system call be cancellable? So this proposed fix would have a side benefit of giving me cancellability for free? Or would this be a bug or at least a breaking change?
Steven, I don't think this bug is related to your issue. If bug 9712 (of which your 17168 seems to be a duplicate) is resolved by adding futex and glibc makes futex a cancellation point, THEN the resolution of this bug (12683) would make it safe to use cancellation with the futex function in a way that's race-free. I think this is a strong argument for resolving 9712 by adding futex: unless it's part of libc, there's no safe way to make it cancellable, because wrapping syscall() with async cancellation will introduce an application-level bug comparable to this bug.
Okay Rich Felker, I was confused because the implementation in Musl seems to look it'd would make syscall users that block cancellation points. So, if GLibc does something similar to your solution they would have to explicitly block cancels in the syscall function to preserve compatibility (and in the future GLibc might possibly consider moving over to making syscall automatically cancellable for blocking system calls but that'd be a separate issue)?
Steven, I'm not sure I understand what you're saying. This issue report is not about changing which syscalls/functions are cancellable. For the standard functions that is specified by POSIX, and for extensions, the natural choices were already made and changing them would be problematic to their users. The topic at hand is just fixing the mechanism by which cancellation is performed so that there are not race conditions. If your question is about the syscall() function that applications can use to make syscalls directly, there is no open issue for making it cancellable, and as above, changing this would be problematic. One could envision a request for a separate version of the syscall() function which is cancellable, but as far as I know nobody has requested this and I think it's a bad idea to be adding features that encourage applications to make syscalls directly (since this is usually non-portable between archs due to subtle differences in the calling conventions and other issues like whether the libc-level structs match the syscall-level ones for a given arch).
I am currently working on a fix based on musl implementation, which from comments #16 and #17 seems a good approach. My initial idea is to use PC markers instead of DWARF2, since it see it as more clean approach. However, this is require a lot of cleanup. I plan push implementations for powerpc64 and x86_64 and ask for arch maintainers for more arch specific work. I also plan to write a wiki page describing the work done and summarizing the discussion on this bug report.
(In reply to Adhemerval Zanella Netto from comment #23) > I am currently working on a fix based on musl implementation, which from > comments #16 and #17 seems a good approach. My initial idea is to use PC > markers instead of DWARF2, since it see it as more clean approach. However, > this is require a lot of cleanup. > > I plan push implementations for powerpc64 and x86_64 and ask for arch > maintainers for more arch specific work. I also plan to write a wiki page > describing the work done and summarizing the discussion on this bug report. The biggest problem with DWARF2 is the parser, and making it accessible from the signal handler. I strongly suggest using PC, and a list of exception regions generated from markers in the assembly (similar to kernel exception regions).
I think we have stubmled upon this bug, or something related to it. Can someone please confirm I'm on the right track here? We have a multithreaded server application which calls recv() and poll() from async cancellable threads, each thread handles a single connection with a master thread accpeting new connections and adding them to a job queue. More and more often now we are seeing the server lock up and on inspection two or more threads seem deadlocked in some race condition inside libc recv() and or poll(). One example here shows two back traces from gdb from the two threads that seemed deadlocked chewing 100% CPU: Thread 1 bt: #0 __pthread_disable_asynccancel () at ../nptl/sysdeps/unix/sysv/linux/x86_64/cancellation.S:98 #1 0x00007f895ba987fd in __libc_recv (fd=0, fd@entry=33, buf=buf@entry=0x7cada02b, n=n@entry=1024, flags=1537837035, flags@entry=16384) at ../sysdeps/unix/sysv/linux/x86_64/recv.c:35 #2 0x000000000040ec54 in recv (__flags=16384, __n=1024, __buf=0x7cada02b, __fd=33) at /usr/include/x86_64-linux-gnu/bits/socket2.h:44 [snip] Thread 2 bt: #0 0x00007f895ba987eb in __libc_recv (fd=fd@entry=31, buf=buf@entry=0x7ca5e02b, n=n@entry=1024, flags=-1, flags@entry=16384) at ../sysdeps/unix/sysv/linux/x86_64/recv.c:33 #1 0x000000000040ec54 in recv (__flags=16384, __n=1024, __buf=0x7ca5e02b, __fd=31) at /usr/include/x86_64-linux-gnu/bits/socket2.h:44 [snip] There can be more than two threads involved, but I'm unsure if it can happen with just one thread locked up, but it's always inside recv() or poll() and sometimes in __pthread_disable_asynccancel() within either of those. Could I work around this problem by changing the threads to syncronmous cancellable or try to work around the need to cancel the treads at all?
I don't think this is the bug you're seeing. If it were, use of async cancellation would only make it worse. But the symptoms you'd see from this bug would be things like side effects of a function having happened despite it getting cancelled. If you're seeing 100% cpu load from threads in recv, the most likely explanation is that the socket you're reading from is in EOF status (remote sending end closed), so that recv immediately returns zero. Repeatedly attempting to read in this situation would be an application bug, not anything related to glibc.
(In reply to Rich Felker from comment #26) > I don't think this is the bug you're seeing. If it were, use of async > cancellation would only make it worse. But the symptoms you'd see from this > bug would be things like side effects of a function having happened despite > it getting cancelled. > > If you're seeing 100% cpu load from threads in recv, the most likely > explanation is that the socket you're reading from is in EOF status (remote > sending end closed), so that recv immediately returns zero. Repeatedly > attempting to read in this situation would be an application bug, not > anything related to glibc. Thanks Rich, your suggestion made me think to look through the code paths again and you are quite right, there was an infinite loop in there, not obvious but I found it. In light of the current problems with cancellable threads and syscalls, I'm going to disable cancelation during the main job execution (where all the recv() can poll()) call are, just in case this bug is causing problems I'm unaware of. Many thanks, you saved me a lot of hair pulling :)
Back to the issue reported on the bug report, I am still working on a fix for this issue and I am currently rebasing my changes against master and checking if there is any missing issue that I need to take care of. I have decided to extend this fix in basically two subtasks that complement each other: 1. Consolidate all the cancellation syscall implementations on C code to remove the requirement of syscall-cancel.h header. This is orthogonal to cancellation fix and aims to make the architecture adjustment simpler (less assembly-macro code that emulates functions calls). 2. The cancellation (bz12683) fix itself which will require multiple patches for each architecture. For 1. I have a set of 33 patches, which 2 I already sent for review without reply [1] [2]. The rest is similar to consolidation patches I have been working to set a C implementation plus architecture defines to set the correct syscall usage. For 2. the patch is incomplete regarding architecture support. Based on current implementation, I have added fixes for only x86, powerpc, s390, aarch64, and arm. I summarize the steps required for each architecture fix at [3] and I plan to review it after the patch rebase I am working it. However for correct fix I am foreseeing some issues that I will need to take care when more architectures are fixed: 1. i386 syscall code when using cancellation is suboptimal using this cancellation mechanism due the not usage of vDSO. This has been discussed on LKML on how kernel could help userland to accomplish it, but afaik discussion was stalled. 2. Current code assumes maximum of 6 arguments syscalls for the helper bridge. This is true for most architecture, however afaik MIPS has 7 argument cancellable syscall, which will require code adjustment. To avoid more performance issues, ideally both the bridge function and related declaration should be arch-specific (6 arguments if not specific, otherwise 7). I am currently tracking some issues that have arise from code rebase and I will check on the architectures I have access (x86, powerpc64le, arm, aarch64). For 1. I also plan to have a full make/check using all the supported architectures and as soon I have all fixed I plan to update my sourceware branch [5] and re-start discussion for 2.26. My plan is at least to have 1. on 2.26. I will update the bugzilla with this report. [1] https://sourceware.org/ml/libc-alpha/2016-08/msg00443.html [2] https://sourceware.org/ml/libc-alpha/2016-08/msg00442.html [3] https://sourceware.org/glibc/wiki/PortStatus [4] https://lkml.org/lkml/2016/3/8/1105 [5] https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/azanella/bz12683
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "GNU C Library master sources". The branch, azanella/bz12683 has been created at d95b8c5f0fa8a8a2b99e63461df3fbe917990a5e (commit) - Log ----------------------------------------------------------------- https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=d95b8c5f0fa8a8a2b99e63461df3fbe917990a5e commit d95b8c5f0fa8a8a2b99e63461df3fbe917990a5e Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Tue Jan 24 21:05:56 2017 -0200 nptl: Remove THREAD_ATOMIC_* macros This patch removes the ununsed THREAD_ATOMIC_* macros now that nptl code is using C11 atomics. Checked on x86_64-linux-gnu and i686-linux-gnu. * manual/pattern.texi (THREAD_ATOMIC_BIT_SET): Remove. (THREAD_ATOMIC_CMPXCHG_VAL): Likewise. * nptl/pthreadP.h (THREAD_ATOMIC_CMPXCHG_VAL): Likewise. (THREAD_ATOMIC_BIT_SET): Likewise. * sysdeps/i386/nptl/tls.h (THREAD_ATOMIC_CMPXCHG_VAL): Likewise. (THREAD_ATOMIC_AND): Likewise. * sysdeps/x86_64/nptl/tls.h (THREAD_ATOMIC_CMPXCHG_VAL): Likewise. (THREAD_ATOMIC_AND): Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=fdafb04cb8269446e056273bcd923a4186113790 commit fdafb04cb8269446e056273bcd923a4186113790 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Tue Jan 24 09:17:37 2017 -0200 nptl: cancelhandling refactor This patch basically uses glibc C11 atomic function to access and modify struct pthread cancelhandling variable. All plain access are handled using the atomic_load_* functions and THREAD_ATOMIC_BIT_SET is replaced by atomic_fetch_or_acquire. Checked on i686-linux-gnu, x86_64-linux-gnu, x86_64-linux-gnux32, aarch64-linux-gnu, arm-linux-gnueabihf, and powerpc64le-linux-gnu. * nptl/descr.h (CANCELED_BITMASK): Rename to THREAD_CANCELED. (EXITING_BITMASK): Rename to THREAD_EXITING. (TERMINATED_BITMARK): Rename to THREAD_TERMINATED. * nptl/allocatestack.c (setxid_mark_thread): Use THREAD_* instead of *_BITMASK and use glibc C11 atomic function to access pthread cancelhandling. * nptl/libc-cancellation.c (__syscall_cancel): Likewise. * nptl/nptl-init.c (sigcancel_handler): Likewise. * nptl/pthread_cancel.c (pthread_cancel): Likewise. * nptl/pthread_create.c (__free_tcb): Likewise. (START_THREAD_DEFN): Likewise. (__pthread_create_2_1): Likewise. * nptl/pthread_detach.c (pthread_detach): Likewise. * nptl/pthread_exit.c (__pthread_exit): Likewise. * nptl/pthread_join.c (pthread_join): Likewise. * nptl_db/td_thr_get_info.c (td_thr_get_info): Likewise. * nptl_db/td_thr_getfpregs.c (td_thr_getfpregs): Likewise. * nptl_db/td_thr_getgregs.c (td_thr_getregs): Likewise. * nptl_db/td_thr_setfpregs.c (td_thr_setfpregs): Likewise. * nptl_db/td_thr_setgregs.c (td_thr_setgregs): Likewise. * sysdeps/unix/sysv/linux/arm/syscall_cancel.S (__syscall_cancel_arch): Change CANCELED_BITMASK to THREAD_CANCELED in comment. * sysdeps/unix/sysv/linux/syscall_cancel.c (__syscall_cancel_arch): Likewise. * sysdeps/x86_64/nptl/tcb-offsets.sym (TCB_CANCELED_BITMASK): Remove. (TCB_EXITING_BITMASK): Likewise. (TCB_TERMINATED_BITMASK): Likewise. * sysdeps/unix/sysv/linux/i386/syscall_cancel.S (__syscall_cancel_arch): Likewise. * sysdeps/unix/sysv/linux/powerpc/syscall_cancel.S (__syscall_cancel_arch): Likewise. * sysdeps/unix/sysv/linux/sh/syscall_cancel.S (__syscall_cancel_arch): Likewise. * sysdeps/unix/sysv/linux/sparc/sparc32/syscall_cancel.S (__syscall_cancel_arch): Likewise. * sysdeps/unix/sysv/linux/sparc/sparc64/syscall_cancel.S (__syscall_cancel_arch): Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=9037ec95f5d0e8ac5d846f3b2181e468a15e6819 commit 9037ec95f5d0e8ac5d846f3b2181e468a15e6819 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Tue Jul 18 16:33:41 2017 -0300 nptl: Remove setxid bit from cancelhandling This patch removes the setxid signaling out of cancelhandling flag and use its own member of pthread instead. The idea is 1 simplify setxid handling to now mix atomic operation with thread cancellation since both are orthogonal, 2. isolate cancelhandling variable to only track thread cancel state. It also uses glibc C11 atomic operation while accessing new setxid_op field. Checked on i686-linux-gnu, x86_64-linux-gnu, x86_64-linux-gnux32, aarch64-linux-gnu, arm-linux-gnueabihf, and powerpc64le-linux-gnu. * nptl/allocatestack.c (get_cached_stack): Set sexid_op on new allocated stack. (setxid_mark_thread): Use setxid_op to mark thread as executing a sexid operation insteaf of using cancelhandling. Also adapt algorithm to use C11 atomic analogous functions. (setxid_unmark_thread): Likewise. (setxid_signal_thread): Likewise. * nptl/nptl-init.c (sighandler_setxid): Likewise. * nptl/pthread_create.c (sighandler_setxid): Likewise. * nptl/descr.h (SETXID_BIT): Remove define. (SETXID_BITMASK): Likewise. (setxid_op): New member. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c3f33edb679e95ded02b15fd5fe8d760a759ac3c commit c3f33edb679e95ded02b15fd5fe8d760a759ac3c Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Fri Jan 20 17:24:35 2017 -0200 nptl: Move cancel state and type out cancelhandling This patch move both the cancel state (PTHREAD_CANCEL_ENABLE and PTHREAD_CANCEL_DISABLE) and cancel type (PTHREAD_CANCEL_DEFERRED and PTHREAD_CANCEL_ASYNCHRONOUS) out of cancelhandling member. The idea is to avoid atomic handling of cancelhandling where these states are concerned since with exclusive member they are not concurrent accessed: * pthread_setcancel{type,state} and internal functions __pthread_{un}register_cancel_defer (and its analogous compat versions) only change the type/state of the calling thread. * __syscall_cancel also only requires to check the state of calling thread. * sigcancel_handler is executed only for the thread about to be potentially canceled. * pthread_join deadlock checks only requires access the state for the calling thread. * Same behavior for pthread_testcancel. With this change the cancelhandling member from pthread struct is used solely for cancelation handling and setxid signaling. Checked on i686-linux-gnu, x86_64-linux-gnu, x86_64-linux-gnux32, aarch64-linux-gnu, arm-linux-gnueabihf, and powerpc64le-linux-gnu. * nptl/allocatestack.c (get_cached_stack): Set both cancelstate and canceltype on new stack allocation. * nptl/cleanup_defer.c (__pthread_register_cancel_defer): Set canceltype directly instead of using cancelhandling member. (__pthread_unregister_cancel_restore): Likewise. * nptl/cleanup_defer_compat.c (_pthread_cleanup_push_defer): Likewise. (_pthread_cleanup_pop_restore): Likewise. * nptl/descr.h (CANCELSTATE_BIT): Remove flag. (CANCELSTATE_BITMASK): Likewise. (CANCELTYPE_BIT): Likewise. (CANCELTYPE_BITMSK): Likewise. (CANCEL_RESTMASK): Likewise. (CANCEL_ENABLED_AND_CANCELED): Likewise. (CANCEL_ENABLED_AND_CANCELED_AND_ASYNCHRONOUS): Likewise. (cancelstate): New member. (canceltype): Likewise. * nptl/libc-cancellation.c (__syscall_cancel): Use cancelstate instead of cancelhandling member. * nptl/nptl-init.c (sigcancel_handler): Likewise. * nptl/pthreadP.h (__do_cancel): Likewise. (CANCELLATION_P): Likewise. * nptl/pthread_join.c (pthread_join): Remove CANCELLATION_P usage. * nptl/pthread_testcancel.c (__pthread_testcancel): Likewise. * nptl/pthread_setcancelstate.c (__pthread_setcancelstate): Use cancelstate member instead of cancelhandling. * nptl/pthread_setcanceltype.c (__pthread_setcanceltype): Use canceltype member instead of canceltype. * sysdeps/x86_64/nptl/tcb-offsets.sym (TCB_CANCELSTATE_BITMASK): Remove. (TCB_CANCELTYPE_BITMASK): Likewise. (TCB_CANCEL_RETMASK): Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=ca81de5ed9021e4b96e7e3b0ce49eea806f92460 commit ca81de5ed9021e4b96e7e3b0ce49eea806f92460 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Feb 26 16:24:15 2018 -0300 nptl: riscv: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the riscv modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function. The default syscall_cancel.c should be as expected for RISCV. With GCC 7.3.1 syscall_cancel.c generates the following code: 0000000000000000 <__GI___syscall_cancel_arch>: 0: 8346 mv t1,a7 0000000000000002 <__syscall_cancel_arch_start>: 2: 00052883 lw a7,0(a0) 6: 0048f893 andi a7,a7,4 a: 00089c63 bnez a7,22 <.L0 > e: 88ae mv a7,a1 10: 8532 mv a0,a2 12: 85b6 mv a1,a3 14: 863a mv a2,a4 16: 86be mv a3,a5 18: 8742 mv a4,a6 1a: 879a mv a5,t1 1c: 00000073 ecall 0000000000000020 <__syscall_cancel_arch_end>: 20: 8082 ret 22: 1141 addi sp,sp,-16 24: e406 sd ra,8(sp) 26: 00000097 auipc ra,0x0 2a: 000080e7 jalr ra # 26 <.L0 > Checked with a build for riscv64-linux-gnu with run-built-tests=no. [BZ #12683] * sysdeps/unix/sysv/linux/riscv/sigcontextinfo.h (ucontext_get_pc): New fuction. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=33154acabe37c1bc4b4c180229d7f120b512dab9 commit 33154acabe37c1bc4b4c180229d7f120b512dab9 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Fri Feb 3 21:07:07 2017 -0200 nptl: hppa: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the hppa modifications required for the BZ#12683. It basically adds the required __pthread_get_pc function. HPPA requires an arch-specific syscall_cancel because the INTERNAL_SYSCALL_NCS adds some instruction to fetch the returned syscalls value. The implementation were based on on default C version built with GCC 6.1 Checked on hppa-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/hppa/sigcontextinfo.h: New file. * sysdeps/unix/sysv/linux/hppa/syscall_cancel.S: Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=5f51f6b99358e17c177010167bd5dc56fecf1788 commit 5f51f6b99358e17c177010167bd5dc56fecf1788 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:03:07 2017 -0200 nptl: mips: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the mips modifications required for the BZ#12683. It adds the required ucontext_get_pc function, a mips32 cancellable syscall wrapper and 7 argument cancellable syscall support. To avoid code pessimization and add a requirement on all architectures to support {INLINE,INTERNAL)_SYSCALL with 7 argument, its support is added through a flag, HAVE_CANCELABLE_SYSCALL_WITH_7_ARGS, which changes the signature and prototype of the requires macros and functions (SYSCALL_CANCEL, __syscall_cancel and __syscall_cancel_arch). As default 6 arguments cancellable syscalls are use. MIPS o32 requires an arch-specific implementation because INTERNAL_SYSCALL_NCS adds an 'addiu' just after the syscall instruction which invalidates the checks on sigcancel_handler. Checked against a build and make check run-built-tests=no for mips-gnu-linux, mips64-linux-gnu, mips64-n32-linux-gnu. I also ran some basic o32 and n64 cancellation tests on a simulated mips64 qemu system. [BZ #12683] * nptl/libc-cancellation.c (__syscall_cancel): Define and use 7 argument syscall if architecture requires it. * nptl/pthreadP.h (__syscall_cancel_arch): Likewise. * sysdeps/unix/sysdep.h (__syscall_cancel, __SYSCALL_CANCEL*): Define with 7 argument if architecture requires it. (__SYSCALL_CANCEL7_ARG_DEF): New macro. (__SYSCALL_CANCEL7_ARG): Likewise. (__SYSCALL_CANCEL7_ARG7): Likewise. * sysdeps/unix/sysv/linux/syscall_cancel.c (__syscall_cancel_arch): Likewise. * sysdeps/mips/nptl/tls.h (READ_THREAD_POINTER): Check __mips_isa_rev existance for macro definition. * sysdeps/unix/sysv/linux/mips/sigcontextinfo.h (ucontext_get_pc): New function. * sysdeps/unix/sysv/linux/mips/mips32/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/mips/mips32/sysdep.h (HAVE_CANCELABLE_SYSCALL_WITH_7_ARGS): Define. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=e4db079ad7eea4a58dc6713dc2f8f1817e8de480 commit e4db079ad7eea4a58dc6713dc2f8f1817e8de480 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:03:03 2017 -0200 nptl: sh: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the sh modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function and a cancellable syscall wrapper. SH requires an arch-specific syscall_cancel because the INTERNAL_SYSCALL_NCS adds the required or instruction to workaround a hardware bug [1]. The implementation were based on on default C version built with GCC 6.2.1. Checked against a build and make check run-built-tests=no for sh4-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/sh/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/sh/sigcontextinfo.h (ucontext_get_pc): New function. [1] http://documentation.renesas.com/eng/products/mpumcu/tu/tnsh7456ae.pdf https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=793aeb3010780a615f2f4b2236cad3f1707e88e4 commit 793aeb3010780a615f2f4b2236cad3f1707e88e4 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:02:53 2017 -0200 nptl: nios2: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the nios2 modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function. The default syscall_cancel.c should be as expected for NIOS2. With GCC 6.2.1 syscall_cancel.c generates the following code: --- 00000000 <__GI___syscall_cancel_arch>: 0: defffe04 addi sp,sp,-8 4: dd800015 stw r22,0(sp) 8: 002ce03a nextpc r22 c: 02000034 movhi r8,0 10: 42000004 addi r8,r8,0 14: dfc00115 stw ra,4(sp) 18: b22d883a add r22,r22,r8 0000001c <__syscall_cancel_arch_start>: 1c: 20c00017 ldw r3,0(r4) 20: 18c0010c andi r3,r3,4 24: 18000f1e bne r3,zero,64 <__syscall_cancel_arch_end+0x18> 28: 3015883a mov r10,r6 2c: 2805883a mov r2,r5 30: da400517 ldw r9,20(sp) 34: 380b883a mov r5,r7 38: da000417 ldw r8,16(sp) 3c: d9c00317 ldw r7,12(sp) 40: d9800217 ldw r6,8(sp) 44: 5009883a mov r4,r10 48: 003b683a trap 0 0000004c <__syscall_cancel_arch_end>: 4c: 38000126 beq r7,zero,54 <__syscall_cancel_arch_end+0x8> 50: 0085c83a sub r2,zero,r2 54: dfc00117 ldw ra,4(sp) 58: dd800017 ldw r22,0(sp) 5c: dec00204 addi sp,sp,8 60: f800283a ret 64: b0800017 ldw r2,0(r22) 68: 103ee83a callr r2 --- Checked against a build and make check run-built-tests=no for nios2-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/nios2/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=14f629ccf56a06dba93cf6c1808496acb81edec8 commit 14f629ccf56a06dba93cf6c1808496acb81edec8 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:02:36 2017 -0200 nptl: sparc: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the sparc modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function, cancellable syscall wrappers, and a fix for pause. Sparc requires arch-specific syscall_cancel implementation because INLINE_SYSCALL_NCS uses the __SYSCALL_STRING (defined different for sparc32 and sparc64) and it issues additional instructions after the syscall one to check the resulting error code. When used in the default syscall_cancel.c implementation the label __syscall_cancel_arch_end is not placed just after the syscall as expected. Both 32 and 64 bits version were based on default C version built with GCC 6.1. Also, different than other architectures, SPARC passes the sigcontext_t struct pointer as third argument in the signal handler set with SA_SIGINFO (some info at [1]) for 64 bits and the pt_regs in 32 bits. From Linux code: * arch/sparc/kernel/signal_64.c 428 /* 3. signal handler back-trampoline and parameters */ 429 regs->u_regs[UREG_FP] = ((unsigned long) sf) - STACK_BIAS; 430 regs->u_regs[UREG_I0] = ksig->sig; 431 regs->u_regs[UREG_I1] = (unsigned long) &sf->info; 432 433 /* The sigcontext is passed in this way because of how it 434 * is defined in GLIBC's /usr/include/bits/sigcontext.h 435 * for sparc64. It includes the 128 bytes of siginfo_t. 436 */ 437 regs->u_regs[UREG_I2] = (unsigned long) &sf->info; * arch/sparc/kernel/signal_32.c: 392 regs->u_regs[UREG_FP] = (unsigned long) sf; 393 regs->u_regs[UREG_I0] = ksig->sig; 394 regs->u_regs[UREG_I1] = (unsigned long) &sf->info; 395 regs->u_regs[UREG_I2] = (unsigned long) &sf->regs; 396 397 regs->pc = (unsigned long) ksig->ka.sa.sa_handler; 398 regs->npc = (regs->pc + 4); So to access the signal mask in the signal frame, a arch-specific ucontext_get_mask is defined which obtain the sa_mask from the context. Checked on a SPARC T5 for sparc64-linux-gnu and sparcv9-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/sparc/sparc32/sigcontextinfo.h (ucontext_get_pc): New function. * sysdeps/unix/sysv/linux/sparc/sparc64/sigcontextinfo.h (ucontext_get_pc): Likewise. * sysdeps/unix/sysv/linux/sparc/sparc32/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/sparc/sparc64/syscall_cancel.S: Likwise. * sysdeps/unix/sysv/linux/sparc/sparc64/pause.c: New file. [1] https://www.spinics.net/lists/sparclinux/msg05037.html https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=9f21ab0e6f18e0f548d36a6f460b7ac6ae5f476e commit 9f21ab0e6f18e0f548d36a6f460b7ac6ae5f476e Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:02:28 2017 -0200 nptl: tile: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the tile modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function The default syscall_cancel.c should be as expected for TILE. With GCC 6.1 syscall_cancel.c generates the following code: * TILEGX 0000000000000000 <__GI___syscall_cancel_arch>: 0: c7bf46ce576bfd9d { move r29, sp ; addi r28, sp, -24 ; st sp, lr } 8: 283bf825401e0db6 { addi sp, sp, -32 ; move r10, r1 } 10: eeedf85b85c18d9b { addi r27, sp, 24 ; move lr, r2 ; st r28, r29 } 18: eef5f88155bbf0c1 { move r1, r3 ; move r2, r4 ; st r27, r30 } 20: 283bf8c25107f143 { move r3, r5 ; move r4, r6 } 28: 283bf8e2d1483000 { move r5, r7 } 0000000000000030 <__syscall_cancel_arch_start>: 30: 9c5e4000340c3000 { ld4s r11, r0 } 38: 18182165d1483000 { andi r11, r11, 4 } 40: 17c00163d1483000 { bnez r11, 78 <__syscall_cancel_arch_end+0x20> } 48: 283bfee051483000 { move r0, lr } 50: 286b180051485000 { swint1 } 0000000000000058 <__syscall_cancel_arch_end>: 58: 180906ced1401ff7 { subx lr, zero, r1 ; addi r29, sp, 32 } 60: 87b8c6ce4dd77040 { cmovnez r0, r1, lr ; addi r28, sp, 24 ; ld lr, r29 } 68: 9ef6400035cc3000 { ld r30, r28 } 70: 286a6ee040120db6 { addi sp, sp, 32 ; jrp lr } 78: 2000000051483000 { jal 78 <__syscall_cancel_arch_end+0x20> } * TILEGX32 00000000 <__GI___syscall_cancel_arch>: 0: cbbfc6ce576bfd9d { move r29, sp ; addxi r28, sp, -8 ; st sp, lr } 8: 283bf825402f0db6 { addxi sp, sp, -16 ; move r10, r1 } 10: eeedf860d5cbf0b7 { move lr, r2 ; move r1, r3 ; st r28, r29 } 18: 283bf8a1d107f102 { move r2, r4 ; move r3, r5 } 20: 283bf8e2d107f184 { move r4, r6 ; move r5, r7 } 00000028 <__syscall_cancel_arch_start>: 28: 9c5e4000340c3000 { ld4s r11, r0 } 30: 18182165d1483000 { andi r11, r11, 4 } 38: 17c0016351483000 { bnez r11, 68 <__syscall_cancel_arch_end+0x18> } 40: 283bfee051483000 { move r0, lr } 48: 286b180051485000 { swint1 } 00000050 <__syscall_cancel_arch_end>: 50: 28660fe540210d9d { addxi r29, sp, 16 ; subx r10, zero, r1 } 58: 9fbe40004dd4a040 { cmovnez r0, r1, r10 ; ld lr, r29 } 60: 286a6ee040210db6 { addxi sp, sp, 16 ; jrp lr } 68: 2000000051483000 { jal 68 <__syscall_cancel_arch_end+0x18> } Checked against a build and make check run-built-tests=no for tile-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/tile/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=5868f2b379c4dc9fed02585f5a9f6d41951a4b7f commit 5868f2b379c4dc9fed02585f5a9f6d41951a4b7f Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:02:17 2017 -0200 nptl: microblaze: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the microblaze modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function. Microblaze requires an arch specific assembly implementation because the archicture INTERNAL_SYSCALL_NCS implementation adds a nop after the brki instruction because the instruction expect a delay branch slot. I based this implementation on generated assembly using GCC 6.1. Checked against a build and make check run-built-tests=no for microblaze-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/microblaze/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/microblaze/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=a9d375373acbfc69dded5b12efbeef5c6ecdcf87 commit a9d375373acbfc69dded5b12efbeef5c6ecdcf87 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:02:08 2017 -0200 nptl: m68k: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the m68k modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function The default syscall_cancel.c should is as expected for m68k with GCC 6.2.1. It generates the following code for syscall_cancel.os: --- 00000000 <__GI___syscall_cancel_arch>: 0: 4e56 0000 linkw %fp,#0 4: 48e7 3c00 moveml %d2-%d5,%sp@- 00000008 <__syscall_cancel_arch_start>: 8: 206e 0008 moveal %fp@(8),%a0 c: 2010 movel %a0@,%d0 e: 0800 0002 btst #2,%d0 12: 6628 bnes 3c <__syscall_cancel_arch_end+0xa> 14: 206e 0024 moveal %fp@(36),%a0 18: 2a2e 0020 movel %fp@(32),%d5 1c: 282e 001c movel %fp@(28),%d4 20: 262e 0018 movel %fp@(24),%d3 24: 242e 0014 movel %fp@(20),%d2 28: 222e 0010 movel %fp@(16),%d1 2c: 202e 000c movel %fp@(12),%d0 30: 4e40 trap #0 00000032 <__syscall_cancel_arch_end>: 32: 4cee 003c fff0 moveml %fp@(-16),%d2-%d5 38: 4e5e unlk %fp 3a: 4e75 rts 3c: 61ff 0000 0000 bsrl 3e <__syscall_cancel_arch_end+0xc> --- Checked against a build and make check run-built-tests=no for m68k-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/m68k/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=5bd68272946551afa1d5578fc67b4acacf46e09a commit 5bd68272946551afa1d5578fc67b4acacf46e09a Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:01:58 2017 -0200 nptl: alpha: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the alpha modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function The default syscall_cancel.c should be fine for alpha and GCC 6.1 generates the following cod for syscall_cancel.os: --- 0000000000000000 <__GI___syscall_cancel_arch>: 0: 00 00 bb 27 ldah gp,0(t12) 4: 00 00 bd 23 lda gp,0(gp) 8: f0 ff de 23 lda sp,-16(sp) c: 00 04 f1 47 mov a1,v0 10: 00 00 5e b7 stq ra,0(sp) 0000000000000014 <__syscall_cancel_arch_start>: 14: 00 00 30 a0 ldl t0,0(a0) 18: 01 00 e1 43 sextl t0,t0 1c: 01 90 20 44 and t0,0x4,t0 20: 0f 00 20 f4 bne t0,60 <__syscall_cancel_arch_end+0x20> 24: 10 04 f2 47 mov a2,a0 28: 11 04 f3 47 mov a3,a1 2c: 12 04 f4 47 mov a4,a2 30: 10 00 9e a6 ldq a4,16(sp) 34: 13 04 f5 47 mov a5,a3 38: 18 00 be a6 ldq a5,24(sp) 3c: 83 00 00 00 callsys 0000000000000040 <__syscall_cancel_arch_end>: 40: 21 05 e0 43 negq v0,t0 44: 00 00 5e a7 ldq ra,0(sp) 48: c0 04 61 46 cmovne a3,t0,v0 4c: 10 00 de 23 lda sp,16(sp) 50: 01 80 fa 6b ret 54: 00 00 fe 2f unop 58: 1f 04 ff 47 nop 5c: 00 00 fe 2f unop 60: 00 00 7d a7 ldq t12,0(gp) 64: 00 40 5b 6b jsr ra,(t12),68 <__syscall_cancel_arch_end+0x28> 68: 1f 04 ff 47 nop 6c: 00 00 fe 2f unop --- Checked on alpha-linux-gnu, no regression found. [BZ #12683] * sysdeps/unix/sysv/linux/alpha/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=3a5a8aa8f6e665fb0fdf3ab6befbaa4a9b91a913 commit 3a5a8aa8f6e665fb0fdf3ab6befbaa4a9b91a913 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:01:44 2017 -0200 nptl: ia64: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the s390 modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function. a workaround for mismatched sigcontext::sa_flags defiition between kernel and GLIBC (tracked by BZ#21634), and an arch specific syscall_cancel implementation. IA64 requires an arch-specific syscall_cancel implemetantion because {INLINE,INTERNAL}_SYSCALL is implemented by branching to a gate DSO (similar to i386) which renders the pointer comparison in SIGCANCEL handler wrong. This incurs in performance penalty due the use of a break instruction insteaf of a eds one, however cancellable syscalls should potentially incur in syscalls blocking. Checked with a ia64-linux-gnu build with run-tests-built=no. [BZ #12683] * sysdeps/unix/sysv/linux/ia64/sigcontextinfo.h (ucontext_get_pc, ucontext_get_mask): New functions. * sysdeps/unix/sysv/linux/ia64/syscall_cancel.S: New file. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=d47c1d0fa0b4ce173b9fd06caaf7a999a6a78af8 commit d47c1d0fa0b4ce173b9fd06caaf7a999a6a78af8 Author: Adhemerval Zanella <adhemerval.zanella@linaro.com> Date: Wed Aug 12 10:51:38 2015 -0300 nptl: s390: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the s390 modifications required for the BZ#12683 fix. It basically adds the required ucontext_get_pc function. The built cancelable syscall wrapper for s390 using GCC 7.2.1 and default configuration flags shows the wrappers on expected corrected places: --- __GI___syscall_cancel_arch: .LFB39: .cfi_startproc stm %r6,%r15,24(%r15) .cfi_offset 6, -72 .cfi_offset 7, -68 .cfi_offset 8, -64 .cfi_offset 9, -60 .cfi_offset 10, -56 .cfi_offset 11, -52 .cfi_offset 12, -48 .cfi_offset 13, -44 .cfi_offset 14, -40 .cfi_offset 15, -36 ahi %r15,-96 .cfi_def_cfa_offset 192 .global __syscall_cancel_arch_start .type __syscall_cancel_arch_start,@function __syscall_cancel_arch_start: l %r0,0(%r2) tml %r0,4 jne .L5 lr %r1,%r3 lr %r2,%r4 lr %r3,%r5 lr %r4,%r6 l %r5,192(%r15) l %r6,196(%r15) l %r7,200(%r15) svc 0 .global __syscall_cancel_arch_end .type __syscall_cancel_arch_end,@function __syscall_cancel_arch_end: l %r4,152(%r15) lm %r6,%r15,120(%r15) .cfi_remember_state .cfi_restore 15 .cfi_restore 14 .cfi_restore 13 .cfi_restore 12 .cfi_restore 11 .cfi_restore 10 .cfi_restore 9 .cfi_restore 8 .cfi_restore 7 .cfi_restore 6 .cfi_def_cfa_offset 96 br %r4 .L5: .cfi_restore_state brasl %r14,__syscall_do_cancel .cfi_endproc --- The s390x version also shows similar placement: --- __GI___syscall_cancel_arch: .cfi_startproc stmg %r6,%r15,48(%r15) .cfi_offset 6, -112 .cfi_offset 7, -104 .cfi_offset 8, -96 .cfi_offset 9, -88 .cfi_offset 10, -80 .cfi_offset 11, -72 .cfi_offset 12, -64 .cfi_offset 13, -56 .cfi_offset 14, -48 .cfi_offset 15, -40 aghi %r15,-160 .cfi_def_cfa_offset 320 .global __syscall_cancel_arch_start .type __syscall_cancel_arch_start,@function __syscall_cancel_arch_start: l %r0,0(%r2) tmll %r0,4 jne .L5 lgr %r1,%r3 lgr %r2,%r4 lgr %r3,%r5 lgr %r4,%r6 lg %r5,320(%r15) lg %r6,328(%r15) lg %r7,336(%r15) svc 0 .global __syscall_cancel_arch_end .type __syscall_cancel_arch_end,@function __syscall_cancel_arch_end: lg %r4,272(%r15) lmg %r6,%r15,208(%r15) .cfi_remember_state .cfi_restore 15 .cfi_restore 14 .cfi_restore 13 .cfi_restore 12 .cfi_restore 11 .cfi_restore 10 .cfi_restore 9 .cfi_restore 8 .cfi_restore 7 .cfi_restore 6 .cfi_def_cfa_offset 160 br %r4 .L5: .cfi_restore_state brasl %r14,__syscall_do_cancel .cfi_endproc --- Checked on s390-linux-gnu and s390x-linux-gnu build with run-tests-built=no. [BZ #12683] * sysdeps/unix/sysv/linux/s390/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=4c686923b51f81cca01ad125f403b50fe05eeef4 commit 4c686923b51f81cca01ad125f403b50fe05eeef4 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Fri May 8 17:12:31 2015 -0300 nptl: arm: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the ARM modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function and adjust the generic syscall_cancel build. For ARM we need to build syscall_cancel in ARM mode (-marm) to avoid INTERNAL_SYSCALL to issue the syscall through the helper gate __libc_do_syscall (which invalidates the mark checks on SIGCANCEL handler). Checked on arm-linux-gnueabihf. [BZ #12683] * sysdeps/unix/sysv/linux/arm/Makefile (CFLAGS-syscall_cancel.c): New rule. * sysdeps/unix/sysv/linux/arm/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=1147b8b7f935fadfc4bc75f44bd50e5bebf4474e commit 1147b8b7f935fadfc4bc75f44bd50e5bebf4474e Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Wed May 6 17:51:29 2015 -0300 nptl: aarch64: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the aarch64 modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function. The built cancelable syscall wrapper for aarch64 using GCC 7.2.1 and default configuration flags shows an expected optimized version: --- __GI___syscall_cancel_arch: .LFB38: .cfi_startproc .global __syscall_cancel_arch_start .type __syscall_cancel_arch_start,@function __syscall_cancel_arch_start: ldr w9, [x0] tbnz x9, 2, .L7 mov x8, x1 mov x0, x2 mov x1, x3 mov x2, x4 mov x3, x5 mov x4, x6 mov x5, x7 svc 0 // syscall nr .global __syscall_cancel_arch_end .type __syscall_cancel_arch_end,@function __syscall_cancel_arch_end: ret --- A similar code is obtained with GCC 5.3.1, so I see to need to provide an arch-specific syscall_cancel.S for aarch64. Checked on aarch64-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/aarch64/sysdep-cancel.h (PSEUDO): Redefine to call __syscall_cancel function for cancellable syscalls. (__pthread_get_ip): Add implementation. * sysdeps/unix/sysv/linux/aarch64/sysdep.h (SYSCALL_CANCEL_ERROR): Add definition. (SYSCALL_CANCEL_ERRNO): Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c31e9f5f8549cfe2a3585d497d0764f2496c86a1 commit c31e9f5f8549cfe2a3585d497d0764f2496c86a1 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Fri Sep 18 18:14:19 2015 -0300 nptl: powerpc: Fix Race conditions in pthread cancellation (BZ#12683) This patches adds the powerpc64 modification required for the BZ#12683. It basically adds the required __pthread_get_pc function and a arch specific syscall_cancel implementation. The powerpc requires an arch-specific syscall_cancel because INTERNAL_SYSCALL_NCS adds a mfcr just after the sc instruction to get the CR0.SO bit information from kernel (which signals the error return status). So for cancelled syscalls with side effects, __pthread_get_pc will point to mcfr and thus invalidating the checks on sigcancel_handler. Checked on powerpc64le-linux-gnu and powerpc-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/powerpc/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/powerpc/sysdep-cancel.h (__pthread_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=3bf87610132f283253be992d73762c2792752c14 commit 3bf87610132f283253be992d73762c2792752c14 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon May 4 16:30:13 2015 -0300 nptl: i386: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the i386 modifications required for the BZ#12683. It basically provides the required ucontext_get_pc symbol, add the cancelable syscall wrapper and fix a thread atomic update macro. On i386 an arch-specific cancellation implementation is required because depending of the glibc configuration and underlying kernel the syscall may be done using a vDSO symbol (__kernel_vsyscall). By using the vDSO symbol the resulting PC value for an interrupted syscall points to an adress outside the expected markers in __syscall_cancel_arch. It has been discussed in LKML [1] on how kernel could help userland to accomplish it, but afaik discussion was stalled. Also, since glibc supports i486, the old 'int 0x80' should be used in the syscall wrapper. One option could make minimum default chip to pentium II (which implements sysenter) or add a runtime check on syscall_cancel.S to use 'int 0x80' or sysenter. Similar to x86_64, it also remove bogus arch-specific THREAD_ATOMIC_BIT_SET where it always reference to current thread instead of the one referenced by input 'descr' argument. Checked on i686-linux-gnu. [BZ #12683] * sysdeps/i386/nptl/tls.h (THREAD_ATOMIC_CMPXCHG_VAL, THREAD_ATOMIC_BIT_SET): Remove macro. * sysdeps/unix/sysv/linux/i386/Makefile [$(subdir) = elf] (sysdep-rtld_routines): Add libc-do-syscall object. * sysdeps/unix/sysv/linux/i386/lowlevellock.h (lll_wait_tid): Use cancellable futex syscall macro. * sysdeps/unix/sysv/linux/i386/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/i386/sigcontextinfo.h (ucontext_get_pc): New function. [1] https://lkml.org/lkml/2016/3/8/1105 https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c3619359a7a15ed73b7c5e7356173dcc2adf5d2c commit c3619359a7a15ed73b7c5e7356173dcc2adf5d2c Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Sat May 9 14:20:26 2015 -0300 nptl: x32: Fix Race conditions in pthread cancellation (BZ#12683) This patches adds the x32 modification required for the BZ#12683. It basically adjust the syscall size used to pass the arguments to the syscall cancel wrappers by zero extending pointers type while preserting values for default types (such as off_t). Checked on x86_64-linux-gnu-x32. [BZ #12683] * sysdeps/unix/sysv/linux/x86_64/x32/sysdep.h (__syscall_arg_t): Define type for x32. (__SSC): Add platform specific macro. * include/libc-pointer-arith.h (__integer_if_pointer_type_sub, __integer_if_pointer_type, cast_to_integer): Parametrize integer type cast. (cast_to_uinteger): New macro. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=13c8c4809c7703c09d77a31601cba75cdc3de5d4 commit 13c8c4809c7703c09d77a31601cba75cdc3de5d4 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Sep 29 09:48:34 2014 -0300 nptl: x86_64: Fix Race conditions in pthread cancellation (BZ#12683) This patches adds the x86_64 modification required for the BZ#12683. It basically provide the required ucontext_get_pc symbol and remove the arch-specific libc-cancellation implementations. It also remove bogus arch-specific THREAD_ATOMIC_BIT_SET where it always reference to current thread instead of the one referenced by input 'descr' argument. It works as long the input is the self thread pointer, however it creates wrong code is it used along with a description to a different one (as on nptl/pthread_cancel.c). The code generated create an additional load to reference to TLS segment, for instance the code: THREAD_ATOMIC_BIT_SET (THREAD_SELF, cancelhandling, CANCELED_BIT); Compiles to: lock;orl $4, %fs:776 Where with patch changes it now compiles to: mov %fs:16,%rax lock;orl $4, 776(%rax) If some usage indeed proves to be a hotspot we can add an extra macro with a more descriptive name (THREAD_ATOMIC_BIT_SET_SELF for instance) where x86_64 might optimize it. In fact all x86_64 THREAD_ATOMIC_* macros do not respect the input descr and possible will fail when used with a 'descr' difference than THREAD_SELF. Checked on x86_64-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/x86_64/cancellation.S: Remove file. * sysdeps/unix/sysv/linux/x86_64/libc-cancellation.S: Remove file. * sysdeps/unix/sysv/linux/x86_64/librt-cancellation.S: Remove file. * sysdeps/unix/sysv/linux/x86_64/lowlevellock.h (lll_wait_tid): Use cancellable futex wait call. * sysdeps/unix/sysv/linux/x86_64/sigcontextinfo.h (ucontext_get_pc): New function. * sysdeps/x86_64/nptl/tcb-offsets.sym (TCB_CANCELING_BITMASK): Remove. * sysdeps/x86_64/nptl/tls.h (THREAD_ATOMIC_CMPXCHG_VAL, THREAD_ATOMIC_BIT_SET): Remove macros. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=5c75e9c9cbecc95437a4bc5c9b030bd3af63116b commit 5c75e9c9cbecc95437a4bc5c9b030bd3af63116b Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Fri Sep 18 18:26:35 2015 -0300 nptl: Fix Race conditions in pthread cancellation (BZ#12683) This patches fixes some race conditions in NPTL cancellation code by redefining how cancellable syscalls are defined and handled. Current approach is to enable asynchronous cancellation prior to making the syscall and restore the previous cancellation type once the syscall returns. As decribed in BZ#12683, this approach shows 2 important problems: 1. Cancellation can act after the syscall has returned from kernel, but before userspace saves the return value. It might result in a resource leak if the syscall allocated a resource or a side effect (partial read/write), and there is no way to program handle it with cancellation handlers. 2. If a signal is handled while the thread is blocked at a cancellable syscall, the entire signal handler runs with asynchronous cancellation enabled. This can lead to issues if the signal handler call functions which are async-signal-safe but not async-cancel-safe. For cancellation to work correctly, there are 5 points at which the cancellation signal could arrive: 1. Before the final "testcancel" and before the syscall is made. 2. Between the "testcancel" and the syscall. 3. While the syscall is blocked and no side effects have yet taken place. 4. While the syscall is blocked but with some side effects already having taken place (e.g. a partial read or write). 5. After the syscall has returned. And GLIBC wants to act on cancellation in cases 1, 2, and 3 but not in case 4 or 5. The proposed solution follows: * Handling case 1 is trivial: do a conditional branch based on whether the thread has received a cancellation request; * Case 2 can be caught by the signal handler determining that the saved program counter (from the ucontext_t) is in some address range beginning just before the "testcancel" and ending with the syscall instruction. * In this case, except for certain syscalls that ALWAYS fail with EINTR even for non-interrupting signals, the kernel will reset the program counter to point at the syscall instruction during signal handling, so that the syscall is restarted when the signal handler returns. So, from the signal handler's standpoint, this looks the same as case 2, and thus it's taken care of. * In this case, the kernel cannot restart the syscall; when it's interrupted by a signal, the kernel must cause the syscall to return with whatever partial result it obtained (e.g. partial read or write). * In this case, the saved program counter points just after the syscall instruction, so the signal handler won't act on cancellation. This one is equal to 4. since the program counter is past the syscall instruction already. Another case that needs handling is syscalls that fail with EINTR even when the signal handler is non-interrupting. In this case, the syscall wrapper code can just check the cancellation flag when the errno result is EINTR, and act on cancellation if it's set. The proposed GLIBC adjustments are: 1. Remove the enable_asynccancel/disable_asynccancel function usage in syscall definition and instead make them call a common symbol that will check if cancellation is enabled (__syscall_cancel at nptl/libc-cancellation.c), call the arch-specific cancellable entry-point (__syscall_cancel_arch) and cancel the thread when required. 2. Provide a arch-specific symbol that contains global markers. These markers will be used in SIGCANCEL handler to check if the interruption has been called in a valid syscall and if the syscalls has been completed or not. A default version is provided (sysdeps/unix/sysv/linux/syscall_cancel.c), however the markers may not be set on correct expected places depeding of how INTERNAL_SYSCALL_NCS is implemented by the underlying architecture. In this case arch-specific implementation should be provided. 3. Rewrite SIGCANCEL asynchronous handler to check for both cancelling type and if current IP from signal handler falls between the global markes and act accordingly (sigcancel_handler at nptl/nptl-init.c). 4. Adjust nptl/pthread_cancel.c to send an signal instead of acting directly. This avoid synchronization issues when updating the cancellation status and also focus the logic on signal handler and cancellation syscall code. 5. Adjust pthread code to replace CANCEL_ASYNC/CANCEL_RESET calls to appropriated cancelable futex syscalls. 6. Adjust libc code to replace LIBC_CANCEL_ASYNC/LIBC_CANCEL_RESET to appropriated cancelable syscalls. 7. Adjust 'lowlevellock-futex.h' arch-specific implementations to provide cancelable futex calls (used in libpthread code). This patch adds the proposed changes to NPTL. The code leaves all the ports broken without further patches in the list. [BZ #12683] * nptl/Makefile [routines]: Add syscall_cancel object. [libpthread-routines]: Remove cancellation object. (CFLAGS-cancellation.c): Remove rule. (CFLAGS-syscall_cancel.c): New rule. (tests): Add tst-cancel28. * nptl/Versions [GLIBC_PRIVATE] (libc): Add __syscall_cancel, __syscall_cancel_arch_start, and __syscall_cancel_arch_end. * nptl/cancellation.c: Remove file. * nptl/descr.h (CANCELING_BIT): Remove define. (CANCELING_BITMASK): Likewise. (CANCEL_RESTMASK): Adjust value with CANCELED_BIT remove. * nptl/libc-cancellation.c (__syscall_cancel): Add non-cancellable implementation for loader and cancellable one for libc. (__syscall_do_cancel): New function: cancel call for syscall wrappers. * nptl/lll_timedlock_wait.c (__lll_timedlock_wait): Use cancellable futex operation. (__lll_timedwait_tid): Likewise. * nptl/nptl-init.c (sigcancel_handler): Rewrite function to avoid race conditions. (__pthread_initialize_minimal_internal): Add SA_RESTART to SIGCANCEL handler. * nptl/pt-system.c [LIBC_CANCEL_HANDLED]: Remove definition. * io/creat.c (LIBC_CANCEL_HANDLED): Likewise. * io/ppoll.c [ppoll] (LIBC_CANCEL_HANDLED): Likewise. * misc/pselect [__pselect] (LIBC_CANCEL_HANDLED): Likewise. * sysdeps/posix/pause.c (LIBC_CANCEL_HANDLED): Likewise. * sysdeps/unix/sysv/linux/generic/creat.c (LIBC_CANCEL_HANDLED): Likewise. * nptl/pthreadP.h (__do_cancel): Rewrite to both disable asynchronous cancellation and setting the thread as cancelled. (CANCEL_ASYNC): Remove definition. (CANCEL_RESET): Likewise. (LIBC_CANCEL_ASYNC): Likewise. (LIBC_CANCEL_RESET): Likewise. (LIBC_CANCEL_HANDLED): Likewise. (__syscall_cancel_arch): Add prototype. (__pthread_enable_asynccancel): Remove prototype. (__pthread_disable_asynccancel): Likewise. (__libc_enable_asynccancel): Likewise. (__libc_disable_asynccancel): Likewise. (__librt_enable_asynccancel): Likewise. (__librt_disable_asynccancel): Likewise. (__syscall_cancel_arch): Add prototype. (__syscall_do_cancel): Likewise. * nptl/pthread_cancel.c (pthread_cancel): Rewrite to just set CANCELLED_BIT and call __pthread_kill. * nptl/pthread_create.c (start_thread): Likewise. * nptl/pthread_timedjoin.c (pthread_timedjoin_np): Likewise. * nptl/sem_timedwait.c (sem_timedwait): Likewise. * nptl/sem_wait.c (__new_sem_wait): Likewise. * nptl/sem_waitcommon.c (futex_abstimed_wait): Likewise. * sysdeps/nptl/aio_misc.h (AIO_MISC_WAIT): Likewise. * sysdeps/nptl/gai_misc.h (GAI_MISC_WAIT): Likewise. * sysdeps/posix/sigpause.c (do_sigpause): Likewise. * sysdeps/posix/sigwait.c (__sigwait): Likewise. * sysdeps/posix/waitid.c (__waitid): Likewise. * sysdeps/nptl/lowlevellock.h (lll_wait_tid): Likewise. * sysdeps/posix/open64.c (__libc_open64): Likewise. * sysdeps/unix/sysv/linux/sigwait.c (__sigwait): Likewise. * nptl/pthread_exit.c (pthread_exit): Rewrite to set EXITING_BIT before call __pthread_unwind. * nptl/pthread_join.c (pthread_join): Remove CANCEL_ASYNC/CANCEL_RESET usage. * rt/Makefile [CFLAGS-librt-cancellation.c]: Remove rule. * sysdeps/generic/sysdep-cancel.h (LIBC_CANCEL_ASYNC): Remove define. (LIBC_CANCEL_RESET): Likewise. (LIBC_CANCEL_HANDLED): Likewise. * sysdeps/unix/sysv/linux/clock_nanosleep.c (__clock_nanosleep): Likewise. * sysdeps/unix/sysv/linux/fcntl.c (__libc_fcntl): Likewise. * sysdeps/unix/sysv/linux/generic/wordsize-32/fcntl.c (__libc_fcntl): Likewise. * sysdeps/nptl/Makefile [$(subdir) = rt] (librt-sysdep_routines): Remove librt-cancellation object. [$(subdir) = rt] (librt-cancellation.c): Remove rule. * sysdeps/nptl/librt-cancellation.c: Remove file. * sysdeps/unix/sysv/linux/futex-internal.h (lll_futex_wait_cancel): Use lll_futex_timed_wait_cancel. (futex_reltimed_wait_cancelable): Likewise. (futex_abstimed_wait_cancelable)): Use lll_futex_timed_wait_bitset_cancel. * sysdeps/unix/sysv/linux/lowlevellock-futex.h (lll_futex_wait_cancel): New macro. (lll_futex_timed_wait_cancel): Likewise. (lll_futex_timed_wait_bitset_cancel): Likewise. * sysdeps/unix/sysdep.h (SYSCALL_CANCEL): New macro: cancelable syscall calls. (INTERNAL_SYSCALL_NCS_CALL): New macro. (__syscall_cancel): New prototype. * sysdeps/unix/sysv/linux/socketcall.h (SOCKETCALL): Use __SSC macros. (SOCKETCALL_CANCEL): Use SYSCALL_CANCEL macros. * sysdeps/generic/sysdep-cancel.h (LIBC_CANCEL_ASYNC): Remove define. (LIBC_CANCEL_RESET): Likewise. (LIBC_CANCEL_HANDLED): Likewise. * sysdeps/unix/sysv/linux/pthread_kill.c (__pthread_kill): Allow SIGCANCEL to be sent. * nptl/tst-cancel28.c: New file. * sysdeps/unix/sysv/linux/syscall_cancel.c: Likewise. * support/temp_file.c (create_temp_fifo): New function. * support/temp_file.h (create_temp_fifo): Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=0796a7c793e817c84a4e06e405ed902c41d0a8de commit 0796a7c793e817c84a4e06e405ed902c41d0a8de Author: Adhemerval Zanella <adhemerval.zanella@linaro.com> Date: Mon Sep 21 15:55:58 2015 -0700 nptl: Fix testcases for new pthread cancellation mechanism With upcoming fix for BZ#12683, pthread cancellation does not act for: 1. If syscall is blocked but with some side effects already having taken place (e.g. a partial read or write). 2. After the syscall has returned. The main change is due the fact programs need to act in syscalls with side-effects (for instance, to avoid leak of allocated resources or handle partial read/write). This patch changes the NPTL testcase that assumes the old behavior and also remove the tst-cancel-wrappers.sh test (which checks for symbols that will not exist anymore). For tst-cancel{2,3} case it remove the pipe close because it might cause the write syscall to return with side effects if the close is executed before the pthread cancel. It also changes how to call the read syscall on tst-backtrace{5,6} to use syscall instead of read cancelable syscall to avoid need to handle the cancelable bridge function calls. It requires a change on powerpc syscall implementation to create a stackframe, since powerpc backtrace rely on such information. Checked on i686-linux-gnu, x86_64-linux-gnu, x86_64-linux-gnux32, aarch64-linux-gnu, arm-linux-gnueabihf, powerpc64le-linux-gnu, powerpc-linux-gnu, sparcv9-linux-gnu, and sparc64-linux-gnu. * debug/tst-backtrace5.c (handle_signal): Check for syscall instead of read. (fn): Issue the read syscall instead of call the cancellable syscall. * nptl/Makefile [$(run-built-tests) = yes] (tests-special): Remove tst-cancel-wrappers.sh. * nptl/tst-cancel-wrappers.sh: Remove file. * nptl/tst-cancel2.c (do_test): Do not close pipe. * nptl/tst-cancel3.c (do_test): Likewise. * nptl/tst-cancel4.c (tf_write): Handle cancelled syscall with side-effects. (tf_send): Likewise. * sysdeps/unix/sysv/linux/powerpc/syscall.S (syscall): Create stack frame. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=9a5fd16b1327cdcab9cd26687c7f70779c2d63dd commit 9a5fd16b1327cdcab9cd26687c7f70779c2d63dd Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Feb 26 17:52:02 2018 -0300 powerpc: Create stackframe information on syscall This patch adds a minimal stackframe creation on powerpc syscall implementation so backtrace works correctly on a signal handler. Checked on powerpc64le-linux-gnu. * sysdeps/unix/sysv/linux/powerpc/syscall.S (syscall): Create stack frame. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=7a2c0ea11bac560392d4002626129523968f6c22 commit 7a2c0ea11bac560392d4002626129523968f6c22 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Thu Feb 22 15:47:33 2018 -0300 nptl: Fix tst-cancel4 sendto tests Now that send might be implemented calling sendto syscall on Linux, I am seeing some issue in some kernel configurations where tst-cancel4 sendto do not block as expected. The socket used to force the syscall blocking is used with default system configuration for buffer sending size, which might not be correct to force blocking. This patch fixes it by explicit setting buffer socket lower than the buffer size used. It also enabled sendto cancellation tests to work in both ways (since internally send is implemented routing to sendto on Linux kernel). The patch also removes unrequired make rules on some archictures for send/recv. The generic nptl Makefile already set the compiler flags required on some architectures for correct unwinding and libc object are not strictly required to support unwind (since pthread_cancel requires linking against libpthread). Checked on aarch64-linux-gnu and x86_64-linux-gnu. I also did a sniff test with tst-cancel{4,5} on a simulated mips64-linux-gnu. * nptl/tst-cancel4-common.h (set_socket_buffer): New function. * nptl/tst-cancel4-common.c (do_test): Call set_socket_buffer for socketpair endpoint. * nptl/tst-cancel4.c (tf_send): Call set_socket_buffer and use WRITE_BUFFER_SIZE as buffer size for sending socket. (tf_sendto): Use SOCK_STREAM instead of SOCK_DGRAM and fix an issue on system where send is implemented with sendto syscall. * sysdeps/unix/sysv/linux/mips/mips64/Makefile [$(subdir) = socket] (CFLAGS-recv.c, CFLAGS-send.c): Remove rules. [$(subdir) = nptl] (CFLAGS-recv.c, CFLAGS-send.c): Likewise. * sysdeps/unix/sysv/linux/riscv/rv64/Makefile: Remove file. -----------------------------------------------------------------------
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "GNU C Library master sources". The branch, azanella/bz12683 has been created at 2e8c8e9622f0591fdffb7b5e79d1b6937af52e6a (commit) - Log ----------------------------------------------------------------- https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=2e8c8e9622f0591fdffb7b5e79d1b6937af52e6a commit 2e8c8e9622f0591fdffb7b5e79d1b6937af52e6a Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Tue Jan 24 21:05:56 2017 -0200 nptl: Remove THREAD_ATOMIC_* macros This patch removes the ununsed THREAD_ATOMIC_* macros now that nptl code is using C11 atomics. Checked on x86_64-linux-gnu and i686-linux-gnu. * manual/pattern.texi (THREAD_ATOMIC_BIT_SET): Remove. (THREAD_ATOMIC_CMPXCHG_VAL): Likewise. * nptl/pthreadP.h (THREAD_ATOMIC_CMPXCHG_VAL): Likewise. (THREAD_ATOMIC_BIT_SET): Likewise. * sysdeps/i386/nptl/tls.h (THREAD_ATOMIC_CMPXCHG_VAL): Likewise. (THREAD_ATOMIC_AND): Likewise. * sysdeps/x86_64/nptl/tls.h (THREAD_ATOMIC_CMPXCHG_VAL): Likewise. (THREAD_ATOMIC_AND): Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c905cddb3868b5128b0460239e0cc5d609a7ef65 commit c905cddb3868b5128b0460239e0cc5d609a7ef65 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Tue Jan 24 09:17:37 2017 -0200 nptl: cancelhandling refactor This patch basically uses glibc C11 atomic function to access and modify struct pthread cancelhandling variable. All plain access are handled using the atomic_load_* functions and THREAD_ATOMIC_BIT_SET is replaced by atomic_fetch_or_acquire. Checked on i686-linux-gnu, x86_64-linux-gnu, x86_64-linux-gnux32, aarch64-linux-gnu, arm-linux-gnueabihf, and powerpc64le-linux-gnu. * nptl/descr.h (CANCELED_BITMASK): Rename to THREAD_CANCELED. (EXITING_BITMASK): Rename to THREAD_EXITING. (TERMINATED_BITMARK): Rename to THREAD_TERMINATED. * nptl/allocatestack.c (setxid_mark_thread): Use THREAD_* instead of *_BITMASK and use glibc C11 atomic function to access pthread cancelhandling. * nptl/libc-cancellation.c (__syscall_cancel): Likewise. * nptl/nptl-init.c (sigcancel_handler): Likewise. * nptl/pthread_cancel.c (pthread_cancel): Likewise. * nptl/pthread_create.c (__free_tcb): Likewise. (START_THREAD_DEFN): Likewise. (__pthread_create_2_1): Likewise. * nptl/pthread_detach.c (pthread_detach): Likewise. * nptl/pthread_exit.c (__pthread_exit): Likewise. * nptl/pthread_join.c (pthread_join): Likewise. * nptl_db/td_thr_get_info.c (td_thr_get_info): Likewise. * nptl_db/td_thr_getfpregs.c (td_thr_getfpregs): Likewise. * nptl_db/td_thr_getgregs.c (td_thr_getregs): Likewise. * nptl_db/td_thr_setfpregs.c (td_thr_setfpregs): Likewise. * nptl_db/td_thr_setgregs.c (td_thr_setgregs): Likewise. * sysdeps/unix/sysv/linux/arm/syscall_cancel.S (__syscall_cancel_arch): Change CANCELED_BITMASK to THREAD_CANCELED in comment. * sysdeps/unix/sysv/linux/syscall_cancel.c (__syscall_cancel_arch): Likewise. * sysdeps/x86_64/nptl/tcb-offsets.sym (TCB_CANCELED_BITMASK): Remove. (TCB_EXITING_BITMASK): Likewise. (TCB_TERMINATED_BITMASK): Likewise. * sysdeps/unix/sysv/linux/i386/syscall_cancel.S (__syscall_cancel_arch): Likewise. * sysdeps/unix/sysv/linux/powerpc/syscall_cancel.S (__syscall_cancel_arch): Likewise. * sysdeps/unix/sysv/linux/sh/syscall_cancel.S (__syscall_cancel_arch): Likewise. * sysdeps/unix/sysv/linux/sparc/sparc32/syscall_cancel.S (__syscall_cancel_arch): Likewise. * sysdeps/unix/sysv/linux/sparc/sparc64/syscall_cancel.S (__syscall_cancel_arch): Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=2212de8d6ffc4b29baf170422656d784caab4959 commit 2212de8d6ffc4b29baf170422656d784caab4959 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Tue Jul 18 16:33:41 2017 -0300 nptl: Remove setxid bit from cancelhandling This patch removes the setxid signaling out of cancelhandling flag and use its own member of pthread instead. The idea is simplify setxid handling to not mix atomic operation with thread cancellation since both are orthogonal, and to isolate cancelhandling variable to only track thread cancel state. It also uses glibc C11 atomic operation while accessing new setxid_op field. Checked on i686-linux-gnu, x86_64-linux-gnu, x86_64-linux-gnux32, aarch64-linux-gnu, arm-linux-gnueabihf, and powerpc64le-linux-gnu. * nptl/allocatestack.c (get_cached_stack): Set sexid_op on new allocated stack. (setxid_mark_thread): Use setxid_op to mark thread as executing a sexid operation insteaf of using cancelhandling. Also adapt algorithm to use C11 atomic analogous functions. (setxid_unmark_thread): Likewise. (setxid_signal_thread): Likewise. * nptl/nptl-init.c (sighandler_setxid): Likewise. * nptl/pthread_create.c (sighandler_setxid): Likewise. * nptl/descr.h (SETXID_BIT): Remove define. (SETXID_BITMASK): Likewise. (setxid_op): New member. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=f1b02e0cc90b72148e2755b55dff1faac3b9eb8b commit f1b02e0cc90b72148e2755b55dff1faac3b9eb8b Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Fri Jan 20 17:24:35 2017 -0200 nptl: Move cancel state and type out cancelhandling This patch move both the cancel state (PTHREAD_CANCEL_ENABLE and PTHREAD_CANCEL_DISABLE) and cancel type (PTHREAD_CANCEL_DEFERRED and PTHREAD_CANCEL_ASYNCHRONOUS) out of cancelhandling member. The idea is to simplify their access since for most cases there is no concurrent access with cancellation mode: * pthread_setcancel{type,state} and internal functions __pthread_{un}register_cancel_defer (and its analogous compat versions) only change the type/state of the calling thread. * __syscall_cancel also only requires to check the state of calling thread. * sigcancel_handler is executed only for the thread about to be potentially canceled. * pthread_join deadlock checks only requires access the state for the calling thread. * Same behavior for pthread_testcancel. With this change the cancelhandling member from pthread struct is used solely for cancelation handling and setxid signaling. Checked on i686-linux-gnu, x86_64-linux-gnu, x86_64-linux-gnux32, aarch64-linux-gnu, arm-linux-gnueabihf, and powerpc64le-linux-gnu. * nptl/allocatestack.c (get_cached_stack): Set both cancelstate and canceltype on new stack allocation. * nptl/cleanup_defer.c (__pthread_register_cancel_defer): Set canceltype directly instead of using cancelhandling member. (__pthread_unregister_cancel_restore): Likewise. * nptl/cleanup_defer_compat.c (_pthread_cleanup_push_defer): Likewise. (_pthread_cleanup_pop_restore): Likewise. * nptl/descr.h (CANCELSTATE_BIT): Remove flag. (CANCELSTATE_BITMASK): Likewise. (CANCELTYPE_BIT): Likewise. (CANCELTYPE_BITMSK): Likewise. (CANCEL_RESTMASK): Likewise. (CANCEL_ENABLED_AND_CANCELED): Likewise. (CANCEL_ENABLED_AND_CANCELED_AND_ASYNCHRONOUS): Likewise. (cancelstate): New member. (canceltype): Likewise. * nptl/libc-cancellation.c (__syscall_cancel): Use cancelstate instead of cancelhandling member. * nptl/nptl-init.c (sigcancel_handler): Likewise. * nptl/pthreadP.h (__do_cancel): Likewise. (CANCELLATION_P): Likewise. * nptl/pthread_join.c (pthread_join): Remove CANCELLATION_P usage. * nptl/pthread_testcancel.c (__pthread_testcancel): Likewise. * nptl/pthread_setcancelstate.c (__pthread_setcancelstate): Use cancelstate member instead of cancelhandling. * nptl/pthread_setcanceltype.c (__pthread_setcanceltype): Use canceltype member instead of canceltype. * sysdeps/x86_64/nptl/tcb-offsets.sym (TCB_CANCELSTATE_BITMASK): Remove. (TCB_CANCELTYPE_BITMASK): Likewise. (TCB_CANCEL_RETMASK): Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=7102b569bba27db931ae98460469796757709741 commit 7102b569bba27db931ae98460469796757709741 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Feb 26 16:24:15 2018 -0300 nptl: riscv: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the riscv modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function. The default syscall_cancel.c should be as expected for RISCV. With GCC 7.3.1 syscall_cancel.c generates the following code: 0000000000000000 <__GI___syscall_cancel_arch>: 0: 8346 mv t1,a7 0000000000000002 <__syscall_cancel_arch_start>: 2: 00052883 lw a7,0(a0) 6: 0048f893 andi a7,a7,4 a: 00089c63 bnez a7,22 <.L0 > e: 88ae mv a7,a1 10: 8532 mv a0,a2 12: 85b6 mv a1,a3 14: 863a mv a2,a4 16: 86be mv a3,a5 18: 8742 mv a4,a6 1a: 879a mv a5,t1 1c: 00000073 ecall 0000000000000020 <__syscall_cancel_arch_end>: 20: 8082 ret 22: 1141 addi sp,sp,-16 24: e406 sd ra,8(sp) 26: 00000097 auipc ra,0x0 2a: 000080e7 jalr ra # 26 <.L0 > Checked with a build for riscv64-linux-gnu with run-built-tests=no. [BZ #12683] * sysdeps/unix/sysv/linux/riscv/sigcontextinfo.h (ucontext_get_pc): New fuction. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=1092ac32af74ddaa5ccfb63b8ef33b40b9befc8a commit 1092ac32af74ddaa5ccfb63b8ef33b40b9befc8a Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Fri Feb 3 21:07:07 2017 -0200 nptl: hppa: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the hppa modifications required for the BZ#12683. It basically adds the required __pthread_get_pc function. HPPA requires an arch-specific syscall_cancel because the INTERNAL_SYSCALL_NCS adds some instruction to fetch the returned syscalls value. The implementation were based on on default C version built with GCC 6.1 Checked on hppa-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/hppa/sigcontextinfo.h: New file. * sysdeps/unix/sysv/linux/hppa/syscall_cancel.S: Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=f74aa232cde071899f8137920e84bbba3c8ed679 commit f74aa232cde071899f8137920e84bbba3c8ed679 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:03:07 2017 -0200 nptl: mips: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the mips modifications required for the BZ#12683. It adds the required ucontext_get_pc function, a mips32 cancellable syscall wrapper and 7 argument cancellable syscall support. To avoid code pessimization and add a requirement on all architectures to support {INLINE,INTERNAL)_SYSCALL with 7 argument, its support is added through a flag, HAVE_CANCELABLE_SYSCALL_WITH_7_ARGS, which changes the signature and prototype of the requires macros and functions (SYSCALL_CANCEL, __syscall_cancel and __syscall_cancel_arch). As default 6 arguments cancellable syscalls are use. MIPS o32 requires an arch-specific implementation because INTERNAL_SYSCALL_NCS adds an 'addiu' just after the syscall instruction which invalidates the checks on sigcancel_handler. Checked against a build and make check run-built-tests=no for mips-gnu-linux, mips64-linux-gnu, mips64-n32-linux-gnu. I also ran some basic o32 and n64 cancellation tests on a simulated mips64 qemu system. [BZ #12683] * nptl/libc-cancellation.c (__syscall_cancel): Define and use 7 argument syscall if architecture requires it. * nptl/pthreadP.h (__syscall_cancel_arch): Likewise. * sysdeps/unix/sysdep.h (__syscall_cancel, __SYSCALL_CANCEL*): Define with 7 argument if architecture requires it. (__SYSCALL_CANCEL7_ARG_DEF): New macro. (__SYSCALL_CANCEL7_ARG): Likewise. (__SYSCALL_CANCEL7_ARG7): Likewise. * sysdeps/unix/sysv/linux/syscall_cancel.c (__syscall_cancel_arch): Likewise. * sysdeps/mips/nptl/tls.h (READ_THREAD_POINTER): Check __mips_isa_rev existance for macro definition. * sysdeps/unix/sysv/linux/mips/sigcontextinfo.h (ucontext_get_pc): New function. * sysdeps/unix/sysv/linux/mips/mips32/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/mips/mips32/sysdep.h (HAVE_CANCELABLE_SYSCALL_WITH_7_ARGS): Define. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=7806615561d4f40f4d53f3375536cfe08fe127e0 commit 7806615561d4f40f4d53f3375536cfe08fe127e0 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:03:03 2017 -0200 nptl: sh: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the sh modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function and a cancellable syscall wrapper. SH requires an arch-specific syscall_cancel because the INTERNAL_SYSCALL_NCS adds the required or instruction to workaround a hardware bug [1]. The implementation were based on on default C version built with GCC 6.2.1. Checked against a build and make check run-built-tests=no for sh4-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/sh/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/sh/sigcontextinfo.h (ucontext_get_pc): New function. [1] http://documentation.renesas.com/eng/products/mpumcu/tu/tnsh7456ae.pdf https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=02badc9d7b0c5379bd46019546ed9d2d8acb6a4a commit 02badc9d7b0c5379bd46019546ed9d2d8acb6a4a Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:02:53 2017 -0200 nptl: nios2: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the nios2 modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function. The default syscall_cancel.c should be as expected for NIOS2. With GCC 6.2.1 syscall_cancel.c generates the following code: --- 00000000 <__GI___syscall_cancel_arch>: 0: defffe04 addi sp,sp,-8 4: dd800015 stw r22,0(sp) 8: 002ce03a nextpc r22 c: 02000034 movhi r8,0 10: 42000004 addi r8,r8,0 14: dfc00115 stw ra,4(sp) 18: b22d883a add r22,r22,r8 0000001c <__syscall_cancel_arch_start>: 1c: 20c00017 ldw r3,0(r4) 20: 18c0010c andi r3,r3,4 24: 18000f1e bne r3,zero,64 <__syscall_cancel_arch_end+0x18> 28: 3015883a mov r10,r6 2c: 2805883a mov r2,r5 30: da400517 ldw r9,20(sp) 34: 380b883a mov r5,r7 38: da000417 ldw r8,16(sp) 3c: d9c00317 ldw r7,12(sp) 40: d9800217 ldw r6,8(sp) 44: 5009883a mov r4,r10 48: 003b683a trap 0 0000004c <__syscall_cancel_arch_end>: 4c: 38000126 beq r7,zero,54 <__syscall_cancel_arch_end+0x8> 50: 0085c83a sub r2,zero,r2 54: dfc00117 ldw ra,4(sp) 58: dd800017 ldw r22,0(sp) 5c: dec00204 addi sp,sp,8 60: f800283a ret 64: b0800017 ldw r2,0(r22) 68: 103ee83a callr r2 --- Checked against a build and make check run-built-tests=no for nios2-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/nios2/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=a6ed5172ed96b96d6fe99bb1e83f0af8aec6bb97 commit a6ed5172ed96b96d6fe99bb1e83f0af8aec6bb97 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:02:36 2017 -0200 nptl: sparc: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the sparc modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function, cancellable syscall wrappers, and a fix for pause. Sparc requires arch-specific syscall_cancel implementation because INLINE_SYSCALL_NCS uses the __SYSCALL_STRING (defined different for sparc32 and sparc64) and it issues additional instructions after the syscall one to check the resulting error code. When used in the default syscall_cancel.c implementation the label __syscall_cancel_arch_end is not placed just after the syscall as expected. Both 32 and 64 bits version were based on default C version built with GCC 6.1. Also, different than other architectures, SPARC passes the sigcontext_t struct pointer as third argument in the signal handler set with SA_SIGINFO (some info at [1]) for 64 bits and the pt_regs in 32 bits. From Linux code: * arch/sparc/kernel/signal_64.c 428 /* 3. signal handler back-trampoline and parameters */ 429 regs->u_regs[UREG_FP] = ((unsigned long) sf) - STACK_BIAS; 430 regs->u_regs[UREG_I0] = ksig->sig; 431 regs->u_regs[UREG_I1] = (unsigned long) &sf->info; 432 433 /* The sigcontext is passed in this way because of how it 434 * is defined in GLIBC's /usr/include/bits/sigcontext.h 435 * for sparc64. It includes the 128 bytes of siginfo_t. 436 */ 437 regs->u_regs[UREG_I2] = (unsigned long) &sf->info; * arch/sparc/kernel/signal_32.c: 392 regs->u_regs[UREG_FP] = (unsigned long) sf; 393 regs->u_regs[UREG_I0] = ksig->sig; 394 regs->u_regs[UREG_I1] = (unsigned long) &sf->info; 395 regs->u_regs[UREG_I2] = (unsigned long) &sf->regs; 396 397 regs->pc = (unsigned long) ksig->ka.sa.sa_handler; 398 regs->npc = (regs->pc + 4); So to access the signal mask in the signal frame, a arch-specific ucontext_get_mask is defined which obtain the sa_mask from the context. Checked on a SPARC T5 for sparc64-linux-gnu and sparcv9-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/sparc/sparc32/sigcontextinfo.h (ucontext_get_pc): New function. * sysdeps/unix/sysv/linux/sparc/sparc64/sigcontextinfo.h (ucontext_get_pc): Likewise. * sysdeps/unix/sysv/linux/sparc/sparc32/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/sparc/sparc64/syscall_cancel.S: Likwise. * sysdeps/unix/sysv/linux/sparc/sparc64/pause.c: New file. [1] https://www.spinics.net/lists/sparclinux/msg05037.html https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=fbdf0307b3e6b880a5c0effce603f441c1dce7ed commit fbdf0307b3e6b880a5c0effce603f441c1dce7ed Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:02:17 2017 -0200 nptl: microblaze: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the microblaze modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function. Microblaze requires an arch specific assembly implementation because the archicture INTERNAL_SYSCALL_NCS implementation adds a nop after the brki instruction because the instruction expect a delay branch slot. I based this implementation on generated assembly using GCC 6.1. Checked against a build and make check run-built-tests=no for microblaze-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/microblaze/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/microblaze/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=838fce23efc583cb8f016914fc374319b4b85f96 commit 838fce23efc583cb8f016914fc374319b4b85f96 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:02:08 2017 -0200 nptl: m68k: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the m68k modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function The default syscall_cancel.c should is as expected for m68k with GCC 6.2.1. It generates the following code for syscall_cancel.os: --- 00000000 <__GI___syscall_cancel_arch>: 0: 4e56 0000 linkw %fp,#0 4: 48e7 3c00 moveml %d2-%d5,%sp@- 00000008 <__syscall_cancel_arch_start>: 8: 206e 0008 moveal %fp@(8),%a0 c: 2010 movel %a0@,%d0 e: 0800 0002 btst #2,%d0 12: 6628 bnes 3c <__syscall_cancel_arch_end+0xa> 14: 206e 0024 moveal %fp@(36),%a0 18: 2a2e 0020 movel %fp@(32),%d5 1c: 282e 001c movel %fp@(28),%d4 20: 262e 0018 movel %fp@(24),%d3 24: 242e 0014 movel %fp@(20),%d2 28: 222e 0010 movel %fp@(16),%d1 2c: 202e 000c movel %fp@(12),%d0 30: 4e40 trap #0 00000032 <__syscall_cancel_arch_end>: 32: 4cee 003c fff0 moveml %fp@(-16),%d2-%d5 38: 4e5e unlk %fp 3a: 4e75 rts 3c: 61ff 0000 0000 bsrl 3e <__syscall_cancel_arch_end+0xc> --- Checked against a build and make check run-built-tests=no for m68k-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/m68k/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=6da7532402198c74e7c063ade4a1ea6c6906b82a commit 6da7532402198c74e7c063ade4a1ea6c6906b82a Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:01:58 2017 -0200 nptl: alpha: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the alpha modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function The default syscall_cancel.c should be fine for alpha and GCC 6.1 generates the following cod for syscall_cancel.os: --- 0000000000000000 <__GI___syscall_cancel_arch>: 0: 00 00 bb 27 ldah gp,0(t12) 4: 00 00 bd 23 lda gp,0(gp) 8: f0 ff de 23 lda sp,-16(sp) c: 00 04 f1 47 mov a1,v0 10: 00 00 5e b7 stq ra,0(sp) 0000000000000014 <__syscall_cancel_arch_start>: 14: 00 00 30 a0 ldl t0,0(a0) 18: 01 00 e1 43 sextl t0,t0 1c: 01 90 20 44 and t0,0x4,t0 20: 0f 00 20 f4 bne t0,60 <__syscall_cancel_arch_end+0x20> 24: 10 04 f2 47 mov a2,a0 28: 11 04 f3 47 mov a3,a1 2c: 12 04 f4 47 mov a4,a2 30: 10 00 9e a6 ldq a4,16(sp) 34: 13 04 f5 47 mov a5,a3 38: 18 00 be a6 ldq a5,24(sp) 3c: 83 00 00 00 callsys 0000000000000040 <__syscall_cancel_arch_end>: 40: 21 05 e0 43 negq v0,t0 44: 00 00 5e a7 ldq ra,0(sp) 48: c0 04 61 46 cmovne a3,t0,v0 4c: 10 00 de 23 lda sp,16(sp) 50: 01 80 fa 6b ret 54: 00 00 fe 2f unop 58: 1f 04 ff 47 nop 5c: 00 00 fe 2f unop 60: 00 00 7d a7 ldq t12,0(gp) 64: 00 40 5b 6b jsr ra,(t12),68 <__syscall_cancel_arch_end+0x28> 68: 1f 04 ff 47 nop 6c: 00 00 fe 2f unop --- Checked on alpha-linux-gnu, no regression found. [BZ #12683] * sysdeps/unix/sysv/linux/alpha/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=60eb1c51381fda4131058dffbd46f9d512cef59f commit 60eb1c51381fda4131058dffbd46f9d512cef59f Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Jan 16 17:01:44 2017 -0200 nptl: ia64: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the s390 modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function. a workaround for mismatched sigcontext::sa_flags defiition between kernel and GLIBC (tracked by BZ#21634), and an arch specific syscall_cancel implementation. IA64 requires an arch-specific syscall_cancel implemetantion because {INLINE,INTERNAL}_SYSCALL is implemented by branching to a gate DSO (similar to i386) which renders the pointer comparison in SIGCANCEL handler wrong. This incurs in performance penalty due the use of a break instruction insteaf of a eds one, however cancellable syscalls should potentially incur in syscalls blocking. Checked with a ia64-linux-gnu build with run-tests-built=no. [BZ #12683] * sysdeps/unix/sysv/linux/ia64/sigcontextinfo.h (ucontext_get_pc, ucontext_get_mask): New functions. * sysdeps/unix/sysv/linux/ia64/syscall_cancel.S: New file. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=ee923bad6397f393d8ae8d38b1ea95c8df0dcf74 commit ee923bad6397f393d8ae8d38b1ea95c8df0dcf74 Author: Adhemerval Zanella <adhemerval.zanella@linaro.com> Date: Wed Aug 12 10:51:38 2015 -0300 nptl: s390: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the s390 modifications required for the BZ#12683 fix. It basically adds the required ucontext_get_pc function. The built cancelable syscall wrapper for s390 using GCC 7.2.1 and default configuration flags shows the wrappers on expected corrected places: --- __GI___syscall_cancel_arch: .LFB39: .cfi_startproc stm %r6,%r15,24(%r15) .cfi_offset 6, -72 .cfi_offset 7, -68 .cfi_offset 8, -64 .cfi_offset 9, -60 .cfi_offset 10, -56 .cfi_offset 11, -52 .cfi_offset 12, -48 .cfi_offset 13, -44 .cfi_offset 14, -40 .cfi_offset 15, -36 ahi %r15,-96 .cfi_def_cfa_offset 192 .global __syscall_cancel_arch_start .type __syscall_cancel_arch_start,@function __syscall_cancel_arch_start: l %r0,0(%r2) tml %r0,4 jne .L5 lr %r1,%r3 lr %r2,%r4 lr %r3,%r5 lr %r4,%r6 l %r5,192(%r15) l %r6,196(%r15) l %r7,200(%r15) svc 0 .global __syscall_cancel_arch_end .type __syscall_cancel_arch_end,@function __syscall_cancel_arch_end: l %r4,152(%r15) lm %r6,%r15,120(%r15) .cfi_remember_state .cfi_restore 15 .cfi_restore 14 .cfi_restore 13 .cfi_restore 12 .cfi_restore 11 .cfi_restore 10 .cfi_restore 9 .cfi_restore 8 .cfi_restore 7 .cfi_restore 6 .cfi_def_cfa_offset 96 br %r4 .L5: .cfi_restore_state brasl %r14,__syscall_do_cancel .cfi_endproc --- The s390x version also shows similar placement: --- __GI___syscall_cancel_arch: .cfi_startproc stmg %r6,%r15,48(%r15) .cfi_offset 6, -112 .cfi_offset 7, -104 .cfi_offset 8, -96 .cfi_offset 9, -88 .cfi_offset 10, -80 .cfi_offset 11, -72 .cfi_offset 12, -64 .cfi_offset 13, -56 .cfi_offset 14, -48 .cfi_offset 15, -40 aghi %r15,-160 .cfi_def_cfa_offset 320 .global __syscall_cancel_arch_start .type __syscall_cancel_arch_start,@function __syscall_cancel_arch_start: l %r0,0(%r2) tmll %r0,4 jne .L5 lgr %r1,%r3 lgr %r2,%r4 lgr %r3,%r5 lgr %r4,%r6 lg %r5,320(%r15) lg %r6,328(%r15) lg %r7,336(%r15) svc 0 .global __syscall_cancel_arch_end .type __syscall_cancel_arch_end,@function __syscall_cancel_arch_end: lg %r4,272(%r15) lmg %r6,%r15,208(%r15) .cfi_remember_state .cfi_restore 15 .cfi_restore 14 .cfi_restore 13 .cfi_restore 12 .cfi_restore 11 .cfi_restore 10 .cfi_restore 9 .cfi_restore 8 .cfi_restore 7 .cfi_restore 6 .cfi_def_cfa_offset 160 br %r4 .L5: .cfi_restore_state brasl %r14,__syscall_do_cancel .cfi_endproc --- Checked on s390-linux-gnu and s390x-linux-gnu build with run-tests-built=no. [BZ #12683] * sysdeps/unix/sysv/linux/s390/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=683a8f59a9473e0ff6d6a3df26c0294948ddc8d8 commit 683a8f59a9473e0ff6d6a3df26c0294948ddc8d8 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Fri May 8 17:12:31 2015 -0300 nptl: arm: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the ARM modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function and adjust the generic syscall_cancel build. For ARM we need to build syscall_cancel in ARM mode (-marm) to avoid INTERNAL_SYSCALL to issue the syscall through the helper gate __libc_do_syscall (which invalidates the mark checks on SIGCANCEL handler). Checked on arm-linux-gnueabihf. [BZ #12683] * sysdeps/unix/sysv/linux/arm/Makefile (CFLAGS-syscall_cancel.c): New rule. * sysdeps/unix/sysv/linux/arm/sigcontextinfo.h (ucontext_get_pc): New function. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=0151ef6406a6e32c4601929bcbd3ea2b136f223b commit 0151ef6406a6e32c4601929bcbd3ea2b136f223b Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Wed May 6 17:51:29 2015 -0300 nptl: aarch64: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the aarch64 modifications required for the BZ#12683. It basically adds the required ucontext_get_pc function. The built cancelable syscall wrapper for aarch64 using GCC 7.2.1 and default configuration flags shows an expected optimized version: --- __GI___syscall_cancel_arch: .LFB38: .cfi_startproc .global __syscall_cancel_arch_start .type __syscall_cancel_arch_start,@function __syscall_cancel_arch_start: ldr w9, [x0] tbnz x9, 2, .L7 mov x8, x1 mov x0, x2 mov x1, x3 mov x2, x4 mov x3, x5 mov x4, x6 mov x5, x7 svc 0 // syscall nr .global __syscall_cancel_arch_end .type __syscall_cancel_arch_end,@function __syscall_cancel_arch_end: ret --- A similar code is obtained with GCC 5.3.1, so I see to need to provide an arch-specific syscall_cancel.S for aarch64. Checked on aarch64-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/aarch64/sysdep-cancel.h (PSEUDO): Redefine to call __syscall_cancel function for cancellable syscalls. (__pthread_get_ip): Add implementation. * sysdeps/unix/sysv/linux/aarch64/sysdep.h (SYSCALL_CANCEL_ERROR): Add definition. (SYSCALL_CANCEL_ERRNO): Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=996cfcbe41d1a7561a791ab20de2dab89bde529d commit 996cfcbe41d1a7561a791ab20de2dab89bde529d Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Fri Sep 18 18:14:19 2015 -0300 nptl: powerpc: Fix Race conditions in pthread cancellation (BZ#12683) This patches adds the powerpc64 modification required for the BZ#12683. It basically adds the required __pthread_get_pc function and a arch specific syscall_cancel implementation. The powerpc requires an arch-specific syscall_cancel because INTERNAL_SYSCALL_NCS adds a mfcr just after the sc instruction to get the CR0.SO bit information from kernel (which signals the error return status). So for cancelled syscalls with side effects, __pthread_get_pc will point to mcfr and thus invalidating the checks on sigcancel_handler. Checked on powerpc64le-linux-gnu and powerpc-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/powerpc/sigcontextinfo.h (ucontext_get_pc): New function. * sysdeps/unix/sysv/linux/powerpc/syscall_cancel.S: New file. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=2258ba74573f9265fea31606d2d1352514e9deb0 commit 2258ba74573f9265fea31606d2d1352514e9deb0 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon May 4 16:30:13 2015 -0300 nptl: i386: Fix Race conditions in pthread cancellation (BZ#12683) This patch adds the i386 modifications required for the BZ#12683. It basically provides the required ucontext_get_pc symbol, add the cancelable syscall wrapper and fix a thread atomic update macro. On i386 an arch-specific cancellation implementation is required because depending of the glibc configuration and underlying kernel the syscall may be done using a vDSO symbol (__kernel_vsyscall). By using the vDSO symbol the resulting PC value for an interrupted syscall points to an adress outside the expected markers in __syscall_cancel_arch. It has been discussed in LKML [1] on how kernel could help userland to accomplish it, but afaik discussion was stalled. Also, since glibc supports i486, the old 'int 0x80' should be used in the syscall wrapper. One option could make minimum default chip to pentium II (which implements sysenter) or add a runtime check on syscall_cancel.S to use 'int 0x80' or sysenter. Checked on i686-linux-gnu. [BZ #12683] * sysdeps/i386/nptl/tls.h (THREAD_ATOMIC_CMPXCHG_VAL, THREAD_ATOMIC_BIT_SET): Remove macro. * sysdeps/unix/sysv/linux/i386/Makefile [$(subdir) = elf] (sysdep-rtld_routines): Add libc-do-syscall object. * sysdeps/unix/sysv/linux/i386/lowlevellock.h (lll_wait_tid): Use cancellable futex syscall macro. * sysdeps/unix/sysv/linux/i386/syscall_cancel.S: New file. * sysdeps/unix/sysv/linux/i386/sigcontextinfo.h (ucontext_get_pc): New function. [1] https://lkml.org/lkml/2016/3/8/1105 https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=f43adba514ff665338e6da714551d8c3f6b158ce commit f43adba514ff665338e6da714551d8c3f6b158ce Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Sat May 9 14:20:26 2015 -0300 nptl: x32: Fix Race conditions in pthread cancellation (BZ#12683) This patches adds the x32 modification required for the BZ#12683. It basically adjust the syscall size used to pass the arguments to the syscall cancel wrappers by zero extending pointers type while preserting values for default types (such as off_t). Checked on x86_64-linux-gnu-x32. [BZ #12683] * sysdeps/unix/sysv/linux/x86_64/x32/sysdep.h (__syscall_arg_t): Define type for x32. (__SSC): Add platform specific macro. * include/libc-pointer-arith.h (__integer_if_pointer_type_sub, __integer_if_pointer_type, cast_to_integer): Parametrize integer type cast. (cast_to_uinteger): New macro. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=299d22ce112252b2a950e4395344dcf3791d73a9 commit 299d22ce112252b2a950e4395344dcf3791d73a9 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Sep 29 09:48:34 2014 -0300 nptl: x86_64: Fix Race conditions in pthread cancellation (BZ#12683) This patches adds the x86_64 modification required for the BZ#12683. It basically provide the required ucontext_get_pc symbol and remove the arch-specific libc-cancellation implementations. Checked on x86_64-linux-gnu. [BZ #12683] * sysdeps/unix/sysv/linux/x86_64/cancellation.S: Remove file. * sysdeps/unix/sysv/linux/x86_64/libc-cancellation.S: Remove file. * sysdeps/unix/sysv/linux/x86_64/librt-cancellation.S: Remove file. * sysdeps/unix/sysv/linux/x86_64/sigcontextinfo.h (ucontext_get_pc): New function. * sysdeps/x86_64/nptl/tcb-offsets.sym (TCB_CANCELING_BITMASK): Remove. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=6fe99352106cf8f244418f3708b3d5928e82e831 commit 6fe99352106cf8f244418f3708b3d5928e82e831 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Fri Sep 18 18:26:35 2015 -0300 nptl: Fix Race conditions in pthread cancellation (BZ#12683) This patches fixes some race conditions in NPTL cancellation code by redefining how cancellable syscalls are defined and handled. Current approach is to enable asynchronous cancellation prior to making the syscall and restore the previous cancellation type once the syscall returns. As decribed in BZ#12683, this approach shows 2 important problems: 1. Cancellation can act after the syscall has returned from kernel, but before userspace saves the return value. It might result in a resource leak if the syscall allocated a resource or a side effect (partial read/write), and there is no way to program handle it with cancellation handlers. 2. If a signal is handled while the thread is blocked at a cancellable syscall, the entire signal handler runs with asynchronous cancellation enabled. This can lead to issues if the signal handler call functions which are async-signal-safe but not async-cancel-safe. For cancellation to work correctly, there are 5 points at which the cancellation signal could arrive: 1. Before the final "testcancel" and before the syscall is made. 2. Between the "testcancel" and the syscall. 3. While the syscall is blocked and no side effects have yet taken place. 4. While the syscall is blocked but with some side effects already having taken place (e.g. a partial read or write). 5. After the syscall has returned. And GLIBC wants to act on cancellation in cases 1, 2, and 3 but not in case 4 or 5. The proposed solution follows: * Handling case 1 is trivial: do a conditional branch based on whether the thread has received a cancellation request; * Case 2 can be caught by the signal handler determining that the saved program counter (from the ucontext_t) is in some address range beginning just before the "testcancel" and ending with the syscall instruction. * In this case, except for certain syscalls that ALWAYS fail with EINTR even for non-interrupting signals, the kernel will reset the program counter to point at the syscall instruction during signal handling, so that the syscall is restarted when the signal handler returns. So, from the signal handler's standpoint, this looks the same as case 2, and thus it's taken care of. * In this case, the kernel cannot restart the syscall; when it's interrupted by a signal, the kernel must cause the syscall to return with whatever partial result it obtained (e.g. partial read or write). * In this case, the saved program counter points just after the syscall instruction, so the signal handler won't act on cancellation. This one is equal to 4. since the program counter is past the syscall instruction already. Another case that needs handling is syscalls that fail with EINTR even when the signal handler is non-interrupting. In this case, the syscall wrapper code can just check the cancellation flag when the errno result is EINTR, and act on cancellation if it's set. The proposed GLIBC adjustments are: 1. Remove the enable_asynccancel/disable_asynccancel function usage in syscall definition and instead make them call a common symbol that will check if cancellation is enabled (__syscall_cancel at nptl/libc-cancellation.c), call the arch-specific cancellable entry-point (__syscall_cancel_arch) and cancel the thread when required. 2. Provide a arch-specific symbol that contains global markers. These markers will be used in SIGCANCEL handler to check if the interruption has been called in a valid syscall and if the syscalls has been completed or not. A default version is provided (sysdeps/unix/sysv/linux/syscall_cancel.c), however the markers may not be set on correct expected places depeding of how INTERNAL_SYSCALL_NCS is implemented by the underlying architecture. In this case arch-specific implementation should be provided. 3. Rewrite SIGCANCEL asynchronous handler to check for both cancelling type and if current IP from signal handler falls between the global markes and act accordingly (sigcancel_handler at nptl/nptl-init.c). 4. Adjust nptl/pthread_cancel.c to send an signal instead of acting directly. This avoid synchronization issues when updating the cancellation status and also focus the logic on signal handler and cancellation syscall code. 5. Adjust pthread code to replace CANCEL_ASYNC/CANCEL_RESET calls to appropriated cancelable futex syscalls. 6. Adjust libc code to replace LIBC_CANCEL_ASYNC/LIBC_CANCEL_RESET to appropriated cancelable syscalls. 7. Adjust 'lowlevellock-futex.h' arch-specific implementations to provide cancelable futex calls (used in libpthread code). This patch adds the proposed changes to NPTL. The code leaves all the ports broken without further patches in the list. [BZ #12683] * manual/llio.texi: Adjust comments regarding pthread_enable_asynccancel and pthread_disable_asynccancel. * nptl/Makefile (routines): Add syscall_cancel object. (libpthread-routines): Add pthread_kill_internal and remove cancellation object. (CFLAGS-cancellation.c): Remove rule. (CFLAGS-syscall_cancel.c): New rule. (tests): Add tst-cancel28. * nptl/Versions [GLIBC_PRIVATE] (libc): Add __syscall_cancel, __syscall_cancel_arch_start, and __syscall_cancel_arch_end. * nptl/cancellation.c: Remove file. * sysdeps/nptl/librt-cancellation.c: Likewise. * nptl/descr.h (CANCELING_BIT, CANCELING_BITMASK): Remove define. (CANCELED_BIT, EXITING_BIT, TERMINATED_BIT, SETXID_BIT, CANCEL_RESTMASK): Adjust value with CANCELED_BIT removal. * nptl/libc-cancellation.c (__syscall_cancel): New symbol: symbol brigde for non-cancellable syscalls. (__syscall_do_cancel): New symbol. * nptl/nptl-init.c (sigcancel_handler): Rewrite function to avoid race conditions. (__pthread_initialize_minimal_internal): Add SA_RESTART to SIGCANCEL handler. (UCONTEXT_SIGMASK): New macro. * nptl/pthreadP.h (__do_cancel): Rewrite to both disable asynchronous cancellation and setting the thread as cancelled. (__do_cancel_with_result): New macro. (CANCEL_ASYNC, CANCEL_RESET, LIBC_CANCEL_ASYNC, LIBC_CANCEL_RESET, LIBC_CANCEL_HANDLED): Remove macros. (__syscall_cancel_arch, __syscall_do_cancel, __pthread_kill_internal): New prototypes. (__pthread_enable_asynccancel, __pthread_disable_asynccancel, __libc_enable_asynccancel, __libc_disable_asynccancel, __librt_enable_asynccancel, __librt_disable_asynccancel): Remove prototypes. * nptl/pthread_cancel.c (pthread_cancel): Rewrite to just set CANCELLED_BIT and call __pthread_kill. * nptl/pthread_create.c (START_THREAD_DEFN): Likewise. * nptl/pthread_exit.c (__pthread_exit): Call __do_cancel_with_result. * nptl/pthread_join_common.c (__pthread_timedjoin_ex): Likewise. * nptl/pthread_kill.c (__pthread_kill): Check internal signals with __is_internal_signal, tail call __pthread_kill_internal, and remove stub_warning. * nptl/pthread_kill_internal.c: New file. * nptl/tst-cancel28.c: Likewise. * rt/Makefile [CFLAGS-librt-cancellation.c]: Remove rule. * support/temp_file.c (support_create_temp_fifo): New function. * support/temp_file.h (support_create_temp_fifo): New prototype. * sysdeps/generic/sigcontextinfo.h (ucontext_get_pc): Likewise. * sysdeps/generic/sysdep-cancel.h (LIBC_CANCEL_ASYNC, LIBC_CANCEL_RESET): Remove define. * sysdeps/htl/pthreadP.h (__pthread_kill_internal): New prototype. * sysdeps/nptl/Makefile [$(subdir) = rt] (librt-sysdep_routines): Remove librt-cancellation object. [$(subdir) = rt] (CFLAGS-librt-cancellation.c): Remove rule. * nptl/lll_timedwait_tid.c (__lll_timedwait_tid): Use cancellable futex operation. * sysdeps/unix/sysdep.h (SYSCALL_CANCEL): Rewrite to call __syscall_cancel. (INTERNAL_SYSCALL_NCS_CALL, __INTERNAL_SYSCALL_NCS*, __SYSCALL_CANCEL*): New macros. * sysdeps/unix/sysv/linux/futex-internal.h (fuex_wait_cancel, futex_reltimed_wait_cancelable, futex_abstimed_wait_cancelable): Use cancelable futex wrapper. * sysdeps/unix/sysv/linux/lowlevellock-futex.h (lll_futex_syscall_cp, lll_futex_wait_cancel, lll_futex_timed_wait_cancel, lll_futex_timed_wait_bitset_cancel): New macros. * sysdeps/unix/sysv/linux/pthread_kill_internal.c: New file. * sysdeps/unix/sysv/linux/socketcall.h (SOCKETCALL): Use __SSC macros. (SOCKETCALL_CANCEL): Use SYSCALL_CANCEL macros. (__SOCKETCALL_CANCEL*): New macros. * sysdeps/unix/sysv/linux/syscall_cancel.c: New file. * sysdeps/unix/sysv/linux/sysdep.h (SYSCALL_CANCEL_RET): New macro. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=590b6e0aa785db2164870707244d4ae16a711277 commit 590b6e0aa785db2164870707244d4ae16a711277 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Thu May 10 17:24:56 2018 -0300 nptl: Cleanup cancellation macros This patch wraps all uses of *_{enable,disable}_asynccancel and and *_CANCEL_{ASYNC,RESET} in either already provided macros (lll_futex_timed_wait_cancel) or creates new ones if the functionality is not provided (SYSCALL_CANCEL_NCS, lll_futex_wait_cancel, and lll_futex_timed_wait_cancel). Also for some generic implementations, the direct call of the macros are removed since the underlying symbols are suppose to provide cancellation support. This is a priliminary patch intended to simplify the work required for BZ#12683 fix. It is a refactor change, no semantic changes are expected. Checked on x86_64-linux-gnu and i686-linux-gnu. * nptl/pthread_join_common.c (__pthread_timedjoin_ex): Use lll_wait_tid with timeout. * nptl/sem_wait.c (__old_sem_wait): Use lll_futex_wait_cancel. * sysdeps/nptl/aio_misc.h (AIO_MISC_WAIT): Use futex_reltimed_wait_cancelable for cancelabla mode. * sysdeps/nptl/gai_misc.h (GAI_MISC_WAIT): Likewise. * sysdeps/posix/open64.c (__libc_open64): Do not call cancelation macros. * sysdeps/posix/sigwait.c (__sigwait): Likewise. * sysdeps/posix/waitid.c (__sigwait): Likewise. * sysdeps/unix/sysdep.h (__SYSCALL_CANCEL_CALL, SYSCALL_CANCEL_NCS): New macro. * sysdeps/nptl/lowlevellock.h (lll_wait_tid): Add timeout argument. (lll_timedwait_tid): Remove macro. * sysdeps/unix/sysv/linux/i386/lowlevellock.h (lll_wait_tid): Likewise. (lll_timedwait_tid): Likewise. * sysdeps/unix/sysv/linux/sparc/lowlevellock.h (lll_wait_tid): Likewise. (lll_timedwait_tid): Likewise. * sysdeps/unix/sysv/linux/x86_64/lowlevellock.h (lll_wait_tid): Likewise. (lll_timedwait_tid): Likewise. * sysdeps/unix/sysv/linux/clock_nanosleep.c (__clock_nanosleep): Use SYSCALL_CANCEL_NCS. * sysdeps/unix/sysv/linux/futex-internal.h (futex_reltimed_wait_cancelable): Use LIBC_CANCEL_{ASYNC,RESET} instead of __pthread_{enable,disable}_asynccancel. * sysdeps/unix/sysv/linux/lowlevellock-futex.h (lll_futex_wait_cancel): New macro. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=323ab50e88231adaf93d3bdf42b15612f2795ce9 commit 323ab50e88231adaf93d3bdf42b15612f2795ce9 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Wed May 9 10:39:49 2018 -0300 i386: Remove bogus THREAD_ATOMIC_* macros The x86 defines optimized THREAD_ATOMIC_* macros where reference always the current thread instead of the one indicated by input 'descr' argument. It work as long the input is the self thread pointer, however it generates wrong code is the semantic is to set a bit atomicialy from another thread. This is not an issue for current GLIBC usage, however the new cancellation code expects that some synchronization code to atomically set bits from different threads. If some usage indeed proves to be a hotspot we can add an extra macro with a more descriptive name (THREAD_ATOMIC_BIT_SET_SELF for instance) where i386 might optimize it. Checked on i686-linux-gnu. * sysdeps/i686/nptl/tls.h (THREAD_ATOMIC_CMPXCHG_VAL, THREAD_ATOMIC_AND, THREAD_ATOMIC_BIT_SET): Remove macros. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=b6838cd772c01dca2cf49217d4af8bcfdbe8eeb8 commit b6838cd772c01dca2cf49217d4af8bcfdbe8eeb8 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Wed May 9 10:32:25 2018 -0300 x86: Remove wrong THREAD_ATOMIC_* macros The x86 defines optimized THREAD_ATOMIC_* macros where reference always the current thread instead of the one indicated by input 'descr' argument. It work as long the input is the self thread pointer, however it generates wrong code is the semantic is to set a bit atomicialy from another thread. This is not an issue for current GLIBC usage, however the new cancellation code expects that some synchronization code to atomically set bits from different threads. The generic code generates an additional load to reference to TLS segment, for instance the code: THREAD_ATOMIC_BIT_SET (THREAD_SELF, cancelhandling, CANCELED_BIT); Compiles to: lock;orl $4, %fs:776 Where with patch changes it now compiles to: mov %fs:16,%rax lock;orl $4, 776(%rax) If some usage indeed proves to be a hotspot we can add an extra macro with a more descriptive name (THREAD_ATOMIC_BIT_SET_SELF for instance) where x86_64 might optimize it. Checked on x86_64-linux-gnu. * sysdeps/x86_64/nptl/tls.h (THREAD_ATOMIC_CMPXCHG_VAL, THREAD_ATOMIC_AND, THREAD_ATOMIC_BIT_SET): Remove macros. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=42794ba6346307769c5836e51c0b85b988b00732 commit 42794ba6346307769c5836e51c0b85b988b00732 Author: Adhemerval Zanella <adhemerval.zanella@linaro.com> Date: Mon Sep 21 15:55:58 2015 -0700 nptl: Fix testcases for new pthread cancellation mechanism With upcoming fix for BZ#12683, pthread cancellation does not act for: 1. If syscall is blocked but with some side effects already having taken place (e.g. a partial read or write). 2. After the syscall has returned. The main change is due the fact programs need to act in syscalls with side-effects (for instance, to avoid leak of allocated resources or handle partial read/write). This patch changes the NPTL testcase that assumes the old behavior and also remove the tst-cancel-wrappers.sh test (which checks for symbols that will not exist anymore). It also changes how to call the read syscall on tst-backtrace{5,6} to use syscall instead of read cancelable syscall to avoid need to handle the cancelable bridge function calls. Checked on i686-linux-gnu, x86_64-linux-gnu, x86_64-linux-gnux32, aarch64-linux-gnu, arm-linux-gnueabihf, powerpc64le-linux-gnu, powerpc-linux-gnu, sparcv9-linux-gnu, and sparc64-linux-gnu. * debug/tst-backtrace5.c (handle_signal): Check for syscall instead of read. (fn): Issue the read syscall instead of call the cancellable syscall. * nptl/tst-cancel4.c (tf_write): Handle cancelled syscall with side-effects. (tf_send): Likewise. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=00278aa8a3fffa4cb1d9a7b882b84e512aa09ae9 commit 00278aa8a3fffa4cb1d9a7b882b84e512aa09ae9 Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Tue May 8 16:28:18 2018 -0300 nptl: Remove tst-cancel-wrappers test and related macros With upcoming BZ#12683 fix, syscall cancellation is not more handled by {libc,pthread,librt}_{enable,disable}_asynccancel symbols. This renders both LIBC_CANCEL_HANDLED and empty declaration and tst-cancel-wrappers.sh unrequired. This patch removes both the macro and the nptl test. Checked on x86_64-linux-gnu. * io/creat.c (LIBC_CANCEL_HANDLED): Remove macro. * io/ppoll.c (LIBC_CANCEL_HANDLED): Likewise. * misc/pselect.c (LIBC_CANCEL_HANDLED): Likewise. * nptl/pthreadP.h (LIBC_CANCEL_HANDLED): Likewise. * sysdeps/generic/sysdep-cancel.h (LIBC_CANCEL_HANDLED): Likewise. * sysdeps/posix/pause.c (LIBC_CANCEL_HANDLED): Likewise. * sysdeps/posix/sigpause.c (LIBC_CANCEL_HANDLED): Likewise. * sysdeps/unix/sysv/linux/creat.c (LIBC_CANCEL_HANDLED): Likewise. * sysdeps/unix/sysv/linux/creat64.c (LIBC_CANCEL_HANDLED): Likewise. * sysdeps/unix/sysv/linux/sigwait.c (LIBC_CANCEL_HANDLED): Likewise. * sysdeps/unix/sysv/linux/sigwaitinfo.c (LIBC_CANCEL_HANDLED): Likewise. * nptl/Makefile [$(run-built-tests) = yes] (tests-special): Remove tst-cancel-wrappers.sh. (generated): Remove tst-cancel-wrappers.out. (tst-cancel-wrappers.out): Remove rule. * nptl/tst-cancel-wrappers.sh: Remove file. https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=d6ed8a26a2b63f28721c4adfca00de8da83238ff commit d6ed8a26a2b63f28721c4adfca00de8da83238ff Author: Adhemerval Zanella <adhemerval.zanella@linaro.org> Date: Mon Feb 26 17:52:02 2018 -0300 powerpc: Add CFI information on indirect syscall This patch the required CFI information on powerpc indirect syscall so backtrace works correctly on signal handler. Checked on powerpc-linux-gnu and powerpc64le-linux-gnu. * sysdeps/unix/sysv/linux/powerpc/syscall.S (syscall): Add CFI information. -----------------------------------------------------------------------