[newlib-cygwin/cygwin-3_6-branch] Cygwin: thread: Fix stack alignment for PTHREAD_CANCEL_ASYNCHRONOUS

[Takashi Yano]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=9580d7f3792f6635382ef8220cfbea802be8dd5e

commit 9580d7f3792f6635382ef8220cfbea802be8dd5e
Author: Takashi Yano <takashi.yano@nifty.ne.jp>
Date:   Wed Dec 24 03:14:03 2025 +0900

    Cygwin: thread: Fix stack alignment for PTHREAD_CANCEL_ASYNCHRONOUS
    
    The test case winsup/testsuites/winsup.api/pthread/cancel2 fails
    on Windows 11 and Windows Server 2025, while it works on Windows 10
    and Windows Server 2022. PTHREAD_CANCEL_ASYNCHRONOUS is implemented
    by forcing the target thread's instruction pointer (IP) to pthread::
    static_cancel_self() using [GS]etThreadContext(). static_cancel_self()
    will call Windows API function during thread shutdown. A misaligned
    stack will lead to unexpected exceptions.
    
    Previously, the stack pointer was not maintained to 16-byte alignment,
    even though this is required by 64-bit Windows ABI. At the start of
    the function prologue, the stack is expected to be at an offset of
    8 byte from 16-byte boundary (SP % 16 == 8) in x86_64 architecture,
    as the call instruction has just pushed the return IP onto the stack.
    
    However, this appears to have been overlooked when cygwin first added
    x86_64 support.
    
    This patch fixes this issue by aligning the stack pointer as well as
    the instruction pointer in the PTHREAD_CANCEL_ASYNCHRONOUS handling.
    
    Addresses: https://cygwin.com/pipermail/cygwin/2025-December/259117.html
    Fixes: 61522196c715 ("* Merge in cygwin-64bit-branch.")
    Reported-by: Takashi Yano <takashi.yano@nifty.ne.jp>
    Reviewed-by: Jon Turney <jon.turney@dronecode.org.uk>
    Signed-off-by: Takashi Yano <takashi.yano@nifty.ne.jp>
    (cherry picked from commit b41e3b652c4215556a62a2f47e8f8f11553fa550)

Diff:
---
 winsup/cygwin/thread.cc | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/winsup/cygwin/thread.cc b/winsup/cygwin/thread.cc
index 9ee96504b..d410b9575 100644
--- a/winsup/cygwin/thread.cc
+++ b/winsup/cygwin/thread.cc
@@ -629,7 +629,27 @@ pthread::cancel ()
       threadlist_t *tl_entry = cygheap->find_tls (cygtls);
       if (!cygtls->inside_kernel (&context))
 	{
+#if defined(__x86_64__)
+	  /* Need to maintain the alignment of the stack pointer.
+	     https://learn.microsoft.com/en-us/cpp/build/stack-usage?view=msvc-170
+	     states,
+	       "The stack will always be maintained 16-byte aligned,
+		except within the prolog (for example, after the return
+		address is pushed),",
+	     that is, we need 16n + 8 byte alignment here because the stack
+	     pointer must be maintaiined to the same alignment required by
+	     the function prologue. Since the call instruction pushes the
+	     return address (rip) onto the stack, which is 8 bytes,
+	     an additional 8 bytes is required to emulate this behaviour.
+	     However, we do not need to push return address itself, because
+	     pthread::static_cancel_self() must not return. */
+	  context.Rsp &= ~0x07UL;
+	  if ((context.Rsp & 8) == 0)
+	    context.Rsp -= 8;
 	  context.Rip = (ULONG_PTR) pthread::static_cancel_self;
+#else
+#error unimplemented for this target
+#endif
 	  SetThreadContext (win32_obj_id, &context);
 	}
       cygheap->unlock_tls (tl_entry);