Bug 10165 - windres crashes when processing resource files with UTF-8 (cp 65001) encoding.
Summary: windres crashes when processing resource files with UTF-8 (cp 65001) encoding.
Alias: None
Product: binutils
Classification: Unclassified
Component: binutils (show other bugs)
Version: 2.19
: P2 normal
Target Milestone: ---
Assignee: unassigned
Depends on:
Reported: 2009-05-17 22:48 UTC by Tom Bramer
Modified: 2014-05-28 19:40 UTC (History)
1 user (show)

See Also:
Host: i686-pc-mingw32
Target: i686-pc-mingw32
Build: i686-pc-mingw32
Last reconfirmed:

Proposed patch to windres (409 bytes, patch)
2009-05-17 22:49 UTC, Tom Bramer
Details | Diff
Test case input. (104 bytes, text/plain)
2009-05-17 22:51 UTC, Tom Bramer

Note You need to log in before you can comment on or make changes to this bug.
Description Tom Bramer 2009-05-17 22:48:42 UTC
OS: Windows XP SP2

windres, version 2.18 and up, when processing an rc file that contains at least
one quoted string, will crash with an access violation (on dereference of NULL
pointer) if the code page is explicitly set to 65001 (UTF-8).  

Here is an example input that causes the issue:

======================== BEGIN =========================
#define IDS_STRING1 1
#pragma code_page(65001)

        IDS_STRING1 "Any string will do..."
========================= END ==========================

The cause:

When calling the MultiByteToWideChar function with a code page of 65000, 65001,
or others (see http://msdn.microsoft.com/en-us/library/dd319072(VS.85).aspx),
the dwFlags parameter must be 0, unless when using Windows XP and later, in
which it may be 0 or MB_ERR_INVALID_CHARS.  If the value given for dwFlags is in
fact set to a value other than the aforementioned valid cases, the function will
set the last error to ERROR_INVALID_FLAGS.  The wind_MultiByteToWideChar
function always passes MB_PRECOMPOSED as the dwFlags parameter to
MultiByteToWideChar, causing the function to fail and not convert the UTF-7/8
string to UTF-16.  The wind_MultiByteToWideChar function assumes that the
MultiByteToWideChar function is successful, when in reality, the wide character
buffer still contains uninitialized data.

What worked for me:

======================== BEGIN =========================
--- binutils-2.19.1/binutils/winduni.c	2007-07-05 12:54:44 -0400
+++ binutils-2.19.1-new/binutils/winduni.c	2009-05-17 17:52:33 -0400
@@ -661,7 +661,15 @@ wind_MultiByteToWideChar (rc_uint_type c
   rc_uint_type ret = 0;
 #if defined (_WIN32) || defined (__CYGWIN__)
-  ret = (rc_uint_type) MultiByteToWideChar (cp, MB_PRECOMPOSED,
+  rc_uint_type conv_flags = MB_PRECOMPOSED;
+  /* MB_PRECOMPOSED is not allowed for UTF-7 or UTF-8.  
+     MultiByteToWideChar will set the last error to ERROR_INVALID_FLAGS
+     if we do anyways. */
+  if (cp == CP_UTF8 || cp == CP_UTF7)
+      conv_flags = 0;
+  ret = (rc_uint_type) MultiByteToWideChar (cp, conv_flags,
 					    mb, -1, u, u_len);
   /* Convert to bytes. */
   ret *= sizeof (unichar);
======================== END =========================
Comment 1 Tom Bramer 2009-05-17 22:49:48 UTC
Created attachment 3940 [details]
Proposed patch to windres
Comment 2 Tom Bramer 2009-05-17 22:51:13 UTC
Created attachment 3941 [details]
Test case input.
Comment 3 cvs-commit@gcc.gnu.org 2009-06-09 15:14:45 UTC
Subject: Bug 10165

CVSROOT:	/cvs/src
Module name:	src
Changes by:	nickc@sourceware.org	2009-06-09 15:14:23

Modified files:
	binutils       : ChangeLog winduni.c 

Log message:
	PR 10165
	* winduni.c (wind_MultiByteToWideChar): Do not pass MB_PRECOMPOSED
	to MultiByteToWideChar when using the CP_UTF8 or CO_UTF7 types.


Comment 4 Nick Clifton 2009-06-09 15:16:22 UTC
Hi Tom,

  Thanks for reporting this bug and supplying a patch to fix it.

  I have checked in your patch along with the changelog entry below.


2009-06-09  Tom Bramer  <tjb@postpro.net>

	PR 10165
	* winduni.c (wind_MultiByteToWideChar): Do not pass MB_PRECOMPOSED
	to MultiByteToWideChar when using the CP_UTF8 or CO_UTF7 types.
Comment 5 Jackie Rosen 2014-02-16 17:44:57 UTC Comment hidden (spam)