Fixing the Cygwin/XFree86 mwm numlock bug

[Harold L Hunt <huntharo at msu dot edu>]

I am the project leader and primary developer for the Cygwin/XFree86 project
and the maintainer of the Cygwin setup.exe-installable Lesstif package.  I am
extremely interested in working with you guys to figure out why one cannot
move windows in mwm when the NumLock key is toggled on.

I have subscribed to the Lesstif mailing list for however long it takes to fix
this bug.

I have done a search of the Lesstif mailing list and found that the problem is
acknowledged but that someone on the Cygwin/XFree86 side was needed for
debugging.  I searched for ``mwm cygwin site:terror.hungry.com'' at google.com:

http://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=UTF-8&q=mwm+cygwin+site%3Aterror.hungry.com&btnG=Google+Search


Description of the problem
==========================
Running mwm within Cygwin/XFree86 with the NumLock key toggled on causes mwm
windows to be unmovable with the mouse.  Toggling the NumLock key off allows
the mwm windows to be moved.


The problem seems to arise from the fact that Cygwin/XFree86 treats NumLock as
a modifier key, while, apparently, older versions of X11 did not consider
NumLock to be a modifier key.


Relation to the ``kb'' extension
================================
The kb extension is enabled in Cygwin/XFree86 by default.  I just found out,
and I don't recall ever having been told this before, that if you run
Cygwin/XFree86 with the ``-kb'' parameter, which disables the kb extension,
then mwm windows are movable with the NumLock key toggled on or off. 
Essentially, the problem goes away if you turn off the kb extension.


Pondering as to what could cause the problem
============================================
I do not understand why Cygwin/XFree86 exhibits this problem, but XFree86 on
Linux does not seem to.  As I describe below, Cygwin/XFree86 took its keyboard
modifier setup code directly from XFree86.  Maybe XFree86 on Linux does not
turn on the kb extension by default?


Very thorough details
=====================
Cygwin/XFree86 sets up its modifier keys in winkeybd.c/winGetKeyMappings,
which can be viewed here:
http://cvsweb.xfree86.org/cvsweb/xc/programs/Xserver/hw/xwin/winkeybd.c

Note that WIN_NEW_KEYBOARD_SUPPORT is currently 0, or off, so you need to skip
the #if WIN_NEW_KEYBOARD_SUPPORT section at the top of the file.

In winGetKeyMappings we set the NumLockMask for the XK_Num_Lock key:
	case XK_Num_Lock:
	  pModMap[i] = NumLockMask;
	  break;

We copied this functionality directly from
xc/programs/Xserver/hw/xfree86/common/xf86Kbd.c's xf86KbdGetMapping function.

NumLockMask is defined over in
xc/programs/Xserver/hw/xfree86/common/atKeynames.h as:
     #define AltMask		Mod1Mask
     #define NumLockMask	Mod2Mask
     #define AltLangMask	Mod3Mask
     #define KanaMask	Mod4Mask
     #define ScrollLockMask	Mod5Mask

Of course, the ModXMasks are defined in xc/include/X.h as:
     #define ShiftMask		(1<<0)
     #define LockMask		(1<<1)
     #define ControlMask		(1<<2)
     #define Mod1Mask		(1<<3)
     #define Mod2Mask		(1<<4)
     #define Mod3Mask		(1<<5)
     #define Mod4Mask		(1<<6)
     #define Mod5Mask		(1<<7)

The xkb extension also seems to have some sort of NumLockMask, over in
xc/programs/Xserver/xkb/xkbDflts.h:
     #define	vmod_NumLockMask	(1<<0)
     #define	vmod_AltMask	(1<<1)
     #define	vmod_LevelThreeMask	(1<<2)
     #define	vmod_AltGrMask	(1<<3)
     #define	vmod_ScrollLockMask	(1<<4)

======================================================================

Any help that the Lesstif developers can give would be greatly appreciated.  I
have yet to come across anyone that understands fully what the xkb extension
does and how to interface with it.  Perhaps you guys know more deep magic than
we do :)

Thanks very much in advance,

Harold Hunt