RE: winlayer.c fails to build

[Alexander Gottwald <Alexander dot Gottwald at informatik dot tu-chemnitz dot de>]

On Sun, 29 Jul 2001, Harold Hunt wrote:

> > However, you will still need to apply the following patch, else
> > the X Server
> > will crash on startup if the WIN_LAYER_SUPPORT flag, in win.h, is set to
> > 'YES':
> > http://www.msu.edu/~huntharo/xwin/layerinit.c.diff
> 
> Do be aware that enabling the WIN_LAYER_SUPPORT flag causes the X Server to
> give a STATUS_ACCESS_VIOLATION on shutdown.  That is a known problem, and
> I'd appreciate it if someone would figure out how to fix it so that we can
> enable layer support permanently.

A debug session showed, that PScreen->CloseScreen is called for all initialized 
components. The crash ocurs in layerinit.c in layerCloseScreen:350 
"xfree(shadowGetScrPriv(pScreen))" since the data was already freed in 
shadowCloseScreen. It seem's they wanted to catch this with the if statement
"if (kind != LAYER_SHADOW)". LAYER_SHADOW is defined in layer.h as 1, 
while LAYER_FB is defined as 0. In the debug session, kind is always 0 - 
which equals to LAYER_FB.

It seems they expect the fb-layer to be initialized before the shadow layer. 
The fb initialization was commented out by your patch. What was the reason for
this. 

I see two ways to fix it.
- let shadowCloseScreen set the privPtr to NULL after freeing it (xfree will
        ignore such pointers)

    xfree (pScrPriv);
+   shadowGetScrPriv(pScreen) = NULL;
    return (*pScreen->CloseScreen) (i, pScreen);
            
- let LayerStartInit initialize the fb screen. 
    server start - close cycle worked for engine 1,2 and 4.
    Engine 8 failed with winAllocateFBPrimaryDD-Could not lock primary surface
    But it also does with the fb layer initialization commented out
    
bye
    ago
-- 
Alexander.Gottwald@s1999.tu-chemnitz.de
http://www.gotti.org
phone: +49 3725 3498080     mobile: +49 172 7854017