[PATCH] Ensure rl_get_screen_size() returns the actual terminal dimensions

[Patrick Palka]

On Thu, Sep 4, 2014 at 6:52 AM, Pedro Alves <palves@redhat.com> wrote:
> On 09/04/2014 11:48 AM, Pedro Alves wrote:
>> On 09/01/2014 02:25 AM, Patrick Palka wrote:
>>> We should call rl_resize_terminal() before calling rl_get_screen_size()
>>> to help ensure that rl_get_screen_size() will return the correct
>>> terminal dimensions.  Doing so fixes a couple issues where TUI does not
>>> correctly resize itself due to rl_get_screen_size() returning stale
>>> terminal dimensions.
>>>
>>>      * tui/tui-win.c (tui_resize_all): Call rl_resize_terminal before
>>>      calling rl_get_screen_size.
>>
>> OK.
>
> Actually, I take that back, for a moment.
>
> readline's rl_sigwinch_handler already call rl_resize_terminal, and then
> chains into the TUI's handler.  So, why do we need this?
>
> Thanks,
> Pedro Alves
>

Sometimes we overwrite readline's screen size by calling
tui_update_gdb_sizes() like done in tui_enable().  When that's done
rl_get_screen_size() will not return the real terminal dimensions, but
rather the dimensions that were set by tui_update_gdb_sizes().  So in
the following scenario the TUI will get resized to the wrong
dimensions:

0. Run GDB.
1. Enter TUI mode.
2. Exit TUI mode.
3. Resize the terminal.
4. Enter TUI mode again.
5. Press a key so that the TUI will resize itself.

Here we resize the terminal (causing RL's SIGWINCH handler to run,
updating RL's idea of the terminal size).  Then we enter TUI mode
which calls tui_update_gdb_sizes() which resets RL's idea of the
terminal size.  Then we call rl_get_screen_size() in tui_getc() which
returns not the terminal dimensions but the dimensions of the TUI's
command window as set by tui_update_gdb_sizes().  So in the end the
TUI gets resized to incorrect dimensions.

Also, when we are blocking inside a secondary prompt, readline's
SIGWINCH handler does not seem to trigger for some reason. So when we
resize the window during a secondary prompt, our SIGWINCH handler gets
triggered but readline's handler doesn't which means that the
dimensions returned by rl_get_screen_size() will be wrong.  I'm not
really sure why this happens.

Explicitly calling rl_resize_terminal() fixes both of these issues.

Patrick