Bug 15548

Summary: Limit completions to a pre-defined limit when completions list is very large
Product: gdb Reporter: Phil Muldoon <pmuldoon>
Component: cliAssignee: Not yet assigned to anyone <unassigned>
Status: RESOLVED FIXED    
Severity: enhancement CC: dcl441-bugs, dje, dmalcolm, gbenson, jan, maayan, manu, rscrihf+sourceware, saugustine, xdje42, yhager
Priority: P2    
Version: unknown   
Target Milestone: ---   
Host: Target:
Build: Last reconfirmed:

Description Phil Muldoon 2013-05-29 11:39:51 UTC
Related: http://sourceware.org/bugzilla/show_bug.cgi?id=9007

When you complete on a symbol with a wide-scope expression (ie bt _<tab>), GDB can sometimes take a very long time to compile a list of symbols for completion.  This large symbol list is useless anyway, nobody is going to comb through 70,000 symbols to find the right one.

In these cases, we should limit the number of symbols iterated to a set limit before we bail out, and declare the symbol expression too broad for completion.  This will signal to the user to enter a more tightly defined expression (which they would do, anyway), and not frustrate them with a long wait while GDB grinds through thousands of symbols to provide a completion list that is unusable.
Comment 1 dje 2013-06-03 17:24:36 UTC
IIRC, This has been proposed before, and was rejected. [fyi]

It would be nice to get this implemented.
Comment 2 dje 2013-06-03 19:25:33 UTC
One problem I see with implementing this (and hopefully I'm either wrong or will be easy to fix) is that readline doesn't work the way we want it to for this purpose.  readline first finds all completions and *then* asks if the user wants to see them all.  Instead, we need readline to, say, first fetch the first N+1 items, and then if it returns N+1 then ask the user if s/he wants to see them all (or the next N, or whatever).
Comment 3 dje 2013-06-03 20:20:00 UTC
My bad, seems like this can be handled entirely within gdb.
[no readline changes needed: it's gdb's completers that construct the list of all completions, hopefully there's no reason why we can't just have them query the user whether s/he wants to continue.
doesn't make much sense for e.g., signal_completer, other than wanting the max #completions to apply to all completers - seems excessive]
Comment 4 Manuel López-Ibáñez 2013-06-25 10:08:54 UTC
Any hints how this could be fixed? I may be willing to give it a try. It is such a pain the ass.
Comment 5 Gary Benson 2013-10-25 19:01:20 UTC
(In reply to dje from comment #1)
> IIRC, This has been proposed before, and was rejected. [fyi]
> 
> It would be nice to get this implemented.

Doug, do you remember roughly when this was?  I found this thread:
https://sourceware.org/ml/gdb-patches/2008-05/threads.html#00385
https://sourceware.org/ml/gdb-patches/2008-06/threads.html#00042
but as far as I can see that was approved and committed.
Comment 6 dje 2013-10-25 19:09:22 UTC
(In reply to Gary Benson from comment #5)
> (In reply to dje from comment #1)
> > IIRC, This has been proposed before, and was rejected. [fyi]
> > 
> > It would be nice to get this implemented.
> 
> Doug, do you remember roughly when this was?  I found this thread:
> https://sourceware.org/ml/gdb-patches/2008-05/threads.html#00385
> https://sourceware.org/ml/gdb-patches/2008-06/threads.html#00042
> but as far as I can see that was approved and committed.

I can't remember.  I'd just go ahead.
If you do, let me know because I've been working on it in bits and pieces but am happy for someone else to work on this.
Comment 7 Gary Benson 2013-10-25 19:36:59 UTC
I plan to start Monday.  Have you made much progress?  (Is there a branch somewhere we can maybe collaborate on?)
Comment 8 Gary Benson 2013-11-04 13:02:47 UTC
See also bug 11920.
Comment 9 Gary Benson 2014-02-14 13:36:54 UTC
After having implemented this (https://sourceware.org/ml/gdb-patches/2014-02/msg00398.html) I'm coming to the conclusion that you can't rememdy this problem this way.

Here's an example where it can work:
  $ gdb /usr/lib64/libreoffice/program/soffice.bin 
  (gdb) start
  (gdb) b <Tab>

GDB tries to expand every symbol table.  With a limit of 1000 completions, GDB  quickly hits the limit and returns control to the user.

Here's an example of where this method fails:
  $ gdb /usr/lib64/libreoffice/program/soffice.bin 
  (gdb) start
  (gdb) b m<Tab>

This list only has 875 candidates, but GDB still has to expand pretty much every symbol table to root them all out.  A limit of 1000 here will not stop the user seeing a lengthy pause.

I propose to continue discussing this issue as bug 11920, which refers to the problem (namely that tab completion might stall the CLI without feedback) rather than to this specific solution.

I will close this bug as RESOLVED INVALID in one week if nobody objects.
Comment 10 Jan Kratochvil 2014-02-14 13:45:03 UTC
I think you are hitting here the problem of Fedora debuginfo dwz PR 16405 which I (or anyone) has not fixed yet.  On dwz debuginfo GDB has problem to resolve just several symbols (in a backtrace) because it (needlessly) expands tons of dwz-included CUs.

You should try it on debuginfo without dwz (such as RHEL-6 etc.).
Comment 11 Gary Benson 2014-02-14 16:13:55 UTC
As discussed on IRC, I'm using RHEL 6.5 so there should be no dwz.
Comment 12 Maayan Keshet 2014-09-21 08:54:51 UTC
Maybe it's difficult to resolve thoroughly, as the problem is the amount of symbol tables and not amount of symbols (so it can happen with vastly varying amount of symbols).

But for the meantime, can we have an option to disable the tab completion when the inputted prefix is < $SOME_NUM? e.g. if I enter the command "noshorttab" and do "b foo<tab>" nothing will happen.
Comment 13 Manuel López-Ibáñez 2014-09-29 11:43:14 UTC
(In reply to Gary Benson from comment #9)
> After having implemented this
> (https://sourceware.org/ml/gdb-patches/2014-02/msg00398.html) I'm coming to
> the conclusion that you can't rememdy this problem this way.

At least you can alleviate it!

> This list only has 875 candidates, but GDB still has to expand pretty much
> every symbol table to root them all out.  A limit of 1000 here will not stop
> the user seeing a lengthy pause.

The two issues seem to be important to fix. And the issue here already has a patch that is basically approved!

The problem is not only a lengthy pause (which is annoying but it is a different issue), it is also that after the pause, Emacs may spend a several *minutes* printing candidates, slowing the whole computer down with it. A limit of 1000 seems almost too much, 150 or so seems much more manageable.

Please, can you commit your patch so at least there is some progress on this?
Comment 14 Gary Benson 2014-11-06 11:16:25 UTC
New patch mailed:
https://sourceware.org/ml/gdb-patches/2014-11/msg00113.html
Comment 15 Sourceware Commits 2015-01-31 23:15:07 UTC
The master branch has been updated by Doug Evans <devans@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=ef0b411a110cd2602cb89c3fb237baf8beb28545

commit ef0b411a110cd2602cb89c3fb237baf8beb28545
Author: Gary Benson <gbenson@redhat.com>
Date:   Sat Jan 31 15:07:22 2015 -0800

    Add max-completions parameter, and implement tab-completion limiting.
    
    This commit adds a new exception, MAX_COMPLETIONS_REACHED_ERROR, to be
    thrown whenever the completer has generated too many candidates to
    be useful.  A new user-settable variable, "max_completions", is added
    to control this behaviour.  A top-level completion limit is added to
    complete_line_internal, as the final check to ensure the user never
    sees too many completions.  An additional limit is added to
    default_make_symbol_completion_list_break_on, to halt time-consuming
    symbol table expansions.
    
    gdb/ChangeLog:
    
    	PR cli/9007
    	PR cli/11920
    	PR cli/15548
    	* cli/cli-cmds.c (complete_command): Notify user if max-completions
    	reached.
    	* common/common-exceptions.h (enum errors)
    	<MAX_COMPLETIONS_REACHED_ERROR>: New value.
    	* completer.h (get_max_completions_reached_message): New declaration.
    	(max_completions): Likewise.
    	(completion_tracker_t): New typedef.
    	(new_completion_tracker): New declaration.
    	(make_cleanup_free_completion_tracker): Likewise.
    	(maybe_add_completion_enum): New enum.
    	(maybe_add_completion): New declaration.
    	(throw_max_completions_reached_error): Likewise.
    	* completer.c (max_completions): New global variable.
    	(new_completion_tracker): New function.
    	(free_completion_tracker): Likewise.
    	(make_cleanup_free_completion_tracker): Likewise.
    	(maybe_add_completions): Likewise.
    	(throw_max_completions_reached_error): Likewise.
    	(complete_line): Remove duplicates and limit result to max_completions
    	entries.
    	(get_max_completions_reached_message): New function.
    	(gdb_display_match_list): Handle max_completions.
    	(_initialize_completer): New declaration and function.
    	* symtab.c: Include completer.h.
    	(completion_tracker): New static variable.
    	(completion_list_add_name): Call maybe_add_completion.
    	(default_make_symbol_completion_list_break_on_1): Renamed from
    	default_make_symbol_completion_list_break_on.  Maintain
    	completion_tracker across calls to completion_list_add_name.
    	(default_make_symbol_completion_list_break_on): New function.
    	* top.c (init_main): Set rl_completion_display_matches_hook.
    	* tui/tui-io.c: Include completer.h.
    	(tui_old_rl_display_matches_hook): New static global.
    	(tui_rl_display_match_list): Notify user if max-completions reached.
    	(tui_setup_io): Save/restore rl_completion_display_matches_hook.
    	* NEWS (New Options): Mention set/show max-completions.
    
    gdb/doc/ChangeLog:
    
    	* gdb.texinfo (Command Completion): Document new
    	"set/show max-completions" option.
    
    gdb/testsuite/ChangeLog:
    
    	* gdb.base/completion.exp: Disable completion limiting for
    	existing tests.  Add new tests to check completion limiting.
    	* gdb.linespec/ls-errs.exp: Disable completion limiting.
Comment 16 Sourceware Commits 2015-01-31 23:27:02 UTC
The master branch has been updated by Doug Evans <devans@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=cfb069a8bebfacaf00dee6446e86a856978670be

commit cfb069a8bebfacaf00dee6446e86a856978670be
Author: Gary Benson <gbenson@redhat.com>
Date:   Sat Jan 31 15:24:26 2015 -0800

    ChangeLog entries for max-completions patch.
    
    gdb/ChangeLog:
    
    	PR cli/9007
    	PR cli/11920
    	PR cli/15548
    	* cli/cli-cmds.c (complete_command): Notify user if max-completions
    	reached.
    	* common/common-exceptions.h (enum errors)
    	<MAX_COMPLETIONS_REACHED_ERROR>: New value.
    	* completer.h (get_max_completions_reached_message): New declaration.
    	(max_completions): Likewise.
    	(completion_tracker_t): New typedef.
    	(new_completion_tracker): New declaration.
    	(make_cleanup_free_completion_tracker): Likewise.
    	(maybe_add_completion_enum): New enum.
    	(maybe_add_completion): New declaration.
    	(throw_max_completions_reached_error): Likewise.
    	* completer.c (max_completions): New global variable.
    	(new_completion_tracker): New function.
    	(free_completion_tracker): Likewise.
    	(make_cleanup_free_completion_tracker): Likewise.
    	(maybe_add_completions): Likewise.
    	(throw_max_completions_reached_error): Likewise.
    	(complete_line): Remove duplicates and limit result to max_completions
    	entries.
    	(get_max_completions_reached_message): New function.
    	(gdb_display_match_list): Handle max_completions.
    	(_initialize_completer): New declaration and function.
    	* symtab.c: Include completer.h.
    	(completion_tracker): New static variable.
    	(completion_list_add_name): Call maybe_add_completion.
    	(default_make_symbol_completion_list_break_on_1): Renamed from
    	default_make_symbol_completion_list_break_on.  Maintain
    	completion_tracker across calls to completion_list_add_name.
    	(default_make_symbol_completion_list_break_on): New function.
    	* top.c (init_main): Set rl_completion_display_matches_hook.
    	* tui/tui-io.c: Include completer.h.
    	(tui_old_rl_display_matches_hook): New static global.
    	(tui_rl_display_match_list): Notify user if max-completions reached.
    	(tui_setup_io): Save/restore rl_completion_display_matches_hook.
    	* NEWS (New Options): Mention set/show max-completions.
    
    gdb/doc/ChangeLog:
    
    	* gdb.texinfo (Command Completion): Document new
    	"set/show max-completions" option.
    
    gdb/testsuite/ChangeLog:
    
    	* gdb.base/completion.exp: Disable completion limiting for
    	existing tests.  Add new tests to check completion limiting.
    	* gdb.linespec/ls-errs.exp: Disable completion limiting.
Comment 17 Doug Evans 2015-03-28 22:06:04 UTC
*** Bug 11048 has been marked as a duplicate of this bug. ***
Comment 18 Doug Evans 2015-03-28 22:07:03 UTC
Patch committed.