Re: [patch 0/2] Accelerate symbol lookups 15x

[Doug Evans <xdje42 at gmail dot com>]

On Mon, Oct 20, 2014 at 2:44 PM, Jan Kratochvil
<jan.kratochvil@redhat.com> wrote:
> Hi,
>
> this patchset has been developed+tested only on top of
>         [patchv2] Fix 100x slowdown regression on DWZ files
>         https://sourceware.org/ml/gdb-patches/2014-10/msg00031.html
>
> While this patchset is technically independent the GDB performance and memory
> requirements would not make testing of this patchset possible.
>
> Originally I expected with the patch above (fixing backtrace performance) even
> the sluggish interactive GDB performance would get fixed.  It was not fixed.
> During debugging I get 10-30 seconds for a response to simple commands like:
>         (gdb) print vectorvar.size()
>
> With this patch the performance gets to 1-2 seconds which is somehow
> acceptable.  The problem is that dwarf2_gdb_index_functions.lookup_symbol
> (quick_symbol_functions::lookup_symbol) may return (and returns) NULL even for
> symbols which are present in .gdb_index but which can be found in already
> expanded symtab.  But searching in the already expanded symtabs is just too
> slow when there are 400000+ expanded symtabs.  There would be needed some
> single global hash table for each objfile so that one does not have to iterate
> all symtabs.  Which .gdb_index could perfectly serve for, just its
> lookup_symbol() would need to return authoritative yes/no answers.
>
> Even after such fix these two simple patches are useful for example for
> non-.gdb_index files.
>
> One can reproduce the slugging interactive GDB performance with:
>         #include <string>
>         using namespace std;
>         string var;
>         class C {
>         public:
>           void m() {}
>         };
>         int main() {
>           C c;
>           c.m();
>           return 0;
>         }
> g++ -o slow slow.C -Wall -g $(pkg-config --libs gtkmm-3.0)
> gdb ./slow -ex 'b C::m' -ex 'maintenance set per-command space' -ex 'maintenance set per-command symtab' -ex 'maintenance set per-command time' -ex r
> [...]
> (gdb) p <tab><tab>
> Display all 183904 possibilities? (y or n) n
> (gdb) p/r var
> $1 = {static npos = <optimized out>, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x3a4db073d8 <std::string::_Rep::_S_empty_rep_storage+24> ""}}
> Command execution time: 20.023000 (cpu), 20.118665 (wall)
>                         ^^^^^^^^^
> Space used: 927997952 (+0 for this command)
> #symtabs: 186099 (+0), #primary symtabs: 21573 (+0), #blocks: 290353 (+0)
>
> For a non-trivial application with all symtabs expanded it takes 113 seconds.
>
> Benchmark on non-trivial application without 'p <tab><tab>':
> Command execution time:   0.496000 (cpu),   0.496882 (wall) --- both fixes
> Command execution time:   0.899000 (cpu),   0.908062 (wall) --- just lookup_symbol_aux_objfile fix
> Command execution time:   3.492000 (cpu),   3.491791 (wall) --- FSF GDB HEAD
>
> Benchmark on non-trivial application with    'p <tab><tab>':
> Command execution time:   7.373000 (cpu),   7.395095 (wall) --- both fixes
> Command execution time:  13.572000 (cpu),  13.592689 (wall) --- just lookup_symbol_aux_objfile fix
> Command execution time: 113.036000 (cpu), 113.067995 (wall) --- FSF GDB HEAD
>
>
> No regressions on {x86_64,x86_64-m32,i686}-fedora21pre-linux-gnu in standard
> and .gdb_index-enabled runs.  Neither of the patches should cause any visible
> behavior change.

Hi.
I've been playing with the patches a bit.
I still have some more research to do, I'd like to make sure I have a
deep understanding of why this works before this goes in.
Thanks.

For example, the count of calls to dict_hash before/after goes from 13.8M to 31.
I'm guessing one t hing we're doing here is coping with an artifact of
dwz: what was once one global block to represent the entire objfile is
now N.
[I'm sure the patches help in the non-dwz case, but I suspect it's less.
Which isn't to say the patches aren't useful.
I just need play with a few more examples.]