[RFC][gdb/symtab] Lazy expansion of full symbol table

[Tom de Vries]

On 6/15/21 3:26 PM, Tom Tromey wrote:
>>> I don't have this executable
> 
> Tom> Uploaded to https://ftp.suse.com/pub/people/tdevries/gdb/pr23710/cc1
> 
> Thanks, I grabbed it.
> 
>>> but FWIW my scanner rewrite is ~10x faster
>>> than the current psymtab reader.
> 
> Tom> Interesting, that's
> Tom> https://github.com/tromey/gdb/commits/submit/no-more-psym ?
> 
> Tom> I've tried that branch with the cc1 example, and ran into:
> Tom> ...
> Tom> DW_FORM_strp pointing outside of .debug_str section [in module cc1]
> 
> Thank you for trying it.  This was a bug in the CU importing code.  I've
> fixed it and will push momentarily -- but be warned, I rebase this
> branch a lot...
> 

Sure, understood.

> On my machine, with "maint time 1", /bin/gdb reports:
> 
>     (gdb) file ./cc1
>     2021-06-15 07:18:29.554 - command started
>     Reading symbols from ./cc1...
>     2021-06-15 07:18:35.118 - command finished
>     Command execution time: 5.553344 (cpu), 5.563964 (wall)
> 
> The branch says:
> 
>     (gdb) file ./cc1
>     2021-06-15 07:18:50.688 - command started
>     Reading symbols from ./cc1...
>     2021-06-15 07:18:51.074 - command finished
>     Command execution time: 1.260050 (cpu), 0.386460 (wall)
> 
> So a bit more than 10x in this case. 

Wow, nice :)

FWIW, my measurement is:

1. branch base 739025e89c4
...
$ gdb -q -ex "set trace-commands on" -ex "maint time 1" -ex "file lto/cc1"
+maint time 1
+file lto/cc1
Reading symbols from lto/cc1...
2021-06-16 11:51:54.007 - command finished
Startup time: 2.949213 (cpu), 2.887778 (wall)
(gdb)
...

2. branch 5bc56d745fd
...
$ gdb -q -ex "set trace-commands on" -ex "maint time 1" -ex "file lto/cc1"
+maint time 1
+file lto/cc1
Reading symbols from lto/cc1...
2021-06-16 12:00:35.060 - command finished
Startup time: 0.947957 (cpu), 0.408178 (wall)
(gdb)
...

Which is a factor 7 speedup.

> I wish it were a tiny bit faster,
> so that it wasn't perceptible, but meh, DWARF wasn't designed to be fast
> to read.
> 
> Currently I think the scanner is complete, I just need to make some
> changes to the lookup code, and maybe also how the new index entries are
> stored, so that completion is more efficient.  Of course there may still
> be bugs, but at the moment I don't anticipate any reader slowdowns.

I did an overnight build and test with the updated branch (5bc56d745fd)
and ran into some trouble.  The first internal-error I investigated
happens when parsing the libstdc++ .debug package (so, it was not
specific to the test-case).  It seems the branch has some trouble with
the dwz layout where an abbrev entry is shared between different CUs:
...
src/gdb/dwarf2/abbrev-cache.c:59: internal-error: void
abbrev_cache::add(abbrev_table_up): Assertion `*slot == nullptr' failed.
...

I also could reproduce this problem with "maint set worker-threads 1".

I set a breakpoint on abbrev_cache::add and abbrev_cache::find, both
limited to section offset 0.

I see then a lot of finds, followed by two adds (and the second add
triggers the assert).

[ If you want to reproduce, download f.i.
https://download.opensuse.org/update/leap/15.2/oss_debug/x86_64/libstdc++6-debuginfo-10.2.1+git583-lp152.4.1.x86_64.rpm
. ]

Thanks,
- Tom