PATCH PR27323 debuginfod concurrency++

[Mark Wielaard]

Hi Frank,

On Tue, 2021-02-02 at 16:59 -0500, Frank Ch. Eigler via Elfutils-devel wrote:
>     PR27323 debuginfod: improve query concurrency with grooming
>     
>     Start using a second sqlite3 database connection for webapi query
>     servicing.  This allows much better concurrency when long-running
>     grooming operations are in progress.
>     
>     No testsuite impact.  Grooming times are too short to try to hit with
>     concurrent requests.  OTOH the existing tests did show some
>     interesting regressions that needed fixing, like needing not to
>     dual-wield db and dbq when doing rpm-dwz-related lookups from during
>     scanning, and the way in which corrupted databases are reported.
>     These needed some automated invocations of gdb on the running
>     debuginfod binaries that just failed their testing, for in-situ
>     debugging.
>     
>     Hand-tested for function on a huge 20GB index file.  Allowed webapi
>     queries to be run throughout random points of the grooming process,
>     including especially the long count(*) report loops before & after.
>     
> [...]
> +2021-02-02  Frank Ch. Eigler <fche@redhat.com>
> +
> +	PR27323
> +	* debuginfod.cxx (dbq): New read-only database connection for queries
> +	only.
> +	(signal_handler): Interrupt it.
> +	(main): Open / close it.
> +	(handle_buildid): Use it for webapi queries only.
> +	(database_stats_report): Make more interruptible.  Report sqlite3
> +	operation times to the prometheus metrics.
> +	(groom): Make more interruptible.
> +	(thread_main_fts_source_paths, thread_main_groom): Ensure
> +	state/progress metrics are fresh even in case of exceptions.

This does make sense to me. Looks good.

Thanks,

Mark