caching failed lookups of debuginfo?

[Frank Ch. Eigler]

Hi -

> another debuginfod related question, but unrelated to the other thread I 
> started earlier today. In a work branch I have ported my heaptrack profiler 
> over to elfutils. I have then run the analyzer that uses elfutils (and thus 
> debuginfod internally via dwfl) on a recorded data file to have it download 
> all the debug info files it can find.

Nice.


> These negative lookups are not cached. Meaning rerunning the same process 
> using dwfl and debuginfod on the same data would always incur a significant 
> slowdown, as we would again and again try to look for something that's not 
> there. The lookups take roughly ~200ms for me to realize the data is not on 
> the server.

That's not correct, as of elfutils 0.184 (commit 5f72c51a7e5c0),
with some more recent tweaks in (commit 7d64173fb11c6).

- FChE



> What's worse, I'm seeing multiple lookups for the same buildid *within the 
> same process*. I.e.:
> 
> ```
> export DEBUGINFOD_VERBOSE=1
> ./heaptrack_interpret ... |& egrep "^url 0 https" | sort | uniq -c | sort
> ...
>       6 url 0 https://debuginfod.archlinux.org/buildid/
> 7f4b16b4b407cbae2d7118d6f99610e29a18a56a/debuginfo
>       8 url 0 https://debuginfod.archlinux.org/buildid/
> c09c6f50f6bcec73c64a0b4be77eadb8f7202410/debuginfo
>      14 url 0 https://debuginfod.archlinux.org/buildid/
> 85766e9d8458b16e9c7ce6e07c712c02b8471dbc/debuginfo
> ```
> 
> Here, we are paying roughly `14 * 0.2s = 2.8s` just for a single library.
> 
> Can we find a way to improve this situation somehow generically? I would 
> personally even be OK with caching the 404 error case locally for some time 
> (say, one hour or one day or ...). Then at least we would at most pay this 
> cost once per library, and not multiple times. And rerunning the analysis a 
> second time would become much faster again.
> 
> Was there a deliberate decision against caching negative server side lookups?
> 
> Thanks
> -- 
> Milian Wolff
> mail@milianw.de
> http://milianw.de