This is the mail archive of the
mailing list for the elfutils project.
Re: How to debug broken unwinding?
- From: Milian Wolff <mail at milianw dot de>
- To: elfutils-devel at sourceware dot org
- Cc: Mark Wielaard <mark at klomp dot org>
- Date: Tue, 13 Jun 2017 18:12:46 +0200
- Subject: Re: How to debug broken unwinding?
- Authentication-results: sourceware.org; auth=none
- References: <2244170.M6KXmO4ZXp@agathebauer> <email@example.com>
On Dienstag, 13. Juni 2017 16:20:25 CEST Mark Wielaard wrote:
> Hi Milian,
> On Thu, 2017-06-01 at 22:57 +0200, Milian Wolff wrote:
> > How should I debug this, or how can I report a good bug report for this? I
> > guess I could upload a perf archive and document the steps required to
> > build perf with libdw as the unwinder, as it allows to easily compare
> > libunwind and libdw for unwinding.
> I am reading the rest of you messages in this thread now and it seems
> you did find the issue already. But the above would still be useful if
> you happen to have the instructions already written down. I was actually
> looking at perf recently and couldn't immediately figure out how to
> build it. It seems it depends on a whole kernel build. But if there is
> an easy way to just build perf (with a variant that uses libunwind and
> libdw) that would be useful in general I think.
Perf is, you could say, a two-fold software.
On one hand you have the kernel side of things, which does the PMU
configuration and actual sampling and data recording.
One the other hand you have a bunch of userspace tooling that tells the kernel
side to start recording, but - more importantly - to analyse the recorded
The first part in the kernel is pretty stable - no need to compile your own
kernel. Esp. on a recent distro this is not required.
For the second side, compiling perf's userland is pretty easy to get the
latest and greatest features. And it also allows us to compare libunwind vs.
git clone git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux \
make # will output what dependencies where found
cp perf perf-libunwind
cp perf perf-elfutils
# record data with either version
./perf record --call-graph dwarf <something>
Then you can use different "perf" versions to compare the unwinders. I usually
diff the output of `perf script` or `perf report`, e.g.:
./perf-libunwind stat --inline > libunwind.txt
./perf-elfutils stat --inline > elfutils.txt
diff -u libunwind.txt elfutils.txt
Hope this helps!
> More in general debugging the unwinder is indeed too hard. There are
> some failures that are ignored because it isn't immediately clear
> whether they are fatal. e.g. you might be unable to recover some
> register value that then isn't ever used for a simple backtrace, or you
> might not find the CFI for a pc address, but want to use the fallback
> unwinder in that case. Maybe we should add a debug mode configuration
> that compiles in tracing output of the unwinder operations?
That would be helpful, I guess. Most notably for laymen like me who still have
trouble understanding what exactly is going on here. If there would be some
magic flag I could use (like libunwind's UNW_DEBUG_LEVEL=15) to generate a
trace I could append to a bug report, that would be good to have!