Re: [RFC][PATCH 22/22] Add Infinity notes implementing td_ta_map_lwp2thr

[Adhemerval Zanella <adhemerval dot zanella at linaro dot org>]

On 21-10-2015 12:42, Gary Benson wrote:
> Adhemerval Zanella wrote:
>> I like the idea of improving debugging capabilities with something
>> new than libpthread_db, however I do not see the best approach of
>> using the i8 files directly in GLIBC. If I understood correctly from
>> the link you posted with this documentation, it is still a very WIP
>> project with rough places and the language/compiler is still being
>> constantly updated.
> 
> It is a work-in-progress right now.  It's being developed to allow GDB
> to debug multithreaded inferiors, so this patch series (or, rather, the
> complete version that will follow in the next few months) is the work
> that will turn Infinity from WIP to something complete.
> 
> I mailed this series as an RFA to get some early feedback.  I'm
> anticipating that a prerequisite of a GLIBC commit will be a GDB that
> fully supports Infinity (and a prerequisite of a GDB commit will be a
> GLIBC with Infinity notes) and what I *didn't* want to do was appear
> out of the blue with a complete system I'd spent six months writing
> away by myself only for somebody to point out a killer flaw that meant
> the whole thing was junk.  Also because I recognise that input from
> the wider community will make the final result something way better
> than I could achieve on my own.
> 
> So yes, it's very work-in-progress right now, but by the time it's
> ready to commit it should be pretty well baked and polished.

I did say it was flawed or even half-baked, my concern is due it uses
a non well establish language it may add more maintaining burden than
use assembly drop releases. What happens if had or decided to extend 
I8X to cover a new feature and add this dependency on the GLIBC code? 
Will you always provide backwards compatibility? Will we need to tie
GLIBC releases to some I8X release or will be feature based?    

> 
>> Also I do not like to add another tool dependency for complete GLIBC
>> build and one in such state (GLIBC do use python for benchmark but
>> it is not really required for build afaik). So which is the issue of
>> shipping the ELF notes in assembly format instead?
> 
> The initial reason I opted for a compiled source approach is that the
> note format is somewhat complex.  The format is documented here:
> 
>   https://sourceware.org/gdb/wiki/NoteFormat
> 
> I want Infinity to be future-proof.  Now that libthread_db exists,
> GLIBC is essentially bound to support it *forever*.  I don't want to
> introduce another debugger interface that we'll have to support
> forever unless I can be fairly confident we won't have to replace that
> five or ten years down the line.  So there's a lot of requirements in
> the note format specification that (should!) mean that any new feature
> we require can be added without breaking existing note consumers.
> 
> The second reason for using a compiler (or, more generally, tools) is
> testing.  We could ship the notes in assembly language but then how do
> you test them?  I don't like the idea of GLIBC shipping with untested
> code, especially since the result of a breakage might not be
> immediately obvious: if we ship a broken note in a GLIBC release it
> might well not be noticed until it makes its way into some distro and
> some user finds they can't debug their application because one of the
> thread-debugging notes is invalid.  That's why the required tools are
> a compiler *and* a unit tester.  I don't know if you saw, but the
> testcase for the note in this patch exercises every path through the
> compiled note.  It's not perfect, but it's as close as I can get.  So,
> each note I add to GLIBC will have an accompanying testcase so I can
> be as certain as I can be that if "make check" passed then the notes
> in the files will work.
> 
> For me, the second reason (testing) is the more compelling of the two.
> And if you're going to require an external tool for testing, why not
> require an external compiler too?

AFAIK the only non-trivial build dependency for testing is perl and 
I do not see a way forward to add another interpreter dependency.  It
will be even harder on cross-compiling environment to do native testing
with such dependency.

> 
> I appreciate that people don't want to add more build dependencies to
> GLIBC but I don't really have a way around that.  To mitigate the
> impact I've written the tools without using any external libraries,
> and I've written them to work on as wide a range of Python versions
> as is possible.  It's not possible to write code that works both
> for Python < 2.6 and for Python >= 3.0 simultaneously so 2.6 is the
> oldest you can use.

So the compelling reason I see so far to do not make assembly releases
and to not add the python as build/tests requirement is the testcases.
How hard would be to add a non-python testcase on GLIBC to test such
features?

> 
> As a final (not entirely related) point, while I'm talking about
> testing, I'd like to note that while the compiler and tester are both
> works-in-progress, they both have pretty comprehensive test suites.
> The compiler has tests with 98% coverage so I'm pretty sure it's doing
> what I think it is on all supported versions of Python.  I'm using
> Travis CI to test every commit I make with seven different versions of
> Python from 2.7 to the latest nightly:
> 
>   https://travis-ci.org/gbenson/i8c
> 
> The tester has only 87% coverage but I'm working on that :)
> 
> Cheers,
> Gary
>