Re: [RFC][PATCH 22/22] Add Infinity notes implementing td_ta_map_lwp2thr

[Gary Benson <gbenson at redhat dot com>]

Adhemerval Zanella wrote:
> On 21-10-2015 12:42, Gary Benson wrote:
> > Adhemerval Zanella wrote:
> > > I like the idea of improving debugging capabilities with
> > > something new than libpthread_db, however I do not see the best
> > > approach of using the i8 files directly in GLIBC. If I
> > > understood correctly from the link you posted with this
> > > documentation, it is still a very WIP project with rough places
> > > and the language/compiler is still being constantly updated.
> > 
> > It is a work-in-progress right now.  It's being developed to allow
> > GDB to debug multithreaded inferiors, so this patch series (or,
> > rather, the complete version that will follow in the next few
> > months) is the work that will turn Infinity from WIP to something
> > complete.
> > 
> > I mailed this series as an RFA to get some early feedback.  I'm
> > anticipating that a prerequisite of a GLIBC commit will be a GDB that
> > fully supports Infinity (and a prerequisite of a GDB commit will be a
> > GLIBC with Infinity notes) and what I *didn't* want to do was appear
> > out of the blue with a complete system I'd spent six months writing
> > away by myself only for somebody to point out a killer flaw that meant
> > the whole thing was junk.  Also because I recognise that input from
> > the wider community will make the final result something way better
> > than I could achieve on my own.
> > 
> > So yes, it's very work-in-progress right now, but by the time it's
> > ready to commit it should be pretty well baked and polished.
> 
> I did say it was flawed or even half-baked, my concern is due it
> uses a non well establish language it may add more maintaining
> burden than use assembly drop releases. What happens if had or
> decided to extend I8X to cover a new feature and add this dependency
> on the GLIBC code?  Will you always provide backwards compatibility?
> Will we need to tie GLIBC releases to some I8X release or will be
> feature based?

I wasn't intending to ever break backwards compatibility.  That would
mess things up for people (and I'd have to deal with it!)

If some new language feature was added then some configure check could
be written, either by looking at "i8c --version" or by trying to
compile some code that had the new feature and seeing if it failed.
It would be much as we do for the other tools we use.

> > > Also I do not like to add another tool dependency for complete
> > > GLIBC build and one in such state (GLIBC do use python for
> > > benchmark but it is not really required for build afaik). So
> > > which is the issue of shipping the ELF notes in assembly format
> > > instead?
> >
> > The initial reason I opted for a compiled source approach is that
> > the note format is somewhat complex.  The format is documented
> > here:
> > 
> >   https://sourceware.org/gdb/wiki/NoteFormat
> > 
> > I want Infinity to be future-proof.  Now that libthread_db exists,
> > GLIBC is essentially bound to support it *forever*.  I don't want to
> > introduce another debugger interface that we'll have to support
> > forever unless I can be fairly confident we won't have to replace that
> > five or ten years down the line.  So there's a lot of requirements in
> > the note format specification that (should!) mean that any new feature
> > we require can be added without breaking existing note consumers.
> > 
> > The second reason for using a compiler (or, more generally, tools) is
> > testing.  We could ship the notes in assembly language but then how do
> > you test them?  I don't like the idea of GLIBC shipping with untested
> > code, especially since the result of a breakage might not be
> > immediately obvious: if we ship a broken note in a GLIBC release it
> > might well not be noticed until it makes its way into some distro and
> > some user finds they can't debug their application because one of the
> > thread-debugging notes is invalid.  That's why the required tools are
> > a compiler *and* a unit tester.  I don't know if you saw, but the
> > testcase for the note in this patch exercises every path through the
> > compiled note.  It's not perfect, but it's as close as I can get.  So,
> > each note I add to GLIBC will have an accompanying testcase so I can
> > be as certain as I can be that if "make check" passed then the notes
> > in the files will work.
> > 
> > For me, the second reason (testing) is the more compelling of the two.
> > And if you're going to require an external tool for testing, why not
> > require an external compiler too?
> 
> AFAIK the only non-trivial build dependency for testing is perl and 
> I do not see a way forward to add another interpreter dependency.  It
> will be even harder on cross-compiling environment to do native testing
> with such dependency.

I8C and I8X run on the host, not the target.  You only need Python on
the host.  It's probably already there :)

And on the host, if you have Python and pip installed then you can
install I8C and I8X with a single shell command.  It takes about 20
seconds on my <5Mbit connection and you don't need root access.

> > I appreciate that people don't want to add more build dependencies to
> > GLIBC but I don't really have a way around that.  To mitigate the
> > impact I've written the tools without using any external libraries,
> > and I've written them to work on as wide a range of Python versions
> > as is possible.  It's not possible to write code that works both
> > for Python < 2.6 and for Python >= 3.0 simultaneously so 2.6 is the
> > oldest you can use.
> 
> So the compelling reason I see so far to do not make assembly releases
> and to not add the python as build/tests requirement is the testcases.
> How hard would be to add a non-python testcase on GLIBC to test such
> features?

I8X has 1230 statements (lines of code that do something, as opposed
to comments, blank lines and documentation).  I don't have figures for
how Python compares to C in terms of line count but I figure you're
looking at 2500-5000 lines of C and that's not including the testcases
themselves or something to parse and execute them if you go that
route.  Easily a month of work, and the result would likely be less
flexible.

Cheers,
Gary

-- 
http://gbenson.net/