Re: [RFC][PATCH 22/22] Add Infinity notes implementing td_ta_map_lwp2thr

[Gary Benson <gbenson at redhat dot com>]

Adhemerval Zanella wrote:
> On 22-10-2015 06:56, Gary Benson wrote:
> > Adhemerval Zanella wrote:
> > > Em 21 de out de 2015, às 19:28, Gary Benson <gbenson@redhat.com> escreveu:
> > > > Adhemerval Zanella wrote:
> > > > > I think one strategy that we can use to decouple the need
> > > > > for a python binary in build is:
> > > > > 
> > > > > 1. Provide assembly generated notes from a defined I8X
> > > > >    compiler along with I8X sources.
> > > > > 2. Add a makefile rule to autogenerate the assembly file
> > > > >    if a I8C compiler is presented in the system.
> > > > > 3. For tests, only run is I8X is presented.
> > > >
> > > > That's not really viable for the reasons I detailed above.
> > > > You would have to ship generated assembly for every platform,
> > > > and even that is not enough.  For example, on some platforms
> > > > "sizeof (struct pthread)" is a constant that gets into notes.
> > > > If someone adds a field to struct pthread and doesn't
> > > > regenerate the notes then you're shipping GLIBC with broken
> > > > notes.  Are there CFLAGS the user could set that would change
> > > > structure offsets?  Setting these would mean broken notes if
> > > > you didn't have I8C installed.  There are constants in kernel
> > > > headers that end up in notes.  If one of these changes, you're
> > > > shipping GLIBC with broken notes.
> > > >
> > > > There are so many silent failure modes here--and broken notes
> > > > are far worse than no notes at all.  The only robust way to do
> > > > this is to build the notes with the constants and offsets that
> > > > you are building the rest of GLIBC with, and that means saying
> > > > if you want a GLIBC with notes then you need to install this
> > > > package on your build system.
> > >
> > > This can be done with proper documentation and in the release
> > > cycle: the release maintainer will recreate the assembly
> > > annotations for the release architectures.  My only concern is
> > > the kernel notes dependency, however even those we need to take
> > > care since you also do not know prior hand which version you
> > > will end using.
> > 
> > That's fragile.  It's also burdensome for the release maintainer.
> > It also raises the barrier for people hacking on GLIBC, because
> > not only do they need to know all the other things they need to
> > know, they also now need to know about recreating notes.  Say
> > somebody is hacking on NPTL, they don't know about updating notes,
> > so the first thing they see is the debugger won't attach.  How
> > long do they spend going round in circles until somebody says "oh,
> > you need to run this command in this directory."?
> > 
> > It also doesn't solve the problem that if people are working
> > without I8C and I8X installed then they are creating GLIBCs
> > containing unchecked notes.  My position is users create GLIBCs
> > containing fully-tested notes or they create GLIBCs creating no
> > notes at all.
> > 
> > In either setup there's a process to create notes.  What you're
> > saying is, "let's encode this process in documentation and let
> > people figure it out."  What I'm saying is, "let's encode this in
> > code and let the build host figure it out."
> 
> Yes it is a tradeoff of adding external dependency and making GLIBC
> build independently of them. Also the exported NPTL structures do
> not change in a dynamic way (struct pthread, for instance, changes
> it's size twice in seven years).

I've given a bunch of reasons why pre-generated notes might become
broken.  The fact that *one* of those reasons might not occur very
often is irrelevant.

The impact of a broken note in an application is that debugging that
application may be difficult or impossible.  GLIBC is used by almost
every application, so a broken note in GLIBC could mean it becomes
impossible to debug *almost any application* on your system.
Including the debugger which you might like to use to locate the
problem.  So it's important that any notes in GLIBC are thoroughly
tested.

Pregenerated notes have identified failure modes which cannot be
detected at generation time.  The only way to detect such failures
is by testing the build.

Notes are complex enough that they cannot be thoroughly tested
without tooling.  If that tooling is not present, "make check"
cannot check the build.

The impact of broken notes is sufficiently severe that if notes
cannot be tested then they should not be present in the build.
So you require tooling to have a build with notes.  And if you
require tooling to have a build with notes, you may as well
have tooling to build the notes in the first place.

> Also, currently if someone is hacking on GLIBC and he will also need
> to adjust libpthread_db anyway. I know it is not the best approach
> and I do want to see a better strategy to handle it, but adding an
> external tools with python dependency IMHO is not the best one.

I hope this isn't all about Python.  Python is pretty mainstream,
and I'm pretty sure most people will either have it on their build
system already or could install it with a single shell command.
I appreciate that people who are cross-compiling may not wish to
or be able to install Python on their target machines, which is
why I've taken steps to ensure that Python is only used on the
host.

> I would expect such tool to be included in binutils and have only
> binutils own dependencies as requisites.

GLIBC has plenty of dependencies from outside of binutils.

I'm happy for the project to be donated to the FSF and/or hosted
on sourceware.org if that would reassure people.  I don't think,
however, that binutils is a natural place to put a Python project.

> I would also expect notes definition language stability, meaning
> that a notes defined with a older version will always be handled
> by new compiler version (so building a older glibc notes version
> with a new notes compiler won't break the build).

Naturally.  And there's no reason for that not to be the case.

Thanks,
Gary

-- 
http://gbenson.net/