Re: [RFC][PATCH 22/22] Add Infinity notes implementing td_ta_map_lwp2thr

[Adhemerval Zanella <adhemerval dot zanella at linaro dot org>]

On 22-10-2015 11:11, Gary Benson wrote:
> Adhemerval Zanella wrote:
>> On 22-10-2015 06:56, Gary Benson wrote:
>>> Adhemerval Zanella wrote:
>>>> Em 21 de out de 2015, às 19:28, Gary Benson <gbenson@redhat.com> escreveu:
>>>>> Adhemerval Zanella wrote:
>>>>>> I think one strategy that we can use to decouple the need
>>>>>> for a python binary in build is:
>>>>>>
>>>>>> 1. Provide assembly generated notes from a defined I8X
>>>>>>    compiler along with I8X sources.
>>>>>> 2. Add a makefile rule to autogenerate the assembly file
>>>>>>    if a I8C compiler is presented in the system.
>>>>>> 3. For tests, only run is I8X is presented.
>>>>>
>>>>> That's not really viable for the reasons I detailed above.
>>>>> You would have to ship generated assembly for every platform,
>>>>> and even that is not enough.  For example, on some platforms
>>>>> "sizeof (struct pthread)" is a constant that gets into notes.
>>>>> If someone adds a field to struct pthread and doesn't
>>>>> regenerate the notes then you're shipping GLIBC with broken
>>>>> notes.  Are there CFLAGS the user could set that would change
>>>>> structure offsets?  Setting these would mean broken notes if
>>>>> you didn't have I8C installed.  There are constants in kernel
>>>>> headers that end up in notes.  If one of these changes, you're
>>>>> shipping GLIBC with broken notes.
>>>>>
>>>>> There are so many silent failure modes here--and broken notes
>>>>> are far worse than no notes at all.  The only robust way to do
>>>>> this is to build the notes with the constants and offsets that
>>>>> you are building the rest of GLIBC with, and that means saying
>>>>> if you want a GLIBC with notes then you need to install this
>>>>> package on your build system.
>>>>
>>>> This can be done with proper documentation and in the release
>>>> cycle: the release maintainer will recreate the assembly
>>>> annotations for the release architectures.  My only concern is
>>>> the kernel notes dependency, however even those we need to take
>>>> care since you also do not know prior hand which version you
>>>> will end using.
>>>
>>> That's fragile.  It's also burdensome for the release maintainer.
>>> It also raises the barrier for people hacking on GLIBC, because
>>> not only do they need to know all the other things they need to
>>> know, they also now need to know about recreating notes.  Say
>>> somebody is hacking on NPTL, they don't know about updating notes,
>>> so the first thing they see is the debugger won't attach.  How
>>> long do they spend going round in circles until somebody says "oh,
>>> you need to run this command in this directory."?
>>>
>>> It also doesn't solve the problem that if people are working
>>> without I8C and I8X installed then they are creating GLIBCs
>>> containing unchecked notes.  My position is users create GLIBCs
>>> containing fully-tested notes or they create GLIBCs creating no
>>> notes at all.
>>>
>>> In either setup there's a process to create notes.  What you're
>>> saying is, "let's encode this process in documentation and let
>>> people figure it out."  What I'm saying is, "let's encode this in
>>> code and let the build host figure it out."
>>
>> Yes it is a tradeoff of adding external dependency and making GLIBC
>> build independently of them. Also the exported NPTL structures do
>> not change in a dynamic way (struct pthread, for instance, changes
>> it's size twice in seven years).
> 
> I've given a bunch of reasons why pre-generated notes might become
> broken.  The fact that *one* of those reasons might not occur very
> often is irrelevant.
> 
> The impact of a broken note in an application is that debugging that
> application may be difficult or impossible.  GLIBC is used by almost
> every application, so a broken note in GLIBC could mean it becomes
> impossible to debug *almost any application* on your system.
> Including the debugger which you might like to use to locate the
> problem.  So it's important that any notes in GLIBC are thoroughly
> tested.
> 
> Pregenerated notes have identified failure modes which cannot be
> detected at generation time.  The only way to detect such failures
> is by testing the build.
> 
> Notes are complex enough that they cannot be thoroughly tested
> without tooling.  If that tooling is not present, "make check"
> cannot check the build.
> 
> The impact of broken notes is sufficiently severe that if notes
> cannot be tested then they should not be present in the build.
> So you require tooling to have a build with notes.  And if you
> require tooling to have a build with notes, you may as well
> have tooling to build the notes in the first place.

Yes I understand the issue, so in the end it is a matter to allow
external optional tools to add marks and hook up in GLIBC.  I would
prefer something related to systemtap way, however it also have its
defects (testing for instance).

I would prefer something more decoupled of GLIBC, so if we need to
get rid off its support if a new strategy/project shows a superior
approach it would be easier. I would also prefer to no be dependent
of external interpreters to implement such compiler/instrumentation
(since IMHO there is no strong reason to depend of python or any
other interpreter for such project).

But also this I also see it is not a strong reason to really oppose
it.  


> 
>> Also, currently if someone is hacking on GLIBC and he will also need
>> to adjust libpthread_db anyway. I know it is not the best approach
>> and I do want to see a better strategy to handle it, but adding an
>> external tools with python dependency IMHO is not the best one.
> 
> I hope this isn't all about Python.  Python is pretty mainstream,
> and I'm pretty sure most people will either have it on their build
> system already or could install it with a single shell command.
> I appreciate that people who are cross-compiling may not wish to
> or be able to install Python on their target machines, which is
> why I've taken steps to ensure that Python is only used on the
> host.
> 
>> I would expect such tool to be included in binutils and have only
>> binutils own dependencies as requisites.
> 
> GLIBC has plenty of dependencies from outside of binutils.
> 
> I'm happy for the project to be donated to the FSF and/or hosted
> on sourceware.org if that would reassure people.  I don't think,
> however, that binutils is a natural place to put a Python project.
> 
>> I would also expect notes definition language stability, meaning
>> that a notes defined with a older version will always be handled
>> by new compiler version (so building a older glibc notes version
>> with a new notes compiler won't break the build).
> 
> Naturally.  And there's no reason for that not to be the case.
> 
> Thanks,
> Gary
>