This is the mail archive of the gdb-patches@sourceware.org mailing list for the GDB project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [RFA] Add interface for registering JITed code


On Tue, Jul 28, 2009 at 12:00 PM, Eli Zaretskii<eliz@gnu.org> wrote:
>> From: Reid Kleckner <rnk@mit.edu>
>> Date: Mon, 27 Jul 2009 13:40:02 -0700
>> Cc: Tom Tromey <tromey@redhat.com>, gdb-patches@sourceware.org,
>> ? ? ? unladen-swallow@googlegroups.com
>>
>> I mentioned the manual chapter in the NEWS entry, since the
>> explanation of how it works is long.
>
> Thanks. ?I have a few comments.
>
>> +* GDB now has an interface for JIT compilation. ?Applications that
>> +dynamically generate code can create symbol files in memory and register
>> +them with GDB. ?For users, the feature should work transparently, and
>> +for JIT developers, the interface is documented in the GDB manual.
>
> Please state the name of the chapter in the manual where this is
> documented.

Done.

>> +@node JIT Interface
>> +@chapter @value{GDBN}'s JIT Interface
>
> I think a better name is "JIT Compilation Interface".
>
> Please add pertinent index entries here. ?I suggest these:
>
> ?@cindex just-in-time compilation
> ?@cindex JIT compilation interface

Done.

>> +This chapter documents @value{GDBN}'s just-in-time compiler (JIT) interface.
>
> Whenever you introduce new terminology, first usage should be in @dfn,
> to make the term stand out:
>
> ? This chapter documents @value{GDBN}'s @dfn{just-in-time} (JIT)
> ? compilation interface.
>
> Also, how about saying a few words here about JIT compiling, for those
> who maybe hear about that for the first time? ?Just a sentence or two
> would be good enough.

Sure.

>> +Just-in-time compilers (JITs) are normally difficult to debug because they
>
> Are we talking about debugging a JIT compiler or about debugging a
> program that uses a JIT compiler? ?I thought the latter, but it sounds
> like you are talking about the former here.

The line is somewhat blurry, since for our purposes we are trying to
debug Unladen Swallow, which is an interpreter for Python that uses
the LLVM JIT.  It's somewhat appropriate to refer to it as a JIT for
Python, but for the purposes of the documentation it's better to be
clear.

>> +generate code at runtime, and GDB normally gets all of its symbols from object
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ^^^
> "@value{GDBN}" (here and elsewhere).

Done.

>> +files.
>
> Does this rewording of the above sentence catch what you meant to say
> (I'm not sure I understood)?
>
> ?Programs compiled with JIT compilers are normally difficult to debug
> ?because portions of their code are generated at runtime, and not
> ?written to object files, where @value{GDBN} normally finds the debug
> ?information it needs.
>
> The rest of the text needs similar rewording, to make it more clear.
> In particular, you use "JIT" meaning "a JIT compiler" and "executables
> that use JIT" when you really mean "programs that use JIT
> compilation".

I've left the use of JIT as a noun in the later technical part of the
chapter where "JIT" means "the JIT compiler" and not the program using
the JIT.  The audience for those sections should be JIT authors, not
JIT users.  In other cases I've tried to use "program that uses JIT
compilation" or just "program".

> If you can rephrase the text along the above guidelines, that'd be
> great. ?If not, go ahead and commit it with the few technical changes
> I mention here, and I will do the rest.
>
>> + ? ?In order to debug executables that use JITs, GDB has an interface that
>> +allows the JIT to register in-memory symbol files with GDB at runtime. ?If you
>> +are using GDB to debug a program using this interface, then it should just work
>> +so long as you have not stripped the binary. ?If you are developing a JIT, then
>> +the interface is documented in the next section. ?At this time, the only known
>> +client of this interface is the LLVM JIT.
>
> A question out of ignorance: are the special symbols you use to
> interface with the JIT generated code specific to LLVM, or are they
> part of some broader standard? ?If the former, the instructions below
> are actually instructions to develop other JIT compilers that follow
> the LLVM conventions, and we are in effect committing GDB to force
> these conventions on developers of JIT compilers, don't we?

If only there were a standard.  :)  I talked with some GDB developers
here at Google a month and a half ago, and we decided that this would
be a decent interface for GDB, LLVM, and potentially other JIT
compilers out there.  There's nothing intended to be LLVM specific
about it.  Most of the constraints on the interface come from the GDB
side, since GDB thinks about symbols and debug info mostly in terms of
object files.  The symbol names don't actually refer to LLVM or GDB
explicitly, and they are just __jit_debug_register_code and
__jit_debug_descriptor.

The only thing that that the interface forces on JIT authors is that
they need to link in some kind of object file generation like libelf.
LLVM happens to have code for writing ELFs, so it doesn't introduce an
external dependency, it only makes the binary using the JIT a bit
bigger.

If this restriction is a problem for somebody using this interface in
the future, then I think it could be extended to come up with some
kind of symbol "file" format, where the user specifies all of the
section headers, symbols, etc in some standard in memory structure,
and gives pointer/size pairs to the sections.  If you've used libelf,
then you can imagine something similar to the in-memory structures
that you use to describe the ELF before writing it.  That way they
only need one copy of the code and symbols.  This would probably
require implementing a more interesting BFD iovector to read the
structures and follow the pointers, and then maybe even a custom BFD
filetype, which would be a bit of work.

So while I think that would be the ideal because it requires less
effort on the client side, I consider it future work.

>> +@section Using the JIT Interface
>
> Please make each @section also a @node. ?(This will need a @menu be
> added to the parent chapter.) ?I would like to avoid sections that are
> not nodes, because nodes make it easier to navigate the manual in a
> structural way.

Sure.

>> +If the JIT recompiles code throughout its lifetime without unregistering it,
>> +then GDB and the JIT will continue to use extra memory for these symbol files.
>
> Does this simply say "don't recompile without unregistering, or you
> will leak memory", or does it say something else?

Yeah, I didn't use the phrase "leak memory" because to some people it
means specifically that the pointers are lost and cannot be freed,
which is not the case.  But on second thought, saying "leak memory" is
less confusing and less verbose.

Thanks for the review,
Reid

Attachment: jit-patch.txt
Description: Text document


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]