Re: Assembler as shared library

[David Malcolm <dmalcolm at redhat dot com>]

[existing thread uses top-posting, so am going with that]

Philip: thanks for working on this.

Some context for binutils people: I've been working on an experimental
branch of gcc that can be built as a shared library for just-in-time
compilation:
http://gcc.gnu.org/wiki/JIT
The GCC JIT codebase can take client API calls and give assembler,
rather than machine code.  Right now I have an ugly hack that calls out
to (another) gcc to use gcc's harness logic to invoke programs to turn
the .s file into a .so (via a .o), then links it into memory.  IIRC,
this hack takes about half the wallclock, though I don't know to what
extent it's the expense of execing subprocesses vs the inherent work of
the .s to machine code, and ELF manipulation.  So it would be nice to
have the existing binutils code be available as a library that can turn
either a .s file (or even an in-memory string) into machine code.  Hence
my interest in the GNU Assembler as a shared library (although as I
understand it that's only half the battle - turning it into a DSO would
be further effort, and ideally we could skip much of this, since it's
all in-process).

>From the JIT's point of view, what I need is something that can
repeatedly be invoked within one process (perhaps from different
threads), so I think state management is going to be an issue here.
Last time I looked at the gas source code, my recollection is that there
are quite a few global variables.

One way to tackle this is to get rid of all globals, moving them all
into some kind of "context" struct.  Another way is to put a big mutex
around the library, and to purge all state when done.  I'm using a
similar approach in gcc, and the latter especially within my "jit"
branch.

An approach to take might be an outside-in one: what should the API look
like?   From the libgccjit perspective, perhaps something like this
would be good:

  extern gas_context *
  gas_context_acquire (void);

  extern void
  gas_context_release (gas_context *ctxt);

  extern void
  gas_context_set_input_file (gas_context *ctxt,
                              const char *path);

  extern void
  gas_context_set_output_file (gas_context *ctxt,
                               const char *path);

  extern int
  gas_context_assemble (gas_context *ctxt);

??? (caveat: very much a first iteration, no idea if the above is sane,
though there's a stylistic resemblance to the libgccjit.h API)

Hope this is helpful
Dave

On Mon, 2014-02-17 at 16:54 +0000, Philip Herron wrote:
> Hey all
> 
> This is my initial patch it creates a shared library out of gas and
> installs the necessary headers along side it. I am trying to work out
> how to use it so far need more time tracing through the code to figure
> it out but so far i am been copying and pasting bits and pieces to get
> this so far as a client program:
> 
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> 
> #include <as.h>
> #include <subsegs.h>
> #include <output-file.h>
> 
> char * out_file_name;
> 
> int main (int argc, char **argv)
> {
>     printf ("trying...\n");
>     out_file_name = strdup ("./ninja");
> 
>     bfd_init ();
>     md_begin ();
> 
>     symbol_begin ();
>     frag_init ();
>     subsegs_begin ();
>     read_begin ();
>     input_scrub_begin ();
>     expr_begin ();
> 
>     output_file_create (out_file_name);
> 
>     dot_symbol_init ();
> 
>     text_section = subseg_new (TEXT_SECTION_NAME, 0);
>     data_section = subseg_new (DATA_SECTION_NAME, 0);
>     bss_section = subseg_new (BSS_SECTION_NAME, 0);
> 
>     subseg_new (BFD_ABS_SECTION_NAME, 0);
>     subseg_new (BFD_UND_SECTION_NAME, 0);
>     reg_section = subseg_new ("*GAS `reg' section*", 0);
>     expr_section = subseg_new ("*GAS `expr' section*", 0);
>     subseg_set (text_section, 0);
> 
>     read_a_source_file ("./t.s");
>     write_object_file ();
> 
>     free (out_file_name);
> 
>     return 0;
> }
> 
> $ gcc -g -O2 -Wall test.c -o test -I/opt/binutils/include
> -L/opt/binutils/lib -lgas -lbfd
> 
> Its definitely not right but demonstrates i am calling into normal gas
> things. Hope to get more working soon and i will probably need to make
> some wrapper code as this although i am doing it completely wrong is
> fairly involved. I would great if i could get some feedback i reckon
> this patch is probably going to break some stuff.
> 
> Thanks i hope i can get something more use-able soon.
> 
> --Phil
> 
> On 14 February 2014 17:06, Philip Herron <redbrain@gcc.gnu.org> wrote:
> > Ah thanks! I have a github repo:
> >
> > https://github.com/redbrain/binutils-gdb/tree/libasm
> >
> > It now builds libasm.la which as is built off now and as seems to
> > work. So i think i have the first stages of this working. I am
> > #ifdefing out the main so far and need to make a libgas.h so we can
> > use this in David Malcoms jit.
> >
> > On 14 February 2014 15:13, nick clifton <nickc@redhat.com> wrote:
> >> Hi Philip,
> >>
> >>
> >>> But i can find any where to start to make a little mini assembler
> >>> using libopcodes.
> >>
> >>
> >> Take a look at gas/doc/internals.texi, especially the section on GAS
> >> processing.
> >>
> >> Essentially you will need to provide library routines that emulate what is
> >> done in gas/as.c:main().
> >>
> >> Cheers
> >>   Nick
> >>
> >>