Re: Does a FFI_STRUCT type really need the elements array?

[Kaz Kylheku <kaz at kylheku dot com>]

On 22.06.2017 05:01, Anthony Green wrote:
> Hi Kaz,
>
> There are complexities around dealing with structs with nested struct 
> elements.
>
> It sounds like yours is a special case, where you are doing a lot of
> work that is normally done by libffi, which is too bad.

Hi Anthony,

There is nothing too bad about it, the work of doing the struct member
offset and alignment calculations are quite a small and easy subtask,
relative to everything. Most of that "everything" is out of the scope
of libffi.  Those little calculations for offsets and whatnot aren't
convenient if they have to be in a parallel framework of objects in
a separate library.

The main value in libffi is that it provides hand-coded machine language
routines for doing call dispatch.

Memory layouts can be done in portable C easily.

Oh, and another requirement I have is that conversion between
C objects and objects in my higher level language has to be supported
even if libffi is not available (isn't detected by the configure 
script).

I have a "HAVE_LIBFFI" preprocessor symbol, such that everything still
builds when it evaluates to zero. The foreign call/callback capability
isn't there, but other features are, like reading a C struct from a 
file,
for instance.

Why I wrote yesterday's posting is that I started adding support for
unions, and realized that the elements[] array thing is getting in the
way. I cannot fill that array with the union element types; libffi
will miscalculate the size. Basically, the approach I'm going to take
is to calculate the size based on the largest union element, then pad
it according to the worst-case alignment, and stick those values into
the ffi_type, which will be of FFI_TYPE_STRUCT. Then I might as well
treat structs and arrays that way too.

> The libffi API is far from perfect, but this is one area where there
> have been few complaints.  That being said, it would be interesting to
> hear a more fully formed proposal that could be considered prior to a
> major API breaking release.

For the people that rely on the struct member calculations, it would
probably be helpful to have these requirements:

* A way to request the calculation to be done after preparing the
  tree of types: like a void ffi_type_init(ffi_type *) function
  or whatever. You shouldn't have to involve a ffi_type in the
  construction of a call descriptor in order to get libffi to
  calculate its size, alignment and member offsets.

* the elements[] array should be an array of structures which
  provide the offset information. (I have exactly such an array in
  my own struct: it holds offsets, and also shift values and
  masks for bitfields.)  For backward compatibility, elements can
  be kept as-is (array of ffi_type * pointers), but a parallel array
  can be added for the extra info. If that array pointer is not null,
  it gets filled in.

* unions could be supported with a FFI_TYPE_UNION which simply
  puts all the elements at offset zero and does the size and
  alignment calculation accordingly.

I get your comment about API breaking. If members are added to
ffi_type, and something like ELF versioning is not used, then
clients compiled with the old headers cannot work with a new lib.

Any types like ffi_type that are allocated by client code should
contain padding members for future extension.

I see in ffi.h that we have this:

typedef struct _ffi_type
{
  size_t size;
  unsigned short alignment;
  unsigned short type;
  struct _ffi_type **elements;
} ffi_type;

which is somewhat of a mistake; there should be something at the end 
like

  void *pad[4]; /* reserved for future extension */

possibly along with the requirement that all correctly written
client code must do memset(&type->pad, 0, sizeof type->pad) or
equivalent, such as using calloc for these objects or whatever.

That requirement makes it possible, in the future, to add
members to the structure whose initialization responsibility
rests with the application. not just members that are managed
internally by the library.

Other possibilities are to have a version handshake;
have a pair of preprocessor symbols FFI_MAJOR and FFI_MINOR which are
passed to some call like ffi_init(FFI_MAJOR, FFI_MINOR).
Then libffi knows exactly which version of the header file
the binary client was compiled with. It knows that an old
client wouldn't have initialized members that do not exist
in its version of the struct declaration. If the space is
reserved, it can be treated as uninitialized, and used.
If the space is not reserved, the library can avoids
touching it.


Cheers ...