Creating a Versioned Struct Wrapper For a GLIBC Function


In order to accommodate a new feature or ABI change it may be necessary to change the size of a struct in GLIBC.

When a struct is used in a published interface simply changing its size is dangerous because doing so would break any existing application that relies on the previous struct size.

Changing a published interface requires providing two or more versions of a function using the technique of symbol versioning, where older compatibility interfaces are provided via versioned symbols. The new interface is not versioned, and it becomes the new default.

Unfortunately, when the new interface change involves a structure size change, using only symbol versioning to provide symbols for the old interface does not guarantee full compatibility since the static archive of GLIBC (libc.a) doesn't provide symbol versioning.

Statically linking against libc.a always results in linking to the latest interface for a function, i.e. the versioned interfaces may be there but dynamic loader symbol version matching will not find them since the static linker resolves all symbols at link time.

Problems with static linking and structure size changes


Consider the following example:


Calling 'foo()' and passing a pointer to a version 1 'struct bar', allocated by the application, to libc.a which understands version 2 'struct bar' will be problematic given any of the following structure changes:

This entire problem of a static library that understands version 1 'struct bar' being statically linked against a libc.a that understands version 2 'struct bar' may be remedied by an operating system providing static compat archives for libc.a that are built against version 1 'struct bar', but this is not an acceptable general solution because the introduction of multiple interface changes could introduce an explosion of compatibility libraries.

Creating Safe Interfaces For Changing Structure Definitions


How To Define the Compat Function


/* File 'old_foo.c'  */
#include "bar.h"

#if SHLIB_COMPAT (libc, GLIBC_2_9, GLIBC_2_10)
int
attribute_compat_text_section
__dyn_foo_2_9 (struct bar *b)
{
  return __xfoo(_BAR_VERSION_1,b);
}
compat_symbol (libc, __dyn_foo_2_9, foo, GLIBC_2_9);
#endif

Make sure to add this file to the Makefile so it gets built:

routines += old_foo

Add A Versions File Entry for the Versioned Symbol


libc {
  GLIBC_2.9 {
   foo
  }
}

Define the Structure Version Numbers

Introduce struct version definitions onto the bottom of the 'bar.h' header file.


/* File 'bar.h'  */
...
#define _BAR_VERSION_1 1
#define _BAR_VERSION_2 2
#define _BAR_VERSION _BAR_VERSION_2

Color2(red:Once defined, these definitions can-not be changed.)

These are used to tell the as-yet-undescribed wrapper function '__xfoo()', which version of the function it is to execute.
Notice that _BAR_VERSION is defined as the latest _BAR_VERSION_N. This is important, because this means that the default is always the newest interface. When you add a new version 'N' simply add a new definition and increment the default:

#define _BAR_VERSION_3 3
#define _BAR_VERSION _BAR_VERSION_3

Extend The Structure Definition


This gives us three ways to extend the structure:

In the latter two cases where there need to be multiple definitions (whether via version named structs or definition guards) one can not directly inline the implementation in the '__xfoo()' wrapper function, since one can only have a single structure definition in a compiled module.
For the later two, the choice of which method to utilize comes down to how you choose to implement the functions which operate upon 'struct bar'.

Grow the Structure


Define The Structure


/* File 'bar.h'  */

struct bar
  {
    (elements for version 1)
    (new elements for version 2)
  }

Implement The Wrapper Function


/* File 'xfoo.c'  */
#include "bar.h"

export
__xfoo (int n, struct bar *b, ...)
  {
    switch (n)
    {
      case _BAR_VERSION_1:
        (inline implementation for 'struct bar' version 1)
      case _BAR_VERSION_2:
        (inline implementation for 'struct bar' taking into account
         the new elements for Version 2)
    }
  }

When the structure grows, but fields don't change position, there's no danger of writing to the wrong field, so in the _BAR_VERSION_1 case we simply ignore the extra fields.
We need to update the Makefile so that xfoo is compiled:

# Relevant Makefile
routines += xfoo

Define The New Function Stub


This is accomplished by Makefile magic:

static-only-routines += foo

/* File 'foo.c'  */
#include "bar.h"

export
foo (struct bar *b, ...)
  {
    return __xfoo (_BAR_VERSION, b, ...);
  }


Since the default 'foo() is now provided in libc-nonshared.a an application will be linked against this static stub which defines the struct size in perpetuity and it will always call the '__xfoo()' dynamic wrapper with the correct struct version number.

Guard The Structure Definition


Define The Structure


/* File 'bar.h'  */

struct bar
  {
#ifdef _USE_BAR_VERSION_1
    (elements for version 1)
#else
    (elements for version 2)
#endif
  }
...

Implement The Wrapper Function


/* File 'xfoo.c'  */
#include "bar.h"

export
__xfoo (int n, struct bar *b, ...)
{
  switch (n)
  {
    case _BAR_VERSION_1:
      __foo_BAR_VERSION_1(b);
    case _BAR_VERSION_2:
      __foo_BAR_VERSION_2(b);
  }
}

/* File 'foo_v1.c'  */
#define _USE_BAR_VERSION_1 1
#include "bar.h"

export
__foo_BAR_VERSION_1 (struct bar *b, ...)
{
  (operate upon _USE_BAR_VERSION_1 guarded''version 1'' 'struct bar')
}

/* File 'foo_v2.c'  */
#include "bar.h"

export
__foo_BAR_VERSION_2 (struct bar *b, ...)
{
    (operate upon unguarded ''version 2'' 'struct bar')
}

Add the new files to the relevant Makefile:

# Makefile
routines += xfoo foo_v1 foo_v2

Define The New Function Stub


This is accomplished by Makefile magic:

static-only-routines += foo

/* File 'foo.c'  */
#include "bar.h"

export
foo (struct bar *b, ...)
  {
    return __xfoo (_BAR_VERSION, b, ...);
  }


Since the default 'foo() is now provided in libc-nonshared.a an application will be linked against this static stub which defines the struct size in perpetuity and it will always call the '__xfoo()' dynamic wrapper with the correct struct version number.

Use Version Named Definitions


Define The Structure


/* File 'bar.h'  */

struct bar_VERSION_1
  {
    (elements for version 1)
  }

struct bar
  {
    (elements for version 2)
  }
...

Implement The Wrapper Function


/* File 'xfoo.c'  */
#include "bar.h"

export
__xfoo (int n, void *b, ...)
  {
    switch (n)
    {
      case _BAR_VERSION_1:
        {
          struct bar_VERSION_1 *bv1 = (struct bar_VERSION_1 *)b;
          (operate upon ''version 1'' 'struct bar_VERSION_1')
         }
      case _BAR_VERSION_2:
          struct bar *bv2 = (struct bar *)b;
          (operate upon ''version 2'' 'struct bar')
    }
  }

Add the new files to the relevant Makefile:

# Makefile
routines += xfoo

Define The New Function Stub


This is accomplished by Makefile magic:

static-only-routines += foo

This implementation differs from the other since it casts the 'struct bar' implementation to a 'void *'.

/* File 'foo.c'  */
#include "bar.h"

export
foo (struct bar *b, ...)
  {
    return __xfoo (_BAR_VERSION, (void *)b, ...);
  }



Since the default 'foo() is now provided in libc-nonshared.a an application will be linked against this static stub which defines the struct size in perpetuity and it will always call the '__xfoo()' dynamic wrapper with the correct struct version number.

Shortcomings


This struct versioning mechanism won't help those cases where there's an existing static library, namely appv1.a, compiled against structure Version 1 and you attempt to statically link against structure Version 2 from a newer GLIBC (where the struct versioning mechanism has just been introduced).

Statically linking in this scenario will link to the newest instance of 'foo()' provided by libc_nonshared.a. This is incorrect because that instance of foo() is invokes _xfoo() with a Version 2 indicator. This will cause data corruption.

You can solve this by creating a special stub for foo() called foo_stub.oS which is statically linked with appv1.a into a new archive appv1_compat.a.

This stub looks like the new default stub for 'foo()' but it calls '_xfoo(_BAR_VERSION_1,...)' rather than the latest _BAR_VERSION_N.

Finally this is linked against the new libc-nonshared.a (where _BAR_VERSION_2 is the default) and the new libc.a.

Explanation


Now, each version of libc_nonshared.a that corresponds to a libc.so version will contain a statically defined 'foo' function, and it will dynamically invoke '_xfoo' and pass an 'n' version number as an argument. When libc_nonshared.a is statically linked to an application, it will forever have inlined code for 'foo' that will dynamically invoke 'xfoo' with a statically defined version number 'n', guaranteeing backward compatibility.

When the struct size changes again, libc_nonshared.a version of 'foo' will call '_xfoo' passing 'n+1', and so on. So, in each struct change, you need just to increment the version number that libc_nonshared.a statically defined 'foo' passes to '_xfoo', and then implement the new features under a dynamic check in '_xfoo'.

Further Implications


struct biz
  {
    (elements)
    struct bar b;
    (elements)
  }

If you need to version 'struct bar and you've done so by extending it, you've now endangered struct biz.

You will need to provide a similar versioned structure methodology to 'struct biz' and any functions which operate upon 'struct biz'.

Resources

Credits


None: Development/Versioning_A_Structure (last edited 2009-07-10 05:49:28 by RyanScottArnold)