Differences between revisions 27 and 28
Revision 27 as of 2009-01-30 20:13:39
Size: 18753
Comment:
Revision 28 as of 2009-07-10 05:49:28
Size: 18753
Comment:
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
= Creating a versioned struct wrapper for a GLIBC function = = Creating a Versioned Struct Wrapper For a GLIBC Function =

Creating a Versioned Struct Wrapper For a GLIBC Function


In order to accommodate a new feature or ABI change it may be necessary to change the size of a struct in GLIBC.

When a struct is used in a published interface simply changing its size is dangerous because doing so would break any existing application that relies on the previous struct size.

Changing a published interface requires providing two or more versions of a function using the technique of symbol versioning, where older compatibility interfaces are provided via versioned symbols. The new interface is not versioned, and it becomes the new default.

Unfortunately, when the new interface change involves a structure size change, using only symbol versioning to provide symbols for the old interface does not guarantee full compatibility since the static archive of GLIBC (libc.a) doesn't provide symbol versioning.

Statically linking against libc.a always results in linking to the latest interface for a function, i.e. the versioned interfaces may be there but dynamic loader symbol version matching will not find them since the static linker resolves all symbols at link time.

Problems with static linking and structure size changes


Consider the following example:

  • Our system has GLIBC version 2.9 installed. In libc.so and libc.a there is a function foo() defined in foo.c:

    /* File 'foo.c'  */
    #include "bar.h"
    
    export foo (struct bar *b, ...)
      {
        (...)
      }
  • Function foo() take a pointer to struct bar as a parameter and struct bar is defined in bar.h:

    /* File 'bar.h'  */
    
    struct bar /* Version 1 */
      {
        (elements for Version 1)
      }
  • We have created our own static library archive, app.a, that has components built against GLIBC 2.9:

    • This archive is made up of app.oS and libc_nonshared.a

    /* Application 'app.c'  */
    #include "bar.h"
    
    int main() {
      struct bar b;
      struct fiz f;
      foo(&b); /
      return 0;
    }
    • Note: For the sake of demonstration app.oS actually contains the code necessary for an executable, i.e. main().

  • app.oS was created using:

    gcc app.c -c -o app.oS
  • The archive was created using:
    ar cr app.a app.oS /usr/lib/libc_nonshared.a
  • This is linked into an application on our system against libc.a GLIBC 2.9 using the following:

    gcc -static -lc app.c -o app
  • The following figure demonstrates static linking without a version wrapper:

    static-linking-no-struct-wrapper.png

  • Let's say that for GLIBC 2.10 we extend the definition of struct bar:

    • /* File 'bar.h'  */
      
      struct bar /* Version 2.0 */
        {
          (elements for Version 1)
          (new elements for Version 2)
        }
  • The following figure demonstrates static linking with a version wrapper:

    static-linking-with-struct-wrapper.png

  • If the app.a which was built against GLIBC 2.9 is now linked against GLIBC 2.10 libc.a we will encounter data corruption on application execution as explained in the following section.


Calling 'foo()' and passing a pointer to a version 1 'struct bar', allocated by the application, to libc.a which understands version 2 'struct bar' will be problematic given any of the following structure changes:

  • If 'struct bar' grows: libc.a will write off the end of the allocated space for structure version 1.0.

  • If 'struct bar' shrinks: libc.a will only fill in the number of fields it feels necessary which may not be the number of fields necessary for the application to operation successfully.

  • If the fields in 'struct bar' have moved: libc.a will populate the structure in the incorrect way. There are alignment issues, field ordering issues, field size issues, etc. The application will get back a completely corrupted structure.

    • Note: Try not to move fields anyway. Some applications, and GLIBC itself, make use of pre-computed field offsets or even access the first field of a structure via the structure pointer itself.

This entire problem of a static library that understands version 1 'struct bar' being statically linked against a libc.a that understands version 2 'struct bar' may be remedied by an operating system providing static compat archives for libc.a that are built against version 1 'struct bar', but this is not an acceptable general solution because the introduction of multiple interface changes could introduce an explosion of compatibility libraries.

Creating Safe Interfaces For Changing Structure Definitions


  • In order to prevent the problems mentioned earlier without creating compatibility libraries, we need to:
  • Provide compat versions of foo() built against the old structure size which calls __xfoo(_BAR_VERSION_1).

  • Update Versions file to provide a versioned interface for foo().

  • Define the version numbers for the struct bar versions we need to provide.

  • Provide a new non-versioned implementation of foo() which simply calls '__xfoo(_BAR_VERSION)' with the default _BAR_VERSION. This version exists in libc_nonshared.a and is statically linked into all applications, including those which are dynamically and statically linked

  • Extend the structure definition.
  • Modify struct bar in a variety of ways.

  • Provide a wrapper function for 'foo()' called '__xfoo()' which takes _BAR_VERSION_N as a first parameter and implements 'foo()' based upon the version indicated by the parameter.

How To Define the Compat Function


  • First, we must make sure that applications which use the old struct sizes still work when linking dynamically against libc.so.

  • Create a compatibility function '__dyn_foo_2_9()' and tell the build system that this is a compat symbol for 'foo()' in the file 'old_foo.c'.

    • For our example we'll assume that GLIBC_2_10 is the next version of GLIBC, and GLIBC_2_9 is where we're introducing our new 'struct bar' change.

/* File 'old_foo.c'  */
#include "bar.h"

#if SHLIB_COMPAT (libc, GLIBC_2_9, GLIBC_2_10)
int
attribute_compat_text_section
__dyn_foo_2_9 (struct bar *b)
{
  return __xfoo(_BAR_VERSION_1,b);
}
compat_symbol (libc, __dyn_foo_2_9, foo, GLIBC_2_9);
#endif

Make sure to add this file to the Makefile so it gets built:

routines += old_foo

Add A Versions File Entry for the Versioned Symbol


  • Add the symbol 'foo' to the Versions file for export under a specific version. In our case, since we want an exported versioned symbol for 2.9 we need to add that to the GLIBC_2.9 Versions definition:

libc {
  GLIBC_2.9 {
   foo
  }
}

Define the Structure Version Numbers

Introduce struct version definitions onto the bottom of the 'bar.h' header file.


/* File 'bar.h'  */
...
#define _BAR_VERSION_1 1
#define _BAR_VERSION_2 2
#define _BAR_VERSION _BAR_VERSION_2

Color2(red:Once defined, these definitions can-not be changed.)

These are used to tell the as-yet-undescribed wrapper function '__xfoo()', which version of the function it is to execute.
Notice that _BAR_VERSION is defined as the latest _BAR_VERSION_N. This is important, because this means that the default is always the newest interface. When you add a new version 'N' simply add a new definition and increment the default:

#define _BAR_VERSION_3 3
#define _BAR_VERSION _BAR_VERSION_3

Extend The Structure Definition


  • There are three possibilities for structure changes and we'll address all three. The version 2 of 'struct bar' has:

  • fewer elements than version 1.

  • a different element order than version 1.

  • more elements than version 1.

This gives us three ways to extend the structure:

  • Grow The Structure : Applicable to struct change 3.
  • Guard The Structure Definition: Applicable to struct changes 1. and 2.
  • Use Version Named Definitions: Applicable to struct changes 1. and 2.

In the latter two cases where there need to be multiple definitions (whether via version named structs or definition guards) one can not directly inline the implementation in the '__xfoo()' wrapper function, since one can only have a single structure definition in a compiled module.
For the later two, the choice of which method to utilize comes down to how you choose to implement the functions which operate upon 'struct bar'.

Grow the Structure


  • This is the easy case. We can simply extend 'struct bar' and don't need any special version guards or version named structures. For compatibility case, we simply end up allocating more space than necessary for the structure.

Define The Structure


  • New elements must always follow old elements, so simply add the new elements to the definition.

/* File 'bar.h'  */

struct bar
  {
    (elements for version 1)
    (new elements for version 2)
  }

Implement The Wrapper Function


  • The purpose of the wrapper function is to query the first parameter which is the struct version number. It then executes the correct implementation of function 'foo()' based upon the version requested.
    A simple definition of the wrapper function '__xfoo()' in the case where 'struct bar' is extended looks something like the following:

/* File 'xfoo.c'  */
#include "bar.h"

export
__xfoo (int n, struct bar *b, ...)
  {
    switch (n)
    {
      case _BAR_VERSION_1:
        (inline implementation for 'struct bar' version 1)
      case _BAR_VERSION_2:
        (inline implementation for 'struct bar' taking into account
         the new elements for Version 2)
    }
  }

When the structure grows, but fields don't change position, there's no danger of writing to the wrong field, so in the _BAR_VERSION_1 case we simply ignore the extra fields.
We need to update the Makefile so that xfoo is compiled:

# Relevant Makefile
routines += xfoo

Define The New Function Stub


  • We now create a new default implementation of the symbol 'foo()'. This new default is actually provided only by libc_nonshared.a.

This is accomplished by Makefile magic:

static-only-routines += foo

/* File 'foo.c'  */
#include "bar.h"

export
foo (struct bar *b, ...)
  {
    return __xfoo (_BAR_VERSION, b, ...);
  }
  • Notice how this always uses _BAR_VERSION, which is the latest available struct version.


Since the default 'foo() is now provided in libc-nonshared.a an application will be linked against this static stub which defines the struct size in perpetuity and it will always call the '__xfoo()' dynamic wrapper with the correct struct version number.

Guard The Structure Definition


Define The Structure


  • When 'bar.h' is #included by a .c file it will need to choose which version of the structure it wants defined:

/* File 'bar.h'  */

struct bar
  {
#ifdef _USE_BAR_VERSION_1
    (elements for version 1)
#else
    (elements for version 2)
#endif
  }
...

Implement The Wrapper Function


  • '__xfoo()' must invoke a function for each 'struct bar' version dependent implementation where said functions exist in other modules, i.e. other '.c' files. This allows different definitions of 'struct bar' per implementation.

/* File 'xfoo.c'  */
#include "bar.h"

export
__xfoo (int n, struct bar *b, ...)
{
  switch (n)
  {
    case _BAR_VERSION_1:
      __foo_BAR_VERSION_1(b);
    case _BAR_VERSION_2:
      __foo_BAR_VERSION_2(b);
  }
}

/* File 'foo_v1.c'  */
#define _USE_BAR_VERSION_1 1
#include "bar.h"

export
__foo_BAR_VERSION_1 (struct bar *b, ...)
{
  (operate upon _USE_BAR_VERSION_1 guarded''version 1'' 'struct bar')
}

/* File 'foo_v2.c'  */
#include "bar.h"

export
__foo_BAR_VERSION_2 (struct bar *b, ...)
{
    (operate upon unguarded ''version 2'' 'struct bar')
}

Add the new files to the relevant Makefile:

# Makefile
routines += xfoo foo_v1 foo_v2

Define The New Function Stub


  • We now create a new default implementation of the symbol 'foo()'. This new default is actually provided only by libc_nonshared.a.

This is accomplished by Makefile magic:

static-only-routines += foo

/* File 'foo.c'  */
#include "bar.h"

export
foo (struct bar *b, ...)
  {
    return __xfoo (_BAR_VERSION, b, ...);
  }
  • Notice how this always uses _BAR_VERSION, which is the latest available struct version.


Since the default 'foo() is now provided in libc-nonshared.a an application will be linked against this static stub which defines the struct size in perpetuity and it will always call the '__xfoo()' dynamic wrapper with the correct struct version number.

Use Version Named Definitions


Define The Structure


/* File 'bar.h'  */

struct bar_VERSION_1
  {
    (elements for version 1)
  }

struct bar
  {
    (elements for version 2)
  }
...

Implement The Wrapper Function


  • When using version named structures the header file "bar.h" introduces new structure names for old versions of 'struct bar'. The '__xfoo()' wrapper takes a pointer to 'struct bar' as 'void *' and these are cast to a version named 'struct bar', such as 'struct bar_VERSION_1'.

/* File 'xfoo.c'  */
#include "bar.h"

export
__xfoo (int n, void *b, ...)
  {
    switch (n)
    {
      case _BAR_VERSION_1:
        {
          struct bar_VERSION_1 *bv1 = (struct bar_VERSION_1 *)b;
          (operate upon ''version 1'' 'struct bar_VERSION_1')
         }
      case _BAR_VERSION_2:
          struct bar *bv2 = (struct bar *)b;
          (operate upon ''version 2'' 'struct bar')
    }
  }

Add the new files to the relevant Makefile:

# Makefile
routines += xfoo

Define The New Function Stub


  • We now create a new default implementation of the symbol 'foo()'. This new default is actually provided only by libc_nonshared.a.

This is accomplished by Makefile magic:

static-only-routines += foo

This implementation differs from the other since it casts the 'struct bar' implementation to a 'void *'.

/* File 'foo.c'  */
#include "bar.h"

export
foo (struct bar *b, ...)
  {
    return __xfoo (_BAR_VERSION, (void *)b, ...);
  }
  • Notice how this always uses _BAR_VERSION, which is the latest available struct version.



Since the default 'foo() is now provided in libc-nonshared.a an application will be linked against this static stub which defines the struct size in perpetuity and it will always call the '__xfoo()' dynamic wrapper with the correct struct version number.

Shortcomings


This struct versioning mechanism won't help those cases where there's an existing static library, namely appv1.a, compiled against structure Version 1 and you attempt to statically link against structure Version 2 from a newer GLIBC (where the struct versioning mechanism has just been introduced).

Statically linking in this scenario will link to the newest instance of 'foo()' provided by libc_nonshared.a. This is incorrect because that instance of foo() is invokes _xfoo() with a Version 2 indicator. This will cause data corruption.

  • static-linking-danger.png

You can solve this by creating a special stub for foo() called foo_stub.oS which is statically linked with appv1.a into a new archive appv1_compat.a.

  • static_link_compat_stub.png

    • Note: In the figure _xfoo() takes a digit rather than a #define _BAR_VERSION_N. This is to indicate that once foo_stub.oS is compiled _BAR_VERSION_N is expanded into the digit '1' and passed to _xfoo() in perpetuity.

This stub looks like the new default stub for 'foo()' but it calls '_xfoo(_BAR_VERSION_1,...)' rather than the latest _BAR_VERSION_N.

Finally this is linked against the new libc-nonshared.a (where _BAR_VERSION_2 is the default) and the new libc.a.

Explanation


Now, each version of libc_nonshared.a that corresponds to a libc.so version will contain a statically defined 'foo' function, and it will dynamically invoke '_xfoo' and pass an 'n' version number as an argument. When libc_nonshared.a is statically linked to an application, it will forever have inlined code for 'foo' that will dynamically invoke 'xfoo' with a statically defined version number 'n', guaranteeing backward compatibility.

When the struct size changes again, libc_nonshared.a version of 'foo' will call '_xfoo' passing 'n+1', and so on. So, in each struct change, you need just to increment the version number that libc_nonshared.a statically defined 'foo' passes to '_xfoo', and then implement the new features under a dynamic check in '_xfoo'.

  • static_linking_final.png

Further Implications


  • It is naive to think that simply extending a structure is safe and doesn't require guards by considering the following scenario:

struct biz
  {
    (elements)
    struct bar b;
    (elements)
  }

If you need to version 'struct bar and you've done so by extending it, you've now endangered struct biz.

You will need to provide a similar versioned structure methodology to 'struct biz' and any functions which operate upon 'struct biz'.

Resources

Credits


  • This document was originally written by Carlos Eduardo Seo (IBM) and Ryan S. Arnold (IBM).
    All images were originally created by Ryan S. Arnold (IBM) using Inkscape.

None: Development/Versioning_A_Structure (last edited 2009-07-10 05:49:28 by RyanScottArnold)