18.5 User Module Loaders

While writing the module loading code for GNU M4 1.5, I found that libltdl did not provide a way for loading modules in exactly the way I required: As good as the preloading feature of libltdl may be, and as useful as it is for simplifying debugging, it doesn’t have all the functionality of full dynamic module loading when the host platform is limited to static linking. After all, you can only ever load modules that were specified at link time, so for access to user supplied modules the whole application must be relinked to preload these new modules before lt_dlopen will be able to make use of the additional module code.

In this situation, it would be useful to be able to automate this process. That is, if a libltdl using process is unable to lt_dlopen a module in any other fashion, but can find a suitable static archive in the module search path, it should relink itself along with the static archive (using libtool to preload the module), and then exec the new executable. Assuming all of this is successful, the attempt to lt_dlopen can be tried again – if the ‘suitable’ static archive was chosen correctly it should now be possible to access the preloaded code.

18.5.1 Loader Mechanism

Since Libtool 1.4, libltdl has provided a generalized method for loading modules, which can be extended by the user. libltdl has a default built in list of module loading mechanisms, some of which are peculiar to a given platform, others of which are more general. When the ‘libltdl’ subdirectory of a project is configured, the list is narrowed to include only those mechanisms, or simply loaders, which can work on the host architecture. When ‘lt_dlopen’ is called, the loaders in this list are tried, in order, until the named module has loaded, or all of the loaders in the list have been exhausted. The entries in the final list of loaders each have a unique name, although there may be several candidate loaders for a single name before the list is narrowed. For example, the ‘dlopen’ loader is implemented differently on BeOS and Solaris – for a single host, there can be only one implementation of any named loader. The name of a module loader is something entirely different to the name of a loaded module, something that should become clearer as you read on.

In addition to the loaders supplied with libltdl, your project can add more loaders of its own. New loaders can be added to the end of the existing list, or immediately before any other particular loader, thus giving you complete control of the relative priorities of all of the active loaders in your project.

In your module loading API, you might even support the dynamic loading of user supplied loaders: that is your users would be able to create dynamic modules which added more loading mechanisms to the existing list of loaders!

Version 1.4 of Libtool has a default list that potentially contains an implementation of the following loaders (assuming all are supported by the host platform):

dlpreopen

If the named module was preloaded, use the preloaded symbol table for subsequent lt_dlsym calls.

dlopen

If the host machine has a native dynamic loader API use that to try and load the module.

dld

If the host machine has GNU dld(44), use that to try and load the module.

Note that loader names with a ‘dl’ prefix are reserved for future use by Libtool, so you should choose something else for your own module names to prevent a name clash with future Libtool releases.

18.5.2 Loader Management

The API supplies all of the functions you need to implement your own module loading mechanisms to solve problems just like this:

Function: lt_dlloader_t * lt_dlloader_find (const char *loader_name)

Each of the module loaders implemented by libltdl is stored according to a unique name, which can be used to lookup the associated handle. These handles operate in much the same way as lt_dlhandles: They are used for passing references to modules in and out of the API, except that they represent a kind of module loading method, as opposed to a loaded module instance.

This function finds the ‘lt_dlloader_t’ handle associated with the unique name passed as the only argument, or else returns ‘NULL’ if there is no such module loader registered.

Function: int lt_dlloader_add (lt_dlloader_t *place, lt_user_dlloader *dlloader, const char *loader_name)

This function is used to register your own module loading mechanisms with libltdl. If place is given it must be a handle for an already registered module loader, which the new loader dlloader will be placed in front of for the purposes of which order to try loaders in. If place is ‘NULL’, on the other hand, the new dlloader will be added to the end of the list of loaders to try when loading a module instance. In either case loader_name must be a unique name for use with lt_dlloader_find.

The dlloader argument must be a C structure of the following format, populated with suitable function pointers which determine the functionality of your module loader:

struct lt_user_dlloader { const char *sym_prefix; lt_module_open_t *module_open; lt_module_close_t *module_close; lt_find_sym_t *find_sym; lt_dlloader_exit_t *dlloader_exit; lt_dlloader_data_t dlloader_data; };

Function: int lt_dlloader_remove (const char *loader_name)

When there are no more loaded modules that were opened by the given module loader, the loader itself can be removed using this function.

When you come to set the fields in the lt_user_dlloader structure, they must each be of the correct type, as described below:

Type: const char * sym_prefix

If a particular module loader relies on a prefix to each symbol being looked up (for example, the Windows module loader necessarily adds a ‘_’ prefix to each symbol name passed to lt_dlsym), it should be recorded in the ‘sym_prefix’ field.

Type: lt_module_t lt_module_open_t (lt_dlloader_data_t loader_data, const char *module_name)

When lt_dlopen has reached your registered module loader when attempting to load a dynamic module, this is the type of the module_open function that will be called. The name of the module that libltdl is attempting to load, along with the module loader instance data associated with the loader being used currently, are passed as arguments to such a function call.

The lt_module_t returned by functions of this type can be anything at all that can be recognised as unique to a successfully loaded module instance when passed back into the module_close or find_sym functions in the lt_user_dlloader module loader structure.

Type: int lt_module_close_t (lt_dlloader_data_t loader_data, lt_module_t module)

In a similar vein, a function of this type will be called by lt_dlclose, where module is the returned value from the ‘module_open’ function which loaded this dynamic module instance.

Type: lt_ptr_t lt_find_sym_t (lt_dlloader_data_t loader_data, lt_module_t module, const char *symbol_name)

In a similar vein once more, a function of this type will be called by lt_dlsym, and must return the address of symbol_name in module.

Type: int lt_dlloader_exit_t (lt_dlloader_data_t loader_data)

When a user module loader is lt_dlloader_removed, a function of this type will be called. That function is responsible for releasing any resources that were allocated during the initialisation of the loader, so that they are not ‘leaked’ when the lt_user_dlloader structure is recycled.

Note that there is no initialisation function type: the initialisation of a user module loader should be performed before the loader is registered with lt_dlloader_add.

Type: lt_dlloader_data_t dlloader_data

The dlloader_data is a spare field which can be used to store or pass any data specific to a particular module loader. That data will always be passed as the value of the first argument to each of the implementation functions above.

18.5.3 Loader Errors

When writing the code to fill out each of the functions needed to populate the lt_user_dlloader structure, you will often need to raise an error of some sort. The set of standard errors which might be raised by the internal module loaders are available for use in your own loaders, and should be used where possible for the sake of uniformity if nothing else. On the odd occasion where that is not possible, libltdl has API calls to register and set your own error messages, so that users of your module loader will be able to call lt_dlerror and have the error message you set returned:

Function: int lt_dlseterror (int errorcode)

By calling this function with one of the error codes enumerated in the header file, ‘ltdl.h’, lt_dlerror will return the associated diagnostic until the error code is changed again.

Function: int lt_dladderror (const char *diagnostic)

Often you will find that the existing error diagnostics do not describe the failure you have encountered. By using this function you can register a more suitable diagnostic with libltdl, and subsequently use the returned integer as an argument to lt_dlseterror.

libltdl provides several other functions which you may find useful when writing a custom module loader. These are covered in the Libtool manual, along with more detailed descriptions of the functions described in the preceding paragraphs.

In the next chapter, we will discuss the more complex features of Automake, before moving on to show you how to use those features and add libltdl module loading to the Sic project from A Large GNU Autotools Project in the chapter after that.

This document was generated by Ben Elliston on July 10, 2015 using texi2html 1.82.