Guidance on Tools for Implementing Dynamic Plugin System in Firmware

[Nick Clifton]

Hi Max,

> You assume that I have a high-level OS. Although I use FreeRTOS (which
> is little more than a scheduler), for the purposes of the answers I
> seek consider my project as a bare-metal

Ah!  Thanks for the clarification.

>>>   1. *File Format and Loading:* Recommendations for file formats
>> Whatever file format your OS supports.  This will probably be ELF.
> My OS does not support any format, so I was looking for one that is
> "easy" to design and write, or already used in similar contexts.

Well I would still suggest ELF, even though technically speaking it is
actually more complex than other file formats (eg COFF).  The reasons
for this are that the ELF format is very well documented and there are
lots of tools for inspecting and manipulating ELF binaries.

If however simplicity truly is your goal then maybe the a.out format
would be the best choice:  https://wiki.osdev.org/A.out

Also if you really are programming at the bare-metal level then you
may find that defining your own - very simple - file format will be
the best solution.  You could have just the features that you need
and nothing else, and keeping things simple always helps to make a
project easier to manage.

Do you anticipate the need for a file system ?  Ie are you going to
have these plugins exist as separate entities to be loaded on demand
or will there just be a single monolithic binary containing both
firmware and plugins ?

>>>   2. *Function Entry Points:* Best practices for managing function entry points in non-PIC firmware for plugins?
>> I am not sure what you mean here.  But my general advice for
>> implementing a new feature or system is to follow three simple
>> steps:
> In my plugin implementation scenario, unlike typical cases where
> plugins solely provide functionalities to the host, my plugins will
> also need to utilize APIs offered by the firmware. I've deduced that
> the plugins should have two function pointer tables: one for the
> functions they provide and another for the functions they use from the
> firmware.

In linker terminology I think that these would be called "exported" functions
for the API provided by the plugin and "imported" functions for the ones that
the plugin expects to be provided by the firmware's API.

> Hence, I believe the linker/loader should populate this
> second table. 

And the first one too.  Also in some file formats these two tables are
combined into one, with symbols having a type to indicate if they are
exported or imported.

> Additionally, the firmware must be designed to retrieve
> function pointers at runtime for invoking functions provided by the
> plugin.

Really ?  I would have expected it to be the other way around.  ie a plugin,
once it becomes active, should register with the firmware all of the function
pointers that it wants used.  So the firmware is passive and does not have to
retrieve anything.  Instead it is the plugins that must tell the firmware which
parts of its API they want to use.

> How can I effectively manage these function entry points and
> ensure seamless interaction between the firmware and the plugins?

Well this is obviously the job of the linker and loader combined.  Plus you
will need some way to annotate the sources for the plugins and the firmware
so that when they are compiled/assembled they can indicate which functions
they are exporting and which functions they are expected to be able to import.


>>>   4. *Compilation and Linking:* Specific parameters or considerations for compiling and linking, including post-linking steps.

> In light of the requirement for dual function pointer tables in my
> plugins (one for functions they provide and another for functions they
> use from the firmware) I am seeking guidance on the compilation
> process. I anticipate needing to use flags like -fPIC or -fPIE and
> link with options like --unresolved-symbols=ignore-in-object-files and
> --emit-relocs. However, I would like to know what additional
> parameters are necessary. Specifically, are the standard compiler and
> linker sufficient to 'create' these tables, or is there a need for any
> post-build steps?

In theory the standard compiler and linker should be sufficient (if you are
using one of the file formats mentioned above and not rolling your own).
The firmware would be represented as a library file, either shared or static
depending upon how you intend all of this to work.  This library should
provide symbols to satisfy all of the unresolved references found in the
plugin's code.  Meanwhile the plugin's API should be provided as a set of
globally visible symbols in its symbol table.  Handling libraries, resolving
symbols references and creating symbol tables are all things that the
linker should - in theory at least - be able to handle.

> Should I anticipate the need to utilize existing
> tools or develop custom solutions for this purpose?

Possibly.  It is hard to say without a specific design in place.


> I hope this last question is enough to consider this post relevant to
> the mailing list

Fair enough.

I have given my thoughts above.  Maybe others will also be willing to share
their insights.

Cheers
   Nick