Since gdb 7.10, GDB has Python and Scheme APIs that allow language runtimes extending GDB with their own custom stack unwinders. This allows GDB backtracing/unwinding frames in mixed language applications (for example, an application running Java Virtual Machine) that have custom frame layouts that cannot be handled by the built-in GDB unwinders.
This page collect useful tidbits of information around those APIs.
Discussions around the APIs happen on the main gdb mailing list.
Existing users of the APIs
Related GDB bugs
Branches and patches
All on master since 7.10.
Related mailing list discussions