PATCH: Add support for "thin" archives

["Cary Coutant" <ccoutant at google dot com>]

The attached patch adds support to bfd and binutils for a new kind of
archive I call "thin" archives. Before I go into the details about
what exactly a thin archive is, let me first get into the motivation.

Many times over the years, I've run into an issue with archive
libraries where a circular dependency arose among several libraries,
and the only resolutions available were either to combine those
archives into a single archive, or to repeat the archives on the link
line enough times that the circular dependencies were all resolved.
This eventually led to the --start-group and --end-group linker
options (and similar features in other linkers). Each time I ran into
this problem, I found myself wishing that I could build a
"meta-archive" that would combine the indexes of those archives,
without actually combining the archives themselves.

At other times, I've received requests from customers who are using
archive libraries as intermediate collections of .o files during a
build of a large project. In this usage model, the archives aren't
intended for distribution outside that project; they're just used
during the build. In these situations, the .o files will remain where
they are in the build directories, and the copying of the files into
the archive libraries is a waste of time and space -- useful only
because the archive library serves as a useful collection mechanism to
simplify the later link command.

To fill both of these needs, I came up with the idea of a "thin"
archive, which looks like a regular archive except that the individual
member files are not copied into it -- the archive holds proxy entries
instead that point to the original files where they were at the time
the archive was built. When adding another archive to a thin archive,
the file is also flattened -- the members of the member archive are
added individually. The thin archive still contains a symbol index for
all of its relocatable object members (because the file is flattened,
that includes members of archive members). This allows the linker to
process a group of libraries much more efficiently than the iterative
searching it does with --start-group/--end-group.

A thin archive is distinguished by a new magic string ("!<thin>\n"),
but the rest looks almost exactly like a normal archive. Each member
is represented by an archive header (as a proxy), but there is no copy
of the file following, so each archive header is followed immediately
by the next one. Filenames are always treated as long names and stored
in the extended name table. For flattened archives, the ar_name field
contains both the offset of the filename and the actual offset of the
member within its original archive. Filenames in a thin archive are
stored as relative pathnames (relative to the archive file), unless
they are absolute.

As a proof of concept, I built a thin archive containing libgcc.a,
libgcc_eh.a, and libc.a, and replaced the sequence "--start-group
-lgcc -lgcc_eh -lc --end-group" in the standard g++ link command with
a reference to the one thin archive file.

This patch touches a few files in bfd and binutils, and include/ar.h.
Most of the work is in bfd/archive.c. I've updated the documentation
and testsuite, and included a combined ChangeLog entry. This covers
the common ELF platforms with COFF-based archive formats; I've stubbed
out the support for flattening nested archives in binutils/emul_aix.c,
as I have no easy way of testing on AIX.

-cary

Attachment: thin-archive-patch.txt
Description: Text document