This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Temporarily avoid PPC64 LE bootstrap by making GLIBC_2.17 and GLIBC_2.18 equivalent.
- From: "Carlos O'Donell" <carlos at redhat dot com>
- To: Adam Conrad <adconrad at 0c3 dot net>, Michael Matz <matz at suse dot de>, Andreas Jaeger <aj at suse dot com>, Andreas Schwab <schwab at suse dot de>, Steven Munroe <sjmunroe at us dot ibm dot com>, Adhemerval Zanella <azanella at linux dot vnet dot ibm dot com>, Brent Baude <baude at us dot ibm dot com>, Siddhesh Poyarekar <siddhesh at redhat dot com>, Jeff Law <law at redhat dot com>, GNU C Library <libc-alpha at sourceware dot org>
- Cc: Matt Newsome <mnewsome at redhat dot com>
- Date: Fri, 31 Jan 2014 03:40:29 -0500
- Subject: Temporarily avoid PPC64 LE bootstrap by making GLIBC_2.17 and GLIBC_2.18 equivalent.
- Authentication-results: sourceware.org; auth=none
Distribution Maintainers,
Downgrading the PPC64 LE ABI baseline from 2.18 to 2.17
would normally require a bootstrap from scratch as the core
C library is no longer able to run any of the binaries on the
system. Avoiding this bootstrap saves everyone a lot of time.
To that end I've evaluated several method for avoiding a
full bootstrap including:
* Cross compiling.
* Preloading a library to provide the required symbols.
* Load an alternate libc from an alternate path to provide
the required symbols.
* Patch the dynamic loader to make the symbols equivalent.
Patching the dynamic loader is the easiest solution.
The dynamic loader is patched to treat a dependency on
foo@GLIBC_2.18 as satisfied by foo@GLIBC_2.18 or
foo@GLIBC_2.17.
Such a patch should remain applied to the dynamic loader until
all references to GLIBC_2.18 symbols that should be GLIBC_2.17
symbols are GLIBC_2.17 (or until you rebuild everything against
the new glibc).
This is not a hack you want to carry into production since
the semantics of the hack violate the normal symbol resolution
rules. However, for the ABI as it stands the hack will work
perfectly fine to allow you to avoid bootstrapping the
distribution. It works because we did not add any compatibility
symbols in 2.18 that would have been versioned at 2.17. That
would have created two functions with different behaviour and
this patch would select between them arbitrarily (since the
patch would make them equivalent) and break your binary.
We have no such situation here.
Tested in the following scenario:
* Built x86-64 with ABI baseline set to 2.19.
* Built applications against 2.19-based ABI.
* Built x86-64 with ABI baseline set to 2.18 + patch.
* Ran prebuilt applications against 2.18-based ABI and
verified correct operation.
Applications included multi-threaded and single-threaded
test applications using libc, libm, and libpthread.
I'm providing 3 patches.
Apply 0, then 1 or 2.
0 - Make it possible to load libraries that have a dependency
on another library from the equivalent version set.
1 - Make it possible to resolve a symbol from the equivalent
version set.
2 - Make it possible to resolve a symbol from the equivalent
version set -- debugging version.
A - Refactor check_match into full function for easier debugging.
- Tested for months and already proposed as a WIP.
https://sourceware.org/ml/libc-alpha/2013-12/msg00559.html
B - Add symbol equivalency checking on top of A.
Comments?
This needs some more thorough testing, but I'm happy to provide assistance.
Cheers,
Carlos.
Patch #0
2014-01-31 Carlos O'Donell <carlos@redhat.com>
* elf/dl-version.c (match_symbol): Treat GLIBC_2.17 and
GLIBC_2.18 as equivalent.
diff --git a/elf/dl-version.c b/elf/dl-version.c
index 651d4cf..d0e4443 100644
--- a/elf/dl-version.c
+++ b/elf/dl-version.c
@@ -124,16 +124,25 @@ no version information available (required by ", name, ")");
goto call_cerror;
}
+ /* HACK: Also compare the strings and make 2.17 equivalent to 2.18. */
+ ElfW(Verdaux) *aux = (ElfW(Verdaux) *) ((char *) def + def->vd_aux);
+ int equiv = ((strcmp (string, "GLIBC_2.17") == 0
+ || strcmp (string, "GLIBC_2.18") == 0)
+ && (strcmp (strtab + aux->vda_name, "GLIBC_2.17") == 0
+ || strcmp (strtab + aux->vda_name, "GLIBC_2.18") == 0));
+
/* Compare the hash values. */
- if (hash == def->vd_hash)
+ if (hash == def->vd_hash || equiv)
{
- ElfW(Verdaux) *aux = (ElfW(Verdaux) *) ((char *) def + def->vd_aux);
-
/* To be safe, compare the string as well. */
if (__builtin_expect (strcmp (string, strtab + aux->vda_name), 0)
== 0)
/* Bingo! */
return 0;
+
+ if (equiv)
+ /* Not the same, but equivalent. */
+ return 0;
}
/* If no more definitions we failed to find what we want. */
---
Patch #1
2014-01-31 Carlos O'Donell <carlos@redhat.com>
* elf/dl-lookup.c (do_lookup_x): Treat GLIBC_2.17 and
GLIBC_2.18 as equivalent.
diff --git a/elf/dl-lookup.c b/elf/dl-lookup.c
index a58e5bc..820f5a0 100644
--- a/elf/dl-lookup.c
+++ b/elf/dl-lookup.c
@@ -177,8 +177,17 @@ do_lookup_x (const char *undef_name, uint_fast32_t new_hash,
|| strcmp (map->l_versions[ndx].name, version->name))
&& (version->hidden || map->l_versions[ndx].hash
|| (verstab[symidx] & 0x8000)))
- /* It's not the version we want. */
- return NULL;
+ {
+ /* HACK: Treat GLIBC_2.17 and GLIBC_2.18 as equivalent. */
+ if ((strcmp (map->l_versions[ndx].name, "GLIBC_2.17") == 0
+ || strcmp (map->l_versions[ndx].name, "GLIBC_2.18") == 0)
+ && (strcmp (version->name, "GLIBC_2.17") == 0
+ || strcmp (version->name, "GLIBC_2.18") == 0))
+ return sym;
+
+ /* It's not the version we want. */
+ return NULL;
+ }
}
}
else
---
Patch #2-A
2014-01-31 Carlos O'Donell <carlos@redhat.com>
* elf/dl-lookup.c (check_match): New function.
(do_lookup_x): Remove nested function check_match.
Use non-nested function check_match.
diff --git a/elf/dl-lookup.c b/elf/dl-lookup.c
index a58e5bc..5d676c5 100644
--- a/elf/dl-lookup.c
+++ b/elf/dl-lookup.c
@@ -68,6 +68,116 @@ struct sym_val
# define bump_num_relocations() ((void) 0)
#endif
+/* Utility function for do_lookup_x. The caller is called with undef_name,
+ ref, version, flags and type_class, and those are passed as the first
+ five arguments. The caller then computes sym, symidx, strtab, and map
+ and passes them as the next four arguments. Lastly the caller passes in
+ versioned_sym and num_versions which are modified by check_match during
+ the checking process. */
+static const ElfW(Sym) *
+check_match (const char *const undef_name,
+ const ElfW(Sym) *const ref,
+ const struct r_found_version *const version,
+ const int flags,
+ const int type_class,
+ const ElfW(Sym) *const sym,
+ const Elf_Symndx symidx,
+ const char *const strtab,
+ const struct link_map *const map,
+ const ElfW(Sym) **const versioned_sym,
+ int *const num_versions)
+{
+ unsigned int stt = ELFW(ST_TYPE) (sym->st_info);
+ assert (ELF_RTYPE_CLASS_PLT == 1);
+ if (__builtin_expect ((sym->st_value == 0 /* No value. */
+ && stt != STT_TLS)
+ || (type_class & (sym->st_shndx == SHN_UNDEF)),
+ 0))
+ return NULL;
+
+ /* Ignore all but STT_NOTYPE, STT_OBJECT, STT_FUNC,
+ STT_COMMON, STT_TLS, and STT_GNU_IFUNC since these are no
+ code/data definitions. */
+#define ALLOWED_STT \
+ ((1 << STT_NOTYPE) | (1 << STT_OBJECT) | (1 << STT_FUNC) \
+ | (1 << STT_COMMON) | (1 << STT_TLS) | (1 << STT_GNU_IFUNC))
+ if (__builtin_expect (((1 << stt) & ALLOWED_STT) == 0, 0))
+ return NULL;
+
+ if (sym != ref && strcmp (strtab + sym->st_name, undef_name))
+ /* Not the symbol we are looking for. */
+ return NULL;
+
+ const ElfW(Half) *verstab = map->l_versyms;
+ if (version != NULL)
+ {
+ if (__builtin_expect (verstab == NULL, 0))
+ {
+ /* We need a versioned symbol but haven't found any. If
+ this is the object which is referenced in the verneed
+ entry it is a bug in the library since a symbol must
+ not simply disappear.
+
+ It would also be a bug in the object since it means that
+ the list of required versions is incomplete and so the
+ tests in dl-version.c haven't found a problem.*/
+ assert (version->filename == NULL
+ || ! _dl_name_match_p (version->filename, map));
+
+ /* Otherwise we accept the symbol. */
+ }
+ else
+ {
+ /* We can match the version information or use the
+ default one if it is not hidden. */
+ ElfW(Half) ndx = verstab[symidx] & 0x7fff;
+ if ((map->l_versions[ndx].hash != version->hash
+ || strcmp (map->l_versions[ndx].name, version->name))
+ && (version->hidden || map->l_versions[ndx].hash
+ || (verstab[symidx] & 0x8000)))
+ /* It's not the version we want. */
+ return NULL;
+ }
+ }
+ else
+ {
+ /* No specific version is selected. There are two ways we
+ can get here:
+
+ - a binary which does not include versioning information
+ is loaded
+
+ - dlsym() instead of dlvsym() is used to get a symbol which
+ might exist in more than one form
+
+ If the library does not provide symbol version information
+ there is no problem at all: we simply use the symbol if it
+ is defined.
+
+ These two lookups need to be handled differently if the
+ library defines versions. In the case of the old
+ unversioned application the oldest (default) version
+ should be used. In case of a dlsym() call the latest and
+ public interface should be returned. */
+ if (verstab != NULL)
+ {
+ if ((verstab[symidx] & 0x7fff)
+ >= ((flags & DL_LOOKUP_RETURN_NEWEST) ? 2 : 3))
+ {
+ /* Don't accept hidden symbols. */
+ if ((verstab[symidx] & 0x8000) == 0
+ && (*num_versions)++ == 0)
+ /* No version so far. */
+ *versioned_sym = sym;
+
+ return NULL;
+ }
+ }
+ }
+
+ /* There cannot be another entry for this symbol so stop here. */
+ return sym;
+}
/* Inner part of the lookup functions. We return a value > 0 if we
found the symbol, the value 0 if nothing is found and < 0 if
@@ -123,106 +233,9 @@ do_lookup_x (const char *undef_name, uint_fast32_t new_hash,
const ElfW(Sym) *symtab = (const void *) D_PTR (map, l_info[DT_SYMTAB]);
const char *strtab = (const void *) D_PTR (map, l_info[DT_STRTAB]);
-
- /* Nested routine to check whether the symbol matches. */
- const ElfW(Sym) *
- __attribute_noinline__
- check_match (const ElfW(Sym) *sym)
- {
- unsigned int stt = ELFW(ST_TYPE) (sym->st_info);
- assert (ELF_RTYPE_CLASS_PLT == 1);
- if (__builtin_expect ((sym->st_value == 0 /* No value. */
- && stt != STT_TLS)
- || (type_class & (sym->st_shndx == SHN_UNDEF)),
- 0))
- return NULL;
-
- /* Ignore all but STT_NOTYPE, STT_OBJECT, STT_FUNC,
- STT_COMMON, STT_TLS, and STT_GNU_IFUNC since these are no
- code/data definitions. */
-#define ALLOWED_STT \
- ((1 << STT_NOTYPE) | (1 << STT_OBJECT) | (1 << STT_FUNC) \
- | (1 << STT_COMMON) | (1 << STT_TLS) | (1 << STT_GNU_IFUNC))
- if (__builtin_expect (((1 << stt) & ALLOWED_STT) == 0, 0))
- return NULL;
-
- if (sym != ref && strcmp (strtab + sym->st_name, undef_name))
- /* Not the symbol we are looking for. */
- return NULL;
-
- const ElfW(Half) *verstab = map->l_versyms;
- if (version != NULL)
- {
- if (__builtin_expect (verstab == NULL, 0))
- {
- /* We need a versioned symbol but haven't found any. If
- this is the object which is referenced in the verneed
- entry it is a bug in the library since a symbol must
- not simply disappear.
-
- It would also be a bug in the object since it means that
- the list of required versions is incomplete and so the
- tests in dl-version.c haven't found a problem.*/
- assert (version->filename == NULL
- || ! _dl_name_match_p (version->filename, map));
-
- /* Otherwise we accept the symbol. */
- }
- else
- {
- /* We can match the version information or use the
- default one if it is not hidden. */
- ElfW(Half) ndx = verstab[symidx] & 0x7fff;
- if ((map->l_versions[ndx].hash != version->hash
- || strcmp (map->l_versions[ndx].name, version->name))
- && (version->hidden || map->l_versions[ndx].hash
- || (verstab[symidx] & 0x8000)))
- /* It's not the version we want. */
- return NULL;
- }
- }
- else
- {
- /* No specific version is selected. There are two ways we
- can got here:
-
- - a binary which does not include versioning information
- is loaded
-
- - dlsym() instead of dlvsym() is used to get a symbol which
- might exist in more than one form
-
- If the library does not provide symbol version information
- there is no problem at all: we simply use the symbol if it
- is defined.
-
- These two lookups need to be handled differently if the
- library defines versions. In the case of the old
- unversioned application the oldest (default) version
- should be used. In case of a dlsym() call the latest and
- public interface should be returned. */
- if (verstab != NULL)
- {
- if ((verstab[symidx] & 0x7fff)
- >= ((flags & DL_LOOKUP_RETURN_NEWEST) ? 2 : 3))
- {
- /* Don't accept hidden symbols. */
- if ((verstab[symidx] & 0x8000) == 0
- && num_versions++ == 0)
- /* No version so far. */
- versioned_sym = sym;
-
- return NULL;
- }
- }
- }
-
- /* There cannot be another entry for this symbol so stop here. */
- return sym;
- }
-
const ElfW(Sym) *sym;
const ElfW(Addr) *bitmask = map->l_gnu_bitmask;
+
if (__builtin_expect (bitmask != NULL, 1))
{
ElfW(Addr) bitmask_word
@@ -246,7 +259,10 @@ do_lookup_x (const char *undef_name, uint_fast32_t new_hash,
if (((*hasharr ^ new_hash) >> 1) == 0)
{
symidx = hasharr - map->l_gnu_chain_zero;
- sym = check_match (&symtab[symidx]);
+ sym = check_match (undef_name, ref, version, flags,
+ type_class, &symtab[symidx], symidx,
+ strtab, map, &versioned_sym,
+ &num_versions);
if (sym != NULL)
goto found_it;
}
@@ -268,7 +284,11 @@ do_lookup_x (const char *undef_name, uint_fast32_t new_hash,
symidx != STN_UNDEF;
symidx = map->l_chain[symidx])
{
- sym = check_match (&symtab[symidx]);
+ sym = check_match (undef_name, ref, version, flags,
+ type_class, &symtab[symidx], symidx,
+ strtab, map, &versioned_sym,
+ &num_versions);
+
if (sym != NULL)
goto found_it;
}
---
Patch #2-B
2014-01-31 Carlos O'Donell <carlos@redhat.com>
* elf/dl-lookup.c (check_match): Treat GLIBC_2.17 and
GLIBC_2.18 as equivalent.
--- elf/dl-lookup.c.orig 2014-01-31 01:58:03.391902187 -0500
+++ elf/dl-lookup.c 2014-01-31 01:58:18.455912963 -0500
@@ -135,8 +135,17 @@
|| strcmp (map->l_versions[ndx].name, version->name))
&& (version->hidden || map->l_versions[ndx].hash
|| (verstab[symidx] & 0x8000)))
- /* It's not the version we want. */
- return NULL;
+ {
+ /* HACK: Treat GLIBC_2.17 and GLIBC_2.18 as equivalent. */
+ if ((strcmp (map->l_versions[ndx].name, "GLIBC_2.17") == 0
+ || strcmp (map->l_versions[ndx].name, "GLIBC_2.18") == 0)
+ && (strcmp (version->name, "GLIBC_2.17") == 0
+ || strcmp (version->name, "GLIBC_2.18") == 0))
+ return sym;
+
+ /* It's not the version we want. */
+ return NULL;
+ }
}
}
else
---