Conversely, perhaps ldconfig should use ${prefix}-based file names when ${prefix}!=/usr so that you would get a separate ld.so.cache universe for your separate libc installation.