[hjl@gnu-6 gold]$ cat main.c 
extern void foo (void);

int
main (void)
{
  foo ();
  return 0;
}
[hjl@gnu-6 gold]$ cat tst-quadmod2.S
#define BIAS 0x7fff0000

	.section	.data.rel.local,"aw",@progbits
	.align 8
.Ljmp:
	.quad	func + BIAS
	.text
	.type	func, @function
func:
	.cfi_startproc
	xorl	%edi, %edi
	jmp	exit@PLT
	.cfi_endproc
	.size	func, .-func
	.globl	foo
	.type	foo, @function
foo:
	.cfi_startproc
	.cfi_def_cfa_register 6
	movq	.Ljmp(%rip), %rax
	subq	$BIAS, %rax
	jmp	*%rax
	.cfi_endproc
	.size	foo, .-foo
[hjl@gnu-6 gold]$ make
gcc -B./ -mx32 -g   -c -o main.o main.c
gcc -B./ -mx32 -c -g -o tst-quadmod2.o tst-quadmod2.S
./ld -shared -m elf32_x86_64 -o libtst-quadmod2.so tst-quadmod2.o
gcc -B./ -mx32 -g -o x2 main.o libtst-quadmod2.so -Wl,-rpath,.
./readelf -r --wide libtst-quadmod2.so

Relocation section '.rela.dyn' at offset 0x164 contains 1 entries:
 Offset     Info    Type                Sym. Value  Symbol's Name + Addend
000012f0  00000008 R_X86_64_RELATIVE                  7fff01a0

Relocation section '.rela.plt' at offset 0x170 contains 1 entries:
 Offset     Info    Type                Sym. Value  Symbol's Name + Addend
000012e8  00000107 R_X86_64_JUMP_SLOT     00000000   exit + 0
./x2
make: *** [all] Segmentation fault
[hjl@gnu-6 gold]$ 

The problem is

[hjl@gnu-6 gold]$ readelf -rW tst-quadmod2.o
...
 Offset     Info    Type                Sym. Value  Symbol's Name + Addend
00000000  00000101 R_X86_64_64            00000000   .text + 7fff0000

Symbol value + addend > 32bits at run-time.  BFD linker generates:

[hjl@gnu-6 gold]$ make LD=ld
gcc -B./ -mx32 -g   -c -o main.o main.c
gcc -B./ -mx32 -c -g -o tst-quadmod2.o tst-quadmod2.S
ld -shared -m elf32_x86_64 -o libtst-quadmod2.so tst-quadmod2.o
gcc -B./ -mx32 -g -o x2 main.o libtst-quadmod2.so -Wl,-rpath,.
./readelf -r --wide libtst-quadmod2.so

Relocation section '.rela.dyn' at offset 0x158 contains 1 entries:
 Offset     Info    Type                Sym. Value  Symbol's Name + Addend
002002b0  00000026 R_X86_64_RELATIVE64                 7fff0190

Relocation section '.rela.plt' at offset 0x164 contains 1 entries:
 Offset     Info    Type                Sym. Value  Symbol's Name + Addend
[hjl@gnu-6 gold]$ 

The difference is R_X86_64_RELATIVE64 vs R_X86_64_RELATIVE.  For
R_X86_64_64, we need to generate R_X86_64_RELATIVE64.