When setting the base address to something else than 0x400000, ld will insert a
large (some 1MiB) hole right after the elf header. This is not directly a
problem, except if some third-party program (e.g. grub) relies on finding
secondary headers (e.g. the multiboot header) early in the file.

Example linker script:
OUTPUT_FORMAT("elf64-x86-64", "elf64-x86-64",
          "elf64-x86-64")
OUTPUT_ARCH(i386:x86-64)
ENTRY(_start)
SECTIONS
{
  . = 0x100000 + SIZEOF_HEADERS;
  .text : { *(*) }
}